4-5. Embeddings and Word2Vec

Welcome back everyone. So now you are going to actually start changing the data into batches. You are going to be creating batches for this to pass into the network. So remember, with skip-gram architecture, we’re going to take one word and then we’re going to use the words that are around that word in … Read more

Okay, so the rest of this is the validation code so we can, kind of, keep track of the performance of our word2vec network as it’s training. And then here is some training code. This is, you know, fairly, fairly typical way of doing it. So you just create your batches and then create a … Read more

Hello again. So, here I’m going to be showing you how to actually build this network, so go through the inputs and the embedding layer and the soft max. So, as usual, we have our tensor flow placeholders, so we’re using integers in both of these and so just tf.n32 and then the labels have … Read more

Welcome back. So this is the last part that you’ll be implementing yourself. In our architecture, we have this you know softmax layer on the output and you know since we’re working with, you know tens of thousands of words the softmax layer is going to have tens of thousands of units. But in any … Read more

Hello everyone. Welcome back. So now we’re going to actually start building the network. So here we can see the general structure of the network that we’re going to build. So we have our inputs, and as I said before this, we’re not actually going to be doing this you know one hot encoding, we’re … Read more

Hey there, welcome back. So, now, here’s my solution for the get_target function. So the first thing I did is generate our random window R so using random.randint. So, we want to go from one to our window size, but when you actually, like, do randint, it’s not inclusive, so you wouldn’t get, like, if … Read more

Hi everyone welcome back. So here are my solutions for the subsampling part. So here I am basically just going to, you know just calculate this probability and then scroll through the entire, like into words our data set and drop words that are frequent. So the first thing I did here was calculate the … Read more

Welcome back everyone. So this week you’re going to be exploring embeddings and implementing the word2vec model to understand them. So in this notebook you’ll actually be using TensorFlow to implement the word2vec model. Here are some resources for you to read. So you should check these out either beforehand or while you’re working on … Read more