3 – Making Batches

Welcome back everyone. So now you are going to actually start changing the data into batches. You are going to be creating batches for this to pass into the network. So remember, with skip-gram architecture, we’re going to take one word and then we’re going to use the words that are around that word in … Read more

8 – Training Results

Okay, so the rest of this is the validation code so we can, kind of, keep track of the performance of our word2vec network as it’s training. And then here is some training code. This is, you know, fairly, fairly typical way of doing it. So you just create your batches and then create a … Read more

7 – Building The Network Solution

Hello again. So, here I’m going to be showing you how to actually build this network, so go through the inputs and the embedding layer and the soft max. So, as usual, we have our tensor flow placeholders, so we’re using integers in both of these and so just tf.n32 and then the labels have … Read more

6 – Negative Sampling

Welcome back. So this is the last part that you’ll be implementing yourself. In our architecture, we have this you know softmax layer on the output and you know since we’re working with, you know tens of thousands of words the softmax layer is going to have tens of thousands of units. But in any … Read more

5 – Building The Network

Hello everyone. Welcome back. So now we’re going to actually start building the network. So here we can see the general structure of the network that we’re going to build. So we have our inputs, and as I said before this, we’re not actually going to be doing this you know one hot encoding, we’re … Read more

4 – Batches Solution

Hey there, welcome back. So, now, here’s my solution for the get_target function. So the first thing I did is generate our random window R so using random.randint. So, we want to go from one to our window size, but when you actually, like, do randint, it’s not inclusive, so you wouldn’t get, like, if … Read more

2 – Subsampling Solution

Hi everyone welcome back. So here are my solutions for the subsampling part. So here I am basically just going to, you know just calculate this probability and then scroll through the entire, like into words our data set and drop words that are frequent. So the first thing I did here was calculate the … Read more

1 – Implementing Word2Vec

Welcome back everyone. So this week you’re going to be exploring embeddings and implementing the word2vec model to understand them. So in this notebook you’ll actually be using TensorFlow to implement the word2vec model. Here are some resources for you to read. So you should check these out either beforehand or while you’re working on … Read more