(instructor) Hello again, everyone. So this week we are going to be talking about Sentiment Analysis with Recurrent Neural Network. So hopefully this will give you some more insight, and some more understanding about how recurrent neural networks work. So you’ve seen a lot of this before. We’re going to be using the same data that Andrew Trask used. But in his case, he wasn’t using a recurrent neural network, he was just a normal feed forward network. Sarraj also implemented a network very similar to this in one of his videos. This is what the network looks like. So the idea is that we’re going to pass in a bunch of words from some movie review, and then we have labels for these reviews. They’re either positive or negative. So what we want to do with this network is that we want to take other movie reviews that we haven’t seen and predict if they’re positive or negative. So the way we’re going to do this is we’re going to pass in a bunch of movie reviews with labels and train the network. So the idea is here is our network, and we have kind of a typical setup. So we have our words and then these are going to go into an embedding layer. So we learned about embeddings before when you were implementing Word2Vec. So the problem we have is that when you have words, you end up with tens of thousands of different classes. And so if you try to one hot encode that, it’s going to be really inefficient. So instead, we’re going to encode our words as integers, and then pass those to an embedding layer so that we can get out these embedded vectors, these representations for our words. Then those are passed to LSTM cells. So these are our long and short term memory cells. And they act basically exactly like a hidden layer. And the idea here is that since it’s recurrent, then we pass the state of the hidden layers to the next step in our recurrent network. And that goes to the next step, and eventually goes out. And for an output layer we’re going to use just one unit with a Sigmoid activation. So again we’re just trying to classify things are positive or negative. And so in that case we just need to have a zero or a one output. And what the Sigmoid does is it squishes things between zero and one so you get this probability of being positive, or probability of being negative. Right? So we’re just going use the Sigmoid as our output. And the deal here is that, and you can see I don’t have anything labeled above these, so we’re really only interested in the very last output. So we’re just going to pass in a sequence of words, and then the last output we’re going to use as our labels or target. So in this case best movie ever is probably a positive review. So we’re going to have it as positive. And this is the general idea of our network. We’re just going to pass this in. It’s going to go through our embedding layer to the LSTM hidden layers, and then to our Sigmoid output layer, and then we’re going to calculate the cost from the very last output. OK. Let’s get started building this thing. So here I’m just importing libraries I need, NumPy and TensorFlow, and then loading the data, reviews, and labels. And here you can see an example of some of the reviews. One thing I didn’t notice from this is that you see these backslash n. So these are new line characters. And what they’re doing, they’re actually delimiting the different reviews in this data set. So this backlash n is the second review. And then we see another backslash n here, and this is the third review. So we know how to– so from this we can tell how to separate our reviews, like just this giant block of text in two separate reviews if we split this giant string with the newline character. So you might have seen before, when we look at all of our text, there’s a whole bunch of periods and other punctuations. So first we want to get rid of that. And it’s pretty easy to do. So we’re just going to get, this is basically a string of all punctuations, and then we can go through our reviews. And then if some character in reviews is not a punctuation, then we keep it in. Otherwise, we get rid of it. And then we can split our text into reviews. Just split it on the newline character, and then we’re just going to join back their reviews into its full text thing. So the reason I’m doing this is because if we try to split these things into words, then this is a word. Right, so this new line homelessness is a word. And then this new line story is another word. So when we’re trying to find our vocab, these are going to end up as words, but we don’t really want our words to have new lines in them. So basically what I did is I just split it into two reviews, and then I just put it all back together without new lines. So now we have, just. This all text now is just one long string of words. Then we can get our words by, again, just splitting. So now these are all of our words, so you can see here this is all text. So again, all of our text is one long string that has no periods and no new lines. So we can get all of our words just from this. And you can see the words here. So Brummel High is a cartoon comedy, et cetera.