5 – Training The Network

(speaker) So here I am calculating the output. So like I said before, we’re only really concerned with the final output, which is what we’re going to use to predict our sentiment. So we just need to grab the last one and we can do that using outputs. So this is saying, give us all the rows, so all the batches. And this is saying, give us the last column. Remember that we’re passing a bunch of reviews in at the same time because that’s our batch, and then the last column is just the last output from our LSTM cells. So this last output is this arrow here, is the last output from the LSTM cells, and we’re ignoring all these other outputs. So then these outputs from the LSTM are just going to a fully connected layer that only has one unit because we’re just doing this one like sigmoid predictor unit. So the activation function is here, tf.sigmoid, and this gives us our prediction. So remember that sigmoid is always between 0 and 1, so you can treat it as a probability that is true, it’s a probability that is positive. Then here we’re just using our labels and our predictions to calculate the cost with the mean squared error. And finally, we have our AdamOptimizer that is going to minimize our cost and train the weights. And here I added this little bit to use as a validation accuracy. So I’m just rounding our predictions, because remember predictions are somewhere between 0 and 1. But we want them to be 0 and 1 so that we can say, is it equal to our labels? So here I’m just rounding predictions, so this makes it either 0 or 1, depending on what the value is. And then I’m going to change the data type, I’m going to cast it to an int because our labels are ints. So we want these to have the same data types so that we can do this check of equality. Remember that we’re passing in a bunch of batches. So we have a batch and we’re passing in a bunch of reviews with each batch. So now we actually want to find the mean for all these accuracies. To do that, we’re going to use tf.reduce_mean, so this just finds the mean of all the predictions, and it’s going to give us our accuracy. So here, correct prediction is actually a boolean, that’s what the data type comes out of equal. So we actually have to cast this to a float to get it to work in reduce_mean. So that’s what’s going on here. Then here is just a function to get batches for us. So you can read this text and read through it, but this is pretty typical. So basically I just have two arrays x and y and then here I’m just making sure that every batch is a full batch, so I’m kind of just getting rid of extra data. And then I’m just looping through each of these arrays and in this case yielding batches. And finally, this code is the training code. So this is a lot of code and it’s pretty standard. This is really similar to training code that I write in every single of these notebooks. So if you want to kind of like an extra challenge to you, just go ahead and delete this entire cell and then try to implement it yourself. One way, probably the best way to learn this stuff, is to actually type everything yourself, because that’s sort of the deal with tutorials, is that you just read these tutorials and you think you learned it but then you try to do it and it turns out you didn’t learn it because you didn’t type it in yourself. So it would probably be a good idea for you to just delete all this training code and try to implement it yourself. And of course, you can copy this to another file so you can look at that and see how you’re doing. And finally, we have testing code down here. So I will let you build all this, do the exercises, and run this code. And if you have problems with this, I’m going to have a solution video also and a solution notebook so you can see how I did all this. OK, cheers!

%d 블로거가 이것을 좋아합니다: