7 – LSTM 7 Use Gate

And finally, we come to the use gate or output gate. This is the one that uses the long term memory that just came out of the forget gate and the short term memory that just came out of the learned gate, to come up with a new short term memory and an output. These are the same thing. In this case, we’ll take what’s useful from the long term memory which is this bear over here, and what’s useful from the short term memory which is these dark wolf, and the squirrel, and that’s what’s going to be our new short term memory. So our output basically says, your image is most likely a wolf but it also carry some of the other animals that I’ve seen recently. And mathematically what this does is the following: it applies a small neural network on the output of the forget gate using the tanh activation function, and it applies to another small neural network on the short term memory and the events using the sigmoid activation function. And as a final step, it multiplies these two in order to get the new output. The output also worth of the new short term memory. And that’s how they use gate works.

%d 블로거가 이것을 좋아합니다: