4-6. PyTorch

5 – PyTorch C API V2

by

Now, let’s see how we actually load our Script Module into C++. So for this, there is a requirement that we need LibTorch, so this is the C++ API for PyTorch and we’ll also want to use CMake for building our actual application. So, here we’re just going to make a very simple C++ application … Read more

4 – PyTorch Script Annotation V2

by

Well, sometimes your model might use forms of control flow that don’t actually work with this tracing method. So, for example, you might have some if statements in your forward method that depend on your input. So, things like this, they’re fairly common in natural language processing problems. So, there is a second way to … Read more

3 – PyTorch Script Tracing V1

by

Welcome to this lesson on using some really cool new features in Pytorch. So, this is specifically features that are being introduced and Pytorch 1.0. What they allow you to do is, export your train models and Pytorch that you’ve trained in Python, and export those to a version that you can then load up … Read more

2 – PyTorch For Production V1

by

Hi there. So, I’m going to be walking you through this tutorial for some new features in PyTorch 1.0. PyTorch 1.0 has been specifically built for making this transition between developing your model in Python, and then converting it into a module that you can load into a C++ environment. The reason I want to … Read more

1 – PyTorch 10 Install V1

by

So, just a quick note about PyTorch. The features I’m demonstrating in this lesson, are only available from PyTorch 1.0. At the time I am recording these videos, hydro 20.0 is not the stable version of PyTorch. So, if you install PyTorch the normal way, so you go to their website, and you click on … Read more

9 – 07 CharRNN Solution V1

by

We wanted to define a character RNN with a two layer LSTM. Here in my solution, I am running this code on GPU and here’s my code for defining our character level RNN. First, I defined an LSTM layer, self.lstm. This takes in an input size, which is going to be the length of a … Read more

8 – 06 Defining Model V2

by

All right. So, we have our mini batches of data and now it’s time to define our model. This is a little diagram of what the model will look like. We’ll have our character’s put into our input layer and then a stack of LSTM cells. These LSTM cells make up our hidden recurrent layer … Read more

7 – 05 Batching Data V1

by

So, this is my complete get_batches code that generates mini-batches of data. So, the first thing I wanted to do here is get the total number of complete batches that we can make in batches. To do that, I first calculated how many characters were in a complete mini-batch. So, in one mini-batch, there’s going … Read more

6 – 04 Implementing CharRNN V2

by

This is a notebook where you’ll be building a characterwise RNN. You’re going to train this on the text of Anna Karenina, which is a really great but also quite sad a book. The general idea behind this, is that we’re going to be passing one character at a time into a recurrent neural network. … Read more

5 – Sequence-Batching

by

One of the most difficult parts of building networks for me is getting the batches right. It’s more of a programming challenge than anything deep learning specific. So here I’m going to walk you through how batching works for RNN. With RNNs we’re training on sequences of data like text, stock values, audio etc. By … Read more

4 – Character-Wise RNN

by

Coming up in this lesson you’ll implement a character-wise RNN. That is, the network will learn about some text one character at a time and then generate new text one character at a time. Let’s say, we want to generate new Shakespeare plays. As an example, to be or not to be. We’d pass the … Read more

3 – 03 Training Memory V1

by

Last time, we defined a model, and next, I want to actually instantiate it and train it using our training data. First, I’ll specify my model hyperparameters. The input and output will just be one, it’s just one sequence at a time that we’re processing and outputting, then I’ll specify a hidden dimension which is … Read more

2 – 02 Time Series Prediction V2

by

To introduce you to RNNs in PyTorch, I’ve created a notebook that will show you how to do simple time series prediction with an RNN. Specifically, we’ll look at some data and see if we can create an RNN to accurately predict the next data point given a current data point, and this is really … Read more

10 – 08 Making Predictions V3

by

Now, the goal of this model is to train it so that it can take in one character and produce a next character and that’s what this next step, Making Predictions is all about. We basically want to create functions that can take in a character and have our network predict the next character. Then, … Read more

1 – M4L31 HSA Implementing RNNs V2 RENDERv1 V2

by

Hi again! So, in the last couple lessons, Ortel and Louise introduce you to recurrent neural networks and LSTMs. In this lesson, Matt and I will be going over some implementations of these networks. Because RNNs have a kind of built-in memory, they’re really useful for tasks that are time or sequence dependent. For example, … Read more

9 – 9 Model Validation Loss V2

by

This is what we want our model to look like. It should take in some inputs and then put those through an embedding layer, which produces some embedded vectors that are sent to a final softmax output layer, and here’s my model definition. You can see that it’s a pretty simple model. First, I’m defining … Read more

8 – 8 Word2vec Model V2

by

Now that we’ve taken the time to preprocess and batch our data, it’s time to actually start building the network. Here, we can see the general structure of the network that we’re going to build. So, we have our inputs, which are going to be like batches of our train word tokens, and as we … Read more

7 – 7 Batching Data Solution V1

by

Here’s how I’m defining the context targets around a given word index. First, according to the excerpt from the paper, I’m going to define a range R. R is going to be a random integer in the range one to c, the window size. randint takes in a range that is not inclusive of the … Read more

6 – 6 Defining Context Targets V1

by

Now that our data is in good shape, we need to get it into the proper form to pass it into our network. With the skip-gram architecture for each word in the text, we want to define a surrounding context and grab all the words in a window around that word with size C. When … Read more

5 – 5 Subsampling Solution V1

by

Here is my solution for creating a new list of train words. First, I calculated the frequency of occurrence for each word in our vocabulary. So, I stored the total length of our text in a variable, total_count. Then, I created a dictionary of frequencies. For each word token and count in the word counter … Read more