2-8. Introduction to Neural Networks

Now, I’m going to leave you with a question. Here is the XOR perceptron, which is very similar to the other two except this one returns a true if exactly one of them is true and the other one is false. So it returns this table. Now, the question is can we turn this into … Read more

So here’s something very interesting about perceptrons and it’s that some logical operators can be represented as perceptrons. Here, for example, we have the AND operator and how does that work? The AND operator takes two inputs and it returns an output. The inputs can be true or false but the output is only true … Read more

So you may be wondering why are these objects called neural networks. Well, the reason why they’re called neural networks is because perceptions kind of look like neurons in the brain. In the left we have a perception with four inputs. The number is one, zero, four, and minus two. And what the perception does, … Read more

So let’s recap. We have our data which is all these students. The blue ones have been accepted and the red ones have been rejected. And we have our model which consists of the equation two times test plus grades minus 18, which gives rise to this boundary which the point where the score is … Read more

Now, you may be wondering what happens if we have more data columns so not just testing grades, but maybe something else like the ranking of the student in the class. How do we fit three columns of data? Well the only difference is that now, we won’t be working in two dimensions, we’ll be … Read more

So, let us go back to our neural network with our weights and our input. And recall that the weights with superscript 1 belong to the first layer, and the weights with superscript 2 belong to the second layer. Also, recall that the bias is not called b anymore. Now, it is called W31, W32 … Read more

So before we start calculating derivatives, let’s do a refresher on the chain rule which is the main technique we’ll use to calculate them. The chain rule says, if you have a variable x on a function f that you apply to x to get f of x, which we’re gonna call A, and then … Read more

Okay. So, now we’ll do the same thing as we did before, painting our weights in the neural network to better classify our points. But we’re going to do it formally, so fasten your seat belts because math is coming. On your left, you have a single perceptron with the input vector, the weights and … Read more

So now we’re finally ready to get our hands into training a neural network. So let’s quickly recall feedforward. We have our perceptron with a point coming in labeled positive. And our equation w1x1 + w2x2 + b, where w1 and w2 are the weights and b is the bias. Now, what the perceptron does … Read more

So, our goal is to train our neural network. In order to do this, we have to define the error function. So, let’s look again at what the error function was for perceptrons. So, here’s our perceptron. In the left, we have our input vector with entries x_1 up to x_n, and one for the … Read more

So now that we have defined what neural networks are, we need to learn how to train them. Training them really means what parameters should they have on the edges in order to model our data well. So in order to learn how to train them, we need to look carefully at how they process … Read more

We briefly mentioned multi-class classification in the last video but let me be more specific. It seems that neural networks work really well when the problem consist on classifying two classes. For example, if the model predicts a probability of receiving a gift or not then the answer just comes as the output of the … Read more

Neural networks have a certain special architecture with layers. The first layer is called the input layer, which contains the inputs, in this case, x1 and x2. The next layer is called the hidden layer, which is a set of linear models created with this first input layer. And then the final layer is called … Read more

So in the previous session we learn that we can add to linear models to obtain a third model. As a matter of fact, we did even more. We can take a linear combination of two models. So, the first model times a constant plus the second model times a constant plus a bias and … Read more

So, first let’s add some math. We’re going to label the horizontal axis corresponding to the test by the variable x1, and the vertical axis corresponding to the grades by the variable x2. So this boundary line that separates the blue and the red points is going to have a linear equation. The one drawn … Read more

Now I’m going to show you how to create these nonlinear models. What we’re going to do is a very simple trick. We’re going to combine two linear models into a nonlinear model as follows. Visually it looks like this. The two models over imposed creating the model on the right. It’s almost like we’re … Read more

So, let’s go back to this example of where we saw some data that is not linearly separable. So a line can not divide these red and blue points and we looked at some solutions, and if you remember, the one we considered more seriously was this curve over here. So what I’ll teach you … Read more

Now we’ve been dealing a lot with data sets that can be separated by a line, like this one over here. But as you can imagine the real world is much more complex than that. This is where neural networks can show their full potential. In the next few videos we’ll see how to deal … Read more

So, this is just a small recap video that will get us ready for what’s coming. Recall that if we have our data in the form of these points over here and the linear model like this one, for example, with equation 2×1 + 7×2 – 4 = 0, this will give rise to a … Read more

So let’s compare the Perceptron algorithm and the Gradient Descent algorithm. In the Gradient Descent algorithm, we take the weights and change them from Wi to Wi_ plus_ alpha_ times_ Y hat_ minus_ Y_ times_ Xi. In the Perceptron algorithm, not every point changes weights, only the misclassified ones. Here, if X is misclassified, we’ll … Read more