5 – DL 53 Q Regularization

Now let me show you a subtle way of overfitting a model. Let’s look at the simplest data set in the world, two points, the point one one which is blue and the point minus one minus one which is red. Now we want to separate them with a line. I’ll give you two equations and you tell me which one gives a smaller error and that’s going to be the quiz. Equation one is x1 plus x2. That means w1 equals w2 is one and the bias b is zero. And then equation two is 10×1 plus 10×2. So that means w1 equals w2 equals 10 and the bias b equals zero. Now the question is, which prediction gives a smaller error? This is not an easy question but I want you to think about it and maybe make some calculations, if necessary.