7 – M4 L2b 09 Writing It Down Pt 2 V2

Now for simplicity, let’s look at just one of these mean centered data points. We’re looking for a new basis for the data and we want to write down the coordinates of this point in a new basis. This is the same thing we discussed earlier, we are looking for two new basis vectors for the 2D plane which give us an alternate way of describing our original points location. Okay. Let’s say we’ve run PCA and it has spit out the new basis directions, these are called Principal Components PCs or just components. They are the directions of the new basis as written in the language of the old basis. It’s required by the algorithm that these have length 1 and be perpendicular to each other. So, let’s show them slightly smaller. We’ll consider them to be length 1 now. Now, let’s find our points’ coordinates in the new basis. Let’s do one coordinate at a time. We’ll start with the red basis vector. Let’s write down some names, let’s call the vector pointing to our original data point X and the red basis vector W. Notice that what we’ve formed here is a right triangle. If X is the length of the X vector and Theta is the angle between the X vector and the W vector and you remember your trigonometry, you’ll see that this distance is X cosine Theta. Now, let’s remember the formula for the dot product. X.W is the length of X times the length of W times the cosine of the angle between them, Theta. So this length equals X.W divided by the length of W. This quantity is the projection of our original point onto the new basis direction, and it turns out it’s exactly the quantity we’re looking to maximize over all the data points when we pick our new basis direction. In fact, when I say maximize the variance, I actually mean that if we have all these dot products for all the X vectors pointing to each of our data points, we want the variance of this new set of numbers to be as large as possible.