So, how exactly does optical flow work? We know that it tries to track points from one image frame to another based on the intensity levels of points in each image. Let’s see this in a simple example. Say we have two image frames from a video, and for one point on object in image one, we want to find out where it is in image two. Once we do, we can calculate a motion vector that describes the velocity of this point from the first frame to the next. The math works like this, a point in our first image (x,y), will move some amount from this frame to the next. It will have moved some distance u horizontally, and some distance v vertically. So in the second image frame, that point will be at the coordinate x plus u and y plus v. This motion can be described by a motion vector (u, v). Vector is a quantity that has a magnitude and a direction. Let’s say that a point has moved three pixels to the right, and four pixels up, then our point (x,y) in the first frame will be at x plus three y plus four in the second frame. The motion vector will be (3,4). The magnitude of any vector in x, y space can be found by the Pythagorean theorem for sides of a triangle. A-squared plus b-squared equals c-squared. So in this case, the magnitude of the vector equals the square root of three squared plus four squared, that’s the square root of 25, which is five. The orientation of this vector can be found using trigonometry. Where the angle equals the inverse tangent of four over three, which is about 53.13 degrees. Knowing the magnitude and direction of a moving point or a set of moving points, is all you really need to know to track an object. Next, we’ll learn more about how optical flow estimates motion vectors like these.