Let’s look at the architecture of a classification CNN that takes in an image and outputs a predicted class for that image. You’ve seen how to train CNN’s like this on sets of labeled image data, and you’ve learned about the layers that make up a CNN. Things like convolutional layers that learned to filter images and pooling layers that reduced the size of your input. We know in theory what’s happening in these layers, but after a network has been trained and especially in networks with many layers, how can we know exactly what each layer has learned and why is this important? Take this classifier as an example. Say it’s only been training for a little while and it’s learning to distinguish images of dogs from images of wolves, but it’s making some mistakes. Here are a few images from the training set for comparison. Well can you see that distinguishes these images? You may look at the mouth of the wolf, and the dog, the amount of fur or overall shape. But what does the network see? It turns out that since this model was training on a set of images, in which wolves were primarily pictured on a snowy landscape, it actually learned that snow was an important feature to extract. If there was snow, then an image was most likely classified as a wolf, even if it was really an image of a dog. So because of the limited training data, the model learned to identify snow, rather than really identifying the different features of a dog and a wolf and if we had some insight into what features as model was extracting we could actually visualize this shortcoming. So how can we see what kind of features and network has learned to recognize? This area of study is called feature visualization and it’s all about techniques that allow you to see at each layer of the model. What kinds of image features and network has learned to extract? And we’ll cover a variety of these techniques in the next few videos.