More Deep Learning Models

Semantic segmentation is the task of assigning meaning to part of an object. This can be done at the pixel level where we assign each pixel to a target class such as road, car, pedestrian, sign, or any number of other classes. Semantic segmentation help us derive valuable information about every pixel in the image … Read more

Our first task is object detection. And for that, we can use bounding boxes. They are a simpler method of scene understanding compared to segmentation. In neural network, just has to figure out where an object is and draw a type box around it. There are already great open source state of the art solutions, … Read more

Let’s take a few moments to talk about using FCNs in practice. An FCN has two components, the encoder and the decoder. We mentioned that encoder extracts features that will later be used by the decoder. This may sound familiar to transfer learning, and it is. In fact, we can borrow techniques from transfer learning … Read more

The third special technique that fully convolution on networks use is the skip connection. One effect of convolutions or encoding in general is you narrow down the scope by looking closely at some picture and lose the bigger picture as a result. So even if we were to decode the output of the encoder back … Read more

Now, using the second special technique, we can create decoder of FCN’s using transposed convolution. A transpose convolution is essentially a reverse convolution in which the forward and the backward passes are swapped. Hence, we call it transpose convolution. Some people may call it deconvolution because it undoes the previous convolution. Since all we’re doing … Read more

Let’s use the first special technique in SCNs by replacing a fully connected layer with one by one convolutional layers. This will result in the output value with the tensor will remain 4D instead of flattening to 2D, so spatial information will be preserved. This may sound daunting but it’s actually simpler than you might … Read more

Fully Convolutional Networks have achieved state of the art results in computer vision tasks, such as semantic segmentation. FCNs take advantage of three special techniques; one, replace fully connected layers with one by one convolutional layers, two, up-sampling through the use of transposed convolutional layers, three, skip connections. These skip connections allow the network to … Read more

A typical convolutional neural network might consist of a series of convolution layers. Followed by fully connected layers and ultimately a soft max activation function. This is a great architecture for a classification task like, is this a picture of a hotdog? But what if we want to change our task ever so slightly. We … Read more

Congratulations. Semantic segmentation is the cutting edge of perception in autonomous vehicles and beyond. This is a tremendous skill to demonstrate to employers and to use on the job. Even better, you can build on this knowledge to construct faster and more powerful segmentation networks. In the next lesson, you’ll learn about inference optimizations, which … Read more

IOU, not the monetary kind, is Intersection Over Union Metric, commonly used to measure the performance of a model on the semantic segmentation task. It is literally just the intersection set divided by the union set. Intersection of two sets is an AND operation. If it exists in both sets, then we put it into … Read more

One approach to scene understanding is to train multiple decoders. Each decoder trains on a separate task. We might have one decoder for segmentation and another for depth measurement. This way we can have a single network which not only predicts the class of a pixel but additionally how far away it is. You can … Read more

Welcome to advanced deep learning. You’ll be learning about cutting edge deep neural networks with Kelvin Lwin, the lead Autonomous Vehicle Instructor in NVIDIA. In this module, we will cover fully convolutional networks, object detection, semantics segmentation and inference optimizations. Finally, you’ll apply these concepts and techniques to train and supervise semantic segmentation network and … Read more