3-5. Hyperparameters

Two main choices we need to make when we want to build a recurrent neural network are choosing a cell type, so a long short term memory cell or a vanilla RNN cell or a gated recurrent unit cell, and how deep the model is. How many layers will we stack? And since we’ll need … Read more

Two main choices we need to make when we want to build a recurrent neural network are choosing a cell type, so a long short term memory cell or a vanilla RNN cell or a gated recurrent unit cell, and how deep the model is. How many layers will we stack? And since we’ll need … Read more

Let’s now talk about the hyperparameters that relates to the model itself rather than the training or optimization process. The number of hidden units, in particular, is the hyperparameter I felt was the most mysterious when I started learning about machine learning. The main requirement here is to set a number of hidden units that … Read more

Let’s now talk about the hyperparameters that relates to the model itself rather than the training or optimization process. The number of hidden units, in particular, is the hyperparameter I felt was the most mysterious when I started learning about machine learning. The main requirement here is to set a number of hidden units that … Read more

To choose the right number of iterations or number of epochs for our training step, the metric we should have our eyes on is the validation error. The intuitive manual way is to have the model train for as many epochs or iterations that it takes, as long as the validation error keeps decreasing. Luckily … Read more

To choose the right number of iterations or number of epochs for our training step, the metric we should have our eyes on is the validation error. The intuitive manual way is to have the model train for as many epochs or iterations that it takes, as long as the validation error keeps decreasing. Luckily … Read more

Minibatch size is another hyper parameter that, no doubt, you’ve run into a number of times already. It has an effect on the resource requirements of the training process but also impacts training speed and number of iterations in a way that might not be as trivial as you may think. It’s important to review … Read more

Minibatch size is another hyper parameter that, no doubt, you’ve run into a number of times already. It has an effect on the resource requirements of the training process but also impacts training speed and number of iterations in a way that might not be as trivial as you may think. It’s important to review … Read more

The learning rate is the most important hyper parameter. Even if you apply models that other people built to your own data set, you’ll find that you’ll probably have to try a number of different values for the learning rate to get the model to train properly. If you took care to normalize the inputs … Read more

The learning rate is the most important hyper parameter. Even if you apply models that other people built to your own data set, you’ll find that you’ll probably have to try a number of different values for the learning rate to get the model to train properly. If you took care to normalize the inputs … Read more

Hello, I’m Jay. In this lesson, we will be talking about a number of different hyper parameters and identifying possible starting values and intuitions for a number of hyper parameters that you may have already come across, and that you’ll need to know in your work with deep learning. A hyper parameter is a variable … Read more

Hello, I’m Jay. In this lesson, we will be talking about a number of different hyper parameters and identifying possible starting values and intuitions for a number of hyper parameters that you may have already come across, and that you’ll need to know in your work with deep learning. A hyper parameter is a variable … Read more