2 – M3 L2 C02 V1

So, first things first. How might we approach this idea of estimating an optimal policy? Let’s consider this cart pole example. In this case, the agent has two possible actions, he can push the cart either left or right. So, at each time step, the agent picks one of these two options. We can construct a neural network that approximates the policy, that accepts the state as input. As output, it can return the probability that the agent selects each possible action. So, if there are two possible actions, the output layer will have two nodes. The agent uses this policy to interact with the environment by just passing the most recent state to the network. It outputs action probabilities and then the agent samples from those probabilities to select an action in response. So in this case, there’s a 90 percent chance that the agent selects action left and a 10 percent chance it decides to push the cart right. Our objective then is to determine appropriate values for the network weights so that for each state that we pass into the network it returns action probabilities where the optimal action is most likely to be selected. This will help the agent with its goal to maximize expected return. This is an iterative process where the weights are initially set to random values. Then, as the agent interacts with the environment and learns more about what strategies are best for maximizing reward, it amends those weights. As a direct result, the agent starts to choose the appropriate action for each date and it gradually masters the task. In this lesson, you’ll learn about many different approaches that we can take towards optimizing these weights.

%d 블로거가 이것을 좋아합니다: