1-7. The RL Framework: The Solution

In reinforcement learning, agents learn to prioritize different decisions based on the rewards and punishments associated with different outcomes.

1-7-1. Introduction

by

You’ve already learned about how to formulate a real-world problem so it can be solved with reinforcement learning. In this lesson, you’ll begin to think about ways to solve this problem. It’s important to note that this lesson is significantly more technical than the previous one. If this is initially uncomfortable to you, don’t worry, … Read more

8 – Optimal Policies

by

Several concepts ago, I’ve mentioned that we needed to define the action value function before talking about how the agent could search for an optimal policy, and we will see most of the detail for a later lesson. The main idea is this. The agent interacts with the environment. And from that interaction, it estimates … Read more

7 – Action-Value Functions

by

So far we’ve been working with the state value function for a policy. For each state s, it yields the expected discounted return If the agent starts in state as and then uses the policy to choose its actions for all time steps. You’ve seen a few examples and know how to calculate the state … Read more

6 – Optimality

by

So far in this lesson, we’ve looked at a particular policy Pi, and calculated its corresponding value function. In the quiz, you calculated the value function corresponding to a different policy which we denoted by Pi-Prime. And if you look at each of these value functions, you may notice a pattern or trend. Take the … Read more

5 – Bellman Equations

by

If you take the time yourself to calculate the value function for this policy, you might notice that you don’t need to start your calculations from scratch every time. In particular, you don’t need to look at the first state then add up all the rewards along the way. Then look at the second state, … Read more

4 – State-Value Functions

by

So we’re working with this grid world example and looking for the best policy that leads us to a goal state as quickly as possible. So, let’s start with a very, very bad policy so that we can understand why it’s bad, and then work to improve it. Specifically, we’ll look at a policy where … Read more

3 – Gridworld Example

by

To understand how to go about searching for the best policy, it will help to have a running example. So consider this very very small world and an agent who lives in it. Say the world is primarily composed of nice patches of grass, but two out of the nine locations in the world have … Read more

2 – Policies

by

We’ve seen that we use a Markov decision process or MDP as a formal definition of the problem that we’d like to solve with reinforcement learning. In this video, we specify a formal definition for the solution to this problem. We can start to think of the solution as a series of actions that need … Read more