Module 2 · Lesson 5
Supervised learning gives a model examples with target answers. Reinforcement learning is different. An agent acts inside an environment, observes what happens, receives a reward signal, and gradually learns which actions tend to lead to better long-term outcomes.
The basic loop
- State: what the agent currently observes about the environment.
- Action: a choice available to the agent.
- Reward: numerical feedback after an action or sequence of actions.
- Policy: the strategy the agent uses to choose actions.
Agent
↓ action
Environment
↓ new state + reward
Agent
↓ next action
...The difficulty is that a good action may not produce an immediate reward. An agent often has to learn which sequence of decisions leads to a valuable outcome later.
Exploration versus exploitation
An RL agent faces a recurring tradeoff. Should it choose the action that currently looks best, or try something different that might reveal an even better strategy? This is the classic balance between exploitation and exploration.
Games made RL famous, but the details matter
DeepMind's AlphaGo became famous for defeating elite Go players. It was built for Go, not chess. Later, AlphaZero demonstrated a more general self-play approach across chess, shogi, and Go.
Games are attractive research environments because rules, actions, and outcomes can be defined clearly. Reinforcement learning is also explored in robotics, resource allocation, control systems, and other sequential decision problems. Real-world applications can be much harder because rewards may be incomplete, delayed, unsafe to explore, or difficult to specify correctly.
Does RL always require millions of simulations?
No. Some famous systems have used enormous numbers of interactions, but the amount of experience required depends on the problem, algorithm, environment, existing data, model, and whether simulation is available.
The deeper idea is more important than a particular number: reinforcement learning improves behavior through interaction and feedback over sequences of actions.
Later in this series we will return to reinforcement learning when we discuss RLHF, where feedback is used to shape the behavior of language models after pre-training.
Demystifying AI series