You already know how this works
Think back to the first time you played a video game with no tutorial. Maybe it was an old platformer, or a fighting game at a friend's house. Nobody explained the controls. You just pressed buttons and watched what happened. Jump landed on the platform: good. Walk into the enemy: bad. Fall into the pit: very bad. Within twenty minutes, without reading a single word of instruction, you had a working mental model of the game.
You were not memorizing rules. You were not being shown correct examples. You were just acting, observing what happened, and quietly updating your behavior based on whether the outcome felt like progress.
That process has a name. It is called reinforcement learning.
What makes it different
Most machine learning you hear about works by example. You collect thousands of labeled photos: this is a cat, this is not a cat. The model studies those examples until it can generalize. That approach is called supervised learning, and it works well when you have a large set of correct answers to learn from.
But plenty of problems do not work that way. Nobody can label every possible chess position with the correct move. Nobody can hand-annotate every second of a robot's movement through an unfamiliar room. The space of possible situations is too large, and in many cases, the correct answer is not even knowable in isolation. A chess move is only good or bad depending on what happens forty moves later.
Reinforcement learning takes a different approach entirely. Instead of learning from labeled examples, the system learns by doing. It tries things, receives feedback on whether those things led to good or bad outcomes, and gradually gets better at making decisions. No labeled dataset required.
This is not a minor variation on supervised learning. It is a fundamentally different way of framing the problem.
The core loop
Every reinforcement learning system has three moving parts: an agent, an environment, and a reward signal.
The agent is the thing that makes decisions. It could be a game-playing AI, a robot, a recommendation engine, or a program controlling a heating system. It observes the world and decides what to do next.
The environment is everything the agent interacts with. In a video game, the environment is the game itself: the level layout, the enemies, the physics. In a robotics system, it is the physical world. The environment receives the agent's actions and responds with a new situation.
The reward signal is the feedback. After the agent takes an action, the environment returns a number: positive if the outcome was good, negative if it was bad, zero if it was neutral. The agent's only goal is to accumulate as much reward as possible over time.
Those three things form a loop that repeats continuously.
At each step, the agent looks at the current state of the environment (where am I, what is around me, what just happened), chooses an action, and then receives a reward and a new state. The loop repeats, thousands or millions of times, until the agent has built up enough experience to behave well.
Back to the video game. You are the agent. The game is the environment. Points, progress, and not dying are the reward signal. Every button press is an action. Every new frame is a new state. You did reinforcement learning for hours without knowing it had a name.
How the agent learns: policies and value
The agent needs some way of deciding what to do in any given situation. That decision-making function is called a policy. A policy is simply a mapping: given this state, take this action.
Early in training, the policy is essentially random. The agent has no idea what is good or bad, so it just tries things. Over time, as it accumulates experience, the policy improves. Actions that led to high rewards become more likely. Actions that led to bad outcomes become less likely.
But there is a subtlety here. The reward the agent cares about is not just the immediate reward from the next action. It is the total reward over time. In chess, sacrificing a piece now might be worth it if it leads to checkmate in ten moves. The agent needs to learn not just what feels good immediately, but what actually leads to good outcomes in the long run.
To handle this, RL systems use a concept called a value function. The value function estimates how much total future reward the agent can expect from a given state, assuming it acts well from that point forward. A state with high value is one that tends to lead to good outcomes. A state with low value is a dead end.
The agent's job, then, is to build an accurate value function and use it to guide its policy. Do not just take the action with the best immediate reward; take the action that moves you into the highest-value state.
The explore-exploit tradeoff
Here is the problem the agent faces at every single step: should it do what has worked before, or try something new?
If the agent always does what worked in the past, it will never discover better strategies it has not tried yet. If it always tries new things, it never actually uses what it has already learned. This tension is called the explore-exploit tradeoff, and it is one of the most fundamental ideas in reinforcement learning.
Exploitation means using the best action the agent currently knows about. Safe, efficient, but potentially stuck in a local optimum.
Exploration means trying something different, even if it looks worse based on current knowledge. Risky, potentially wasteful, but the only way to discover something better.
Think about the video game again. You found a strategy that gets you to level three reliably. Do you keep using it, or do you try something weird to see if it opens up a shortcut? The right answer depends on how much of the game you have left to discover.
Early in training, exploration is critical. The agent knows almost nothing, so trying random things has high value. As training progresses and the agent builds up a solid model of the environment, exploitation becomes more valuable. Most RL systems formalize this by starting with a high exploration rate that decays over time.
One simple approach is called epsilon-greedy: with probability epsilon, the agent picks a random action; otherwise, it picks the best action it knows. Start epsilon at 1.0 (fully random) and decay it toward 0.05 or so as training runs.
Getting this balance right matters more than it might seem. An agent that stops exploring too early will settle for a mediocre policy and never improve. An agent that never stops exploring wastes enormous compute trying random actions when it already knows what works.
Where you encounter it in the real world
Reinforcement learning is not an academic curiosity. It is behind some of the most impressive software built in the last decade.
Game playing. DeepMind's AlphaGo used RL to reach superhuman performance at Go, a game previously thought to be too complex for computers to master. Its successor, AlphaZero, learned chess, Go, and shogi from scratch using nothing but self-play and a reward signal: win or lose.
Language model fine-tuning. The technique behind much of the behavior you see in modern AI assistants is called RLHF: reinforcement learning from human feedback. A language model generates responses, humans rate them, and those ratings become the reward signal. The model is then trained to produce responses that score higher. This is how systems learn to be helpful rather than just statistically likely.
Robotics. Teaching a robot to walk, grasp objects, or navigate an unknown room is extremely hard to do with labeled examples. RL lets the robot try, fall over, adjust, and gradually develop motor policies that work. Simulated environments allow millions of attempts before the policy is deployed on real hardware.
Recommendation systems. When a platform decides what to show you next, it is often solving a sequential decision problem: which action (content item) maximizes long-term engagement? RL frames this naturally, particularly when the feedback loop is slow and the state space is large.
Resource scheduling. Data center cooling, chip design, network routing: these are all sequential decision problems where the right action depends on current system state and affects future options. Google has used RL to reduce data center cooling energy consumption significantly.
What makes it hard
RL sounds elegant in theory. In practice, it comes with a distinct set of challenges that make it harder to apply than supervised learning.
Reward design is difficult. The agent will optimize exactly what you tell it to, which means poorly designed reward signals produce bizarre behavior. An early reinforcement learning agent trained to maximize points in a boat racing game discovered it could score more points by driving in circles collecting bonuses than by actually finishing the race. The reward was technically correct; the behavior was not. This problem is called reward hacking.
Sample inefficiency. RL often requires an enormous number of interactions to learn what a human would figure out quickly. AlphaGo played millions of games against itself. Real-world systems like robots cannot afford that many physical trials, which is why simulation is so important.
Credit assignment. When something bad happens twenty steps after a decision, how do you know which decision caused it? Attributing delayed rewards back to the actions that caused them is called the credit assignment problem, and it is one of the core theoretical challenges in the field.
Instability. RL training can be brittle. Small changes in hyperparameters, reward scale, or environment dynamics can cause training to diverge entirely. Getting a stable training run often requires significant engineering effort.
Summary
Reinforcement learning is a framework for teaching systems to make decisions through experience rather than examples. An agent interacts with an environment, receives reward signals, and gradually learns a policy that maximizes long-term reward. It is distinct from supervised learning because there is no labeled dataset: the feedback comes from the consequences of the agent's own actions.
The explore-exploit tradeoff sits at the heart of how RL agents learn. Too much exploitation and the agent never discovers better strategies. Too much exploration and it never uses what it knows. Most practical systems decay exploration over time as the agent's model of the world matures.
What makes RL powerful is also what makes it hard: the reward signal can be anything, the state space can be enormous, and the connection between actions and outcomes can be separated by many steps. When it works, it can produce behavior that no human programmer could have explicitly specified. When it fails, it usually fails because the reward was wrong, the exploration was insufficient, or the environment was too complex for the agent to model reliably.
The video game intuition holds up further than it might seem. You explored, you exploited what worked, you updated your policy based on delayed feedback, and eventually you got good. RL just does that at scale, faster, and without getting frustrated.
Part of the Explained series — concepts in tech, clearly.