Reinforcement learning
Reinforcement learning
Reinforcement learning (RL) is a subfield of machine learning concerned with how agents ought to take actions in an environment in order to maximize cumulative reward. Unlike supervised learning, RL does not rely on labelled data but instead learns through trial and error using feedback from its own actions. It is inspired by behavioural psychology and has applications in robotics, game playing, autonomous driving, and resource management.
Key concepts
- Agent – The learner or decision-maker that interacts with the environment.
- Environment – The world in which the agent operates, defined by states and transition dynamics.
- State – A representation of the current situation.
- Action – A move or decision the agent can make.
- Reward – A scalar feedback signal indicating immediate benefit.
- Policy – A strategy mapping states to actions.
- Value function – An estimate of expected cumulative future reward from a given state or state-action pair.
The goal is to learn an optimal policy that maximizes the long-term expected return, often discounted with a factor γ.
Algorithms
Common RL algorithms include Q-learning, SARSA, Deep Q-Networks, Policy gradient methods (e.g., REINFORCE, Proximal Policy Optimization), and Actor-critic architectures. Model-based RL methods construct an explicit model of the environment to simulate outcomes before acting.
History
The roots of reinforcement learning lie in the work of Richard Bellman on dynamic programming and John von Neumann on game theory. In the 1980s, Christopher Watkins introduced Q-learning, a model-free algorithm. The field gained prominence after AlphaGo defeated a world champion in the game of Go in 2016, using deep RL. Since then, RL has been central to advances in large language model training (e.g., RLHF) and robotics.
Applications
- Game playing (chess, Go, video games)
- Robotics control
- Autonomous vehicles
- Finance (portfolio optimization)
- Healthcare (treatment planning)