Back

A (Long) Peek into Reinforcement Learning How do AI agents master games like Go, control robots, or optimize trading strategies? The answer lies in Reinforcement Learning (RL)—where agents learn by interacting with environments to maximize rewards. Key Concepts: ✓Agent & Environment: The agent takes actions; the environment responds. ✓States & Actions: The agent moves between states by choosing actions. ✓Rewards & Policy: Rewards guide learning; a policy defines the best action strategy. ✓Value Function & Q-Value: Measures how good a state/action is in terms of future rewards. ✓MDPs & Bellman Equations: Most RL problems follow Markov Decision Processes (MDPs), solved using Bellman equations. Solving RL Problems: ✓Dynamic Programming: Requires a known model, iteratively improves policies. ✓Monte-Carlo Methods: Learn from complete episodes. ✓TD Learning (SARSA, Q-Learning, DQN): Learns from incomplete episodes, bootstraps value estimates. ✓Policy Gradient (REINFORCE, A3C): Directly optimizes policies using gradient ascent. ✓Evolution Strategies: Model-agnostic, inspired by natural selection. Challenges: ✓Exploration-Exploitation Tradeoff: Balancing new knowledge vs. maximizing rewards. ✓Deadly Triad: Instability when combining off-policy learning, function approximation, and bootstrapping. Case Study: AlphaGo Zero ✓DeepMind’s AlphaGo Zero achieved superhuman Go-playing skills using self-play and Monte Carlo Tree Search (MCTS), without human supervision. link :https://lilianweng.github.io/posts/2018-02-19-rl-overview/