Reinforcement Learning in Python — Monte Carlo Methods

Led by Claude Shannon Simulacrum

About this course

Model-free learning from experience — Monte Carlo policy evaluation, Monte Carlo control with and without exploring starts.

Hosted by the Claude Shannon Simulacrum, drawing on his foundational work in probability and information theory.

1. Module 1

   Monte Carlo Prediction and Control

   Led by Claude Shannon Simulacrum

   The question

   Monte Carlo introduction (learning from complete episodes) · first-visit vs every-visit MC · Monte Carlo policy evaluation (estimating V(s) from returns) · MC policy evaluation in code · Monte Carlo control (using Q(s,a) to improve policies) · MC con...

   Outcome

   Demonstrates understanding and implementation of monte carlo prediction and control.

   Sub-units

   1. ○ 1.1 Monte Carlo Prediction and Control Open this unit → ✓ ↻
   Begin the course → Print certificate of completion
2. Module 2

   Monte Carlo without Exploring Starts

   Led by Claude Shannon Simulacrum

   The question

   The exploring starts assumption and why it is impractical · Monte Carlo control without exploring starts (epsilon-soft policies) · on-policy vs off-policy methods · MC without exploring starts in code · Monte Carlo summary · connection to TD methods...

   Outcome

   Demonstrates understanding and implementation of monte carlo without exploring starts.

   Sub-units

   1. ○ 2.2 Monte Carlo without Exploring Starts Open this unit → ✓ ↻
   Open this module → Print certificate of completion