Reinforcement Learning: From Bandits to Q-learning

You hear RLHF, RLVR and agentic RL every day — and they all stand on the same
foundation: one value-update rule, and the eternal explore-vs-exploit dilemma.
This is a build-it-yourself course: no gym, no GPU. Every world — a 10-armed
bandit, a 4x4 grid, a cliff edge, a shape-shifting maze — is one you assemble in
pure numpy; every algorithm is one you complete by filling in its most critical
lines, then actually run and watch converge in your own sandbox. You will watch
epsilon-greedy lift the optimal-arm hit rate from 36% to 81%, watch V-values take
shape sweep by sweep as policy arrows point to the goal, watch SARSA and
Q-learning develop two personalities at the cliff edge, then hand your agent a
world model so it can practice in imagination. The course follows the knowledge
skeleton of Sutton & Barto, but every session is an original hands-on script
written for the online sandbox: formulas typeset, algorithms drawn, key knobs
draggable. The final stop lands in the present: with the policy gradient and
reward model you built by hand, you will see exactly what RLHF/RLVR optimizes.