About this course
Why can the same model solve problems it previously couldn't — just by trying multiple
times, adding a "grader", or getting tools and memory? This course follows the skeleton
of Stanford CS329A (Self-Improving AI Agents): test-time compute scaling and verifiers,
self-critique, planning, STaR bootstrapping and RL with verifiable rewards, then
open-ended evolution à la AlphaEvolve, deep-research agents, software-engineering
agents, memory, and agentic evaluation. Every session is hands-on with an AI mentor:
formulas typeset, mechanisms drawn, key loops turned into draggable mini-experiments,
and code that really runs in your own sandbox.
What you'll learn
- Explain the four levers of self-improvement — sampling, verifiers, tool feedback, training bootstrap — and when each applies
- Run best-of-N with a verifier and explain why coverage scales log-linearly
- Implement minimal critique→revise and multi-step planning loops, and tell when they help vs fail
- Use STaR/RLVR to explain how DeepSeek-R1-style reasoners teach themselves
- Run a mini evolutionary search (LLM mutation + evaluator selection) and map it to AlphaEvolve/FunSearch
- Describe how SWE agents, memory augmentation and agentic evals (pass^k, time horizon) work in practice
Syllabus
1Foundations: Sample More, Then Grade3 sessions
2Feedback Loops: Tools, Self-Critique & Planning3 sessions
3Training Bootstrap: Bake Wins Back into the Model3 sessions
4Open-Ended Evolution: Programs That Grow Themselves3 sessions
Same series · 世界名校知名实验室系列
Flagship University Lab Series
Modeled on Stanford / MIT / Berkeley syllabi — learn from scratch with a mentor Agent guiding you in real time.
◎ Modeled on Stanford CS336
LLM from ScratchHand-build every piece of an LLM, along the 17-lecture skeleton of Stanford CS336View course →◎ Modeled on Berkeley’s LLM Agents course
LLM Agents: From Reasoning to Agentic SystemsAlong the 12-lecture skeleton of Berkeley's LLM Agents course — reasoning, frameworks, applications and safety, all hands-onView course →◎ Modeled on MIT’s How to AI (Almost) Anything (multimodal)
Multimodal AI: Teaching Models to See, Hear, and ConnectTurn images, sound and text into tensors by hand — then build alignment, fusion, cross-modal retrieval and multimodal models piece by pieceView course →◎ Modeled on Princeton COS 511 (Theoretical ML)
Learning Theory: PAC, Boosting & Online LearningHand-derive PAC bounds, hand-build AdaBoost, hand-run regret curves — turn learning theory from formulas into experiments you runView course →◎ Modeled on Stanford CS234 (Reinforcement Learning)
Deep Reinforcement Learning: MDPs to Policy GradientsProve convergence by hand, backprop a DQN from scratch, give policy gradient a critic — pure numpy, zero gym, zero GPUView course →◎ Modeled on CMU’s ML in Production (MLIP)
Machine Learning in ProductionTurn a model that merely runs into a system that survives productionView course →