About this course
A build-it-yourself course: no passive lectures. In every session an AI mentor walks you
through writing code, tuning knobs, and reading curves — building an LLM from characters
all the way to alignment. It follows the full 17-lecture skeleton of Stanford CS336
(Language Modeling from Scratch) 1:1 — tokenization, architecture, systems, scaling laws,
inference, data, alignment — with every session rewritten as original hands-on teaching
for the online sandbox.
What you'll learn
- Hand-implement BPE tokenization, embedding similarity, and a full gradient-descent training loop
- Account for a model's FLOPs/memory and reason about architecture & hyperparameters (incl. MoE)
- Explain how GPUs, kernels and parallelism determine training throughput
- Use scaling-law reasoning to trade off model size vs data
- Understand inference optimization (KV cache/batching), evaluation (perplexity/benchmarks) and data pipelines
- Explain what SFT/RLHF/RLVR and multimodal alignment do, and why they work
Syllabus
1Foundations: Tokens, Embeddings & the Training Loop3 sessions
2Attention & the Transformer: Build the Model's Heart by Hand4 sessions
3Architecture: Resource Accounting, Hyperparameters & MoE3 sessions
4Systems: GPUs, Kernels & Parallelism5 sessions
5Scaling Laws: Spending Compute Wisely2 sessions
6Inference & Evaluation2 sessions
7Data: Sources, Curation & Mixing2 sessions
Same series · 世界名校知名实验室系列
Flagship University Lab Series
Modeled on Stanford / MIT / Berkeley syllabi — learn from scratch with a mentor Agent guiding you in real time.
◎ Modeled on Berkeley’s LLM Agents course
LLM Agents: From Reasoning to Agentic SystemsAlong the 12-lecture skeleton of Berkeley's LLM Agents course — reasoning, frameworks, applications and safety, all hands-onView course →◎ Modeled on Stanford CS329A (Self-Improving AI Agents)
Self-Improving AI AgentsHand-build AI agents that make themselves better, along the skeleton of Stanford CS329AView course →◎ Modeled on MIT’s How to AI (Almost) Anything (multimodal)
Multimodal AI: Teaching Models to See, Hear, and ConnectTurn images, sound and text into tensors by hand — then build alignment, fusion, cross-modal retrieval and multimodal models piece by pieceView course →◎ Modeled on Princeton COS 511 (Theoretical ML)
Learning Theory: PAC, Boosting & Online LearningHand-derive PAC bounds, hand-build AdaBoost, hand-run regret curves — turn learning theory from formulas into experiments you runView course →◎ Modeled on Stanford CS234 (Reinforcement Learning)
Deep Reinforcement Learning: MDPs to Policy GradientsProve convergence by hand, backprop a DQN from scratch, give policy gradient a critic — pure numpy, zero gym, zero GPUView course →◎ Modeled on CMU’s ML in Production (MLIP)
Machine Learning in ProductionTurn a model that merely runs into a system that survives productionView course →