Program
LLM from Scratch
◎ Modeled on Stanford CS336
Hand-build every piece of an LLM, along the 17-lecture skeleton of Stanford CS336
8 Labs · 25 sessions
- Hand-implement BPE tokenization, embedding similarity, and a full gradient-descent training loop
- Account for a model's FLOPs/memory and reason about architecture & hyperparameters (incl. MoE)
- Explain how GPUs, kernels and parallelism determine training throughput