Build an Autograd Engine: Your Own Mini-PyTorch

You call loss.backward() every day — but what actually happens inside that one line? In this
course you don't memorize concepts, you build the thing: first a Value class that remembers
how it was computed, so gradients flow backwards through the graph automatically; then a
finite-difference audit that proves, op by op, that your backprop is correct. Next you upgrade
the scalar engine into a tensor engine that eats numpy arrays — backprop through broadcasting
and the two transpose rules of matmul, no hand-waving. On top of it you build your own torch.nn
and torch.optim: Module, Linear, numerically-stable cross-entropy, SGD and Adam. Finally you use
this mini-PyTorch — every line yours — to train XOR, a two-moons classifier and a character-level
name generator, and map it all back to real PyTorch: detach, no_grad, and the memory bill.
Pure Python + numpy, fully offline, every step runs in your own sandbox in under a second.
By the end you'll see it plainly: the heart of PyTorch is a few dozen lines you wrote and verified yourself.