ml-foundations · Parallight Academy

About this course

A build-it-yourself refresher: no term-memorizing, no flash cards. In every session an
AI mentor walks you through deriving a gradient, writing a few dozen lines of numpy, and
running a seeded toy experiment — building classic machine learning from the loss function
all the way to random forests and SVMs. You'll manufacture an overfit on purpose and rescue
it, watch a straight decision boundary get bent by the kernel trick, and finally unify every
model you built behind your own mini scikit-learn. The course follows the classic topic
skeleton of the Stanford CS229 notes, but every session is an original hands-on script
written for the online sandbox: formulas typeset, algorithms drawn, hyperparameters
draggable, code actually running — with every experiment seeded so your numbers match the
script exactly. After this course, .fit() is no longer a black box: you've hand-written
the inside of every one.

What you'll learn

Derive the gradients of MSE and cross-entropy from scratch and hand-write batch gradient descent
Verify first-hand that iterative and closed-form solutions (normal equation, Newton's method) meet at the same optimum
Manufacture an overfit on purpose, then rescue it with ridge regularization and cross-validation
Hand-write kNN, decision trees, random forests, the perceptron and SVMs — and learn the shape and temperament of each decision boundary
Hand-write k-means, PCA and EM to find structure in unlabeled data
Unify every model you built behind a fit/predict interface — your own mini scikit-learn

Syllabus

1The Gradient-Descent Family: from Regression to Classification3 sessions

1Linear Regression by Hand: Loss Surface and Gradient Descent30 minStart →
2Logistic Regression and the Decision Boundary30 minStart →
3Newton's Method: Second-Order Information, Giant Steps30 minStart →

2Overfitting, Regularization & Model Selection3 sessions

1Overfitting, Live: the Temptation of Polynomials30 minStart →
2Ridge Regression: Reining In the Weights30 minStart →
3Cross-Validation: Let the Data Pick Its Own Hyperparameters30 minStart →

3Neighbors & Trees: Classifiers That Don't Draw Lines3 sessions

1k-Nearest Neighbors: the Laziest Classifier30 minStart →
2Decision Trees: Slicing Space with Information Gain30 minStart →
3Random Forests: the Wisdom of the Trees30 minStart →

4Margins & Kernels: from the Perceptron to SVMs0 sessions

Sessions are on the way.

5Unsupervised Learning: Learning Without Labels0 sessions

Sessions are on the way.

6Build Your Own Mini scikit-learn0 sessions

Sessions are on the way.

Same series · 人工智能全透析

AI, End to End

Synthesized from the best material across the web — rebuild the core of AI from scratch, the efficient way.

Classical AI: Search, Games, and ConstraintsWrite A* and alpha-beta pruning in a few dozen lines of stdlib — the bedrock under LLM planners and AGV obstacle avoidanceView course →Computer Vision: From Pixels to ConvolutionsAn image is just a numpy array — hand-write conv kernels for edge detection, then train a classifier that reads digitsView course →Reinforcement Learning: From Bandits to Q-learningTo understand RLHF/RLVR, first understand Q-learning — zero gym, a world you build in pure numpyView course →Build an Autograd Engine: Your Own Mini-PyTorchWrite every line behind loss.backward() yourself — PyTorch stops being a black boxView course →Raft from Scratch: Consensus over an Unreliable NetworkYour multi-agent system IS a distributed system — elect a leader on a deterministic simulated networkView course →Build a Search Engine: First Principles of RAG's "R"Everyone builds RAG but never opens the retrieval core — hand-write an inverted index and BM25, and finally know why your retrieval missesView course →Diffusion and Flow Matching from ScratchYou use Stable Diffusion but don't get why — on a 2D toy distribution, turn data into noise and back, by handView course →