Build a deep learning framework from scratch.
You'll build the engine that powers PyTorch and TensorFlow, automatic differentiation, and train a neural network with it. This course teaches the ideas, not the implementation: by the end you'll understand every part well enough to write the framework yourself.
The loop every framework runs, over and over
inputs prediction how wrong gradients better weights Forward to a prediction, measure the error, send gradients backward, nudge the weights. Repeat until it learns.
Unit 0, Foundations
What a framework is, what "learning" means, and the calculus backprop is built on.
What a Framework Actually Is
The whole thing in one picture: represent, run, differentiate, update.
Learning Is Just Fitting a Function
Parameters, error, and the idea of turning knobs to be less wrong.
Derivatives, Intuitively
Nudge the input, watch the output. The one number that says which way to move.
Gradients: Derivatives With Many Knobs
The gradient points uphill. Learning walks the other way.
The Chain Rule
The single rule backpropagation rests on. Rates multiply along a chain.
Unit 1, The Computation Graph
Values as nodes, operations as edges, and the forward pass that fills it in.
Computation as a Graph
Nodes, edges, leaves; the graph as the framework's memory.
The Forward Pass
Compute the output and leave a trail. Interactive.
Parameters vs Activations
Knobs you turn vs readouts you compute.
Local Gradients
Each operation's own little derivative rule.
Unit 2, Backpropagation
Walk the graph backward, accumulate, and prove it correct.
The Backward Pass
Seed 1, walk in reverse, push local gradients.
Fan-out & Accumulation
When a value is used twice, gradients add. Interactive.
Topological Order
Why backprop must visit nodes in the right order.
Gradient Checking
Verify the engine, and why not finite differences.
Unit 3, Neural Networks
Neurons into layers into an MLP, all on top of the engine.
Unit 4, Training
The task, the loss, gradient descent, and the loop that ties it together.
The Task & Data
Classification, the two moons, the decision boundary.
Loss Functions
MSE, hinge, cross-entropy: defining "good."
Gradient Descent & SGD
Step downhill; the learning rate. Interactive.
Initialization & Regularization
Breaking symmetry; avoiding overfitting.
The Training Loop
Forward, loss, zero, backward, step. Everything meets here.
Unit 5, Beyond v1
From scalars to tensors, then to a fusing compiler.
What you need before starting
You can read and write basic Python: variables, functions, loops, classes.
Functions, graphs, slopes. That's the floor.
Not required upfront. Derivatives and the chain rule are taught from zero in Unit 0.
What you'll understand by the end
that records every operation
and computes its own gradients
built on top of the engine
with a clean parameter API
and a loop that teaches the net
to separate two classes
This course teaches the concepts. You write the framework, that's where the understanding sticks.