Lesson 4.5
The Training Loop
Everything in this course meets here. A handful of steps, repeated, that turn a random network into one that separates the two moons. This is where you stop reading and start building.
The whole thing, in five moves.
Training is one short loop, run many times (each full pass over the data is an epoch). Every concept you've met plays its part:
Forward pass.
Run each input through the MLP to get a prediction. Builds the graph (Units 1, 3).
Compute the loss.
Score how wrong the predictions are, plus a little L2 penalty. This is the root node (Units 4.2, 4.4).
Zero the gradients.
Reset every parameter's gradient to 0 first, because they accumulate (Units 2.2, 4.3).
Backward pass.
Call backward() on the loss; the engine fills in every parameter's gradient (Unit 2).
Step the parameters.
Nudge each one downhill: w ← w − lr × w.grad (Unit 4.3). Then repeat.
That's it. Forward, loss, zero, backward, step, looped a few hundred times. The loss falls, the boundary curves into place, and the random network becomes one that knows the two moons apart.
Press Train, the loss falls and the boundary bends into place
forward → loss → zero grads → backward → step, a few hundred times, that loop is all of training
What success looks like.
As epochs pass, watch the loss trend down and accuracy climb. Plot the decision boundary every so often and you'll see it bend from a meaningless line into a clean curve threading between the crescents. When held-out points land on the right side too, the network has genuinely learned the shape.
monktensor v1 is done when: the net separates make_moons, your gradient check passes (lesson 2.4), the README explains backprop in plain English, and a notebook shows the decision boundary. Tests, a benchmark plot, and a real README are what turn a toy into a portfolio-grade repo.
Now build it.
This is where the course hands off. You've seen every idea: the graph that remembers, the backward pass that multiplies and accumulates, the neuron and the MLP, the loss, and the loop that ties them together. None of it was implementation, that part is yours to write, which is exactly where the understanding becomes real. Unit 5 is a short look at where monktensor goes after v1.
Key takeaways
The training loop is five repeated moves: forward, loss, zero grads, backward, step.
Run it for many epochs and the loss falls while the decision boundary curves into place.
v1 is done when the net separates make_moons, the gradient check passes, and the boundary plot looks right.