Lesson 4.1
The Task & Data
Before training anything, you need a problem to solve and data to learn from. Ours is a classic: tell apart two interleaving crescents of points. Small, visual, and just hard enough to need a real network.
Two moons.
The task is classification: given a point, predict which group it belongs to. The dataset is make_moons, points scattered in two interleaving crescent shapes, one class each. Given a point's two coordinates, the network must answer "moon A or moon B?"
The two coordinates are the features (the inputs). Which moon a point belongs to is its label (the answer). A training example is a point together with its correct label.
The decision boundary.
A trained classifier effectively draws a decision boundary, a line or curve where it switches its answer from one class to the other. For two straight-separable blobs, a straight line would do. But the moons interleave, so no straight line can split them. The boundary has to curve between the crescents. That's why this task needs a network with a nonlinearity and a hidden layer, not a single neuron.
Drag the line to split the two moons
coral is one class, blue the other, points on the wrong side flash red
0 points on the wrong side
try every angle, a straight line always leaves some wrong. The moons interleave, so the boundary has to curve (that's what a hidden layer buys you).
make_moons is the perfect first task for exactly these reasons: it's 2D so you can literally see the boundary the model draws; it's small so training is fast; and it's not linearly separable, so it genuinely exercises depth and nonlinearity. When your boundary curves cleanly between the moons, monktensor works.
Don't just memorize, generalize.
You hold out some labeled points the network never trains on (a validation set). Doing well on training points is easy, a model can memorize. The real test is the held-out points: a boundary that also classifies those correctly has learned the shape, not the specific dots. This is the "generalization, not memorization" idea from lesson 0.2, made concrete.
Key takeaways
The task is classification: features (the coordinates) in, a label (which class) out.
A classifier draws a decision boundary; the moons interleave, so it must curve, needing depth and nonlinearity.
Hold out a validation set to check the model generalizes instead of memorizing.