Lesson 3.2
Layers and the MLP
Put many neurons side by side and you have a layer. Stack layers and you have a multi-layer perceptron. It's still one computation graph, so nothing about backprop changes.
A layer is a row of neurons.
One neuron gives one output. A layer is several neurons looking at the same inputs, each with its own weights and bias, each producing one number. A layer of 4 neurons turns an input into 4 outputs, four different weighted views of the same data.
Stack layers so each layer's outputs become the next layer's inputs, and you have a multi-layer perceptron (MLP), the simplest real neural network.
Why stack them?
A single layer can only carve the input space with straight cuts, it draws lines. Many real problems need curved, wiggly boundaries. Stacking layers, each followed by a nonlinearity, lets the network bend and combine those cuts into complex shapes. Depth is what turns "draw a line" into "draw any boundary you need." (The nonlinearity is essential here, that's the next lesson.)
All the knobs in one list.
An MLP's parameters are simply every weight and bias of every neuron in every layer, gathered into one list. That list is what the optimizer steps each round. A net with input 2 → hidden 4 → hidden 4 → output 1 has, for instance, a few dozen parameters; a real one has millions. The mechanism is identical.
An MLP is just neurons composed into layers composed into a network, and the whole thing is still one computation graph. So the forward pass runs it, and the single backward() from Unit 2 computes the gradient of every parameter end to end. You don't write new differentiation for networks; you reuse the engine.
Key takeaways
A layer is several neurons sharing inputs; an MLP is layers stacked so each feeds the next.
Depth plus nonlinearity lets a network represent curved, complex boundaries, not just straight cuts.
The MLP's parameters are all weights and biases in one list; it's one graph, so backprop just works.