Lesson 3.1
The Neuron
The smallest piece of a neural network. A neuron takes some inputs, weighs each by importance, adds them up, and passes the total through a squasher. It's a tiny graph, so the engine you built already handles it.
A weighted vote.
A neuron is a small decision-maker. It hears several inputs, decides how much each one matters (a weight per input), adds up the weighted opinions, tips the scale with a baseline lean (the bias), and finally passes the total through a squashing function (the activation, from the last lesson of this unit).
One neuron: weigh the inputs, sum, add bias, squash.
The weights say how much (and which way) each input pulls; a positive weight means "more of this input raises my output," negative means the opposite. The bias sets how easily the neuron leans high or low regardless of input.
A worked neuron.
Two inputs, weights w = [2, -1], bias b = 0.5, using tanh as the squasher. Feed it x = [1, 1]:
Drag the inputs and weights, watch the neuron fire
wire thickness = how strongly that input pushes (|w·x|); coral pushes up, blue pushes down
out = tanh(w₁x₁ + w₂x₂ + b) = 0.00
when the squashed sum is near ±1, the neuron is firing; near 0 it's quiet
Every piece here, each multiply, each add, the tanh, is one of the operations from Unit 1. So a neuron is just a small computation graph, and the backward pass you already understand computes the gradient of every weight and the bias for free.
The neuron's parameters.
A neuron's weights and bias are its parameters, the knobs from lesson 1.3. Learning a neuron means adjusting exactly those numbers. The inputs x are data (fixed); the output is an activation (computed). When you build it, a neuron holds a small list of learnable values and knows how to produce its output from them.
Key takeaways
A neuron weighs its inputs, sums them, adds a bias, and squashes the result through an activation.
Its weights and bias are its parameters; inputs are data, output is an activation.
A neuron is just a small computation graph, so your backward pass already differentiates it.