Lesson 0.1
Fewer bits, on purpose
Start at the very beginning: why anyone would store a number less precisely, and what 'a short list of allowed values' actually means.
The problem: too many numbers
Everything an AI system works with, it stores as numbers. A sentence becomes a list of a few hundred numbers, and so does an image, a user, or a product. A search database is just a huge stack of these lists, often tens of millions of them, which together can fill tens of gigabytes of memory.
Here is where it hurts. A normal number on a computer takes 32 bits to store. A bit is the smallest possible unit of storage, a single 0 or 1, so one ordinary number already eats 32 of them. Multiply that by billions of numbers and you are renting a lot of memory: bigger machines, higher cost, slower systems. The entire point of this project is to make each number take fewer bits, so the whole stack shrinks. This lesson is the one idea that makes that possible.
A bit is one unit of storage, a single 0 or 1. A normal number costs 32 bits. The goal of quantization is to spend fewer bits per number, on purpose.
The idea: allow only a few values
You cannot shrink a number for free. Something has to give, and the thing you give up is exactness. The trick is to stop allowing every possible value and instead agree on a short list of allowed values. Then you store each real number as the closest value on that list.
A concrete example. Suppose the only values you are allowed to write down are 0, 2, 4, 6, 8 and 10. That is six allowed values, and nothing in between is permitted. Now a number like 6.5 has no exact home, so you record the nearest allowed value instead, which is 6. Each allowed value is called a level, and replacing a number with its nearest level is quantization. That is the whole idea. Every method in this course is just a smarter way of choosing the levels.
Drag the dot below to pick a real value. The purple marker underneath shows the nearest level it snaps to, the value you would actually store. Then press the buttons to change how many levels there are, from a coarse 2 up to a finer 16.
Drag the value, watch it snap to a level
value 6.5 → stored 6.7 · error 0.2 · 2 bits
A level is one of the allowed values you let yourself store. Quantization replaces each real number with its nearest level.
The price you pay: error
Snapping to the nearest level is almost never exact, and the gap you create has a name: the quantization error. With only a few levels spread across 0 to 10, the value 6.5 lands on 6 and you are off by 0.5. Add more levels and the nearest one sits closer, so the error shrinks. That is the demo's other lever: 2 levels are coarse and wrong by a lot, 16 are fine and barely wrong.
value 6.5, many levels → nearest 6.5, error ~0
Why this saves bits
Now the part that makes the title literal. You never store the level's value, like 6. You store which level you picked: its position in the list, the first allowed value or the third and so on. That position is just a small whole number, a label.
And a label only needs enough bits to count up to the number of levels. Two levels need a 1-bit label, just 0 or 1. Four levels need 2 bits. Two hundred and fifty-six levels need 8 bits, which is one byte. So a shorter list of levels means a shorter label, which means fewer bits per number. "Fewer bits, on purpose" is exactly that: choose a short list of levels, and pay only for the label.
Key takeaways
AI systems store everything as huge piles of numbers, and a normal number costs 32 bits; the goal is to spend fewer.
Quantization picks a short list of allowed values (levels) and stores each number as the nearest level.
You store the level's label, not its value, so fewer levels mean a shorter label and fewer bits, at the cost of a bigger error.