Lesson 3.1
Splitting a vector into chunks
One codebook cannot cover a long vector. Product quantization's trick is to cut the vector into pieces and give each piece its own.
Why one codebook is not enough
Last unit ended on a wall. To represent a 128-dimensional vector well with a single codebook, you would need a dictionary with more entries than there are atoms in a drawer of sand. The space is just too big to cover. Product quantization walks around the wall instead of climbing it: cut the long vector into a handful of short chunks (small slices of the vector, M of them, where M is simply how many pieces you choose) and quantize each chunk on its own.
A 128-number vector cut into 8 chunks is eight little 16-number vectors. Each lives in a small enough space that a 256-entry codebook covers it comfortably. Press the M buttons below to see a 12-number vector regroup, and watch the stored size: one byte per chunk.
12 numbers, split into M chunks
4 chunks → 4 bytes · 12× smaller than 12 floats
Product quantization splits a vector into M chunks and quantizes each chunk independently with its own small codebook. The stored size is M bytes, one per chunk.
The name, and the knob
It is called "product" quantization because the full vocabulary is the combination, the product, of all the per-chunk codebooks. Eight chunks with 256 entries each can describe 256 to the eighth power distinct vectors, an enormous range, while storing only 8 small dictionaries.
M is the one knob. More chunks mean shorter, easier-to-cover pieces and lower error, but more bytes stored. Fewer chunks compress harder and lose more. It is the memory-versus-accuracy trade from Unit 0, now wearing the letter M.
Key takeaways
Product quantization cuts a long vector into M short chunks and quantizes each chunk separately.
Each short chunk lives in a small space, so a 256-entry codebook covers it well.
M is the trade-off knob: more chunks mean lower error but more stored bytes.