Rotate, then round (TurboQuant)

Spread the error evenly

A plain quantizer treats every coordinate the same, which only works well if every coordinate carries a similar share of the spread. Real vectors are lumpy: one direction may hold most of the variation while another holds almost none, so a fixed quantizer wastes precision on the quiet coordinates and runs short on the loud ones. A random rotation fixes this. A rotation is a mathematical spin of the whole vector, like turning a photo around its centre; doing it at random means picking the spin direction blindly, without looking at the data. After spinning the vector, the spread is shared evenly across all coordinates, and a single per-coordinate quantizer becomes close to the best you could do.

Drag the handle below to rotate an elongated cloud. When all the spread sits on one axis the error is lopsided; rotate it and the two axes balance. In hundreds of dimensions a random rotation does this balancing automatically, without ever looking at the data.

The two stages of TurboQuant

TurboQuant (Zandieh, Daliri, Hadian and Mirrokni at Google Research, arXiv 2504.19874) puts this to work in two stages. First, rotate the vector and apply an optimal per-coordinate scalar quantizer, the part the demo shows. Second, take the small leftover error (called the residual) and correct it with a single extra bit per coordinate, which removes the bias from the distance estimate. The paper reports a distortion within a small constant factor of the information-theoretic best possible.

The same method was first aimed at compressing an LLM's attention memory, then shown to apply to search vectors too. Widely repeated headline figures, such as specific memory multipliers or wins over a particular baseline, come from vendor write-ups rather than the paper, so treat them as claims to verify, not facts.

Key takeaways