Lesson 1.2
Measuring Closeness
If meaning is distance, you need a way to measure it. Cosine similarity reads the angle between two meanings.
Drag the question and watch the nearest meanings light up.
Last lesson, meaning became a map of points. The natural next question: given a question dropped anywhere on that map, which passages are closest? Grab the coral dot below and move it. The three nearest passages light up, and a closeness number updates as you go. Drag it into the dog corner and the dog words win; drag it toward the car corner and they take over.
Drag the question around the map of meaning
Nearest passages right now:
Closeness is illustrative (distance on this little map), not a measured embedding score.
That is retrieval in miniature. The question is a point, every passage is a point, and we keep the ones that sit nearest. The rest of this lesson is about how to put a real number on "nearest".
Why the angle, not the raw distance.
The obvious move is straight-line distance, the gap between two points. It works on the toy map, but it has a flaw in real embeddings: longer texts tend to produce longer vectors (points further from the origin), so a chatty passage can look "far" just for being long, even when it is about exactly the right thing. Length should not decide meaning.
So instead of distance we read direction. Picture an arrow from the origin out to each point. Two texts about the same thing point the same way, whatever their length. The standard measure is cosine similarity: it reads the angle between the two arrows and ignores how long they are.
same direction (angle 0) → cosine 1, very similar.
at a right angle (angle 90) → cosine 0, unrelated.
opposite direction (angle 180) → cosine -1, opposite meaning.
The formula, then a worked number.
Cosine similarity is a short recipe. Multiply the two vectors piece by piece and add up the results (that sum is the dot product). Then divide by each vector's length, to cancel out how long they are.
dot product on top, the two vector lengths on the bottom
Take two tiny vectors, a = (3, 4) for the question and b = (4, 3) for a passage. Top line, the dot product: 3×4 + 4×3 = 12 + 12 = 24. Bottom line, the lengths: each is √(3² + 4²) = √25 = 5, so the bottom is 5 × 5 = 25.
cos(a, b) = 24 / 25 = 0.96
Close to 1, so these two nearly point the same way: a strong match.
Now double the passage to b = (8, 6), twice as long. The dot product doubles to 48 and the bottom doubles too (5 × 10 = 50), so the score stays 48 / 50 = 0.96. Length changed, the verdict did not. That is exactly the property we wanted.
Cosine similarity scores meaning by direction, on a scale from 1 (same) through 0 (unrelated) to -1 (opposite). Because it divides out length, a long passage and a short one are judged on what they say, not how much.
Key takeaways
Retrieval ranks passages by closeness to the question, both treated as points in the embedding space.
Cosine similarity reads the angle between two vectors: 1 is same direction, 0 is unrelated, -1 is opposite.
It divides out vector length, so a passage is scored on meaning, not on how long it happens to be.