Lesson 1.1
Meaning as Geometry
Words and sentences turn into points in space, and things that mean similar things land close together. That is an embedding.
A library shelved by meaning, not by spelling.
Walk into a good library and the cookbooks sit next to each other, the physics texts cluster on their own shelf, and the poetry is two rooms over. Nobody filed them alphabetically by title. They filed them by what they are about, so a book you want sits near other books you would also want.
That is the whole trick behind retrieval. Before we can find the passages that answer a question, we need a way to put every passage on a shelf by meaning. The tool that does it is an embedding: it turns a piece of text into a point in space, and arranges things so that close together means similar in meaning.
Text becomes a list of numbers.
A point in space is just its coordinates. On a flat map that is two numbers, an across and an up. An embedding does the same thing, only with many more numbers, often hundreds. That list of numbers is called a vector, and the model that produces it (an embedding model) is trained so that texts about similar things get similar lists.
Hundreds of dimensions sounds intimidating, but the idea does not change from two. More room just lets the model separate finer shades of meaning. We will reason about it in two numbers because that is something you can draw on paper; the math is identical when there are five hundred.
"a small dog" → embedding model → [0.81, 0.12, ..., 0.04]
One text in, one fixed-length list of numbers out. Same text always gives the same list.
A tiny worked example.
Pretend we have an embedding model with only two dimensions, so every word is a point on a flat map. Here are some made-up coordinates. The numbers are invented for the picture, but the arrangement is the point: the dog words huddle together, and the road word sits far off on its own.
| word | point (x, y) |
|---|---|
| puppy | (0.9, 0.8) |
| dog | (0.8, 0.9) |
| cat | (0.7, 0.7) |
| highway | (0.1, 0.2) |
Eyeball it: puppy at (0.9, 0.8) is almost on top of dog at (0.8, 0.9), and cat is nearby. highway at (0.1, 0.2) sits in a different corner. Nothing here matched letters. "Puppy" and "dog" share no spelling, yet they landed next to each other because they mean similar things. That is what we mean by meaning as geometry.
An embedding is a map of meaning. Each text is a point; texts about similar things sit close. Find the right passage and you have turned a reading problem into a distance problem.
What an embedding is not
Keyword matching
Keyword search compares letters: "dog" only matches "dog". An embedding compares meaning, so "puppy" and "canine" land near "dog" even with no shared characters.
A lookup hash
A hash scatters inputs at random so two similar texts get unrelated codes. An embedding does the opposite: similar texts get nearby points, on purpose. Nearness is the whole feature.
Key takeaways
An embedding turns a piece of text into a point in space (a vector, a fixed-length list of numbers).
The space is arranged so that close together means similar in meaning, not similar in spelling.
That reframes finding the right passage as a distance problem, which the next lessons measure and search.