Stuffing the Context

Hand someone the pages, then ask.

Picture asking a colleague a question about your company's billing rules. You could ask cold and hope they remember. Or you could slide the two relevant pages across the desk first and say: "answer from these." The second way is calmer and far more likely to be right. That handing-over is exactly what augmentation does for the model.

By now retrieval has done its job: you have a short list of passages that look relevant to the question (the top results from your search). Augmentation is the next hop. It takes those passages and weaves them into a single block of text, the prompt, that the model will read before it writes a word. Nothing about the model changes. You are just being careful about what it sees.

Three parts in one prompt.

A grounded prompt almost always has the same three parts. First an instruction (often called the system instruction) that sets the rules of the game. Then the context, the retrieved passages pasted in, usually each one labeled so the model can point back to it. Then the question the reader actually asked. Order matters, but the shape is steady.

Here is a tiny skeleton of that shape. It is just to show the parts, not a working system: the real serving code is yours to write.

Three lines of instruction, the labeled passages, the question. That is the whole template. The art is in those instruction lines, which is where the next two parts come in.

Answer only from the context.

The single most important sentence in the prompt is the one that tells the model to answer only from what you gave it. Without it, the model treats your passages as a suggestion and happily fills gaps from its own memory, which is the exact behavior RAG exists to stop. With it, you draw a fence: the answer must come from inside the context.

Two more instructions ride alongside it. Tell the model to cite the passage it used (that is what those [1] and [2] labels are for, so the reader can check the source). And tell it to say "I do not know" when the context simply does not contain the answer. That last one is the hardest behavior to get and the most valuable. A model that admits a gap is worth far more than one that invents a confident, wrong reply.

A worked example.

Say the reader asks: "what is the refund window?" Retrieval hands back two passages. The first says "refunds are accepted within 30 days of purchase." The second is about shipping and is not really relevant, but it scored close enough to make the list. You build the prompt with both passages labeled, the three instruction lines on top, and the question at the bottom.

The model reads it and answers: "The refund window is 30 days from purchase [1]." It used passage one, ignored the off-topic passage two, and pointed back to its source. Now imagine the answer was not in either passage. A well-instructed model replies "I do not know based on the provided context" instead of guessing. Same model, same passages: the instruction is what bought you the honesty.

Key takeaways