Lesson 0.2
A request's journey
Follow one chat request from your app, through the gateway, to a provider and back.
What a request is made of.
Before we trace the trip, look at what actually travels. A chat request has two parts: the body (what you are asking) and the headers (who is asking).
The body says which model you want, the conversation so far (the messages), and a few dials
that shape the answer. The two you will meet first are temperature (how adventurous the
model is, from steady to creative) and max_tokens (a ceiling on how long the answer can
run). The headers carry your auth key, the secret that proves the request is allowed.
{
"model": "gpt-4o-mini",
"messages": [{ "role": "user", "content": "Name one moon of Mars." }],
"temperature": 0.2,
"max_tokens": 50
}
That is the whole package: a key in the header, a model and messages and a couple of dials in the body. Now watch where it goes.
The round trip, step by step.
One request, five stops. Notice that the gateway is the middle three. The client only ever sees the first and the last.
The client sends the request.
Your app sends the body and the auth header to the gateway's address. As far as the app knows, this is the whole journey: ask, wait, answer.
The gateway authenticates and picks a provider.
The gatekeeper checks the key is valid, then reads the model name and decides which provider should handle it. For now, picking is simple: the model name maps to one provider. Unit 3 makes this choice smarter.
The gateway forwards to the provider.
It sends the request on to the real provider, using the provider's own key, in the shape that provider expects. The next unit is all about this forwarding step.
The provider runs the model and replies.
The provider runs the model and returns a
completion, plus a usage block that counts the tokens used: how many went in
(prompt_tokens) and how many came out (completion_tokens). That count is
what the bill is based on later.
The gateway returns the answer to the client.
The gatekeeper passes the completion back to your app. The app gets one clean answer and never had to know which door it came from.
What comes back.
Here is a small response body for the request above. The answer is in choices, and the
token count is in usage.
"choices": [
{ "message": { "role": "assistant", "content": "Phobos." } }
],
"usage": { "prompt_tokens": 14, "completion_tokens": 3 }
}
Look at the two bodies side by side. The request had model, messages, and a few
dials. The response has choices and usage. Same family of shapes going in and
coming out, which is exactly why a gateway can sit in the middle without your app noticing.
Same shape in, same shape out. The request and response have a predictable structure, and the gateway's job is everything that happens between: authenticate, pick a provider, forward, and hand the answer back.
Key takeaways
A request is a body (model, messages, dials like temperature and max_tokens) plus headers (the auth key). The response carries the answer in choices and the token count in usage.
The round trip is five stops: client sends, gateway authenticates and picks a provider, gateway forwards, provider replies with a completion and usage, gateway returns the answer.
The client only sees the first and last stop. Everything in between is the gateway's work, which is what the rest of the course builds up.