What a gateway is

You talk to one thing. It deals with the rest.

Think of a hotel concierge. You want a dinner table, a taxi, and tickets for a show. You do not call three restaurants, two cab companies, and a box office yourself. You tell the concierge once, and the concierge talks to all of them behind the desk. To you it feels like one helpful person. Behind the desk it is a web of phone calls you never see.

A travel-booking site works the same way. You search one box, and it quietly queries a dozen airlines, then hands you one clean list. One front desk, many suppliers.

In our story, that front desk is the gatekeeper monk. Every traveler arrives at one door. He greets each one, and he is the only one who knows what lies down each path behind the gate. An LLM gateway is that gatekeeper for AI models.

One API in front of many providers.

Plainly: an LLM gateway is a single place you send your request to, and it forwards that request to the real AI provider for you. The precise version: a gateway is one API endpoint that sits in front of many model providers, such as OpenAI, Anthropic, Google, or a model running on your own machine.

Your app never calls the providers directly. It calls the gateway. The gateway calls the real provider, waits for the answer, and passes it back to you. From your app's point of view, there is only ever one address to talk to.

Toggle a provider down and send again. The door stays the same, but the request finds whichever path is still open. That is the gatekeeper's job in one picture: one door, many providers behind it, and he always knows where to send you.

Real products do exactly this. OpenRouter gives you one endpoint that reaches hundreds of models. LiteLLM is an open-source gateway you run yourself. Cloudflare AI Gateway sits in front of your provider calls as a managed layer. Different shapes, same idea.

Why bother with one door?

Two reasons, and both come from having a single choke point that every request flows through.

First, you can swap providers without touching your app. If your app talks straight to one provider, switching to another means editing the app. With a gateway in the middle, you change the destination in one place and your app never notices. The model behind the door changed; the door did not.

Second, you can add features that should apply to every call, in one place. Retrying a failed request, remembering a recent answer (caching), capping how much someone can spend (limits): if every request passes through the same door, you build these once at the door instead of copying them into every app. We spend the rest of the course on exactly these jobs.

Follow one request through.

Your app sends the gateway a small request: which model it wants, and the conversation so far.

The gateway reads the model name, picks the matching provider, and forwards the request. The provider runs the model and sends back a completion. The gateway hands that completion straight back to your app.

Your app asked one address and got one answer. Whether the model lived at OpenAI, Anthropic, or on a box in the next room is the gateway's business, not the app's.

Key takeaways