Lesson 0.3
Why a gateway at all
The problems one front door solves: provider lock-in, repeated failures, runaway cost.
What goes wrong without one.
You can call a provider directly. Plenty of apps do, and for a single small app it is fine. The trouble shows up the moment you have more than one app, or you want to change anything.
Without a gateway, each app hardcodes one provider. The provider's address and quirks are baked into the app, so switching means editing code in every app that calls it. You are locked in.
Each app also re-implements retries on its own. When a call fails, every app needs its own logic to try again, and they all drift apart and get it slightly wrong in different ways.
They share no cache. App A asks a question and pays for it. App B asks the exact same question an hour later and pays again, because neither knew the other had already seen the answer.
And there is no spend control. Nobody is counting across all the apps, so a runaway loop in one of them can burn through the budget before anyone notices.
What one front door fixes.
Put a gateway in the middle and every one of those problems moves to a single place where you solve it once.
- →Swap providers freely. Change the destination at the door, and every app follows. No app edits.
- →Retry and fallback in one place. Write the retry logic once, at the door, and every app inherits it correctly.
- →A cache shared across everything. One app's answer can serve another's identical question. You pay the model once.
- →Cost and limits enforced centrally. One ledger counts every token from every app, and one set of caps holds the whole system to its budget.
The pattern underneath all four: a gateway is the one spot every request passes through, so it is the one spot worth fixing.
The gatekeeper's five jobs.
Everything a gateway does maps onto five things the gatekeeper monk does at the gate. The rest of the course is one unit per job. Here is the whole map up front.
Greet at one door
Every request arrives at the same place and gets forwarded on. One address for your whole app.
Pick a path
Choose which provider handles each request, by cost, speed, or quality.
Reroute when a door is shut
When a call fails, try again, or send it down a different path. The app never sees the stumble.
Recognize a traveler
Remember answers already given, so the same question does not pay for the model twice.
Keep the ledger
Count every token, tally the spend, and stop anyone from running past their budget.
A gateway is where you put everything that should be true for every request, written once. Greet at the door, pick a path, reroute on failure, recognize a repeat, keep the ledger. Five jobs, one door.
Key takeaways
Without a gateway, every app hardcodes a provider, re-implements retries, shares no cache, and has no central spend control. The pain grows with every new app.
A gateway moves provider choice, retries, caching, and limits to one front door, so you solve each problem once for every request.
The gatekeeper's five jobs (proxy, routing, retry and fallback, caching, cost and limits) are the five units ahead. Next we start with the first one: forwarding a request.