Lesson 5.3
Budgets and fairness
Per-key spend ceilings, and making sure one heavy user does not starve everyone else.
A ceiling on what each key can spend.
Rate limiting holds the pace. A budget holds the total. It is a spend ceiling tied to an API key: key A may spend $50 this month, and not a cent more. Each key gets its own line on the ledger and its own cap.
The mechanism is the ledger from lesson 5.1, read backwards. That ledger has been summing the cost of every request per key all along. A budget just adds a check: before a call goes out, would it push this key over its ceiling? If yes, the gateway rejects or throttles it. The counting was already happening; the budget gives it teeth.
A budget is a per-key spend ceiling (say $50 a month). The gateway tracks each key's spend from the cost ledger and turns a call away once it would cross the cap.
The call that tips over.
Say key A has a $50 monthly budget and has spent $48 so far. A new request comes in that would cost $3. The gateway checks before sending.
The check happens before the money is spent, not after. The point of a ceiling is that you never sail past it and find out later; the gateway stops the call that would breach it.
Three keys, three ledgers.
Each key carries its own bar: how much of its budget is used against its limit. Key A is brushing the ceiling and the next big call may be refused. The others have room.
One heavy user should not starve the rest.
Budgets cap each key's spend. Fairness is the other half: who gets the shared capacity right now. A provider gives you only so much throughput. If one tenant floods the gateway with requests, they can soak up that whole allowance and leave everyone else waiting, even if no single key is over its money budget.
The fix is to share on purpose. Per-key limits stop any one caller from hogging the line, and fair scheduling takes turns across keys rather than serving whoever shouts loudest. Together they keep the shared capacity actually shared, so a busy neighbor cannot crowd out a quiet one.
Budgets turn the cost ledger into a control: per-key limits keep one user from spending everyone else's capacity, and fair scheduling shares the line. This is the gatekeeper's last job, the ledger with teeth.
Try it yourself
Key B has a $50 budget and has spent $40. Three calls are queued, each costing $5. Which of them go through, and where does the gateway stop? Then think: if all three keys share one provider that can handle 100 requests a minute, why is a money budget alone not enough to keep things fair?
Key takeaways
A budget is a per-key spend ceiling, tracked from the cost ledger. When a call would push a key over its cap, the gateway rejects or throttles it before the spend happens.
Key A at $48 of $50 plus a $3 call would hit $51, over the ceiling, so the call is refused. The check is before the money moves, not after.
Fairness is the other half: per-key limits plus fair scheduling keep one heavy tenant from soaking up shared provider capacity. Budgets are the ledger with teeth, the gatekeeper's last job.