Lesson 2.1
Ways a call fails
Not all failures are equal. Some are worth retrying, and some are your own bug. Telling them apart is the whole skill.
A busy signal is not a wrong number.
You call a shop. Sometimes the line is busy, so you hang up and try again in a minute and it goes through. Sometimes you dialed a digit wrong, and no amount of redialing the same wrong number will ever reach the shop. Both are "the call failed." Only one of them is worth trying again.
Calls from the gateway to a provider fail in exactly these two flavors. A busy signal, where the provider is fine but momentarily can't take your call, and a wrong number, where your request itself is broken and will be broken every single time you send it. The gatekeeper monk has to read which kind of "no" he just got before he decides whether to knock again.
The provider tells you which kind it is, mostly through a status code: a small number attached to every response. Learn to read it and the retry decision almost makes itself.
Two families: their side, or yours.
Group the failures by who is at fault, because that is what decides everything.
Their side, or just bad timing. The provider got overwhelmed (status 429, rate limited), one of their servers hiccuped (status 500, 502, or 503), or the network stalled and the answer never arrived in time (a timeout or a dropped connection). None of these are caused by your request being wrong. The same request, sent again a moment later, has a real shot at succeeding.
Your side. You sent a malformed body or an unknown model name (status 400, bad request), or your API key is wrong or expired (status 401, unauthorized), or the model refused on safety grounds (a content-filter refusal). These come back from a perfectly healthy provider. Sending the identical request again gets you the identical refusal. Retrying just wastes time and, if it is a 429 you misread, can dig the hole deeper.
Click a failure to see the verdict
Click through the cases. The green ones might succeed on a second try because nothing about your request was wrong. The red ones will fail the same way every time, because the problem is baked into what you sent.
The decision table.
Here is the whole rule on one card. The status code on the left, whether to retry in the middle, and the reason on the right.
| Failure | Retry? | Why |
|---|---|---|
| 429 | yes, after a wait | Rate limited. Slow down and it clears. |
| 500 / 502 / 503 | yes | Server hiccup on their side. Usually temporary. |
| timeout / conn error | yes | No answer in time. The next attempt may land. |
| 400 | no | Bad request. Your bug. Same body, same failure. |
| 401 | no | Bad or missing key. Retrying will not fix it. |
| content filter | no | A deliberate refusal, not a glitch. It will refuse again. |
One pattern runs through it: retry the server-side and timing failures, never retry the client errors. The only client-side code worth retrying is 429, and even that one only after you wait, which is the subject of a later lesson.
Retry the failures that might succeed next time: 429, 5xx, timeouts. Never retry the ones that will fail the same way: 400, 401, content refusals. The question is always "did my request cause this, or did their side just stumble?"
Key takeaways
Failures split into two families: transient or server-side (their stumble) and client errors (your bug). The status code tells you which.
Retry 429 (after a wait), 5xx, and timeouts. Do not retry 400, 401, or a content refusal: the same request fails the same way.
Knowing a failure is retryable is only half the job. Next we ask whether retrying is even safe, because some calls do real work twice.