Backoff and jitter

Hammering a busy door does not open it.

A provider just told you it is overwhelmed (a 429). The naive move is to fire the request straight back, then again, then again, as fast as your loop runs. You are now pounding on a door that is already jammed with people. Every retry adds to the pile that caused the 429 in the first place, so you get more 429s, not fewer. The cure made the disease worse.

The fix is patience that grows. Wait a moment before the first retry. If that fails too, wait longer. Keep doubling the pause. You give the struggling provider room to recover instead of crowding it, and you stop after a few tries instead of looping forever.

This pattern has a name: exponential backoff, with a sprinkle of randomness called jitter on top. The gatekeeper monk, turned away at a busy door, steps back and waits a little longer each time rather than rattling the handle.

Backoff: double the wait each time.

The rule is simple. Pick a base wait, say one second. After the first failure wait one second. After the next, two. Then four, then eight. Each pause is double the last. This is "exponential" because the wait grows by multiplying, not by adding a fixed amount.

Two guardrails keep it sane. A cap stops the wait from growing forever (you do not want a retry scheduled for two minutes from now), and a limit on attempts means you give up after, say, five or six tries and report a clean failure instead of retrying into the void. Backoff is not "retry until it works." It is "retry a few times, politely, then stop."

Drag to add attempts and watch each bar double the one before it. The early retries come fast; the later ones space out, giving the provider real breathing room before the final try.

A schedule by hand.

With a one-second base and doubling, here is what the waits look like across attempts. Read it as: the attempt failed, so wait this long before the next one.

Four retries cost about fifteen seconds total, most of it in the last wait. That total matters: it is the budget the caller is paying in patience, which is exactly what the next lesson on deadlines is about.

Jitter: do not all retry in lockstep.

Backoff alone has a hidden trap. Imagine a thousand clients all hit the same provider, all get a 429 at nearly the same instant, and all run the same "wait one second" rule. One second later, all thousand fire again at the exact same moment. You have rebuilt the original spike. This synchronized stampede has a name: the thundering herd.

Jitter breaks the lockstep. Instead of waiting exactly one second, each client waits a random amount inside a band, somewhere between half a second and one second, say. Now the thousand retries are smeared across a window instead of landing on the same tick. The provider sees a gentle stream it can absorb rather than a wall.

Toggle jitter on in the demo above. The crisp doubling bars turn into ranges, each retry landing somewhere inside its band. That little bit of noise is what keeps a crowd of clients from retrying as one fist.

Key takeaways