Idempotency: safe to retry?

Pressing the button twice.

Think about a lift. You press the call button, nothing seems to happen, so you press it again. No harm done: the lift was already coming, and a second press changes nothing. Pressing it once or five times lands you in the same place.

Now think about a "Buy now" button on a checkout page. It hangs. You press it again. If the site is careless, you just bought the thing twice and got charged twice. Same gesture, very different outcome, because this button does something with a lasting effect each time it is pressed.

That difference has a name. The lift button is idempotent: doing it again has the same effect as doing it once. The buy button is not. Last lesson we learned that a timeout is safe to retry. This lesson asks a different question: when you do retry, is doing the work twice actually harmless?

Idempotent means "again equals once."

A read is naturally idempotent. Asking "what is the balance?" twice gives you the balance twice and changes nothing. Reading a page, fetching a record, looking something up: repeat it as often as you like, the world is unchanged.

An action with a side effect usually is not. "Transfer ten dollars" run twice transfers twenty. "Send this email" run twice sends two emails. "Book this flight" run twice books two seats. Each repeat leaves a fresh dent in the world.

Here is the catch for retries. When an attempt times out, you often have no idea whether the work already happened. Maybe the provider never got your request. Maybe it did the work and the answer got lost on the way back. From where you sit, both look identical: silence. If the action was not idempotent, a blind retry might be the second charge, not a harmless redo.

Where this bites an LLM gateway.

A plain completion is usually safe to retry. If you ask a model "summarize this paragraph" and the call times out, retrying just gets you another summary. You spend a little extra money on tokens, but nothing in the world broke. The result is just text handed back to you.

The danger shows up when a call has a side effect beyond returning text. Modern models can call tools: the model decides to run a function you gave it. If that tool charges a card, sends a message, places an order, or writes a row to a database, then the completion is no longer a harmless read. Retry it blindly and you might charge the card twice or send the email twice. The text part was safe; the action the text triggered was not.

So "is this retryable?" (last lesson) and "is this safe to repeat?" (this lesson) are two separate gates. A call can pass the first and fail the second. A tool-calling request that times out is retryable in principle, but not safe to blindly resend.

The fix: an idempotency key.

You cannot always make an action idempotent on its own. But you can make the server recognize a repeat and refuse to do the work twice. The tool for that is an idempotency key: a unique string the client makes up and attaches to a request. It is a label that says "this is operation number abc."

The server remembers keys it has already finished. The first time it sees a key, it does the work and files the result under that key. If the same key arrives again, it skips the work entirely and hands back the first result. The retry becomes safe because the server, not you, is the one that notices it is a duplicate.

Notice the key has to come from the client and stay the same across the retry. If you make a fresh key for the retry, the server sees a brand-new operation and does the work again. Same key is the whole point.

Key takeaways