Idempotent Request Handling

Definition

Idempotent request handling makes repeated attempts at the same operation produce no additional intended business effect after the first successful application. It is especially useful when a client times out and cannot tell whether the server completed the work. The problem is uncertainty, not merely a user clicking a button twice.

HTTP method semantics provide a starting point, but the method name does not make an implementation safe. A payment, booking or order-creation endpoint often needs explicit application-level protection. An operation may also emit messages or call downstream services, so protecting one database row may not protect the entire business workflow.

Define What Counts as the Same Operation

Choose the identity scope before writing retry code. A client-generated key may identify a logical request within a particular account and operation. A domain identifier, such as an order reference, may provide a stronger business invariant. Do not use a globally unscoped key that lets one customer's request collide with another's.

Bind the key to the relevant request content. Reusing a key with a different amount or destination should be rejected or handled through an explicit documented policy, not silently treated as the original request. Keys must not grant access to another user's result; authentication and authorisation still apply.

A Reliable Processing Pattern

1. Validate identity and input. Establish the caller, operation scope and request fingerprint before processing.
2. Claim the operation atomically. Use database constraints and transactional state, or another appropriate durable mechanism, to prevent concurrent attempts from both becoming the first processor.
3. Record progress and outcome. Distinguish in-progress, completed and failed states. Define what a retry receives in each case.
4. Protect the business effect. Keep local state changes and the operation record consistent, and address downstream effects with their own guarantees.
5. Replay or reconcile. Return the stored outcome when appropriate, or expose a safe status check while work remains unresolved.
6. Define retention. Keep operation records for a period that matches realistic retries and business risks, with separate durable uniqueness rules where needed.

Worked Payment Example

In an illustrative checkout, a client submits a payment request and loses its connection before receiving the response. The server may already have charged the customer. A new request with a new identity could create a second charge; simply refusing all retries could leave the customer unable to finish checkout.

The client retries with the original operation key. The application finds the recorded operation and reconciles the payment provider's status using the provider's supported idempotency and lookup mechanisms. It returns the existing outcome when known, or an explicit pending state while reconciliation continues.

A local database transaction cannot atomically include an arbitrary external payment service. The design must handle the gap between the provider accepting the charge and the application recording it. A transactional outbox can reliably hand off local work, but the receiver still needs appropriate deduplication or reconciliation. “Exactly once” is not a substitute for describing these failure windows.

Concurrency Is the Test That Matters

A lookup followed by an insert is vulnerable when two requests arrive together. Both can see no existing record and both can perform the effect. Use the storage system's atomic guarantees to claim the identity, and test competing requests deliberately. A button disabled in the browser cannot protect requests from multiple devices or automated clients.

Handle an in-progress duplicate honestly. Depending on the API contract, it might receive a pending response, a retryable conflict or a status location. Avoid returning success before the business outcome is known. Document this behaviour as part of the API contract.

Failures, Expiry and Side Effects

Decide which failures consume a key and which allow another attempt. Validation rejection and an uncertain downstream result are different states. Retrying an uncertain result as if nothing happened is dangerous. Keep enough evidence to reconcile it without storing unnecessary sensitive request content.

Expiry requires thought. Deleting deduplication records may make an old retry look new. For high-consequence operations, a longer-lived domain uniqueness rule can protect the underlying business action after the response cache expires. Emails, stock reservations and event consumers also need consideration; a deduplicated order row does not prevent two confirmation emails by itself.

Practical Lessons Learned

Test the lost-response case, not only two clean sequential submissions. Simulate connection loss after committing the effect, concurrent requests, process restart and downstream timeout. Verify what the caller sees and what actually happened in the business system.

Keep monitoring focused on unresolved operations and conflicting key reuse. A high retry rate may indicate network instability, slow processing or a poor client policy. Link diagnosis with the operational runbook so support can reconcile without manually repeating the action.

Expert Tips

• Use a stable logical operation identity across retries, not a new random key for every attempt.
• Test atomic ownership with concurrent requests against the actual storage mechanism.
• Make pending and uncertain outcomes visible to clients and support teams.

Common Mistakes

• Calling a pre-insert lookup deduplication without a concurrency guarantee.
• Protecting local rows while ignoring external charges or messages.
• Allowing the same key to represent different request content.
• Deleting records without considering late retries and durable business uniqueness.

Key Takeaways

• Retries must preserve operation identity and respect failure uncertainty.
• Atomic local control and downstream reconciliation solve different problems.
• Verify business effects, not merely repeated HTTP responses.