A timeout does not mean an email failed. I prevent blind retries by recording each intended send before contacting the provider, then checking that record before another attempt.
My approach has three parts:
- Record and guard the send: save the request under one stable key and allow only one worker to submit it.
- Use provider idempotency where supported: reuse the same key and unchanged request within the provider’s stated limits.
- Resolve uncertain outcomes: check provider records before resending. If acceptance remains unclear, mark the send
unknownand follow a recovery policy.
I keep retry decisions inside the email tool - not with the agent. An expired worker claim is not permission to resend. Before launch, I test concurrent calls, timeouts and crashes; afterwards, I track unresolved sends and reconciliation times.
The goal is fewer duplicate submissions, not a promise of exactly-once delivery.
::: @figure
{AI Agent Email Tools: Safe Retry Workflow}
:::
1. Record each intended email before sending
Choose a stable logical-send key
Build the key from the order or case ID, recipient and message purpose. For a one-off email, generate a UUID once and save it.
Enforce the key with a database uniqueness constraint. An application-level “check then insert” can race if two workers run at the same time. Write the intent record before any provider call.
Store email details and provider responses
Record enough context for a retry to reproduce the same request. Include the intent ID, send key, recipient, sender, subject, provider name, account or region, and provider idempotency key where supported.
Store either an immutable rendered message or a versioned template reference with the exact variables needed to recreate the original request. Add a SHA-256 content hash to detect changes before retrying. Also record provider message IDs when returned, response codes, attempt times, retry count, last error and agent-run correlation IDs.
Store timestamps in UTC and display them as 31/01/2026 14:30. Restrict access, redact content from logs and set retention limits. Where protected references are enough, avoid storing full bodies or attachments. UK GDPR requires data minimisation and justified retention.[1][2][3]
Define each state against the provider’s documented behaviour, then use those states to decide whether a retry is safe.
| Status | Meaning |
|---|---|
pending |
Intent exists; no worker has claimed it. |
in_flight |
A worker holds the claim and is attempting submission. |
accepted |
Provider acknowledged submission; recipient delivery is not proven. |
sent |
Use only where documented provider evidence supports this label. |
failed |
Confirmed rejection before acceptance; retry eligibility depends on the error. |
unknown |
Acceptance cannot be established or ruled out. |
Check the ledger before each send attempt
Once the record exists, claim it atomically before sending. Use row locking or a conditional update to check that the state allows sending and no active worker lease exists. Store the worker ID, claim time and lease expiry, then commit before contacting the provider.
If a second worker finds an accepted or sent record, it must return the stored result without submitting again.
An expired lease does not prove the previous send failed. Route stale in_flight records to recovery rather than resetting them to pending. Likewise, a draft ID proves only that a draft was created - not that the provider accepted the send.
Retry only confirmed pre-acceptance failures under documented rules. Keep uncertain outcomes unknown until provider evidence or recovery resolves them.
AI Engineer Interview: Idempotency Keys for Agent Tools
2. Add provider idempotency to send_email
A database claim cannot make the provider call and ledger write atomic. A worker can crash after the provider accepts an email but before your application records it. Provider-supported idempotency protects this gap within the provider’s documented limits, preventing a retry from sending another email.
Reuse the idempotency key and request
Persist one provider key before the first attempt. Reuse that key with the unchanged request, including attachments, headers and provider account. Check the provider’s documentation for key length, scope and retention limits.
A timeout must not generate a new key. A documented successful replay represents the original send, not another email.
| Situation | Documented provider behaviour to establish | Safe application action |
|---|---|---|
| First request | When the key is stored and which response proves acceptance | Mark the intent as in_flight, submit the request once, persist the response and message ID, then mark it accepted. |
| Identical retry | Whether the original response is replayed without a new send | Reuse the key and request; treat a documented replay as the original send. |
| Changed request | Whether mismatches are rejected or the original response is replayed | Reject locally before submission; never reuse the key for changed content. |
| Concurrent requests | Whether matching keys are serialised or reported as in progress | Use an atomic database claim; allow only the claimant to call the provider. |
| Expired key | Whether the request can now be processed as new | Query provider status first; otherwise route the intent to recovery. |
Write guarded send_email pseudocode
Keep retry decisions inside the tool rather than leaving them to the agent’s judgement. The atomic claim must recheck the current state and issue a fencing token. Later writes check this version so an old worker cannot overwrite newer results.
Recovery can authorise a retry only after confirmed non-acceptance or while documented idempotency protection still applies.
send_email(input):
validate input.recipient, input.sender, input.content, input.logical_send_key
payload = canonicalise(input)
intent = insert_intent_if_absent(
logical_key = input.logical_send_key,
payload_hash = hash(payload),
provider_key = new_key()
)
if intent.payload_hash != hash(payload):
reject("logical send key already exists with different content")
if intent.state == "accepted":
return recorded_result(intent)
if intent.state == "unknown":
return require_recovery_or_provider_lookup(intent)
claim = atomic_claim(intent.id, lease, fencing_token)
if claim.failed:
return already_in_progress()
try:
response = provider.send(
payload,
idempotency_key = intent.provider_key
)
persist_response_and_message_id(
intent.id, response, fencing_token,
state = "accepted"
)
return success(response)
except timeout_or_connection_error:
mark_unknown(intent.id, fencing_token)
return unknown_outcome
except provider_rejection as error:
persist_failure(intent.id, error, fencing_token)
return failure
Check provider status before resending
If the ledger leaves acceptance unresolved, first check the stored provider message ID through a supported status endpoint. Then try a supported idempotency-key lookup, account activity or message search. Read the results carefully: provider records may take time to appear. Persist the recovered response and message ID before returning success.
An empty search result is not proof of non-acceptance. Replay an uncertain request only while its original key remains covered by documented guarantees. Otherwise, keep it unknown and route it to recovery.
An expired worker lease does not extend the provider’s idempotency window. Unresolved sends must stay in recovery until the provider state is clear. If there’s no safe replay path, leave the intent as unknown and follow the recovery policy in the next section. If the provider cannot prove acceptance, use recovery - not a blind resend.
3. Handle sends without provider idempotency
When a provider does not support idempotency, treat any timeout as unresolved until recovery confirms acceptance.
Trace a timeout after provider acceptance
Record and claim the intent before sending. The provider may accept the email, but the connection can time out before its message ID arrives. That leaves the ledger without proof of acceptance, so resending could create a duplicate. Mark the intent unknown and follow the recovery policy.
A database lock cannot undo acceptance at the provider. Keep the original request payload, attempt timestamps, timeout details and correlation ID for investigation. A client-generated Message-ID or custom header can help trace the send, but it does not prevent duplicates unless the provider documents that behaviour.[4][5]
Set a recovery policy for unknown sends
Make the trade-off explicit: resending risks a duplicate; suppressing the resend risks a missed email. Stop automatic retries while recovery gathers evidence.
Use the evidence below to choose the least harmful recovery action.
| Recovery action | Duplicate risk | Missed-email risk | Evidence needed before resending |
|---|---|---|---|
| Manual review | Low to medium, depending on evidence quality | Medium if review is delayed | Provider logs and event history matched to the intent |
| Delayed reconciliation | Low to medium | Medium to high for time-sensitive messages | Defined waiting period, delivery events and queue status |
| Provider account search | Medium: results may be incomplete or delayed | Medium | Results matched by recipient, subject, timestamp and correlation ID |
| Intentional suppression | Very low | High if the original was never accepted | Evidence of acceptance or a documented decision that duplication is worse than omission |
Give each unknown record a named owner responsible for recovery and a deadline. Require review within a defined period, followed by escalation to the service owner.
Neither review nor search proves that a resend is safe. If uncertainty remains, require an authorised decision based on whether omission would cause more harm than duplication. Record the evidence, decision-maker, time and chosen action.
Document what each provider guarantees
Record whether the send endpoint supports idempotency. If it does, document key retention and scope, request-matching rules and concurrent-request behaviour. Specify which responses are replayed and what happens after expiry. List the events, webhooks and search APIs available to investigate unknown outcomes.
Keep API deduplication separate from mailbox delivery. Preventing a second outbound message does not guarantee exactly-once delivery through downstream mail systems. Store provider message IDs, email Message-ID values and delivery events separately.
The send tool must read these documented guarantees before deciding whether a resend is allowed.
Conclusion: Test retry safety before deployment
Before deployment, check that you have durable intent records, stable send keys, atomic send gating, stored provider message IDs and a defined recovery path for unknown outcomes.
Test that these controls fail closed under race conditions. Run concurrent calls against the same intent: only one worker should call the provider. Then change the recipient, subject or attachment while keeping the same key. The request should be rejected.
Test unresolved sends, too. Simulate provider acceptance followed by a timeout, then test a crash before the ledger update. Block blind resends in both cases. Treat worker-claim expiry and provider idempotency expiry separately: neither proves a resend is safe.
After launch, track the number and age of unknown sends, alongside median and 95th-percentile reconciliation times. Set alerts for a growing backlog or excessive reconciliation times.
For help with a custom agent implementation, Hokstad Consulting can build these controls into your workflow.
FAQs
How should I handle an intentional resend?
Before resending, check the external service using your saved operation ID or business reference to find out whether the previous attempt succeeded. If it did, record the existing receipt and do not send again.
If the outcome is still unknown or the service doesn’t support native idempotency, set the transaction aside for manual review. Always use stable idempotency keys for state-changing operations so the provider can handle repeated requests correctly.
What if provider status checks contradict my send record?
Treat the provider’s record as the source of truth. A timeout or missing response means the outcome is unknown - not that the transaction failed. Before retrying, check with the provider using your business reference or operation ID.
If the provider confirms completion, record the result and receipt immediately. Do not retry. If the status is still unclear, quarantine the transaction for manual review.
Use stable idempotency keys, but don’t rely on them alone: they cannot guarantee safe retries unless the provider supports idempotency.
When should I resend an unresolved email without idempotency?
Only resend when you’ve confirmed that it’s safe to repeat or that the send failed without side effects. If the provider doesn’t support idempotency, don’t retry blindly. First, check the external send status using the saved operation ID or business reference.
If the outcome is still unclear, quarantine the request for manual review instead of resending it. This prevents duplicate sends and keeps records consistent.