📘 CASE STUDY — Part IV: Idempotent Write API (Outbox + Worker)
SECTION 0 — SCENARIO
A user clicks “Submit” and sees a spinner. The network flakes.
They retry. Now you have:
-
duplicate records
-
duplicated side effects (emails/webhooks)
You need a backend design that treats retries as normal.
SECTION 1 — INVARIANTS
-
Create request is idempotent for the same logical action.
-
Side effects happen at most once.
SECTION 2 — API CONTRACT
-
POST /resource-
header:
Idempotency-Key -
body: request payload
-
Rules:
-
store
(key, userId, requestHash, response) -
same key returns same response
-
different payload returns deterministic error
SECTION 3 — OUTBOX PATTERN
Goal:
- DB write and “event to process side effects” commit atomically.
Mechanism:
-
write resource row
-
write outbox row in same transaction
-
worker reads outbox and emits email/webhook
Worker is idempotent:
- dedupe by outbox id
SECTION 4 — FAILURE MODES
-
worker crash after emitting webhook → dedupe downstream or store delivery records
-
outbox backlog grows → alert on lag
SECTION 5 — OBSERVABILITY
-
idempotency reuse rate
-
outbox lag
-
side-effect delivery failures
SECTION 6 — EXERCISE
Apply this pattern to:
-
signup welcome email
-
file upload processing
-
checkout fulfillment