Use an outbox when database success and message delivery diverge

An order exists in the database, but the fulfillment service never hears about it. Retrying the original request risks creating another order. Publishing the message first creates the reverse problem: fulfillment may act on an order that was never committed. This is the dual-write gap. Neither ordering makes two independent systems behave like one transaction.

An outbox makes the intention to send a message part of the local durable change. A separate dispatcher delivers that intention later. This article uses an illustrative order and fulfillment message. It describes the application responsibilities around the pattern; it does not claim that using an outbox alone guarantees exactly one delivery or immediate consistency across systems.

Locate the actual atomic boundary

Trace the current request and mark the database commit and message publication. Ask what happens if the process stops between them. Include timeouts where the publisher cannot tell whether the broker accepted a message. These uncertain outcomes are usually the reason a simple retry cannot solve the issue safely.

Identify which writes can share a transaction in the chosen database. The order and its outbox entry must commit together for the proposed pattern to close this gap. If they live in unrelated storage systems, placing a table named outbox beside one of them does not create atomicity. Check the database's actual transaction scope and deployment requirements. Keep the initial design within a proven storage boundary rather than building another distributed transaction by accident.

Store business intent with an event identity

Create the order and an outbox record in the same transaction. Give the event a stable identifier, event type, schema version, creation time, and reference to the business operation. Store only the data needed by the consumer, avoiding unnecessary private information. Decide whether the payload is a snapshot or whether consumers fetch current state later.

A practical artifact is a sample event contract written with synthetic values. State which fields are required, which actor owns the event, and whether later changes create a new event. Do not use the order identifier as the identity of every event about that order; creation, cancellation, and refund are distinct intentions. Consumers need to distinguish repeated delivery of one event from a legitimately new state change.

Dispatch without losing the retry path

The dispatcher finds eligible outbox entries, claims bounded work, publishes, and records delivery progress. Define how multiple dispatchers avoid conflicting claims and how a claim expires after a crash. Use transaction or conditional-update behavior appropriate to the database. Do not mark an entry delivered before the publication outcome is known.

Even with correct ordering, a crash after publication but before marking delivery can cause a duplicate. This is expected under an at-least-once design. Preserve the stable event identifier across retries. Record attempt count, next eligible time, and a controlled error category. Use finite retries and an operational recovery path for persistent failures. A stuck outbox is durable, but durability alone does not tell the user or operator that work needs attention.

Make consumers repeatable

The consumer must decide what repeated delivery means. For fulfillment, record the event identifier with the resulting operation in an atomic boundary where possible. A second receipt should recognize prior processing instead of creating another shipment. If the consumer calls a provider, use a business idempotency key and verify the provider's semantics.

Avoid treating a pre-processing lookup as sufficient duplicate protection. Two consumers may both observe no prior record and act concurrently. A uniqueness constraint, conditional write, or transaction must protect the actual invariant. The outbox addresses missing publication after local persistence; it does not solve every downstream race. See retry-safe endpoint design for the related distinction between operation identity and repeated transport attempts.

Decide ordering and compatibility deliberately

Some consumers can process independent events in any order. Others must not see cancellation before creation. Define ordering per business entity only where required, and choose a sequence or version scheme the consumer can validate. Global ordering may impose unnecessary throughput and availability costs.

Version the payload when a consumer can be deployed independently. Prefer additive compatible fields during a transition and define what older consumers do with unknown event types. Do not change a field's meaning while retaining the same contract label. Keep enough event history or diagnostic metadata to investigate disagreements, but apply a retention policy. An outbox is not automatically an audit log or an event-sourced database; those uses require their own guarantees and access rules.

Monitor age and business impact

Measure the oldest undelivered event, delivery attempt rates, persistent errors, and backlog by relevant event type. Queue size alone may hide one old critical event among many recent successful events. Connect these signals to user outcomes: how long can an accepted order wait before fulfillment is considered late?

Provide an operator view that identifies an event and its last safe diagnostic without exposing customer payloads broadly. Define who may replay or suppress an event, and record those actions. Replaying a batch should be bounded and respect consumer duplicate protection. If the application says an order is fulfilled when it has only committed local intent, correct that state contract. The UI should reflect the actual business stage, not the technical existence of an outbox row.

Verify interruption points

In an isolated environment, interrupt the producer before and after commit and confirm that the order and outbox entry appear together or neither appears. Interrupt the dispatcher after successful publication but before its delivery update, then resume it. Verify that the consumer handles the repeated event without a second consequential operation.

Test malformed payloads, an unavailable broker, and a consumer running an older compatible schema. Confirm that permanent failures remain discoverable instead of being retried forever without visibility. Document the scope of the exercise: a fake publisher can verify application state transitions but cannot prove broker acknowledgments or redelivery behavior. Use an integration environment for those boundaries, with synthetic orders and controlled provider behavior.

Use AI to enumerate the gaps

Give an assistant the producer transaction, dispatcher claim sequence, event contract, and consumer duplicate logic. Ask what durable state exists at every interruption point. Require separate analysis of missing delivery, duplicate delivery, and out-of-order delivery. A recommendation that merely says use exactly once should be challenged for the precise boundary it describes.

Have the assistant draft failure-injection cases and identify private fields in the payload. Review all proposed APIs against the installed database and messaging libraries. The pattern introduces storage growth, background operations, and eventual consistency. Use it when reliable side effects justify those costs, rather than adding an outbox to a read-only operation or an effect whose loss is explicitly acceptable.

Sources

AWS transactional outbox guidance provides the primary pattern reference.

Michael Nygard's architecture decision guidance is useful for recording the atomic boundary, accepted delivery semantics, and recovery costs.