Transactional Outbox Pattern

A signup handler does two things: commit a User row to Postgres, and publish UserRegistered to a broker. Each is reliable on its own — Postgres commits durably, the broker acknowledges durably — but nothing ties them into one atomic operation. Four orderings are possible: commit then publish (the happy path), publish then commit, commit succeeds and publish fails, or publish succeeds and commit fails. Only the first is safe if you publish after commit; the other three all produce an event with no matching row, or a row with no event.

The common instinct — "publish inside the same code path right after commit, wrapped in a try/catch" — does not remove the gap. The process can still crash, the network can still drop the publish call, or the broker can be unreachable, all after the commit already succeeded. The row exists; the event is gone. Nothing will ever re-send it, because nothing recorded that it needed to be sent.

The fix moves the event into the same transaction as the business write. The signup handler inserts the User row and an outbox_events row — { id, type: "UserRegistered", payload, created_at } — in one local database transaction. Both commit together or neither does; there is no window where one exists without the other, because they were never two separate commits to begin with.

A separate relay process is responsible for actually reaching the broker. It either polls the outbox table for unpublished rows (WHERE published_at IS NULL ORDER BY created_at) and publishes each one, or it tails the database's write-ahead log directly (Debezium reading Postgres logical replication, for example) and reacts to outbox inserts as they're written — no polling delay, but a CDC connector to operate. Either way, the relay marks a row published only after the broker confirms receipt.

The outbox pattern removes the dual-write gap, but it does not make delivery exactly-once. The relay can publish a row successfully, then crash before it runs the UPDATE ... SET published_at = now() that marks the row sent. On restart, the relay sees an unpublished row and publishes it again — the broker now has two UserRegistered messages for the same signup.

This is a deliberate trade: the outbox guarantees the event is never lost, at the cost of sometimes delivering it more than once. Every consumer of an outbox-relayed event must be idempotent — checking a processed-event table keyed by the event's id before acting, so a redelivered UserRegistered creates one welcome email and one loyalty account, not two.

Trigger. The relay publishes UserRegistered for user-771 to the broker, which acknowledges receipt. Before the relay's UPDATE outbox_events SET published_at = now() commits, the relay process is killed. Symptom. On restart, the relay's next poll finds user-771's outbox row still has published_at = NULL, publishes it again, and the broker now holds two messages for the same signup.

Root mechanism. Publish-then-mark-sent is itself two steps with no atomicity between them — the exact shape of problem the outbox pattern solved for the original business write, recurring one layer down. Recovery: the welcome-email consumer's idempotency check finds user-771's event id already in its processed-events table from the first delivery, and skips sending a second email; the loyalty-account consumer does the same. Verification: query the welcome-email service for user-771 and confirm exactly one sent email, despite two broker messages. Prevention: alert if any outbox row stays unpublished past its expected relay interval — that is the outbox pattern's equivalent of a stuck saga, and it means the relay itself is down, not that an event was lost.

An outbox table grows forever if nothing removes old rows. Delete or archive rows once they are published and past a retention window generous enough to survive a relay outage — deleting immediately after publish removes your only record if the relay's mark-sent step itself needs to be replayed. A periodic job that archives rows older than a few days keeps the table small without deleting evidence you might still need.

The outbox is not the only way to get this right. CDC without an outbox table tails the domain table's own change log directly (no outbox_events table at all) — it removes one table but couples the relay to your domain schema instead of a stable event shape. Two-phase commit could in theory make the database and broker commit together, but almost no message broker participates in a 2PC protocol, and even where XA support exists, it reintroduces the coordinator-lock blocking problem from Saga vs Two-Phase Commit — for a cross-network publish, that lock can be held far longer than a local database transaction.