Skip to main content

Outbox relay internals

OutboxRelayEngine in shared-outbox is the algorithm. outbox-relay-service is the process that runs it for every producer schema.

Per source (auth, social, feed, messaging, expense):

  1. Adaptive sleep between min and max delay. Reset to min when a batch was non-empty. OUTBOX_RELAY_DRAIN_WHEN_BUSY can loop until empty.
  2. Claim a batch in one statement: update locked_at where pending and FOR UPDATE SKIP LOCKED, returning the rows.
  3. Publish each envelope to Redis outside that transaction.
  4. Mark published, or set next_retry_at with exponential backoff. At OUTBOX_RELAY_MAX_ATTEMPTS, set failed_at.
  5. A separate sweep clears locks older than OUTBOX_RELAY_LOCK_TIMEOUT_MINUTES so a crashed worker does not pin rows.
  6. On SIGTERM, SmartLifecycle stops claiming and waits OUTBOX_RELAY_SHUTDOWN_GRACE_PERIOD_MS.

Worker counts are higher for messaging (3) and feed (2) because those streams are hotter. Health is degraded when any schema's lag is over 30 seconds or any dead-letter row exists. OutboxArchivalJob moves old published rows so the pending index stays small.

Relay downtime stalls every integration side effect and does not stall the user write that already committed. Run at least two instances when you leave a single VM. Do not run a legacy in-process relay at the same time: both would publish.

Admin replay:

  • GET /admin/outbox/dead-letters/
  • POST /admin/outbox/dead-letters/replay
  • POST /admin/outbox/dead-letters/replay/bulk

Replay is safe only because consumers dedupe on event_id. Fix the consumer before you replay a poison payload.