Skip to main content

On-call checklist

First five minutes​

  1. Read the alert name and the service label. Open Alerting.
  2. Open Service Health and Async Pipeline.
  3. If there is a user report, take traceId from the error body.
  4. Decide: user-facing (5xx, login, send message, create expense) or delayed side effect (push, email, feed projection). Side effects usually mean the relay or a consumer, not a rollback of the write.

While it is happening​

  • One person changes production. Say what you changed.
  • Restart a single service before you restart the host.
  • Do not truncate event_outbox, streams, or inbox tables.
  • Do not flip Pub/Sub toggles unless you are executing the ADR 0003 rollback on purpose.
  • Capture the image tag that is running before you roll back.

After​

  • Note the start, the mitigation, and the follow-up.
  • If the fix is a code change, add or adjust the runbook section that would have saved the next person ten minutes.
  • If an alert did not fire, add one. If an alert fired and nothing was wrong, narrow it.

Escalation is a human list kept by the team, not a name baked into git.