Prometheus metrics
infra/monitoring/prometheus.yml scrapes:
| Job | Target |
|---|---|
kong | :9542/metrics |
spring-apps | Each service /actuator/prometheus (auth 8081 through messaging 8088, relay on its container port) |
node | Host |
cadvisor | Containers |
postgres | Database |
redis | Redis |
pgbouncer | Pooler |
Use-case metrics come from UseCaseObservationAspect. HTTP histograms are enabled with SPLICK_METRICS_HISTOGRAMS. Rust services expose Prometheus from splick-shared so a cutover does not drop the RED dashboard.
Rules file: infra/monitoring/prometheus/rules/splick-alerts.yml. Groups: api-health, jvm, database, host-resources, containers, notifications, async-pipeline, dependency-slo, data-plane, host-saturation, observability-coverage.
Names you will see in incidents: HighErrorRate5xx, HighErrorRate4xx, HighP95Latency, HighP99Latency, ServiceDown, JvmHeapPressure, HikariPoolExhausted, PushDeadLetterRateHigh, OutboxLagHigh, OutboxLagCritical, OutboxFailedBacklog, InboxBacklogHigh, StreamQueueWaitHigh, HttpClientP95High, JdbcP95High, LettuceP95High, PostgresConnectionsHigh, PgBouncerClientsWaiting, RedisMemoryHigh, RedisEvictions, UseCaseErrorRate, ScheduledJobFailing, KongHigh5xxRate.
Alertmanager config is infra/monitoring/alertmanager/alertmanager.yml. Wire a receiver there. Do not page on a single 4xx.