Transactional emails are being delivered with a median delay of 11 minutes. No messages lost; the queue is draining steadily.
5 entries · newest last
Alert fired: notification-worker queue depth 8420 > 5000
Acknowledged. Queue depth trending up; SendGrid response times normal.
Scaled worker replicas 8 -> 16 and raised concurrency to 24.
Queue depth peaked at 9,120 at 14:02 UTC, now declining.
Backlog down to 1,284. Monitoring until fully drained.
Visible to every responder on the incident channel
Customer and budget impact
Transactional emails are being delivered with a median delay of 11 minutes. No messages lost; the queue is draining steadily.
Users affected
0
Error budget burn
26.9%
1 engaged
A marketing campaign produced a 6x spike in notification volume that exceeded the worker concurrency limit of 12.
Incident is still open.
1 events
Queue depth above 5000
Near the incident window
No deployments nearby