Back to all scenarios
Scenario #420
Scaling & Load
Kubernetes v1.26, RabbitMQ + consumers

Scale-In Happened Before Queue Was Drained

Consumers scaled in while queue still had unprocessed messages.

Find this helpful?
What Happened

Queue depth remained, but pods were terminated.

Diagnosis Steps
  • 1Observed message backlog after autoscaler scale-in.
  • 2Consumers had no shutdown hook to drain queue.
Root Cause

Scale-in triggered without consumer workload cleanup.

Fix/Workaround
• Added preStop hook to finish queue processing.
Lessons Learned

Consumers must handle shutdown gracefully.

How to Avoid
  • 1Track message queues with KEDA or custom metrics.
  • 2Add drain() logic on signal trap in consumer code.
Previous ScenarioNext Scenario