Back to all scenarios
Scenario #487
Scaling & Load
Kubernetes v1.25, AWS EKS

Scaling Causing Unbalanced Pods Across Availability Zones

Pods became unbalanced across availability zones during scaling, leading to higher latency for some traffic.

Find this helpful?
What Happened

During scaling, the pod scheduler did not evenly distribute pods across availability zones, leading to pod concentration in one zone and increased latency in others.

Diagnosis Steps
  • 1Reviewed pod placement logs and found that the scheduler was not balancing pods across zones as expected.
  • 2Traffic logs showed increased latency in one of the availability zones.
Root Cause

Misconfigured affinity rules leading to unbalanced pod distribution.

Fix/Workaround
• Reconfigured pod affinity rules to ensure an even distribution across availability zones.
• Implemented anti-affinity rules to avoid overloading specific zones.
Lessons Learned

Proper pod placement is crucial for high availability and low latency.

How to Avoid
  • 1Use affinity and anti-affinity rules to ensure even distribution across availability zones.
  • 2Regularly monitor pod distribution and adjust scheduling policies as needed.
Previous ScenarioNext Scenario