Back to all scenarios
Scenario #115
Networking
K8s v1.22, Google GKE

Service Load Balancer Failing to Route Traffic to New Pods

Service load balancer failed to route traffic to new pods after scaling up.

Find this helpful?
What Happened

After scaling up the application pods, the load balancer continued to route traffic to old, terminated pods.

Diagnosis Steps
  • 1Verified pod readiness using kubectl get pods and found that new pods were marked as ready.
  • 2Inspected the load balancer configuration and found it was not properly refreshing its backend pool.
Root Cause

The service’s load balancer backend pool wasn’t updated when the new pods were created.

Fix/Workaround
• Manually refreshed the load balancer’s backend pool configuration.
• Monitored the traffic routing to ensure that it was properly balanced across all pods.
Lessons Learned

Load balancer backends need to be automatically updated with new pods.

How to Avoid
  • 1Configure the load balancer to auto-refresh backend pools on pod changes.
  • 2Use health checks to ensure only healthy pods are routed traffic.