Back to all scenarios
Scenario #115
Networking
K8s v1.22, Google GKE
Service Load Balancer Failing to Route Traffic to New Pods
Service load balancer failed to route traffic to new pods after scaling up.
Find this helpful?
What Happened
After scaling up the application pods, the load balancer continued to route traffic to old, terminated pods.
Diagnosis Steps
- 1Verified pod readiness using kubectl get pods and found that new pods were marked as ready.
- 2Inspected the load balancer configuration and found it was not properly refreshing its backend pool.
Root Cause
The service’s load balancer backend pool wasn’t updated when the new pods were created.
Fix/Workaround
• Manually refreshed the load balancer’s backend pool configuration.
• Monitored the traffic routing to ensure that it was properly balanced across all pods.
Lessons Learned
Load balancer backends need to be automatically updated with new pods.
How to Avoid
- 1Configure the load balancer to auto-refresh backend pools on pod changes.
- 2Use health checks to ensure only healthy pods are routed traffic.