Back to all scenarios
Scenario #149
Networking
K8s v1.21, AWS EKS
Intermittent Pod Network Connectivity Due to Cloud Provider Issues
Pod network connectivity was intermittent due to issues with the cloud provider's network infrastructure.
Find this helpful?
What Happened
Pods experienced intermittent network connectivity, and communication between nodes was unreliable.
Diagnosis Steps
- 1Used AWS CloudWatch to monitor network metrics and identified sporadic outages in the cloud provider’s network infrastructure.
- 2Verified that the Kubernetes network infrastructure was working correctly.
Root Cause
Cloud provider network outages affecting pod-to-pod communication.
Fix/Workaround
• Waited for the cloud provider to resolve the network issue.
• Implemented automatic retries in application code to mitigate the impact of intermittent connectivity.Lessons Learned
Be prepared for cloud provider network outages and implement fallback mechanisms.
How to Avoid
- 1Set up alerts for cloud provider outages and implement retries in critical network-dependent applications.
- 2Design applications to be resilient to network instability.