Back to all scenarios
Scenario #152
Networking
K8s v1.19, AWS EKS
DNS Latency Due to Overloaded CoreDNS Pods
CoreDNS latency increased due to resource constraints on the CoreDNS pods.
Find this helpful?
What Happened
CoreDNS started experiencing high response times due to CPU and memory resource constraints, leading to DNS resolution delays.
Diagnosis Steps
- 1Checked CoreDNS pod resource usage and found high CPU usage.
- 2Verified that DNS resolution was slowing down for multiple namespaces and services.
- 3Increased logging verbosity for CoreDNS and identified high query volume.
Root Cause
CoreDNS pods did not have sufficient resources allocated to handle the query load.
Fix/Workaround
• Increased CPU and memory resource limits for CoreDNS pods.
• Restarted CoreDNS pods to apply the new resource limits.
Lessons Learned
CoreDNS should be allocated appropriate resources based on expected load, especially in large clusters.
How to Avoid
- 1Set resource requests and limits for CoreDNS based on historical query volume.
- 2Monitor CoreDNS performance and scale resources dynamically.