Back to all scenarios
Scenario #196
Networking
K8s v1.22, AWS EKS
High CPU Usage in Nodes Due to Overloaded Network Plugin
Nodes experienced high CPU usage due to an overloaded network plugin that couldn’t handle traffic spikes effectively.
Find this helpful?
What Happened
The network plugin was designed to handle a certain volume of traffic, but when the pod-to-pod communication increased, the plugin was unable to scale efficiently, leading to high CPU consumption.
Diagnosis Steps
- 1Monitored node metrics with kubectl top nodes and noticed unusually high CPU usage on affected nodes.
- 2Checked logs for the network plugin and found evidence of resource exhaustion under high traffic conditions.
Root Cause
The network plugin was not adequately resourced to handle high traffic spikes, leading to resource exhaustion.
Fix/Workaround
• Increased resource allocation (CPU/memory) for the network plugin.
• Configured scaling policies for the network plugin to dynamically adjust resources.Lessons Learned
Network plugins need to be able to scale in response to increased traffic to prevent performance degradation.
How to Avoid
- 1Regularly monitor network plugin performance and resources.
- 2Configure auto-scaling and adjust resource allocation based on traffic patterns.