Back to all scenarios
Scenario #76
Cluster Management
K8s v1.22, AWS EKS

Kubernetes Master Node Unresponsive After High Load

The Kubernetes master node became unresponsive under high load due to excessive API server calls and high memory usage.

Find this helpful?
What Happened

The Kubernetes master node was overwhelmed by API calls and high memory consumption, leading to a failure to respond to management requests.

Diagnosis Steps
  • 1Checked the control plane resource usage and found high memory and CPU consumption on the master node.
  • 2Analyzed API server logs and found a spike in incoming requests.
Root Cause

Excessive incoming requests caused API server memory to spike, rendering the master node unresponsive.

Fix/Workaround
• Implemented API rate limiting to control excessive calls.
• Increased the memory allocated to the master node.
Lessons Learned

Ensure that the control plane is protected against overloads and is properly scaled.

How to Avoid
  • 1Use API rate limiting and load balancing techniques for the master node.
  • 2Consider separating the control plane and worker nodes for better scalability.
Previous ScenarioNext Scenario