Back to all scenarios
Scenario #5
Cluster Management
K8s v1.26, Multi-tenant cluster
Misconfigured Taints Blocking Pod Scheduling
Critical workloads weren’t getting scheduled due to incorrect node taints.
Find this helpful?
What Happened
A user added taints (NoSchedule) to all nodes to isolate their app, but forgot to include tolerations in workloads. Other apps stopped working.
Diagnosis Steps
- 1Pods stuck in Pending state.
- 2Used kubectl describe pod <pod> – reason: no nodes match tolerations.
- 3Inspected node taints via kubectl describe node.
Root Cause
Lack of required tolerations on most workloads.
Fix/Workaround
• Removed the inappropriate taints.
• Re-scheduled workloads.Lessons Learned
Node taints must be reviewed cluster-wide.
How to Avoid
- 1Educate teams on node taints and tolerations.
- 2Restrict RBAC for node mutation.