Back to all scenarios
Scenario #32
Cluster Management
K8s v1.24, multi-zone GKE
Inconsistent Node Labels Causing Scheduling Bugs
Zone-aware workloads failed to schedule due to missing zone labels on some nodes.
Find this helpful?
What Happened
Pods using topologySpreadConstraints for zone balancing failed to find valid nodes because some nodes lacked the topology.kubernetes.io/zone label.
Diagnosis Steps
- 1Pod events showed no matching topology key errors.
- 2Compared node labels across zones – found inconsistency.
Root Cause
A few nodes were manually added without required zone labels.
Fix/Workaround
• Manually patched node labels to restore zone metadata.Lessons Learned
Label uniformity is essential for topology constraints.
How to Avoid
- 1Automate label injection using cloud-init or DaemonSet.
- 2Add CI checks for required labels on node join.