Back to all scenarios
Scenario #328
Storage
Kubernetes v1.22, Local PVs
Pod Eviction Due to DiskPressure Despite PVC
Node evicted pods due to DiskPressure, even though app used dedicated PVC backed by a separate disk.
Find this helpful?
What Happened
Node root disk filled up with log data, triggering eviction manager. The PVC itself was healthy and not full.
Diagnosis Steps
- 1Node describe: showed DiskPressure condition true.
- 2Application pod evicted due to node pressure, not volume pressure.
- 3Root disk had full /var/log.
Root Cause
Kubelet doesn’t distinguish between root disk and attached volumes for eviction triggers.
Fix/Workaround
• Cleaned logs from root disk.
• Moved logging to PVC-backed location.
Lessons Learned
PVCs don’t protect from node-level disk pressure.
How to Avoid
- 1Monitor node root disks in addition to volume usage.
- 2Redirect logs and temp files to PVCs.