Back to all scenarios
Scenario #307
Storage
Kubernetes v1.23, NFS-backed PVCs
ReadWriteMany PVCs Cause IO Bottlenecks
Heavy read/write on a shared PVC caused file IO contention and throttling across pods.
Find this helpful?
What Happened
Multiple pods used a shared ReadWriteMany PVC for scratch space. Concurrent writes led to massive IO wait times and high pod latency.
Diagnosis Steps
- 1High pod latency and CPU idle time.
- 2Checked NFS server: high disk and network usage.
- 3Application logs showed timeouts.
Root Cause
No coordination or locking on shared writable volume.
Fix/Workaround
• Partitioned workloads to use isolated volumes.
• Added cache layer for reads.Lessons Learned
RWX volumes are not always suitable for concurrent writes.
How to Avoid
- 1Use RWX volumes for read-shared data only.
- 2Avoid writes unless using clustered filesystems (e.g., CephFS).