PVC Resize Broke StatefulSet Ordering

When resizing PVCs, StatefulSet pods restarted in parallel, violating ordinal guarantees.

Find this helpful?

What Happened

PVC expansion triggered pod restarts, but multiple pods came up simultaneously, causing database quorum failures.

Diagnosis Steps

1Checked StatefulSet controller behavior — PVC resize didn’t preserve pod startup order.
2App logs: quorum could not be established.

Root Cause

StatefulSet controller didn’t serialize PVC resizes.

Fix/Workaround

• Manually controlled pod restarts during PVC resize.
• Added readiness gates to enforce sequential boot.

Lessons Learned

StatefulSets don't coordinate PVC changes well.

How to Avoid