Back to all scenarios
Scenario #17
Cluster Management
K8s v1.24, Velero-based etcd backup
Inconsistent Cluster State After Partial Backup Restore
A partial etcd restore led to stale object references and broken dependencies.
Find this helpful?
What Happened
etcd snapshot was restored, but PVCs and secrets weren’t included. Many pods failed to mount or pull secrets.
Diagnosis Steps
- 1Pods failed with “volume not found” and “secret missing”.
- 2kubectl get pvc --all-namespaces returned empty.
- 3Compared resource counts pre- and post-restore.
Root Cause
Restore did not include volume snapshots or Kubernetes secrets, leading to an incomplete object graph.
Fix/Workaround
• Manually recreated PVCs and secrets using backups from another tool.
• Redeployed apps.Lessons Learned
etcd backup is not enough alone.
How to Avoid
- 1Use backup tools that support volume + etcd (e.g., Velero with restic).
- 2Periodically test full cluster restores.