Inconsistent Service Discovery Due to CoreDNS Misconfiguration

Service discovery was inconsistent due to misconfigured CoreDNS settings.

Find this helpful?

What Happened

The CoreDNS configuration was updated to use an external resolver, but the external resolver had intermittent issues, leading to service discovery failures.

Diagnosis Steps

1Checked CoreDNS logs with kubectl logs -n kube-system <coredns-pod> and noticed errors with the external resolver.
2Used kubectl get svc to check service names and found that some services could not be resolved reliably.

Root Cause

Misconfigured external DNS resolver in CoreDNS caused service discovery failures.

Fix/Workaround

• Reverted CoreDNS configuration to use the internal DNS resolver instead of the external one.
• Restarted CoreDNS pods to apply the changes.

Lessons Learned

External DNS resolvers can introduce reliability issues ; test these changes carefully.

How to Avoid

1Use internal DNS resolvers for core service discovery within the cluster.
2Implement monitoring for DNS resolution health.

Previous Scenario Next Scenario