Back to all scenarios
Scenario #179
Networking
K8s v1.22, Azure AKS
Service Discovery Failures Due to Network Policy Blocking DNS Traffic
Service discovery failed when a network policy was mistakenly applied to block DNS traffic, preventing pods from resolving services within the cluster.
Find this helpful?
What Happened
A network policy was applied to restrict traffic between namespaces but unintentionally blocked DNS traffic on UDP port 53, causing service discovery to fail.
Diagnosis Steps
- 1Ran kubectl get networkpolicy and found an ingress rule that blocked UDP traffic.
- 2Used kubectl exec to test DNS resolution inside the affected pods, which confirmed that DNS queries were being blocked.
Root Cause
The network policy unintentionally blocked DNS traffic due to a misconfigured ingress rule.
Fix/Workaround
• Updated the network policy to allow DNS traffic on UDP port 53.
• Restarted the affected pods to restore service discovery functionality.Lessons Learned
Always carefully test network policies to ensure they don't inadvertently block critical traffic like DNS.
How to Avoid
- 1Review and test network policies thoroughly before applying them in production.
- 2Implement automated tests to verify that critical services like DNS are not affected by policy changes.