Nameserver limits exceeded with more than three DNS servers in /etc/resolv.conf

[alan-wint]

Summary

I have more than 3 DNS servers configured in resolv.conf and that is a hard requirement, also not unusual afaik. My syslog is full of "Nameserver limits exceeded" messages.

What Should Happen Instead?

There should be no problem with more than three DNS servers.

Reproduction Steps

1. Add more than 3 DNS servers to /etc/resolv.conf or via /etc/systemd/resolved.conf
2. Start/restart microk8s

Introspection Report

Skipped as it contains private info that is not for a public forum like this.
Example log line: microk8s.daemon-kubelite[3713]: E0221 17:00:05.944148 3713 dns.go:156] "Nameserver limits exceeded" err="Nameserver limits were exce eded, some nameservers have been omitted, the applied nameserver line is: 7.7.8.9 143.212.212.212 8660:fc::fc"

Can you suggest a fix?

1. Ignore more than the first server
2. Perhaps point to a dedicated resolv.conf file as described here: https://simonfredsted.com/1680

Are you interested in contributing with a fix?

No

[ktsakalozos]

Hi @alan-wint. Thank you for bringing this up. I am sure you have seen that this is a known k8s issue [1]. Please allow us some time to see if/how we can address it in an elegant way.

[1] https://kubernetes.io/docs/tasks/administer-cluster/dns-debugging-resolution/#known-issues

[phpwutz]

thie becomes even more lovely now that systemd adds the ipv6 versions too, so having 2 DNS servers results in 4 entries triggering this warning :/

my current workaround is to have a /etc/resolv-static.conf and add --resolv-conf=/etc/resolv-static.conf to /var/snap/microk8s/current/args/kubelet but this is really not very satisfying

[rahul799]

Hi @ktsakalozos, do we have any workarounds here? It's kind of blocking our deployments. It will be great if you can help.

[RobT2012]

Late reply, I also have a similar issue and was hoping that the systemd version resolved the 3 limit issue. What I've had to do, is use dnsmasq to get around the problem. In many ways dnsmasq provides a better service and even if the 3 limit is resolved, I'll probably continue to use dnsmasq.

To fix the 3 limit with dnsmasq,

1. install dnsmasq

Now configure dnsmaq

2. Edit /etc/resolv.conf, it should ONLY have these 2 lines
   Note: if systemd-resolv is used then edit the conf file it uses if it is not symlinked as /etc/resolv.conf

nameserver ::1
nameserver 127.0.0.1

3. touch /etc/resolv-dnsmasq.conf
   Note: this file can go anywhere and can be named whatever you want

Add the following lines replace with IP addrs to your local DNS's:

nameserver 2001:xxx:xxx:xxx:xxx:xxx
nameserver 192.168.xxx.xxx
(repeat for all your local DNS's - the limit goes past only 3)

4. touch /etc/dnsmasq.d/custom-settings.conf
   Note: this file can be named whatever you want

Add these lines, and adjust according to your preferences

point to the file created in step 3 and use local interface

resolv-file=/etc/resolv-dnsmasq.conf
interface=lo
bind-interfaces
all-servers
cache-size=0
no-negcache

5. systemctl restart dnsmasq

Done!

Note: I also use dnsmasq for my local DNS servers that speak to external authoritative servers such as 8.8.8.8, 8.8.4.4 etc. The configuration is similar, but of course more complicated if you have local services such as DHCP and internal names resolution, etc, but dnsmasq will do the job very well and I've not encountered any frustrating limits.

[adrian-moisa]

Rolling back to an older version helped me. Full explanation here: Github: kube-proxy pods continuously CrashLoopBackOff #118461
sudo apt-get install -y kubelet=1.23.17-00 kubeadm=1.23.17-00 kubectl=1.23.17-00 on Ubuntu LTS 22.04

[bytemarcz]

I tried the dnsmasq solution but no luck. Seems to have made some parts of this work or at least got me past the limits issue but the Calico and Core DNS pods are still showing pod sandbox changed, it will be killed and re-created.

I tried killing the pods and restarting but appears to get me no where. Trying to see if there's any suggestions here.

[splitice]

coredns detects dnsmasq as a loop therefore aborts. It might be possible to run dnsmasq on 127.0.0.2 to avoid this but I havent tested it

[Wentil]

This issue is still outstanding since the start of 2023.

[Mafooq09]

Same issue. Any fix would be really helpful.

[kunthar]

the great saga continues:
http://jpetazzo.github.io/2024/05/12/understanding-kubernetes-dns-hostnetwork-dnspolicy-dnsconfigforming/