Reachability of DNS root servers for zone transfers

Reachability of DNS root servers for zone transfers

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 2916 bytes --]

Hello *,

I am currently working on making Unbound mirror essential DNS root
zones such as ".", "arpa.", "in-addr.arpa." and "ip6.arpa.". Since
Unbound provides the "auth-zone" flag for some time now, I did not
think of this as being too difficult.

However, things use to become difficult while having a closer look
at them. :-\

Having IPFire installations with no direct access to the DNS root
servers turned out to be the biggest issue here. (Besides, not
all root servers can be used, as some of them do not allow zone
transfers via AXFR.) I am currently thinking of this procedure:

(a) If DNS servers are set an known to work, they are used to
fetch mentioned DNS root zones. In case of failures, Unbound
falls back to current behaviour. As DNS resolvers usually do not
allow zone transfers, I expect this to fail in most cases.

(b) In case no DNSSEC-validating or -aware resolvers are available,
Unbound falls back into recursor mode, assuming reachability of
at least one of these servers. In this case, fetching the zones
is easy.

(c) In case of permissive operation (no DNSSEC available), root
zones are not fetched.

It turned out Unbound bumps into validation errors sometime, which
needs some further investigation.

Can/should we always assume DNS root servers are reachable?
Any opinions on this?

Thanks, and best regards,
Peter Müller

P.S.: In case anybody wants to test, here is the corresponding
Unbound configuration snippet (adjust paths if needed):

> # mirror DNS root zones
> auth-zone:
> 	name: "."
> 	for-downstream: no
> 	for-upstream: yes
> 	fallback-enabled: yes
> 	master: b.root-servers.net
> 	master: c.root-servers.net
> 	master: e.root-servers.net
> 	master: f.root-servers.net
> 	master: g.root-servers.net
> 	master: k.root-servers.net
> 	zonefile: "auth-zones/root"
> 
> auth-zone:
> 	name: "arpa."
> 	for-downstream: no
> 	for-upstream: yes
> 	fallback-enabled: yes
> 	master: b.root-servers.net
> 	master: c.root-servers.net
> 	master: e.root-servers.net
> 	master: f.root-servers.net
> 	master: g.root-servers.net
> 	master: k.root-servers.net
> 	master: 80.67.169.40
> 	master: 188.213.49.35
> 	zonefile: "auth-zones/arpa"
> 
> auth-zone:
> 	name: "in-addr.arpa."
> 	for-downstream: no
> 	for-upstream: yes
> 	fallback-enabled: yes
> 	master: f.in-addr-servers.arpa.
> 	zonefile: "auth-zones/in-addr.arpa"
> 
> auth-zone:
> 	name: "ip6.arpa."
> 	for-downstream: no
> 	for-upstream: yes
> 	fallback-enabled: yes
> 	master: f.ip6-servers.arpa.
> 	zonefile: "auth-zones/ip6.arpa"

P.P.S.: See https://unbound.nlnetlabs.nl/pipermail/unbound-users/2018-May/005268.html
for upstream mailinglist thread.
-- 
Microsoft DNS terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

Re: Reachability of DNS root servers for zone transfers

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 3374 bytes --]

Hi,

On Sun, 2018-10-28 at 19:43 +0100, Peter Müller wrote:
> Hello *,
> 
> I am currently working on making Unbound mirror essential DNS root
> zones such as ".", "arpa.", "in-addr.arpa." and "ip6.arpa.". Since
> Unbound provides the "auth-zone" flag for some time now, I did not
> think of this as being too difficult.
> 
> However, things use to become difficult while having a closer look
> at them. :-\
> 
> Having IPFire installations with no direct access to the DNS root
> servers turned out to be the biggest issue here. (Besides, not
> all root servers can be used, as some of them do not allow zone
> transfers via AXFR.) I am currently thinking of this procedure:
> 
> (a) If DNS servers are set an known to work, they are used to
> fetch mentioned DNS root zones. In case of failures, Unbound
> falls back to current behaviour. As DNS resolvers usually do not
> allow zone transfers, I expect this to fail in most cases.

The fallback is essential. This cannot render DNS unusable.

> (b) In case no DNSSEC-validating or -aware resolvers are available,
> Unbound falls back into recursor mode, assuming reachability of
> at least one of these servers. In this case, fetching the zones
> is easy.

In hindsight, this was a bad design decision. We assumed here that this will
always work and that is not true. However, the amount of users is still
relatively small.

> (c) In case of permissive operation (no DNSSEC available), root
> zones are not fetched.

Why?

> It turned out Unbound bumps into validation errors sometime, which
> needs some further investigation.
> 
> Can/should we always assume DNS root servers are reachable?
> Any opinions on this?

Not always, but for the vast majority of users, they should be available.

If not, what are the downsides? Also what are the upsides of this?

> Thanks, and best regards,
> Peter Müller
> 
> P.S.: In case anybody wants to test, here is the corresponding
> Unbound configuration snippet (adjust paths if needed):
> 
> > # mirror DNS root zones
> > auth-zone:
> > 	name: "."
> > 	for-downstream: no
> > 	for-upstream: yes
> > 	fallback-enabled: yes
> > 	master: b.root-servers.net
> > 	master: c.root-servers.net
> > 	master: e.root-servers.net
> > 	master: f.root-servers.net
> > 	master: g.root-servers.net
> > 	master: k.root-servers.net
> > 	zonefile: "auth-zones/root"
> > 
> > auth-zone:
> > 	name: "arpa."
> > 	for-downstream: no
> > 	for-upstream: yes
> > 	fallback-enabled: yes
> > 	master: b.root-servers.net
> > 	master: c.root-servers.net
> > 	master: e.root-servers.net
> > 	master: f.root-servers.net
> > 	master: g.root-servers.net
> > 	master: k.root-servers.net
> > 	master: 80.67.169.40
> > 	master: 188.213.49.35
> > 	zonefile: "auth-zones/arpa"
> > 
> > auth-zone:
> > 	name: "in-addr.arpa."
> > 	for-downstream: no
> > 	for-upstream: yes
> > 	fallback-enabled: yes
> > 	master: f.in-addr-servers.arpa.
> > 	zonefile: "auth-zones/in-addr.arpa"
> > 
> > auth-zone:
> > 	name: "ip6.arpa."
> > 	for-downstream: no
> > 	for-upstream: yes
> > 	fallback-enabled: yes
> > 	master: f.ip6-servers.arpa.
> > 	zonefile: "auth-zones/ip6.arpa"
> 
> P.P.S.: See 
> https://unbound.nlnetlabs.nl/pipermail/unbound-users/2018-May/005268.html
> for upstream mailinglist thread.

Just for the fun of it, I have added all zones to
ns{1,2,3}.lightningwirelabs.com and allow AXFR for everyone.

-Michael

Re: Reachability of DNS root servers for zone transfers

[Peter Müller]

[-- Attachment #1: Type: text/plain, Size: 3644 bytes --]

Hello Michael,

> Hi,
> [...]
>>
>> (a) If DNS servers are set an known to work, they are used to
>> fetch mentioned DNS root zones. In case of failures, Unbound
>> falls back to current behaviour. As DNS resolvers usually do not
>> allow zone transfers, I expect this to fail in most cases.
> 
> The fallback is essential. This cannot render DNS unusable.
True. However, Unbound falls back to simply querying the upstream
DNS servers if zone transfer failed. I do not expect any setup to
break if AXFR is not allowed.
> 
>> (b) In case no DNSSEC-validating or -aware resolvers are available,
>> Unbound falls back into recursor mode, assuming reachability of
>> at least one of these servers. In this case, fetching the zones
>> is easy.
> 
> In hindsight, this was a bad design decision. We assumed here that this will
> always work and that is not true. However, the amount of users is still
> relatively small.
True, but I do not see any usable alternative. We could simply stop
resolving DNS entries, but this is probably even worse.
> 
>> (c) In case of permissive operation (no DNSSEC available), root
>> zones are not fetched.
> 
> Why?
Because we cannot validate them. I consider querying upstream name
servers better than having a ton of probably faked DNS zone data
around.
> 
>> It turned out Unbound bumps into validation errors sometime, which
>> needs some further investigation.
Things work fine so far, except for a really nasty bug: After rebooting
a machine with local root zone mirroring enabled, Unbound falls back
to DNSSEC permissive mode for an unknown reason.

After restarting the daemon manually, everything is fine again. :-\
>>
>> Can/should we always assume DNS root servers are reachable?
>> Any opinions on this?
> 
> Not always, but for the vast majority of users, they should be available.
I suspect this depends on
(a) how many users are located behind some moron ISP which breaks DNSSEC.
(b) how many users restrict outgoing DNS traffic to the two nameservers
they configured (I do so) - since information leaks and bypassing DNSSEC
validation is a threat, I consider this restriction as being useful.

Further, creating firewall groups with _all_ of the root servers IPv4
addresses is error-prone and time-consuming.
(c) how many users are located behind upstream DNS servers which do not
allow AXFR.
> 
> If not, what are the downsides? Also what are the upsides of this?
As far as I am concerned, there is no downside: In case mirroring the
root zones fails, anything (should) just behave as usual.

The upsides of this feature are as follows:
(a) less load on the DNS root servers
(b) faster replies to DNS queries for an invalid TLD (some poorly written
software is doing so)
(c) no disclosure to upstream servers which TLD is queried (paranoia, but hey)
(d) less queries to upstream servers for TLD nameservers

Besides some very minor privacy benefit, this aims to reduce load
and DNS queries. Needless to say, if a TLDs nameserver is already cached
by an upstream nameserver, no information is disclosed to the root servers.
> 
>> [...]
>> P.P.S.: See 
>> https://unbound.nlnetlabs.nl/pipermail/unbound-users/2018-May/005268.html
>> for upstream mailinglist thread.
> 
> Just for the fun of it, I have added all zones to
> ns{1,2,3}.lightningwirelabs.com and allow AXFR for everyone.
Cool, thank you. Let's hope some more resolvers will implement this.
> 
> -Michael
> 

Thanks, and best regards,
Peter Müller
-- 
Microsoft DNS service terminates abnormally when it recieves a response
to a DNS query that was never made.  Fix Information: Run your DNS
service on a different platform.
		-- bugtraq

Re: Reachability of DNS root servers for zone transfers

[Michael Tremer]

[-- Attachment #1: Type: text/plain, Size: 4641 bytes --]

Hey,

On Tue, 2018-10-30 at 15:47 +0100, Peter Müller wrote:
> Hello Michael,
> 
> > Hi,
> > [...]
> > > 
> > > (a) If DNS servers are set an known to work, they are used to
> > > fetch mentioned DNS root zones. In case of failures, Unbound
> > > falls back to current behaviour. As DNS resolvers usually do not
> > > allow zone transfers, I expect this to fail in most cases.
> > 
> > The fallback is essential. This cannot render DNS unusable.
> 
> True. However, Unbound falls back to simply querying the upstream
> DNS servers if zone transfer failed. I do not expect any setup to
> break if AXFR is not allowed.
> > 
> > > (b) In case no DNSSEC-validating or -aware resolvers are available,
> > > Unbound falls back into recursor mode, assuming reachability of
> > > at least one of these servers. In this case, fetching the zones
> > > is easy.
> > 
> > In hindsight, this was a bad design decision. We assumed here that this will
> > always work and that is not true. However, the amount of users is still
> > relatively small.
> 
> True, but I do not see any usable alternative. We could simply stop
> resolving DNS entries, but this is probably even worse.
> > 
> > > (c) In case of permissive operation (no DNSSEC available), root
> > > zones are not fetched.
> > 
> > Why?
> 
> Because we cannot validate them. I consider querying upstream name
> servers better than having a ton of probably faked DNS zone data
> around.

We cannot do that either when we request only one TLD after the other.

> > > It turned out Unbound bumps into validation errors sometime, which
> > > needs some further investigation.
> 
> Things work fine so far, except for a really nasty bug: After rebooting
> a machine with local root zone mirroring enabled, Unbound falls back
> to DNSSEC permissive mode for an unknown reason.
> 
> After restarting the daemon manually, everything is fine again. :-\
> > > 
> > > Can/should we always assume DNS root servers are reachable?
> > > Any opinions on this?
> > 
> > Not always, but for the vast majority of users, they should be available.
> 
> I suspect this depends on
> (a) how many users are located behind some moron ISP which breaks DNSSEC.

Not too many.

> (b) how many users restrict outgoing DNS traffic to the two nameservers
> they configured (I do so) - since information leaks and bypassing DNSSEC
> validation is a threat, I consider this restriction as being useful.

Usually people open port 53 to everything. It is considered to be a low-risk
protocol.

> Further, creating firewall groups with _all_ of the root servers IPv4
> addresses is error-prone and time-consuming.

I have never seen anyone doing that.

> (c) how many users are located behind upstream DNS servers which do not
> allow AXFR.

I could not find a single recursor that allows AXFR. There is no point in doing
that.

> > 
> > If not, what are the downsides? Also what are the upsides of this?
> 
> As far as I am concerned, there is no downside: In case mirroring the
> root zones fails, anything (should) just behave as usual.
> 
> The upsides of this feature are as follows:
> (a) less load on the DNS root servers

Not that that makes a huge difference...

> (b) faster replies to DNS queries for an invalid TLD (some poorly written
> software is doing so)
> (c) no disclosure to upstream servers which TLD is queried (paranoia, but hey)
> (d) less queries to upstream servers for TLD nameservers

All these points make me think of people who use a funny domain for their local
network that they should not be using.

So would you extend the patch and add comments on why there are only a couple of
root servers on there, etc.? Would you also draft a blog post that highlights
the benefits of this?

Is there any contingency option we should offer?

> Besides some very minor privacy benefit, this aims to reduce load
> and DNS queries. Needless to say, if a TLDs nameserver is already cached
> by an upstream nameserver, no information is disclosed to the root servers.
> > 
> > > [...]
> > > P.P.S.: See 
> > > https://unbound.nlnetlabs.nl/pipermail/unbound-users/2018-May/005268.html
> > > for upstream mailinglist thread.
> > 
> > Just for the fun of it, I have added all zones to
> > ns{1,2,3}.lightningwirelabs.com and allow AXFR for everyone.
> 
> Cool, thank you. Let's hope some more resolvers will implement this.

These are not the resolvers. Those are my authoritative name servers.

Best,
-Michael

> > 
> > -Michael
> > 
> 
> Thanks, and best regards,
> Peter Müller