Re: HELP! Resolving Problems

SW> I think it may be failing because queries for
SW> ns1.nameserver.ch to the ascio.com nameservers
SW> return no authority records [...]

The "nameserver.ch." content DNS servers don't add the "NS" resource record
set for "nameserver.ch." to all of their responses, true, but this is
relatively benign. It merely means that the "ch." content DNS servers will
have to be queried afresh every 12 hours.

There is certainly a failure here (albeit that it's possibly not the one that
David Meier is experiencing - he hasn't given us enough information to
determine this). However, it is not related to the "nameserver.ch." content
DNS servers.

SW> klaeui.com has a complex delegation which might also
SW> be causing problems

Actually, what is happening with "klaeui.com." involves just two queries, one
to the root servers and one to the UltraDNS servers. The complex delegation
doesn't actually enter into it at all.

The failure here (with both "klaeui.com." and "pwr.ag.") is a combination of
an over-optimisation in some versions of Sendmail, the fact that BIND passes
through results with the AA bit set to 1 the first time around, and an
outright error in the way that the UltraDNS servers work. Its aetiology is as
follows:

1. To avoid a BIND version 4 bug in the resolution of "CNAME" queries, most
SMTP Client softwares perform an "*" query rather than a "CNAME" query when
canonicalising the domain part of an envelope recipient mailbox, filtering out
from the result those resource records of the type that it actually wanted.
Effectively, a "CNAME" lookup is disguised as a "*" lookup.

However, Sendmail in particular (and, indeed, only certain versions of
Sendmail) _also_ does this in place of issuing explicit "MX" and "A" queries
(even though the BIND version 4 bug does not affect "MX" or "A" queries, only
"CNAME" queries) if it thinks that the response to the first "*" query wasn't
cached along the way. It assumes that "any" means "all" in such
circumstances.

(Not all versions of Sendmail try to optimise their DNS query traffic by doing
this. Also, other MTS softwares, such as "qmail", do not do this. And it's
certainly debatable whether this is a reasonable optimisation to be doing in
the first place, given that best practice is for an SMTP Client to have a
local caching proxy DNS server anyway.)

2. The process of query resolution for an "*" query for "pwr.ag." stops at
the "ag." content DNS servers. In response to other types of queries, the
"ag." content DNS servers return a partial answer comprising a referral for
"pwr.ag.", as expected.

[204.74.112.1:0035] -> [0.0.0.0:0000] 73
Header: 0002 1+0+2+0, R, , query, no_error
Question: pwr.ag. IN A
Authority: pwr.ag. IN NS 86400 ns2.namecenter.ch.
Authority: pwr.ag. IN NS 86400 ns1.namecenter.ch.

However, in response to an "*" query they instead return a complete answer
(i.e. one that doesn't end in a referral) - but one where the relevant
resource record sets ("MX" and "A") are erroneously empty.

[204.74.112.1:0035] -> [0.0.0.0:0000] 123
Header: 0002 1+2+2+0, R, AUTH, query, no_error
Question: pwr.ag. IN *
Answer: pwr.ag. IN NS 86400 ns2.namecenter.ch.
Answer: pwr.ag. IN NS 86400 ns1.namecenter.ch.
Authority: ag. IN NS 86400 TLD2.ULTRADNS.NET.
Authority: ag. IN NS 86400 TLD1.ULTRADNS.NET.

Given that the "*" query will be the first one made, it will be unlikely that
any "pwr.ag." delegation information is already cached. So query resolution
will not reach the "pwr.ag." content DNS servers at all, and will instead stop
at the "ag." content DNS servers when they return that complete answer.

There's a strong argument that the "ag." content DNS servers (run by UltraDNS)
are wrong here. Certainly, a complete answer is not the response that would
be generated by following the algorithm in RFC 1034 section 4.3.2.

3. If this is the first time that the "*" query was made, as it will be in
these particular circumstances, BIND passes through the response leaving the
AA bit set to 1. Sendmail takes this to mean that the response wasn't cached
along the way, and so re-uses the response when performing the "MX" and "A"
lookups, filtering it for resource records of the desired types, instead of
making further queries.

4. The assumptions that Sendmail is making thus break. It is assuming that if
the AA bit is set to 1 in the response, "any" will have really meant "all".
But that's not true in these circumstances.

When Sendmail filters the result of the "*" query looking for "MX" and "A"
resource record sets it finds no resource records of those types. Its
disguised "MX" and "A" lookups thus return empty resource record sets, and it
thus complains that it cannot transport mail addressed to "pwr.ag." and
"klaeui.com." mailboxes.

Note that, as mentioned, this problem relies upon a set of subtle interactions
between a specific combination of softwares. Change any one of them and the
problem goes away:

* Change Sendmail to some other MTA (or change to an appropriate version of
Sendmail), and the assumption that a "passed through" response (i.e. with the
AA bit set to 1) to an "any" query actually contains "all" records goes away.
Other MTAs instead explicitly issue "MX" and "A" queries, which will be
properly resolved because the UltraDNS servers correctly return referrals in
response to "MX" and "A" queries.

* Change BIND to some other proxy DNS server software, and "passed through"
responses from the proxy DNS server go away. ("dnscache" always sets the AA
bit to 0, for example.) Sendmail will thus never assume that "any" has in
fact meant "all", and will thus explicitly issue "MX" and "A" queries rather
than using the result from an "*" query.

* Fix the broken UltraDNS servers so that they always hand out referrals when
appropriate, _even when_ the query type is "*", and query resolution always
ends by asking the "pwr.ag." content DNS servers themselves. They _do_ return
the "MX" and "A" resource record sets in the response to an "*" query, and so
Sendmail's optimisation will happen to work.