RE: Strange DNS resolving behavior

OK, we figured it out. But it was such a nasty problem that I thought
this list might appreciate what happened. Way back when Blaster and
Nachi came out we took Cisco's advice and set up something called a
route-map statement to block packets that the Nachi virus was using to
help find new targets. The statement in essence said "if an inbound
packet is an icmp echo or icmp echo-reply, and it's length is exactly 92
bytes, drop it". This statement had been enabled for months with no
problem.

But then, a few days ago, we made a typo in the course of some other
work that Cisco's IOS didn't complain about, and as a result, we were
dropping ALL packets of length 92 (the clause "must be an icmp packet"
had been turned into a noop by the typo.) And yes, it just so happened
that DNS queries from both Dig and NSLOOKUP for
"amherst.publishingconcepts.com" resulted in responses from certain
nameservers that were exactly 92 bytes long, so these responses never
made it through to us.

You can see why this was so frustrating to debug: only responses to
queries for certain names (amherst.publishingconcepts.com being one of
them) from certain DNS resolvers (in particular the listed servers for
publishingconcepts.com itself) had this precise size. But queries for
other names to the same servers, or for the same name but to other
resolvers, didn't hit the magic size and thus worked fine. And things
were further complicated because I didn't realize that Genuity's servers
that provide secondary DNS service for us are "authoritative only" and
nonrecursive, so some of my testing made it look like their resolvers
were having the same problem, which is what made me think it wasn't just
a local problem to my campus.

I suppose we should count ourselves lucky that this particular DNS name
resolution failure caused us enough trouble to really burrow into the
problem. Otherwise we might have been dropping 92-byte packets for
months before we would have figured out what was going on.

Thanks very much for the help from everyone who responded.

-- John W. Manly <jwmanly@amherst.edu>
Systems and Networking, Amherst College


-----Original Message-----
From: bind-users-bounce@isc.org [mailto:bind-users-bounce@isc.org]On
Behalf Of Simon Waters
Sent: Wednesday, April 21, 2004 1:15 PM
To: comp-protocols-dns-bind@isc.org
Subject: Re: Strange DNS resolving behavior


John Manly wrote:
>
> Finally, I ask Genuity/Level3's name servers. Genuity/Level3 acts
> provides secondary name service for our domains. In one case it
reports
> the domain or name isn't valid, and in another it times out just like
my
> own nameserver does. But again, when I look up a name within the
> publishingconcepts.com domain that I know doesn't exist, I get a
> different kind of error (the expected error) back:
>=20
>=20
> $ nslookup amherst.publishingconcepts.com 4.2.49.3 #
> Genuity/Level3
> Server: 4.2.49.3
> Address: 4.2.49.3#53
> Non-authoritative answer:
> *** Can't find amherst.publishingconcepts.com: No answer # WRONG
ANSER
dig says this server doesn't offer recursion, although that may be
different if you are a customer please use "dig" and show the whole
output where it is useful.

> $ nslookup -q=3D3DA amherst.publishingconcepts.com 4.2.2.1 # =
Another
> Genuity/Level3 server
> ;; connection timed out; no servers could be reached # No answer
> at all (timeout)

Works from here with dig.

> So, I'm completely stumped as to what is going on here. Somehow this
> particular name "amherst.publishingconcepts.com" is not resolving via
> some DNS servers, specifically mine. But other names in the
> publishingconcepts.com domain are resolving properly, and the error
that
> the above DNS name yields is different from the name simply being
> missing.

I think it is just you, so it may just be a local nslookup thing, try
"dig" for more information.

> Any light that anyone can shed, or directions that anyone can sugggest
> for further debugging/troubleshooting/investigation on this issue
would
> be most helpful.

The only oddity in the zone I could see is they list two private name
servers in the zone as authoritative, although even this shouldn't cause
problems, even if you had nameservers on your own network which happened
to be numbered 10.10, this would still work unless you had a private
root, or also tried to master pubishinconcepts.com....

dc01.publishingconcepts.com. 2854 IN A 10.10.0.11
dc02.publishingconcepts.com. 2854 IN A 10.10.0.12


No prizes for guessing the private IP addresses of their domain
controllers I guess ;) Perils of Microsoft DNS servers.




-- Attached file included as plaintext by Ecartis --
-- File: signature.asc
-- Desc: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFAhqwiGFXfHI9FVgYRAn2OAJ9mRnYyt9ARH7AvY/9yW3ljjlk6AgCfSyW7
XmS75iPjpIiLEAprAe/ShiY=3D
=3D6V0N
-----END PGP SIGNATURE-----