Re: administering 1,000 zone files

On Thu, Dec 30, 2004 at 07:08:15PM +0000, Jim Reid wrote:

> >>>>> "Michael" == Michael van Elst <mlelstv@serpens.de> writes:
> Michael> The named.conf on the slaves is split into a general
> Michael> section and many include files.
> This is not a good idea. Having "many include files" is a recipe for
> needless complexity and brittle DNS administration.

Why ?
Do you thing anyone with a mind would deal with 200000 zones manually ?


> Yuk! Why make needless work for yourself determining which zone{}
> statements go in which include files?

Because reading and writing a single much larger configuration file
is too slow. And don't get me wrong, all this was automated, nobody
touched the files by hand.


> Michael> The split is necessary because the whole config is
> Michael> several ten megabytes that you do not want to read and
> Michael> write for every change.
>
> Yuk! This is pointless. If you force the server to reload, it has to
> re-read all these include files -- what if one of them gets lost or
> corrupted?

There is hardly any difference between reading one single large
file or a few dozen small files. Not in complexity and not in time
consumed. After all the server reads hundreds of thousands of
zone files, why should it have problems with a few configuration
files ?

If one file gets lost or corrupted then the same can happen
with the one and single configuration file, destroying everything
instead of just a part.

However, it is unlikely that a file gets lost or corrupted.
There are backups and the files are automatically checksummed.


> -- and check the referenced zone files are current. So you
> needlessly make more work for the name server and yourself ploughing
> through a named.conf file split across say 20 1 Mbyte include files
> than there would be parsing a single named.conf file of 20 Mbytes.

I doubt that the nameserver cares about reading 1 or 20 files.
I myself did never plough through these files, should I ? :)

But the program dealing with the files had the advantage that
instead of ploughing through a few ten MBytes it only had to
work on a few hundred kilobytes, making it 10-20 times as
fast.


> Adding or removing zone{} statements from named.conf implies that the
> name server has to read all of its configuration -- ie the whole file
> -- so that it knows what is the current set of zones to serve.

Sure. But the program creating the configuration file has an advantage
if it has to work on only a small part of the configuration.


> If you're talking about hundreds of thousands of zones and 20 Mbyte
> named.conf files, you really should be using a database to keep track
> of the sheer volume of data and having an SQL script generate the
> config files from that database.

Who says that I didn't ? The hidden primary is (mostly) configured
from a database.

The slave servers (running BIND8 at that time!) use configuration
and zone files stored on a local disk. Even BIND9 cannot do much
better (unless you add the DLZ patches), but why should I rely
on this, really huge, complexity ? For the same functionality
each slave server would have to host a replica(!) of the database(!).

Saying that, this is exactly what I did, except for avoiding
poorly tested patches and using the filesystem and a small
program instead of a database replica.


> Michael> Adding or deleting zones on the master triggered ssh
> Michael> sessions to each slave that ran a small program to
> Michael> add/delete zone statements to/from the appropriate
> Michael> include files and set a flag. The flag was polled
> Michael> regularly to reload the nameserver.
>
> Yuk! What if that small program fails?

What if a large big program fails ? I don't see a point here.
If any program fails that modifies the nameserver configuration
then the result can be disastrous.

You can either modify the configuration or you can create/copy the
configuration. Creating/copying the configuration is too slow with
a few tens of megabytes.

So how do you suggest to modify the configuration files? Manually?
Do you suggest to run a database replica on each slave server
and believe that this will never fail, but insist that the small
program does?


> Wouldn't it be better to
> generate the configuration data in one place?

The configuration data can be easily regenerated (let's say if
the slave servers had both system disks fail at the same time).
In fact, this is absolutely necessary. How would you (== some
automated system) know that a slave server needs to be
configured when there is no data available ?

The configuration data for the hidden primary is generated
in one place, but even there I do not regenerate megabytes
of configuration files for each tiny change. BIND doesn't
store the zones in a single file either, why do you insist
on storing the configuration in a single file?

The configuration data for the slave servers is modified
synchronously with the changes on the hidden primary to
avoid copying tons and tons of megabytes of configuration
files for each single tiny change.

> That way, you wouldn't
> have to worry about the prospect of configurations on servers drifting
> from what was intended because of local, possibly transient errors.

Be assured that the configurations do drift, for some time, because
the slave servers aren't always reachable. Of course this hardly
matters, DNS zone data also "drifts" because zone transfers do not
happen instantly or servers are not reachable. The only thing that
matters is that the differences are automatically corrected.

> Michael> The polling time is offset individually for each slave to
> Michael> avoid reloading (which means an interruption of service)
> Michael> for a whole nameserver set.
>
> Yuk! This is bad. It means there are (long?) windows where the name
> server configuration on disk is not the same as that as the active
> configuration in memory. Worse, the on-disk configuration is more up
> to date than what the server is currently using.

The on-disk configuration is always more up to date. Whenever you
modify it, the nameserver doesn't know it until reloaded. So where
is the problem ?

But if you reload the nameserver every 5 or 10 seconds it doesn't
matter how accurate it is because it will never answer a single
query. So you have to stretch out reloads over say a period of
15 or 30 minutes. Note, this is _reload_, which happens only if there
is a new zone added or an old zone deleted. All the tenthousands
of zones that are not changed are not affected at all except that
there is a small outage on one of the slave servers during the
reload.


> It can also mean lame
> delegations because the master server (say) knows about a newly-added
> zone while one of that zone's slaves knows nothing about it.

Lame delegations happen when a nameserver is not configured for
a zone but which is delegated to it. I don't understand how
this is related.


Greetings,
--
Michael van Elst
Internet: mlelstv@serpens.de
"A potential Snark may lurk in every tree."