Slow sending from client with qmail

Hi qmail experts,

I have a specific issue with slow sending in qmail as follows:

Use Netscape (And IE/Outlook) client to compose message.
Press "send" and wait...wait...wait...

The client will stall for anyywhere from 8 seconds to 15 seconds
while displaying the usual Netscape "progress bar".

Interestingly enough my setup has been OK for several years. There
is a slowdown once in a while happens once in a while but normally
"send" is instant (Within 0.5 seconds).

The network is 100Mb/s so no problem there.

What could cause such a slowdown? It is as if qmail is
trying to do something that takes some time before acknowledging
the Email from the client.

My understanding is that sending an Email from the local network
simply puts it into the queue so there is no need to "wait" for
anything.

I have examined the queue to see if there was anything wrong there,
nothing seems out of order. A couple of test tools were used to check
the queue, nothing wrong.

All emeil sent does get delivered but this slowdown is annoying.

Any tips would welcommed...

Tony

On Mon, 19 Apr 2004 17:27:37 +1000, Tony <tony@no_spam.invalid> may have written:
>
> I have a specific issue with slow sending in qmail as follows:
>
> Use Netscape (And IE/Outlook) client to compose message.
> Press "send" and wait...wait...wait...
>
> The client will stall for anyywhere from 8 seconds to 15 seconds
> while displaying the usual Netscape "progress bar".
>
> Interestingly enough my setup has been OK for several years. There
> is a slowdown once in a while happens once in a while but normally
> "send" is instant (Within 0.5 seconds).
>
> The network is 100Mb/s so no problem there.
>
> What could cause such a slowdown? It is as if qmail is
> trying to do something that takes some time before acknowledging
> the Email from the client.

Are you using any DNSBLs that might not be resolving properly? Are you
trying to do reverse DNS lookups on the clients and there is no reverse
DNS configured for their IP?

> My understanding is that sending an Email from the local network
> simply puts it into the queue so there is no need to "wait" for
> anything.

No, it connects via SMTP just like any other client. Only programs
residing on the mailserver itself can queue mail directly.

--
Brian T Glenn
delink.net Internet Services

On Tue, 20 Apr 2004 00:04:28 +0000, Brian T Glenn wrote:

> On Mon, 19 Apr 2004 17:27:37 +1000, Tony <tony@no_spam.invalid> may
> have written:
>>
>> I have a specific issue with slow sending in qmail as follows:
[...]

>> is a slowdown once in a while happens once in a while but normally
>> "send" is instant (Within 0.5 seconds).
[...]

>> What could cause such a slowdown? It is as if qmail is
>> trying to do something that takes some time before acknowledging

> Are you using any DNSBLs that might not be resolving properly? Are you
> trying to do reverse DNS lookups on the clients and there is no reverse
> DNS configured for their IP?

I do not currently use reverse DNS on the clients but I used to (Redoing
the network). However my box has a reverse DNS entry but it is still slow.

I have been using DNSBL lists:

relays.ordb.org
bl.spamcop.net
rdts.bl.reynolds.net.au

I will check if they are slowing things down but I thaught that these
only got invoked during receipt of Email from outside.

Just checked...

The DNS server reported that it refuses to respond to the query. There's
a problem with the DNS server for bl.spamcop.net.

Tried the reynolds one...

The ordb one does respond.

From memory you can not do a direct lookup with a tool but I think
the slowdown is due to load on these blocklist servers.

If the lookups are invoked when local machines send Email, how
to "exempt" internal addresses (Clients)?

>> My understanding is that sending an Email from the local network simply
>
> No, it connects via SMTP just like any other client. Only programs
> residing on the mailserver itself can queue mail directly.

OK, you are right, I wrote that without thinking to much..:-))

Tony

On Tue, 20 Apr 2004 15:03:15 +1000, Tony <tony@no_spam.invalid> may have written:
> On Tue, 20 Apr 2004 00:04:28 +0000, Brian T Glenn wrote:
> > Are you using any DNSBLs that might not be resolving properly? Are
> > you trying to do reverse DNS lookups on the clients and there is no
> > reverse DNS configured for their IP?
>
> I do not currently use reverse DNS on the clients but I used to (Redoing
> the network). However my box has a reverse DNS entry but it is still slow.

It doesn't have anything to do with the clients themselves. Unless you
pass the -H option to the tcpserver that starts qmail-smtpd, it will
attempt to find a PTR for the address of the client connecting to it. If
these are RFC1918 addresses and the queries get passed to the Internet
(instead of a local nameserver with correct data) chances are you will
have to wait on a timeout.

> I have been using DNSBL lists:
>
> relays.ordb.org
> bl.spamcop.net
> rdts.bl.reynolds.net.au
>
> I will check if they are slowing things down but I thaught that these
> only got invoked during receipt of Email from outside.
>
> Just checked...
>
> The DNS server reported that it refuses to respond to the query. There's
> a problem with the DNS server for bl.spamcop.net.
>
> Tried the reynolds one...
>
> The ordb one does respond.

You can stop lookups on your own clients' IP addresses by creating a
line in /etc/tcp.smtp as follows:

192.168.:allow,RBLSMTPD=""

> From memory you can not do a direct lookup with a tool but I think
> the slowdown is due to load on these blocklist servers.

dig a 2.0.0.127.bl.spamcop.net

> If the lookups are invoked when local machines send Email, how
> to "exempt" internal addresses (Clients)?

See above for the tcpserver line that will stop DNSBL on client
addresses.

> > No, it connects via SMTP just like any other client. Only programs
> > residing on the mailserver itself can queue mail directly.
>
> OK, you are right, I wrote that without thinking to much..:-))

One or both of these problems is causing your slowdowns it seems. If you
don't care about reverse DNS, simple add -H to tcpserver and it will
stop checking.

For the DNSBLs, you will have to make a policy decision on whether or
not you want to lookup client IPs. If they are RFC1918 IPs, the answer
is definitely no.

--
Brian T Glenn
delink.net Internet Services

On Tue, 20 Apr 2004 15:23:28 +0000, Brian T Glenn wrote:

>> On Tue, 20 Apr 2004 00:04:28 +0000, Brian T Glenn wrote:
>> > Are you using any DNSBLs that might not be resolving properly? Are

>> I do not currently use reverse DNS on the clients but I used to (Redoing
>> the network). However my box has a reverse DNS entry but it is still slow.
>
> It doesn't have anything to do with the clients themselves. Unless you
> pass the -H option to the tcpserver that starts qmail-smtpd, it will
> attempt to find a PTR for the address of the client connecting to it. If
> these are RFC1918 addresses and the queries get passed to the Internet
> (instead of a local nameserver with correct data) chances are you will
> have to wait on a timeout.

Well here is the interesting thing, I have always used -v -c40 -UX -x$CDB
for the tcpserver but no "-H" and I have a reverse DNS entry.

It has worked fine in the past and no changes were made.

I suspect that the problem is that the local IP 192.168.x.x is checked
and the slowdown is due to a slower than normal lookup time on the
blocklists.

>
>> I have been using DNSBL lists:
[...]

> You can stop lookups on your own clients' IP addresses by creating a
> line in /etc/tcp.smtp as follows:
>
> 192.168.:allow,RBLSMTPD=""

I use:

192.168.1.:allow,RELAYCLIENT=""

Can I add yours like this:

192.168.:allow,RBLSMTPD,RELAYCLIENT=""

or should I add an extra line with:

192.168.1.:allow,RBLSMTPD=""


>> From memory you can not do a direct lookup with a tool but I think
>> the slowdown is due to load on these blocklist servers.
>
> dig a 2.0.0.127.bl.spamcop.net

I knew there was something there...forgot the finer details...

>> If the lookups are invoked when local machines send Email, how
>> to "exempt" internal addresses (Clients)?
>
> See above for the tcpserver line that will stop DNSBL on client
> addresses.

That is understood.

>> OK, you are right, I wrote that without thinking to much..:-))
>
> One or both of these problems is causing your slowdowns it seems. If you
> don't care about reverse DNS, simple add -H to tcpserver and it will
> stop checking.

As my new TINYDNS is almost ready I will probably leave it and make
sure the reverse lookup works.

> For the DNSBLs, you will have to make a policy decision on whether or
> not you want to lookup client IPs. If they are RFC1918 IPs, the answer
> is definitely no.

Not a hard choice, all our clients are on private networks so yes, I will
use a policy of NO.

This all leads me with just one question as listed above. You have
been most helpfull, thanks.

Tony

On Wed, 21 Apr 2004 19:07:33 +1000, Tony <tony@no_spam.invalid> may have written:
> On Tue, 20 Apr 2004 15:23:28 +0000, Brian T Glenn wrote:
>
>> You can stop lookups on your own clients' IP addresses by creating a
>> line in /etc/tcp.smtp as follows:
>>
>> 192.168.:allow,RBLSMTPD=""
>
> I use:
>
> 192.168.1.:allow,RELAYCLIENT=""
>
> Can I add yours like this:
>
> 192.168.:allow,RBLSMTPD,RELAYCLIENT=""
>
> or should I add an extra line with:
>
> 192.168.1.:allow,RBLSMTPD=""

You were close, I would add the following:

192.168.:allow,RELAYCLIENT="",RBLSMTPD=""

>> dig a 2.0.0.127.bl.spamcop.net
>
> I knew there was something there...forgot the finer details...

DNSBLs are just DNS. In fact, they work exactly like in-addr.arpa
requests except they use a different base zone.

>> One or both of these problems is causing your slowdowns it seems. If you
>> don't care about reverse DNS, simple add -H to tcpserver and it will
>> stop checking.
>
> As my new TINYDNS is almost ready I will probably leave it and make
> sure the reverse lookup works.

It is ideal to have rDNS working of course. Many applications look for
the reverse, and it if is working, they will be so much the faster.

> This all leads me with just one question as listed above. You have
> been most helpfull, thanks.

My pleasure, I hope that clears everything up for you.

--
Brian T Glenn
delink.net Internet Services

On Wed, 21 Apr 2004 22:31:01 +0000, Brian T Glenn wrote:

>>> You can stop lookups on your own clients' IP addresses by creating a
>>> line in /etc/tcp.smtp as follows:
>>>
>>> 192.168.:allow,RBLSMTPD=""
>>
>> I use:
>>
>> 192.168.1.:allow,RELAYCLIENT=""
>>
>> Can I add yours like this:
>>
>> 192.168.:allow,RBLSMTPD,RELAYCLIENT=""

[...]

> You were close, I would add the following:
>
> 192.168.:allow,RELAYCLIENT="",RBLSMTPD=""

I did see something about this in the Bernstein pages
but his doc's are very terse and lack samples:-))

I have done this and it does seem to work just fine.


>> I knew there was something there...forgot the finer details...
>
> DNSBLs are just DNS. In fact, they work exactly like in-addr.arpa
> requests except they use a different base zone.
>
>>> One or both of these problems is causing your slowdowns it seems. If
>>> you don't care about reverse DNS, simple add -H to tcpserver and it
>>> will stop checking.
>>
>> As my new TINYDNS is almost ready I will probably leave it and make
>> sure the reverse lookup works.
>
> It is ideal to have rDNS working of course. Many applications look for
> the reverse, and it if is working, they will be so much the faster.

I know, I have in the past had problems when rDNS failed:-((

As soon as I have DHCP/TINYDNS running together I will "go live"
and replace "bind".

All I can say is that "Qmail Rocks..." to take a quote from an
exellent web site about setting up qmail.

Once again thanks.