Re: SMTP session hangs after MAIL FROM when mysql backend is down

Jeffrey Overman wrote:
> Hi,
>
> I've just started to use postfix, so if I missed something obvious please
> don't shoot me ;)
>
> I've setup postfix to accept mail for a couple of domains. I use a mysql
> backend to store the information for those virtual domains. As a test I
> shut down the mysql backend while postfix was running. I then started a
> SMTP session using telnet and to my surprise the session hung after
> the MAIL FROM command. I'd expect a 4xx error after RCPT TO, but I'm not
> even getting at that stage.
>
> The logs are complaining about trivial-rewrite not being able to talk to
> mysqld, which is understandable. Here's a piece of the log, I've reformatted
> it so it's a bit more readable and doesn't exceed 80 columns :)
>
> ------//------
> Apr 13 21:51:40 [postfix/trivial-rewrite] warning: connect to mysql server
> unix:/var/run/mysqld/mysqld.sock: Can't connect to local MySQL server
> through socket '/var/run/mysqld/mysqld.sock' (2)
> Apr 13 21:51:40 [postfix/trivial-rewrite] fatal:
> mysql:/etc/postfix/mysql/virtual_alias_maps.cf(0,100): table lookup problem
> Apr 13 21:51:41 [postfix/smtpd] warning: premature end-of-input on
> private/rewrite socket while reading input attribute name
> Apr 13 21:51:41 [postfix/smtpd] warning: problem talking to service rewrite:
> Success
> Apr 13 21:51:41 [postfix/master] warning: process
> /usr/lib/postfix/trivial-rewrite pid 30844 exit status 1
> Apr 13 21:51:41 [postfix/master] warning: /usr/lib/postfix/trivial-rewrite:
> bad command startup -- throttling
> ------//------
>
> Is there any way to get postfix to accept the MAIL FROM and then, after the
> RCPT TO, return a 4xx code?
>
> Thanx in advance,
>

The MYSQL.readme says:

The MySQL configuration interface setup allows for multiple mysql databases:
you can use one for a virtual table, one for an access table, and one for an
aliases table if you want.

Since sites that have a need for multiple mail exchangers may enjoy the
convenience of using a networked mailer database, but do not want to introduce
a single point of failure to their system, we've included the ability to have
Postfix reference multiple hosts for access to a single mysql map. This will
work if sites set up mirrored mysql databases on two or more hosts. Whenever
queries fail with an error at one host, the rest of the hosts will be tried in
random order. If no mysql server hosts are reachable, then mail will be
deferred until at least one of those hosts is reachable.

It sounds like it's supposed to time out after some period (5 min?) and then issue a 4xx code.

--
Greg

Jeffrey Overman wrote:
> On Sat, 15 Apr 2006 12:32:26 +0200, Jeffrey Overman wrote:
>
> [...]
>
>
>> daemon_timeout = 18000s
>> smtp_tls_session_cache_timeout = 3600s
>> smtpd_tls_session_cache_timeout = 3600s
>>
>>The only relevant option seems to be the daemon_timeout one. However, I'm
>>a bit reluctant to set that one too low. Maybe 5 hours is a bit too much,
>>but I'd like to see a 4xx response within a minute if my mysql backend is
>>down and I don't want to set the daemon_timeout to 1 minute as virus/spam
>>scanning might take longer than that with large emails.
>
>
> As a test I set the daemon_timeout to 1 minute. That doesn't help much as
> the connection is dropped without issuing a 4xx code.
>
> Greetz

I'm running the Snapshot version of Postfix. On mine, for testing, I configured
in a mysql query to check recipient validity, but purposely don't have the mysql deamon running.

Right after the RCPT TO, the SMTP connection just drops immediately, with no return code sent.


Apr 16 11:28:57 oldhippie postfix/smtpd[29279]: warning: connect to mysql server 127.0.0.1: Can't connect to MySQL server on '127.0.0.1' (111)
Apr 16 11:28:57 oldhippie postfix/smtpd[29279]: fatal: mysql:/etc/postfix/recipients.cf: table lookup problem
Apr 16 11:28:58 oldhippie postfix/master[29216]: warning: process /usr/libexec/postfix/smtpd pid 29279 exit status 1
Apr 16 11:28:58 oldhippie postfix/master[29216]: warning: /usr/libexec/postfix/smtpd: bad command startup -- throttling

--
Greg

> Do you think it's worth mentioning this issue to the postfix developers?

Yes, I just sent them a query.
--
Greg

> Do you think it's worth mentioning this issue to the postfix developers?

The first reply came from Wietse Venema (the author of Postfix) within a
couple of hours:

That code pre-dates the use of SQL for SMTP server access maps;
when a DB or DBM lookup fails the system is so sick that continuing
makes no sense.

It should not be difficult to replace this by code that pretends
that the table returns "450 4.3.5 Server configuration error"
and then logs a nasty warning to the maillog file.


> Or is this behaviour intentional? In which case I'll probably stop using
> postfix, although I like a lot of its features.

Then a few minutes later, he created the following patch for smtpd_check.c
--
Greg



*** ./smtpd_check.c- Fri Mar 24 15:45:43 2006
--- ./smtpd_check.c Mon Apr 17 10:39:17 2006
***************
*** 2258,2265 ****
CHK_ACCESS_RETURN(check_table_result(state, table, value, name,
reply_name, reply_class,
def_acl), FOUND);
! if (dict_errno != 0)
! msg_fatal("%s: table lookup problem", table);
}
CHK_ACCESS_RETURN(SMTPD_CHECK_DUNNO, MISSED);
}
--- 2258,2270 ----
CHK_ACCESS_RETURN(check_table_result(state, table, value, name,
reply_name, reply_class,
def_acl), FOUND);
! if (dict_errno != 0) {
! msg_warn("%s: table lookup problem", table);
! value = "450 4.3.0 Server configuration error";
! CHK_ACCESS_RETURN(check_table_result(state, table, value, name,
! reply_name, reply_class,
! def_acl), FOUND);
! }
}
CHK_ACCESS_RETURN(SMTPD_CHECK_DUNNO, MISSED);
}
***************
*** 2299,2306 ****
CHK_DOMAIN_RETURN(check_table_result(state, table, value,
domain, reply_name, reply_class,
def_acl), FOUND);
! if (dict_errno != 0)
! msg_fatal("%s: table lookup problem", table);
}
/* Don't apply subdomain magic to numerical hostnames. */
if (maybe_numerical
--- 2304,2316 ----
CHK_DOMAIN_RETURN(check_table_result(state, table, value,
domain, reply_name, reply_class,
def_acl), FOUND);
! if (dict_errno != 0) {
! msg_warn("%s: table lookup problem", table);
! value = "450 4.3.0 Server configuration error";
! CHK_DOMAIN_RETURN(check_table_result(state, table, value,
! domain, reply_name, reply_class,
! def_acl), FOUND);
! }
}
/* Don't apply subdomain magic to numerical hostnames. */
if (maybe_numerical
***************
*** 2353,2360 ****
CHK_ADDR_RETURN(check_table_result(state, table, value, address,
reply_name, reply_class,
def_acl), FOUND);
! if (dict_errno != 0)
! msg_fatal("%s: table lookup problem", table);
}
flags = PARTIAL;
} while (split_at_right(addr, delim));
--- 2363,2375 ----
CHK_ADDR_RETURN(check_table_result(state, table, value, address,
reply_name, reply_class,
def_acl), FOUND);
! if (dict_errno != 0) {
! msg_warn("%s: table lookup problem", table);
! value = "450 4.3.0 Server configuration error";
! CHK_ADDR_RETURN(check_table_result(state, table, value, address,
! reply_name, reply_class,
! def_acl), FOUND);
! }
}
flags = PARTIAL;
} while (split_at_right(addr, delim));

> after patching smtpd_check.c, recompiling and reinstalling postfix
> the session still hung after MAIL FROM

If you have the time to explain, what was your particular testing scenario?

The way I tested was to configure the use of a mysql recipient map for RCPT TO,
and then defined a hostname that wasn't running a MySQL server process.

--
Greg

Jeffrey Overman wrote:

> When I use above config, and shutdown mysqld, any SMTP session seems to
> hang after MAIL FROM. I've also tried to add virtual_mailbox_domains
> with fixed values for my 3 domains, it didn't make a difference
> unfortunately.
>
> If you need more information, let me know.


Here's what I've got so far. The problem occurs in the area of using MySQL tables for
virtual_mailbox_domains, virtual_mailbox_maps, virtual_alias_maps, virtual_alias_domains.

One of the code developers, Victor Duchovni, just wrote:

When this uses SQL, the trivial rewrite service (and indirectly the
queue manager) are subject to SQL latency, availability limitations, ...

DO NOT (on critical systems) use SQL or LDAP in this context. Postfix
has no support for an unreliable trivial-rewrite service (it is supposed
to be trivial and correspondingly reliable).

He's also adamant about not using SQL and LDAP for transport maps.

Nonetheless, he had me try something: I changed virtual_mailbox_domains and virtual_alias_domains
to a list of names within the main.cf file, rather than pointing to MySQL tables. After that,
virtual_mailbox_maps and virtual_alias_maps produced an immediate 4XX code when the MySQL server
was unavailable.

More later, maybe.
--
Greg