Re: Spam filtering non-features

Victor Duchovni:
> > It's not hard to make check_client_access smarter about the hostname
> > so it maps (5XX && unknown) into 4xx, but that means we have to
> > update the policy delegation protocol with another attribute so it
> > can do the exact same thing (I like to keep the option to rip out
> > thousands of lines from smtpd into a separate process that plugs
> > into the policy delegation hooks).
>
> This sounds reasonable, I can't think of any compelling cases where
> a REJECT result from an "unknown" key should not be downgraded to
> 4XX if the "unknown" is a result of a failed lookup. Some questions
> will doubtless arise from confused users now and then, but this
> already the case with defer_if_reject, and defer_if_permit, not much
> can top those in complexity :-)

This is easy: if the name match returns REJECT, downgrade into
DEFER if the hostname is "unknown"; plus it needs a new policy
delegation attribute. It covers only the cases where an explicit
action is specified for "unknown" but that is fine.

> > Another concern is that Postfix by design does a lot of sanitization
> > of network information. For example, Postfix rejects hostnames that
> > don't pass the valid_hostname syntax check, and rejects hostnames
> > that pass the valid_hostaddr check.
> >
> > Postfix rejects such names because they could mis-lead humans,
> > because they could be used to evade access controls (a numerical
> > hostname will match a network address pattern), or because they
> > could corrupt audit trail information (hostnames with spaces will
> > not be parsed correctly).
>
> Well we don't use the raw PTR names to construct the name[ip] data
> for logging, but we do send them to the access(5) table, and here
> perhaps it is not unreasonable to skip keys that valid_hostname()?
>
> The key use case is an ISP that supplies bogus generic
> NN-NN-NN-NN.exampl.net PTR records that don't forward resolve.

I see no problems with these particular names. The hostnames are
valid, they can't be confused with network addresses. They don't
mis-lead humans or corrupt audit trails.

I was more concerned with names that are mis-leading, such as names
that pose as a network address, or audit trails that could be
mis-parsed by software that expects Postfix to always produce
logfiles with correctly formatted hostnames.

In particular, I am concerned that the PTR information needs to be
logged when mail is rejected (and perhaps even when it is accepted)
because otherwise trouble shooting will become very difficult.

> There is not much need to pass invalid data into the access table,
> except perhaps as an "invalid" key, that would allow blocking of
> "invalid" PTRs. Comments?

I prefer not to let bad hostnames into Postfix, and to log a warning
so that people know why a name isn't used. This is why valid_hostname()
warnings show up in the maillog file. Specifically, we need to
duplicate the tests that appear at the end of sockaddr_to_hostname().

> > This would be an argument to keep the lookup result from reverse-only
> > lookups strictly separate and not make it available to anything
> > else, lest it be tricked by unsanitized data.
>
> Yes, the "check_ptr_access" mechanism was intended to solve a very
> narrow problem.
>
> > Finally, this sounds like a project of ways to figure out a client
> > identity.
> >
> > - client identity from PTR record
>
> This I really don't view as client identity, I think of it as
> data for an optional local blacklist pure and simple.

I see this as one of several client identities, each with a different
level of credibility. This particular identity has very little
credibility; less than, for example, the client IP address.

> > - client identity from reversible name<->address mapping
>
> This is client identity.

This is another client identity. It is more credible than the one
based on PTR lookups only, but it can be forged.

> > - client identity from EHLO->address mapping
>
> This is even more-so the client identity, and IMHO could
> override the reversed name when the EHLO->address mapping
> matches the client's IP address. The key question is where
> to use it? Received headers? Logs? To record both names or
> just one? ...

This is about as credible as the one from reversible name<->address
mapping except that it is unique, while name<->address mapping may
be ambiguous, or may produce a name that the owner of the mail
server does not like.

Presently, Postfix logs (3, unverified) and (2) in the Received
header. If it were to do anything with (1) then it would probably
use getnameinfo()'s result, for consistency with (2); otherwise
it would create more problems than it solves.

> > That's the easy part. The hard part is a user interface that doesn't
> > create more problems than it solves, and dealing with unsanitized
> > versus sanitized information, as discussed above (valid hostname
> > syntax, hostnames spelled as a network address, corruption of audit
> > trails or otherwise mis-leading humans).
>
> I am not sensing that you are ready to adopt anything concrete in this
> space yet? Are we close to a consensus on what check_reverse_access or
> check_ptr_access (or both) should do. Should check_client_access be
> enhanced to downgrade "unknown" REJECT actions to "DEFER"?

I see no problem with the latter.

We haven't fully analyzed the former. My preference is to consistently
sanitize so that people won't get burned. Those who want unsanitized
PTR bits can use the policy hooks, or write their own MTA.

Wietse