procmail filter behaviour with long messages

There's something weird going on with procmail, and I'm hoping someone
can explain the internal workings of pipes in procmail so I can
understand exactly why this happens.

The problem is that when I have procmail pipe mail to a filter, short
messages get absorbed by the filter but long messages get delivered to
the mailbox - and I don't want them to. This happens when I use filters
that don't read stdin completely, so clearly it all has something to do
with that. Take for example this lone recipe in a .procmailrc

:0
* .*
| /bin/true

This will pipe all incoming mail to the filter /bin/true, which does not
touch stdin at all. I've tried sending mail to this address with varying
sized messages. With short messages ( < 54 lines ) the mail "disappears"
as I would expect it to. No message ends up getting delivered to the
default mailbox.

But with long messages ( >= 54 lines ) the messages DO get delivered to
the mailbox, and I can't figure out what mechanism is causing that to
happen.

OTOH, if I use a filter action such as "| gzip >> archive.gz" there is no
problem, because gzip does read stdin completely. I have some other auto-
reply scripts that use grep to partially read stdin. Again, with those
scripts, long messages end up getting delivered in their entirety to the
mailbox.

What does procmail do with the pipe/filter's data to stdout? How about
the left over data at stdin? When does it decide it wants to deliver the
message to the mailbox, and when does it just leave the mail with the
filter? Clearly there's a threshold involved, maybe someone can explain
why it works this way.

--
Jem Berkes
http://www.pc-tools.net/
Windows, Linux & UNIX software

In comp.mail.misc Jem Berkes <jb@users.pc9.org> wrote:
> There's something weird going on with procmail, and I'm hoping someone
> can explain the internal workings of pipes in procmail so I can
> understand exactly why this happens.

> The problem is that when I have procmail pipe mail to a filter, short
> messages get absorbed by the filter but long messages get delivered to

Your use of the word "filter" here conflicts with what procmail
means by "filter" (consider the 'f' flag on a recipe). This
confused me at first.

Anyway... I could imagine two things happening.
First, if a message is bigger than PIPE_SIZE bytes (or something like
that), and the child program terminates without opening the pipe, I
think you get a write failure into the pipe - or a signal. This is
not a procmail characteristic; it's a "unix" characteristic. Example:

(trap 'echo $i > /dev/tty' 13; i=1; while echo worraworra $i; do i=`expr $i + 1`; done; ) | sleep 15

(the '13' there is SIGPIPE)

On my box this causes 281 to appear on my screen. This would be
about 4096 bytes, which is PIPE_SIZE on my box. IN other words,
it writes and writes and writes, and when it's written 4096 bytes
(after about 281 iterations), it's stuck. This takes just a few
seconds. When the "sleep" terminates, not having drained the pipe
at all, the writing side gets SIGPIPE.


Second, if the child program terminates real soon, before the writer
finishes writing what he wants into the pipe, the writer will get
an error or signal before even getting to 4096 bytes.

(trap 'echo $i > /dev/tty' 13; i=1; while echo worraworra $i; do i=`expr $i + 1`; done; ) | true

I get 6 or 8 or 10, something like that, since "true" terminates right
away.

> the mailbox - and I don't want them to. This happens when I use filters
> that don't read stdin completely, so clearly it all has something to do
> with that. Take for example this lone recipe in a .procmailrc

> :0
> * .*
> | /bin/true

As David told you, the "i" flag is your friend.

I often do something like this:

:0hi
* ^X-I-Am-Spam: true
| : matched X-I-Am-Spam ;

(without the semicolon, either a symlink from "true" to ":" or
something like that is needed)

cheers
====
Collin Park Not a statement of my employer.

[Copie en courrier]

Jem Berkes <jb@users.pc9.org> wrote
news:Xns93C98225A925Bjbuserspc9org@205.200.16.73:

> :0
> * .*
> | /bin/true
....
> What does procmail do with the pipe/filter's data to stdout? How about
> the left over data at stdin? When does it decide it wants to deliver
> the message to the mailbox, and when does it just leave the mail with
> the filter? Clearly there's a threshold involved, maybe someone can
> explain why it works this way.
>

procmail read the stdout filter and process it as a mail. So the
recipiend is used to store the mail in your mailbox.

"Normal" filters don't return data.

/bin/true is not design to filter any input, it should be sensible only
to --help and --version, but may be disturb with "big" mail.

Send mail to /dev/null intead :

:0
* .*
/dev/null


Regards