Re: [AMaViS-user] trouble with file(1) misclassifying gzip data files

Luca,

> I've a Debian 3.1 box with Postfix 2.1.5, Amavisd-new 20030616p10-5.
> A user has sent an email with several attachments, two of these
> attachments are data files in a proprietary format which have been
> recognized from file(1) utility as "gzip compressed data".
> They are not gzip files, and so when amavisd-new tries to decompress them
> it generates an error like this:
> Feb 23 06:54:47 fred amavis[15259]: (15259-03-2) TROUBLE in check_mail:
> decoding2-get-file-types FAILED: 'file' utility (/usr/bin/file) failed,
> status=255 (65280 ) at /usr/sbin/amavisd-new line 3853.
....
> The only solution to this problem was to comment out in the
> /usr/share/misc/file/magic file the section for the gzip recognition.
> file-4.12, magic data from /etc/magic:/usr/share/misc/file/magic

> I know that this could be a file issue rather than an amavisd-new
> trouble, but I was thinking that amavisd-new could have a behaviour more
> graceful regarding this kind of recognition.
>
> May be Amavisd-new 2.2 has a different kind of operation?
> (Debian has not yet packaged this version of Amvisd-new).

Although amavisd-new 2.2.1 calls the file(1) utility slightly differently
than 20030616, it is no more tolerant to utility failures than before.
As the file(1) is one of the key providers of information, guiding
further decoding and 'banned name' checking, I don't think it would be
appropriate to be more permissive to its failures. So far, what issues
were found with it, got fixed by its maintainer, so I think the best approach
is to send bug reports and sample files to its maintainer.

> amavis[14005]: (14005-04-2) Decoding of part-00010 (gzip compressed data,
> was
> "\\342\\360K\\325\\301\\342K\\307\\347K\\311\\323\ \303\\361\\303\\323\\311\
>\360K\\327\\323\\360\\365\\360\\362\\360\\367K\\3 31\\363\\307", from Unix)
> failed, leaving it unpacked: Error running decompressor /bin/gzip -d on
> part-00010, exit status 1 at /usr/sbin/amavisd-new line 4208.

This one is not directly related to file(1) failure, and is treated
as a soft failure - a warning is logged and life goes on.

Mark


-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/...fo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/

Hi Mark,

in the meantime I had a similar problem with simply ASCII text files
beginning with 'BZ' which file(1) recognizes as bzip2 compressed data
:-((.

The real oddity is that file(1) returns an exit code as if it could not
stat() the file, and this is really strange, isn't it?

The real solution for the moment is to create a wrapper to
/usr/bin/file
like this:

-----------------
#!/bin/bash
/usr/bin/file.real $@
exit 0
------------------

So amavisd-new is happy and even if it try to decompress a gzip file
that is not a real gzip file, this will generate a warning but will not
block the delivery of the mail.

I'll file a bug to the file developers.

Thank you for your support.
Regards,
Luca


Mark Martinec wrote:
> Luca,
>
> > I've a Debian 3.1 box with Postfix 2.1.5, Amavisd-new
20030616p10-5.
> > A user has sent an email with several attachments, two of these
> > attachments are data files in a proprietary format which have been
> > recognized from file(1) utility as "gzip compressed data".
> > They are not gzip files, and so when amavisd-new tries to
decompress them
> > it generates an error like this:
> > Feb 23 06:54:47 fred amavis[15259]: (15259-03-2) TROUBLE in
check_mail:
> > decoding2-get-file-types FAILED: 'file' utility (/usr/bin/file)
failed,
> > status=255 (65280 ) at /usr/sbin/amavisd-new line 3853.
> ...
> > The only solution to this problem was to comment out in the
> > /usr/share/misc/file/magic file the section for the gzip
recognition.
> > file-4.12, magic data from /etc/magic:/usr/share/misc/file/magic
>
> > I know that this could be a file issue rather than an amavisd-new
> > trouble, but I was thinking that amavisd-new could have a behaviour
more
> > graceful regarding this kind of recognition.
> >
> > May be Amavisd-new 2.2 has a different kind of operation?
> > (Debian has not yet packaged this version of Amvisd-new).
>
> Although amavisd-new 2.2.1 calls the file(1) utility slightly
differently
> than 20030616, it is no more tolerant to utility failures than
before.
> As the file(1) is one of the key providers of information, guiding
> further decoding and 'banned name' checking, I don't think it would
be
> appropriate to be more permissive to its failures. So far, what
issues
> were found with it, got fixed by its maintainer, so I think the best
approach
> is to send bug reports and sample files to its maintainer.
>
> > amavis[14005]: (14005-04-2) Decoding of part-00010 (gzip compressed
data,
> > was
> >
"\\342\\360K\\325\\301\\342K\\307\\347K\\311\\323\ \303\\361\\303\\323\\311\
> >\360K\\327\\323\\360\\365\\360\\362\\360\\367K\\3 31\\363\\307", from
Unix)
> > failed, leaving it unpacked: Error running decompressor /bin/gzip
-d on
> > part-00010, exit status 1 at /usr/sbin/amavisd-new line 4208.
>
> This one is not directly related to file(1) failure, and is treated
> as a soft failure - a warning is logged and life goes on.
>
> Mark
>
>
> -------------------------------------------------------
> SF email is sponsored by - The IT Product Guide
> Read honest & candid reviews on hundreds of IT Products from real
users.
> Discover which products truly live up to the hype. Start reading now.
> http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
> _______________________________________________
> AMaViS-user mailing list
> AMaViS-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/...fo/amavis-user
> AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
> AMaViS-HowTos:http://www.amavis.org/howto/

Ok, I've setup the file wrapper, so that file returns always exit
code=0, reverted to the original magic database (so file now can
(mis)recognize gzip o bizp2 data files), and now the log from
amavisd-new is this:

Mar 1 09:41:36 fred amavis[19635]: (19635-01-12) Decoding of
part-00002 (bzip compressed data, version: a) failed, lea
ving it unpacked: Error running decompressor /usr/bin/bzip2 -d on
part-00002, exit status 2 at /usr/sbin/amavisd-new li
ne 4208.
Mar 1 09:41:36 fred amavis[19635]: (19635-01-12) decode_parts: NOTICE:
new name requested, but file not created: part-
00003

and the mail is correctly delivered to the recipients.

The alternative was obviously to set $bypass_decode_parts = true, but
I don't like this :-)

Kind regards,
Luca