[AMaViS-user] High memory - high latency and slow processing

Hi All,

I'm running Amavisd-new with clamav 0.83, Spam Assassin 3.0.2, razor2, pyzor
0.40 and dccproc

My filter server running amavisd-new 2.2.0 have a problem : every once in a
while (2-3 days), some process get stuck on high memory consumption and my
server is then very slow on processing the emails.

Out of all the amavisd-new process I run, a couple of them suddently get
stuck :

10811 amavis 25 0 756m 668m 6416 R 99.9 16.6 10:38.81 amavisd
10818 amavis 25 0 756m 729m 6416 R 99.5 18.1 11:12.63 amavisd
10824 amavis 25 0 757m 738m 6416 R 97.2 18.3 10:36.59 amavisd
10810 amavis 25 0 757m 633m 6416 R 58.6 15.7 6:50.30 amavisd
10825 amavis 25 0 756m 625m 6416 R 43.9 15.5 5:46.50 amavisd

(Notice the ~750 memory consumption)

after they get stuck, my processing time goes from an average 1.5 seconds to
10-15 seconds :

Aug 15 06:29:10 epalsws004 amavis[10815]: (10815-44) TIMING [total 32371 ms] -
SMTP EHLO: 3 (0%), SMTP pre-MAIL: 1 (0%), lookup_sql: 5 (0%), SMTP
pre-DATA-flush: 2 (0%), SMTP DATA: 54 (0%), body_hash: 1 (0%), lookup_sql: 5
(0%), mime_decode: 12 (
0%), get-file-type1: 15 (0%), decompose_part: 1 (0%), parts_decode: 0 (0%),
AV-scan-1: 67 (0%), spam-wb-list: 2 (0%), SA msg read: 1 (0%), SA parse: 3
(0%), SA check: 31782 (98%), update_cache: 22 (0%), deal_with_mail_size: 1
(0%), fwd-connect: 1
47 (0%), fwd-mail-from: 1 (0%), fwd-rcpt-to: 1 (0%), write-header: 4 (0%),
fwd-data: 0 (0%), fwd-data-end: 43 (0%), fwd-rundown: 1 (0%), main_log_entry:
17 (0%), update_snmp: 1 (0%), unlink-1-files: 175 (1%), rundown: 4 (0%)


I chek the process causing problem with lsof and strace. The process don't do
anything. The email opened is always spam. Even with SA debugging or
Amavisd-new debugging there is nothing wrong.

If I try to stop amavis with amavisd stop, all the process will die BUT the
one with high memory :
PID 10825: sending SIGKILL in 0 s
Waiting for the process to terminate: 10810, 10811, 10818, 10824, 10825

I don't see anything in amavisd-agent :
sysContact STR
sysDescr STR amavisd-new-2.2.0 (20041102)
sysLocation STR
sysName STR filter.ibm.epals.com
sysObjectID OID 1.3.6.1.4.1.15312.2.1
sysServices INT 64
sysUpTime Timeticks 118041 (0 days, 0:19:40.41)
CacheAttempts 388 1183/h 100.0 % (CacheAttempts)
CacheHits 18 55/h 4.6 % (CacheAttempts)
CacheHitsSpamCheck 18 55/h 4.6 % (CacheAttempts)
CacheHitsSpamMsgs 17 52/h --- % (ContentSpamMsgs)
CacheHitsVirusCheck 14 43/h 3.6 % (CacheAttempts)
CacheMisses 370 1128/h 95.4 % (CacheAttempts)
ContentBadHdrMsgs 7 21/h 1.8 % (InMsgs)
ContentCleanMsgs 377 1150/h 97.2 % (InMsgs)
ContentVirusMsgs 4 12/h 1.0 % (InMsgs)
InMsgs 388 1183/h 100.0 % (InMsgs)
InMsgsRecips 495 1510/h 127.6 % (InMsgs)
OpsDec 374 1141/h 96.4 % (InMsgs)
OpsDecByArZip 2 6/h 0.5 % (InMsgs)
OpsDecByArZipAttempt 6 18/h 1.5 % (InMsgs)
OpsDecByLhaAttempt 4 12/h 1.0 % (InMsgs)
OpsDecByMimeParser 388 1183/h 100.0 % (InMsgs)
OpsDecByUUlibAttempt 336 1025/h 86.6 % (InMsgs)
OpsDecByUnrarAttempt 4 12/h 1.0 % (InMsgs)
OpsSpamCheck 366 1116/h 94.3 % (InMsgs)
OpsSqlSelect 990 3019/h 200.0 % (InMsgsRecips)
OpsVirusCheck 374 1141/h 96.4 % (InMsgs)
OutForwMsgs 384 1171/h 100.0 % (OutMsgs)
OutMsgs 384 1171/h 100.0 % (OutMsgs)
OutMsgsDelivers 384 1171/h 100.0 % (OutMsgs)
QuarMsgs 4 12/h 100.0 % (QuarMsgs)
QuarVirusMsgs 4 12/h 100.0 % (QuarMsgs)
OpsDecType-asc 336 1025/h 86.6 % (InMsgs)
OpsDecType-html 278 848/h 71.6 % (InMsgs)
OpsDecType-txt 22 67/h 5.7 % (InMsgs)
OpsDecType-sgml 8 24/h 2.1 % (InMsgs)
OpsDecType-exe.exe-ms 4 12/h 1.0 % (InMsgs)
OpsDecType-dat 2 6/h 0.5 % (InMsgs)
OpsDecType-zip 2 6/h 0.5 % (InMsgs)
OpsDecType-gif 1 3/h 0.3 % (InMsgs)
Worm.SomeFool.P 2 6/h 50.0 % (ContentVirusMsgs)
Worm.Mytob.T-2 1 3/h 25.0 % (ContentVirusMsgs)
Exploit.HTML.IFrame 1 3/h 25.0 % (ContentVirusMsgs)
Worm.Mytob.AF 1 3/h 25.0 % (ContentVirusMsgs)



If I restart it all and send the email that was stuck back into the filter
server, it will stuck again...

I had this problem on 2 filter servers. Now I updated the first one with all
the newest perl modules, razor2, dcc and spamassassin 3.0.4. The filter
server is really faster and the memory doesn't have high consumption but it
still seems to get affected by some stuck email. Latency goest 10 time
longer..

I hope I gave enough informations...

can anybody help ?

thanks for all your advises.

Rene Bouchard


-------------------------------------------------------
SF.Net email is Sponsored by the Better Software Conference & EXPO
September 19-22, 2005 * San Francisco, CA * Development Lifecycle Practices
Agile & Plan-Driven Development * Managing Projects & Teams * Testing & QA
Security * Process Improvement & Measurement * http://www.sqe.com/bsce5sf
_______________________________________________
AMaViS-user mailing list
AMaViS-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/...fo/amavis-user
AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3
AMaViS-HowTos:http://www.amavis.org/howto/