RE: Help! Outbound rate has slowed to a crawl ...

Hi Joey and John,

Thanks for your replies -- some answers & more
questions below:

--- Joey <Joey@Web56.Net> wrote:
>
> Are you sure your backbone isn't saturated?

How would I know this? Not sure how to determine ...

> CPU getting high?

No, 10-40% idle on avg. Load peaks at 6 or so for
brief periods, but that's normal for this machine.

> Are you sure you are not getting any hard drive
> errors ( bad sectors )?

Assume these would show up in /var/log/messages?
Nothing about bad sectors or anything similar there.

> If you reload DNS with stop & start, then lookup
> something you have never
> looked up before like redshoes.com does it work?

Not sure how to do this, & our regular sysadmin is
sick with the flu :( .

> Do you have extra mailservers that can send out
> email ?
> Possibly with a different IP number?

Unfortunately no. 8(

John Pettitt said "what do the log say?"

Nothing in the syslog (messages), but I noticed a huge
number (3000+) messages all getting set to "(delivery
temporarily suspended: unknown mail transport error)"
all pretty much in a row over a 2 minute period in
maillog. I looked at the log for a while before the
first one got set to that status, but nothing looked
unusual ...

Thanks,

Andy
>
> joey
>
> -----Original Message-----
> From: owner-postfix-users@postfix.org
> [mailto:owner-postfix-users@postfix.org] On Behalf
> Of Andy Kalin
> Sent: Tuesday, February 01, 2005 12:51 AM
> To: postfix-users@postfix.org
> Subject: Help! Outbound rate has slowed to a crawl
> ...
>
> Hi All!
>
> We are running postfix 2.0.16-3.rh8 primarily as an
> outbound relay for
> sending approx 50-100K messages/day. Our outbound
> delivery rate is usually
> around 1000 messages/minute (at peak times). After
> months of steady
> reliable throughput, our outbound became very
> constipated today. Rate has
> dropped to ~60 messages/min and we have a queue that
> has crept up to 36K
> messages over the course of the day. Other
> details:
>
> 1) 16050 of the remaining 17646 requests are flagged
> as "(delivery
> temporarily suspended: unknown mail transport
> error)" (when grepping thru
> mailq output).
> Restarting postfix reduced this number to 1510
> (which will presumably grow)
> 2) The vast majority of the remaining messages are
> now oscillating between
> active & deferred status at long intervals (postfix
> backoff/retry cycles, I
> assume)
> 3) As mentioned, when the active queue is full, the
> outbound rate is
> extremely slow (~60 as noted above)
> 4) we have 50 smtp daemons (default) delivering
> outbound
> 5) this server receives very little inbound mail
> (it's a designated
> outbound)
> 6) the backed up mail requests are being sent to
> multiple different domains
> (see qshape output below)
> 7) the server is not on a ton of blacklists (only
> one, according to
> dnsstuff.com)
> 8) the box's memory is pretty topped out, but this
> is probably to be
> expected given the other symptoms
>
> Can anyone offer some troubleshooting tips? DNS
> lookups seem to be working
> fine, as some messages are being delivered (see
> example at bottom), but I'm
> not sure if my periodic nslookups/etc. would reveal
> any intermittent
> problems ...
>
> Thanks!
>
> Andy
>
>
> [root@euphrates log]# /usr/local/bin/qshape.pl
> active
> | head -20
> T 5 10 20 40
> 80 160 320 640 1280 1280+
> TOTAL 14470 3 2 0 0
>
> 0 1 1 12703 489 1271
> yahoo.com 556 0 0 0 0
>
> 0 0 0 553 3 0
> aol.com 526 2 0 0 0
>
> 0 0 0 524 0 0
> earthlink.net 259 0 0 0 0
>
> 0 0 0 259 0 0
> msn.com 159 0 0 0 0
>
> 0 0 0 159 0 0
> home.com 143 0 0 0 0
>
> 0 0 0 143 0 0
> comcast.net 119 0 1 0 0
>
> 0 0 0 118 0 0
> mindspring.com 117 0 0 0 0
>
> 0 0 0 117 0 0
> juno.com 113 0 0 0 0
>
> 0 0 0 113 0 0
> excite.com 80 0 0 0 0
>
> 0 0 0 80 0 0
> nc.rr.com 77 0 0 0 0
>
> 0 0 0 63 14 0
> bellsouth.net 77 0 0 0 0
>
> 0 0 0 77 0 0
> attbi.com 76 0 0 0 0
>
> 0 0 0 76 0 0
> worldnet.att.net 72 0 0 0 0
>
> 0 0 0 72 0 0
> usa.net 66 0 0 0 0
>
> 0 0 0 66 0 0
> nyc.rr.com 65 0 0 0 0
>
> 0 0 0 22 40 3
> att.net 61 0 0 0 0
>
> 0 0 0 61 0 0
> cs.com 56 0 0 0 0
>
> 0 0 0 56 0 0
> sympatico.ca 55 0 0 0 0
>
> 0 0 0 55 0 0
>
>
>
>
> Jan 31 21:45:21 euphrates postfix/smtp[17936]:
> 8A402E022C: to=<xxxxx@xxxx.net>,
> relay=mx07.xxxx.net[207.69.200.133], delay=27068,
> status=sent (250
> 1cVQQN4ab3Nl3qB0 Message accepted for
> delivery)
> Jan 31 21:45:21 euphrates postfix/qmgr[17845]:
> 8A402E022C: removed
>
>
>
>
> __________________________________
> Do you Yahoo!?
> Yahoo! Mail - 250MB free storage. Do more. Manage
> less.
> http://info.mail.yahoo.com/mail_250
>
>
>


__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com