ipfilter on solaris 10

Hi, (sorry if this arrives twice, my mailserver is acting up)

New user of ipfilter here, sadly started out with some strange
experiences... and I would like to have this group input on the
state of things...

I am running ipfilter under Solaris 10
# ipf -V
ipf: IP Filter: v4.1.9 (592)
Kernel: IP Filter: v4.1.9
Running: yes
Log Flags: 0 = none set
Default: pass all, Logging: available
Active list: 1
Feature mask: 0x107

# showrev
Hostname: xxxx
Hostid: 8xxxxxx6
Release: 5.10
Kernel architecture: sun4us
Application architecture: sparc
Hardware provider: Sun_Microsystems
Domain:
Kernel version: SunOS 5.10 Generic_118833-36

The system(s) are basically mail-relays which handles the
mailtraffic of a lot of customers... (2.5million)

They recently upgraded to Solaris 10 and then ran into trouble
with ipf (solaris version) same rules worked fine under Solaris 8
and a locally compiled ipfilter.

If we cut away thr rules opening for ssh and other, its basically three
rules that control the workings...

pass out quick proto tcp/udp from any to any keep state
pass in quick proto tcp from any to any port = smtp keep state
block in log all

These three rules worked fine under Solaris 8 and ipv v3.4.31 (496)
Now they block _some_ mailservers that answer back... which already
has established connections...

excerpt from ipmon...

05/12/2007 14:16:28.112973 fjgi0 @0:31 b 217.28.202.24,25 ->
81.228.11.98,35703 PR tcp len 20 52 -A IN OOW
05/12/2007 14:16:28.142511 fjgi0 @0:31 b 195.47.247.173,25 ->
81.228.11.98,38021 PR tcp len 20 84 -AFP IN OOW
05/12/2007 14:16:28.612643 fjgi0 @0:31 b 195.47.247.173,25 ->
81.228.11.98,38021 PR tcp len 20 52 -A IN OOW
05/12/2007 14:16:29.221150 fjgi0 @0:31 b 130.235.83.227,25 ->
81.228.11.98,63937 PR tcp len 20 52 -A IN OOW
05/12/2007 14:16:29.472484 fjgi0 @0:31 b 194.116.198.17,25 ->
81.228.11.98,33977 PR tcp len 20 40 -AF IN OOW
05/12/2007 14:16:29.802263 fjgi0 @0:31 b 195.47.247.173,25 ->
81.228.11.98,38021 PR tcp len 20 84 -AFP IN OOW

excerpt from netstat -an at same time...

81.228.11.98.63937 130.235.83.227.25 3051008 1575333 49640 0
ESTABLISHED
81.228.11.98.35703 217.28.202.24.25 102272 87599 49640 0
ESTABLISHED
81.228.11.98.38021 195.47.247.173.25 618368 617321 49640 0
ESTABLISHED
81.228.11.98.33977 194.116.198.17.25 130504 130383 49640 0
ESTABLISHED

I have the patch 125014-03 on the system

These blocks causes some large emails to "stack up" in the mailqueue

I removed keep state from the mail rule and added the rule

pass in quick proto tcp from any port = smtp to any port > 8192

and then it flows freely again...

Why is this happening in this version but not in the old one?

What can I do to ensure the mailflow without having to add the other rule
which does make the system a bit unsecure.

I have increased the state buffers... to have room for the states...
there are more rules for management and ssh etc. etc. which all keep state
and they dont seem to be blocked...

# ipfstat -s
IP states added:
3746980 TCP
976197 UDP
0 ICMP
505164433 hits
46856269 misses
0 maximum
0 no memory
0 max bucket
0 maximum
0 no memory
6518 bkts in use
9573 active
0 expired
0 closed
State logging enabled

State table bucket statistics:
6518 in use
32.57% bucket usage
0 minimal length
7 maximal length
1.469 average length


excerpt from ipf -T list

fr_statemax min 0x1 max 0x7fffffff current 14009
fr_statesize min 0x1 max 0x7fffffff current 20011

Anyone on this list that knows whats going on with smtp?
Are there any timers that I can increase safely to make the states
"hold out" for responses or is this supposed to work like this?
I would have though the first smtp rule would be sufficient to not
block the incoming ack's...

Hoping for some insights...

/Johan A