Re: [Snort-users] Snort and high performance networks

Chris,

Sorry I think may have confused people by using double negative when I
said 800 Mb/s is not impossible. I agree 800 Mb/s is totally possible. It
just requires dealing with the interrupts and a lot of fine tuning . I have
looked into this and I an using a libpcap-mmap version with a buffer at the
NIC. Also the kernel has been modified so that it will only get interuppted
once and then packets will fill the buffer until polled. Once the polling
has happened interupts are again enabled for the NIC.

BUT, when all rules and preprocessors are enabled snort cannot handle this.
The main bottle neck actually appears to be in snort itself . I have even
gone so far as to set the affinity (possible now in the 2.6.x kernel) for
snort to its own processor. It still pegs 100% of the processor and this is
on a lightning fast P4. Now if I drastically cut down the rules and remove
some preprocessors or allocate them enoumous amounts of memory I can get
close to 800 Mb/s .

SCAMPI was able to get snort to run in the mulit-gigE but take a look at
what this required by a large team of PHD networking and security
professionals to achevieve this. They even say snort on a P4 cannot handle
more than 350 Mb/s in a somewhat default configuration. And this was a
year and a half project.
http://www.ist-scampi.org/publicatio...s-fccm2004.pdf

Three research states the bottle neck is the pattern matching in snort, and
this is the reason it cannot handle the high speeds.

Id still like to hear if anyone is actaully acheiving gigE speeds? And can
actuallly state what technology and mods youd did you achieve this.

-- UoC --




>To be fair, I think that 800Mbps is completely and totally possible. I can
>capture 400mbps on a dual P3 800 - I'm not running it through snort but
>my own set of software which might be more computationally intensive. Now,
>my system isn't set up all that well - if I was smart I would have gotten
>a single proc box - the dual system just eats up too many interrupts -
>which is where my main problem is. The syskonnect card just throws off too
>many interrupts for the hardware to keep up. With some modifications
>including more aggressive interrupt coalescing, wedging a larger buffer
>into the libpcap code, and possibly some minor kernel mods I can see it
>hitting 800Mbps. Even better would be getting it into a more up to date
>box with a single proc and PCI-X.
>
>My specific problem is that we're - essentially - an all optical house
>here so I need to use two GigE cards to capture both inbound and outbound
>- which means more hassles with interrupts and so forth and such not. Not
>a good time.
>
>Anyway, I'll be talking with someone who used to be with Enadace and the
>DAG project over at Waikato (en zed university). I saw some of the work
>they've been doing at the PAM 2001 conf and have been keeping an eye on
>them since. After the presentation I'll sit down and see what he has to
>say about this issue and post some highlights here.
>
>
>On May 21, 2004, at 11:58 AM, snort user wrote:
>
>>Hi,
>>
>>Ive snipped out some of the recent posts to this thread. Weve been doing
>>extensive research into snort speeds at my University and to me it seems
>>like these 2 posts are completely innaccurate and absurb. Chad claims to
>>capture all traffic with all rules and preprocessors with a $2500 piece
>>of hardware, while if you buy a $50,000 solution from Sourcefire(home of
>>the creator of snort) you can only get 1 Gig and they disable rules and
>>preprocessors
>>(http://osec.neohapsis.com/results/ni...re-ns3020f-2.6
>>-06.25.2003/productinfo.html).
>>And then when Chris asked you your specs on your box you differ him to
>>TopLayer.
>>
>>Even getting 800 Mb/s as Rafael said is not impossible but really is not
>>feasible without hardcore kernel modifcation and maybe even silicon chips
>>and ASIC cards.
>>
>>Would either of you like to share how your able to do this, I mean the
>>technologies and hardware you using? Also how do you verify these
>>results?
>>
>>-- UoC --
>>
>>
>>-- snip Rafael Ortega--
>>>I'm currently snorting close to 800Mbps with no problem. What to do
>>>with
>>>the amount of info, is another story. I tried ACID, but after 24 hours
>>>and
>>>700,000 events registered, the data base becomes too slow, even after
>>>indexing certain reference fields.
>>-- end snip --
>>
>>-- snip Kreimendahl, Chad --
>>>FWIW... I've got systems that are easily handling between 3-4Gbps each.
>>>That's partially hardware, partially OS, and a little tiny config work.
>>>Very near to all rules enabled on these interfaces, as well as all of
>>>the preprocessors (minus the broken ones), and a database output plugin.
>>>
>>>0 dropped packets. If you check the archives for this list, you'll
>>>find discussions about kernels that can do polling against network
>>>devices, and how this enhances snort performance on high speed links
>>>(network performance in general, really). I believe I mention the OSes,
>>>maybe some config info and hardware used.
>>-- end snip --
>>
>>________________________________________________ _________________
>>FREE pop-up blocking with the new MSN Toolbar – get it now!
>>http://toolbar.msn.click-url.com/go/...ave/direct/01/
>>
>>
>>
>>-------------------------------------------------------
>>This SF.Net email is sponsored by: Oracle 10g
>>Get certified on the hottest thing ever to hit the market... Oracle 10g.
>>Take an Oracle 10g class now, and we'll give you the exam FREE.
>>http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
>>______________________________________________ _
>>Snort-users mailing list
>>Snort-users@lists.sourceforge.net
>>Go to this URL to change user options or unsubscribe:
>>https://lists.sourceforge.net/lists/...fo/snort-users
>>Snort-users list archive:
>>http://www.geocrawler.com/redir-sf.p...st=snort-users
>

__________________________________________________ _______________
Stop worrying about overloading your inbox - get MSN Hotmail Extra Storage!
http://join.msn.click-url.com/go/onm...ave/direct/01/



-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g.
Take an Oracle 10g class now, and we'll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
Snort-users mailing list
Snort-users@lists.sourceforge.net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/...fo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.p...st=snort-users