Re: [Snort-users] making snort go fast

permalink · #1 (**permalink**) 02-21-2008

This is a multipart message in MIME format.
--===============0119853777==
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_041C_01C874B0.20ED5530"
Content-Language: en-us

This is a multipart message in MIME format.
------=_NextPart_000_041C_01C874B0.20ED5530
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: 7bit

Snort-users,

First, let me say that I work for Netronome, one of the companies
mentioned. I agree with Matt's comments on performance and benchmark
testing and the variables involved. You noted that we claim 6 to 8Gbps of
Snort. Vendor claims should be scrutinized. Our performance is ~6.5Gbps
running SNORT with the default rule set examining http traffic with an
average packet size of 440bytes. We can get up to 8Gbps at larger packet
sizes. Additional acceleration beyond 8Gbps is possible by enabling our
'cut-through' for flows that SNORT doesn't need to see (VoIP payloads,
encrypted traffic, etc). This is accomplished via our Open Appliance
which consists of a Netronome Flow Engine PCIe accelerator card in an
Intel x86 platform. If you have any questions, you should contact one of
our engineers.

-dan

David Williams wrote:

> Hello List,

Hello David.

>

> I'm trying to get Snort to go very fast. Has anybody evaluated any of

> these solutions below. I know these vendors are claiming multi-gig

> Snort, but I'm skeptical of vendor claims (obviously).

>

> - Endace's Ninja appliance (they claim 10G, but the webcast seemed to

> contradict this claim by stating just under 2G)

>

Glad to hear someone was listening to that webcast, I was the speaker.

:) Unfortunately I apparently didn't make my point clear enough. The

point of the benchmark I did wasn't to get a max throughput number. In

fact it was specifically NOT to get a max throughput number.

Let me get on my IDS benchmarking soap-box for a moment:

There really is NO way to make a fair and true representation of how an

IDS device of ANY sort (accelerated or not) will perform in Your

environment with YOUR traffic spread and YOUR ruleset. Just NO way,

because those three major variables have significant impact on

throughput. And when I say significant I mean that any single one of

them can bring a nice fast box down to a max throughput of 50meg/sec all

by itself. Down off my soapbox.

So that point made, in all of the benchmarking I've done my goal has

always been specifically NOT to find some mythical max throughput

number. It might look cool, but it means absolutel NOTHING to any

environment beside the test rig. Really really absolutely nothing. With

one rule and stripped down preprocessors and all homogenous traffic you

can make an IDS appliance push traffic as fast as it's bus can move

packets. But that means nothing, abso-frikkin-lutely nothing.

The goal of this endace benchmark I did was to take a baseline with a

REALLY high load ruleset (to make sure we tested all aspects of snort

and it's preprocs, etc) and a VERY hostile traffic corpus (a university

network capture with lots of attacks and 'liberal' users). The baseline

in that test was about 100meg/sec before packets were lost without any

acceleration in the mix.

Then we added the acceleration advantages of the particular platform,

but kept the same traffic and the same really high load ruleset and same

snort config, and found the new max. The percentage gain is the number

we wanted, which in that case was almost 16-fold increase in throughput.

What that number actually was wasn't important. How much better it was

is what was important.

So that tells us that in an environment/ruleset/snortconfig combination

on a 3ghz processor that does about 100meg/sec will see a 16-fold

increase if they introduce this platform's acceleration features. Up to

1.54Gbps+ in this case.

So that gives me as an IDS shopper the information I need to compare.

For instance, if I have a sensor with my ruleset, my traffic load and my

snort config running about 250meg/sec comfortably on a 3ghz processor, I

can likely expect to comfortably reach 4Gbps (250meg * 16) as a rough

number to compare.

So bottom line: My philosophy is that max throughput numbers are

useless, because the variables in the test environment are just too wide

to mean anything to anyone. You have to get a percent gain over a

baseline to make a comparison. There are still flaws in this model, but

it's the best thing I've ever had to use and it's served well enough to

date.

BTW: The endace ninjaboxes I don't believe advertise they can DO 10gbps,

but that they have 10gpbs fiber slots (2x per box I think). If you run a

VERY slim ruleset and a very tuned snort on some really easy traffic, it

might get there. Just as any other device could with the right

environment. Remember to clarify with any salesguy you talk to that he

either does or does not mean to imply that a 10gig port means it can

process packets at 10gbps. I think many sales guys let that implication

go hoping no one notices.

Matt

> - Netronome Systems Open Appliance (claiming 6-8G)

>

> - Bivio Networks B7000 (claiming 10G)

>

> Anybody else I'm missing from the list of vendors claiming to make

> Snort go fast?

>

> thanks,

>

> Dave

------=_NextPart_000_041C_01C874B0.20ED5530
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

<html xmlns:v=3D"urn:schemas-microsoft-com:vml" xmlns:o=3D"urn:schemas-micr=
osoft-com:office:office" xmlns:w=3D"urn:schemas-microsoft-com:office:word" =
xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" xmlns=3D"http:=
//www.w3.org/TR/REC-html40">

<head>
<meta http-equiv=3DContent-Type content=3D"text/html; charset=3Dus-ascii">
<meta name=3DGenerator content=3D"Microsoft Word 12 (filtered medium)">
<style>

</style>

</head>

<body lang=3DEN-US link=3Dblue vlink=3Dpurple>

<div class=3DSection1>

Snort-users,<o:p></o:p>

<o:p> </o:p>

First,
let me say that I work for Netronome, one of the companies mentioned. =
I
agree with Matt's comments on performance and benchmark testing and the
variables involved.  You noted that we claim 6 to 8Gbps of Snort.&nbsp=
; Vendor
claims should be scrutinized.  Our performance is ~6.5Gbps running SNO=
RT
with the default rule set examining http traffic with an average packet siz=
e of
440bytes. We can get up to 8Gbps at larger packet sizes.  Additional a=
cceleration
beyond 8Gbps is possible by enabling our ‘cut-through’ for flow=
s
that SNORT doesn’t need to see (VoIP payloads, encrypted traffic,
etc).  This is accomplished via our Open Appliance which consists of a
Netronome Flow Engine PCIe accelerator card in an Intel x86 platform. =
If
you have any questions, you should contact one of our engineers.<o:p></o:p>=


<o:p> </o:p>

-dan<o:p></o:p>

<o:p> </o:p>

<o:p> </o:p>

David
Williams wrote:<o:p></o:p>

>
Hello List,<o:p></o:p>

<o:p> </o:p>

Hello
David.<o:p></o:p>

<o:p> </o:p>

>
<o:p></o:p>

>
I'm trying to get Snort to go very fast.  Has anybody evaluated any of=
<o:p></o:p>

>
these solutions below.  I know these vendors are claiming multi-gig<o:=
p></o:p>

>
Snort, but I'm skeptical of vendor claims (obviously).<o:p></o:p></p=
>

>
<o:p></o:p>

>
- Endace's Ninja appliance (they claim 10G, but the webcast seemed to<o:p><=
/o:p>

>
contradict this claim by stating just under 2G)<o:p></o:p>

>
<o:p></o:p>

<o:p> </o:p>

Glad
to hear someone was listening to that webcast, I was the speaker.<o:p></o:p=
>

:)
Unfortunately I apparently didn't make my point clear enough. The<o:p></o:p=
>

point
of the benchmark I did wasn't to get a max throughput number. In<o:p></o:p>=


fact
it was specifically NOT to get a max throughput number.<o:p></o:p></=
p>

<o:p> </o:p>

Let
me get on my IDS benchmarking soap-box for a moment:<o:p></o:p>

There
really is NO way to make a fair and true representation of how an<o:p></o:p=
>

IDS
device of ANY sort (accelerated or not) will perform in Your<o:p></o:p></sp=
an>

environment
with YOUR traffic spread and YOUR ruleset. Just NO way,<o:p></o:p></=
p>

because
those three major variables have significant impact on<o:p></o:p></p=
>

throughput.
And when I say significant I mean that any single one of<o:p></o:p><=
/p>

them
can bring a nice fast box down to a max throughput of 50meg/sec all<o:p></o=
:p>

by
itself. Down off my soapbox.<o:p></o:p>

<o:p> </o:p>

So
that point made, in all of the benchmarking I've done my goal has<o:p></o:p=
>

always
been specifically NOT to find some mythical max throughput<o:p></o:p></span=
>

number.
It might look cool, but it means absolutel NOTHING to any<o:p></o:p>=


environment
beside the test rig. Really really absolutely nothing. With<o:p></o:p></spa=
n>

one
rule and stripped down preprocessors and all homogenous traffic you<o:p></o=
:p>

can
make an IDS appliance push traffic as fast as it's bus can move<o:p></o:p><=
/span>

packets.
But that means nothing, abso-frikkin-lutely nothing.<o:p></o:p>

<o:p> </o:p>

The
goal of this endace benchmark I did was to take a baseline with a<o:p></o:p=
>

REALLY
high load ruleset (to make sure we tested all aspects of snort<o:p></o:p></=
span>

and
it's preprocs, etc) and a VERY hostile traffic corpus (a university<o:p></o=
:p>

network
capture with lots of attacks and 'liberal' users). The baseline<o:p></o:p><=
/span>

in
that test was about 100meg/sec before packets were lost without any<o:p></o=
:p>

acceleration
in the mix.<o:p></o:p>

<o:p> </o:p>

Then
we added the acceleration advantages of the particular platform,<o:p></o:p>=


but
kept the same traffic and the same really high load ruleset and same<o:p></=
o:p>

snort
config, and found the new max. The percentage gain is the number<o:p></o:p>=


we
wanted, which in that case was almost 16-fold increase in throughput.<o:p><=
/o:p>

What
that number actually was wasn't important. How much better it was<o:p></o:p=
>

is
what was important.<o:p></o:p>

<o:p> </o:p>

So
that tells us that in an environment/ruleset/snortconfig combination<o:p></=
o:p>

on
a 3ghz processor that does about 100meg/sec will see a 16-fold<o:p></o:p></=
span>

increase
if they introduce this platform's acceleration features. Up to<o:p></o:p></=
span>

1.54Gbps+
in this case.<o:p></o:p>

<o:p> </o:p>

So
that gives me as an IDS shopper the information I need to compare.<o:p></o:=
p>

For
instance, if I have a sensor with my ruleset, my traffic load and my<o:p></=
o:p>

snort
config running about 250meg/sec comfortably on a 3ghz processor, I<o:p></o:=
p>

can
likely expect to comfortably reach 4Gbps (250meg * 16) as a rough<o:p></o:p=
>

number
to compare.<o:p></o:p>

<o:p> </o:p>

So
bottom line: My philosophy is that max throughput numbers are<o:p></o:p></s=
pan>

useless,
because the variables in the test environment are just too wide<o:p></o:p><=
/span>

to
mean anything to anyone. You have to get a percent gain over a<o:p></o:p></=
span>

baseline
to make a comparison. There are still flaws in this model, but<o:p></o:p></=
span>

it's
the best thing I've ever had to use and it's served well enough to<o:p></o:=
p>

date.<o:p></o:p>

<o:p> </o:p>

BTW:
The endace ninjaboxes I don't believe advertise they can DO 10gbps,<o:p></o=
:p>

but
that they have 10gpbs fiber slots (2x per box I think). If you run a<o:p></=
o:p>

VERY
slim ruleset and a very tuned snort on some really easy traffic, it<o:p></o=
:p>

might
get there. Just as any other device could with the right<o:p></o:p><=
/p>

environment.
Remember to clarify with any salesguy you talk to that he<o:p></o:p>=


either
does or does not mean to imply that a 10gig port means it can<o:p></o:p></s=
pan>

process
packets at 10gbps. I think many sales guys let that implication<o:p></o:p><=
/span>

go
hoping no one notices.<o:p></o:p>

<o:p> </o:p>

Matt<o:p></o:p>

<o:p> </o:p>

<o:p> </o:p>

>
- Netronome Systems Open Appliance (claiming 6-8G)<o:p></o:p>

>
<o:p></o:p>

>
- Bivio Networks B7000 (claiming 10G)<o:p></o:p>

>
<o:p></o:p>

>
Anybody else I'm missing from the list of vendors claiming to make<o:p></o:=
p>

>
Snort go fast?<o:p></o:p>

>
<o:p></o:p>

>
thanks,<o:p></o:p>

>
<o:p></o:p>

>
Dave<o:p></o:p>

<o:p> </o:p>

</div>

</body>

</html>

------=_NextPart_000_041C_01C874B0.20ED5530--

--===============0119853777==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
--===============0119853777==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
Snort-users mailing list
Snort-users@lists.sourceforge.net
Go to this URL to change user options or unsubscribe:
https://lists.sourceforge.net/lists/...fo/snort-users
Snort-users list archive:
http://www.geocrawler.com/redir-sf.p...st=snort-users
--===============0119853777==--

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode