Re: synchronous versus asynchronous

So basically if i understand correctly async is using select with
nonblocking sockets and a large set of file descriptors? I assume you
use that since you stated that net-snmp is not thread-safe. And in
addition you just send a whole bunch of udp sendto's in a event loop?

that's really something i would like to have. basically i want to
write a function that does async snmp and in that same function it
calculates the netblock it needs to snmp to by supplying a net/prefix
as argument.

However, i feel really stupid when looking at the net-snmp library or
even example code. It seems bloody complicated :-). I'm now using a
modified version of the sync demo app to use with snmpv1 (Seems to
work for snmpv2 to?) but as stated before, it sucks up cpu bigtime if
it finds a host who it cannot connect to.

Greets, and thanks,
Alef
On Nov 14, 2006, at 7:53 PM, Steve Friedl wrote:

> On Tue, Nov 14, 2006 at 07:10:38PM +0100, Alef Veld wrote:
>> If i wanted to discover a large amount of hosts, which method would
>> be better. How does asynchronous work? Does it just send a bunch of
>> udp packets in a threaded fashion?
>>
>> Another thing; as i start my foray in this snmp world :-), there seem
>> to be _almost_ no librarys who deal directly with snmp. Almost all
>> apps use a library like ucd or net-snmp. Why is this ? Although i'd
>> rather code my own snmp client software i recognize the work and
>> options that have been put into net-snmp so i will probably decide
>> not to invent the wheel again.
>>
>> It's just that i find the net-snmp library pretty difficult to
>> understand, and i wonder if there are some bare bones snmp client
>> whose only hook is the systems headers/libraries.
>
> The clear winner here is async: you can poll hundreds of hosts from
> a modest machine with a single thread. This is because the great bulk
> of the time spent by your program is *waiting for I/O*, and when it
> works synchronously, this time is serialized.
>
> You can ameliorate this somewhat by using multiple threads, but that
> still serializes within a single thread. Threading introduces its own
> complexity - especially since I don't think that NET-SNMP is fully
> threadsafe - and this doesn't scale to hundreds of hosts.
>
> I've seen single-threaded sync applications that have a very hard time
> polling 100 hosts in 5 minutes, especially if one of them is down -
> those retries and timeouts are all fully serialized, and your app is
> doing nothing most of the time.
>
> With async, I/O waits are 100% parallized, so you max out either on
> your network bandwidth, or your CPU, but not on wall-clock time.
>
> The core of your application will be an event loop. You'll have some
> concept of a work-to-do list (hosts with lists of OIDs to request).
> In each loop, you send as many requests as you can (up to some
> reasonable limit), and the library keeps track of which ones are
> waiting for a response.
>
> Then you crank the I/O with a select() call, and hand the results to
> the SNMP library. It runs through the list of file descriptors which
> have fired, reads the responses, and matches them up with the requests
> it's kept track of.
>
> When it matches this response up, it fires your callback to process
> it,
> and you'll take the response PDU, decode it, and store the data
> somewhere.
> Then you tell the library that this request has been satisfied, and it
> frees up one slot for another request you'll submit at the next loop.
>
> Timeouts are handled as well, with the same callback mechanisms.
>
> Make no mistake: async processing is much, much more complicated, and
> even those who have been writing communications controllers for 20
> years
> *cough* me *cough* have some slow slogging.
>
> But async is a slam-dunk win for performance.
>
> Some time ago I wrote an async *DNS* resolver to crank through Apache
> log files (doing IP to name resolutions), and it was able to do on the
> order of 100 resolutions per second on an old slow machine.
>
> fastzolver - Very fast Asynchronous DNS resolver for Webalizer
> http://www.unixwiz.net/tools/fastzolver.html
>
> In fact, most async apps require a way to pace themselves so they
> don't
> blow out the infrastructure.
>
> Steve (async kind of guy)
>
> --
> Stephen J Friedl | Security Consultant | UNIX Wizard | +1 714
> 544-6561
> www.unixwiz.net | Tustin, Calif. USA | Microsoft MVP |
> steve@unixwiz.net
>


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?p...rge&CID=DEVDEV
_______________________________________________
Net-snmp-coders mailing list
Net-snmp-coders@lists.sourceforge.net
https://lists.sourceforge.net/lists/...et-snmp-coders