Re: [rrd-users] Disk I/O

Hi Jeremy,

Yesterday Jeremy wrote:

> Hi,
>
> I'm doing a lot of graphing with RRDTOOL (using PNP for Nagios), updating
> over 10,000 RRD files every few minutes. Just over 6 GB of small .rrd files,
> about 400 writes/sec on average nonstop. Starting to run into disk I/O
> issues, it's keeping up ok but the system is getting less and less
> responsive as the I/O stays pegged at near 100%.
>
> Before I throw hardware at the problem (Gigabyte ramdisks etc), I was
> wondering a few things:
>
> 1) When you do an "rrdtool update" to add new data, does it have to rewrite
> the whole file, or does it just append to the file? Since it rolls up the
> old data I guess it needs to re-write the entire file each time? Either way
> I guess this is not really anything that's configurable that we could try to
> optimize but just curious what's going on under the hood.

rrdtool uses a round robin archive format ... when you write new
data only the data gets written to the file, no rewriting takes
place ...

make sure you use a current version of rrdtool, since there were
some fixes for linux read-a-head behaviour which reduce disk io
dramatically on linux 2.6 boxes ...

also note that you must have enough ram on your machine so that the
os can cache about 20 KByte worth of data for each rrd file in your
set ...

> 2) Is there any way to have rrdtool use a MySQL database to save all the
> data for the various files, instead of a ton of separate small files? Seems
> like that might cause less I/O overhead maybe if it was just doing a bunch
> of inserts but not rewriting all the individual files. I don't think this is
> really possible but I just want to make sure.

there is a patch floating around that enables rrdgraph to pull its
data from a database ...

10'000 rrd file updated every 5 minutes should be no problem given
you have enough ram and a current version of rrdtool.

cheers
tobi

> Thanks!
>

--
Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten
http://it.oetiker.ch tobi@oetiker.ch ++41 62 213 9902

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users