[rrd-users] multiple RRAs of same type

Hi folks,
I'm having difficulty understanding the purpose of having multiple
RRAs of the same type, say AVERAGE, with different values.
example: RRA:AVERAGE:.5:1:288 RRA:AVERAGE:.5:60:4320 RRA:AVERAGE:.5:1440:365
Can someone explain what purpose this serves, and how they're utilized, thanks.

--
aRDy Music and Rick Dicaire present:
http://www.ardynet.com
http://www.ardynet.com:9000/ardymusic.ogg.m3u

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

R Dicaire wrote:

>I'm having difficulty understanding the purpose of having multiple
>RRAs of the same type, say AVERAGE, with different values.
>example: RRA:AVERAGE:.5:1:288 RRA:AVERAGE:.5:60:4320 RRA:AVERAGE:.5:1440:365
>Can someone explain what purpose this serves, and how they're utilized,

It allows you a simple tradeoff between storage space (and associated
processing time) vs length of history. Typically, you aren't too
bothered by knowing to the minute what your data did a whole year
ago, so an average over (say) 2 hour or 12 hour or 24 hour periods
will suffice. But for recent traffic you want as much detail as
possible.

Lets consider a case where your step size is one minute. That's 60
samples every hour, 1440 samples every day, and over half a million
samples every year. Not only would that take a fair bit of storage (I
have an rrd with 512 data series in it), but it would take quite a
bit of processing to reduce it down to a relatively low resolution
graph for the whole year.

Taking the examples you've given above :
RRA:AVERAGE:.5:1:288
Gives you one minute values for 288 minutes (4.8 hours)

RRA:AVERAGE:.5:60:4320
Gives you one hour values for 4320 hours (180 days)

RRA:AVERAGE:.5:1440:365
Gives you one day values for 365 days

If you plot a graph for a year then you can use the low resolution
data that is already stored (less processing required) and it's only
required storing 365 values instead of 525,000 values (less storage
required).

For the last (almost) 1/2 year, you can still plot a graph with
detail down to hourly values. But for the last 4.8 hours you can plot
your one minute values.


There isn't actually anything to stop you keeping every one minute
value for a whole year if your application requires it - it would
allow you to 'drill down'* as I've seen done. RRD will handle it,
it'll just mean more storage and more processing required.

* I've seen graphs done where you can select time span on the graph
and it will then redraw showing just that time span. You can select a
time span on that and it will redraw showing just that selection -
and so on until you reach the limits of the data resolution stored.

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

On Mon, May 19, 2008 at 2:49 AM, Simon Hobson <linux@thehobsons.co.uk> wrote:
> Taking the examples you've given above :
> RRA:AVERAGE:.5:1:288
> Gives you one minute values for 288 minutes (4.8 hours)

I see....if the rrd is updated every 5 min, wouldn't that make one
minute values pointless, or inaccurate?

--
aRDy Music and Rick Dicaire present:
http://www.ardynet.com
http://www.ardynet.com:9000/ardymusic.ogg.m3u

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

R Dicaire wrote:
>On Mon, May 19, 2008 at 2:49 AM, Simon Hobson <linux@thehobsons.co.uk> wrote:
>> Taking the examples you've given above :
>> RRA:AVERAGE:.5:1:288
>> Gives you one minute values for 288 minutes (4.8 hours)
>
>I see....if the rrd is updated every 5 min, wouldn't that make one
>minute values pointless, or inaccurate?

Yes, which is why if you were planning on only updating every five
minutes it wouldn't usually make sense to have a step size of one
minute.

In all cases, your step size and sample frequency should be related
to the data you intend to put in - then the consolidation functions
you use should be related to what you want to get out later.

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

On Mon, May 19, 2008 at 07:49:39AM +0100, Simon Hobson wrote:
> R Dicaire wrote:
>
> >I'm having difficulty understanding the purpose of having multiple
> >RRAs of the same type, say AVERAGE, with different values.
> >example: RRA:AVERAGE:.5:1:288 RRA:AVERAGE:.5:60:4320 RRA:AVERAGE:.5:1440:365
> >Can someone explain what purpose this serves, and how they're utilized,
>
> It allows you a simple tradeoff between storage space (and associated
> processing time) vs length of history. Typically, you aren't too
> bothered by knowing to the minute what your data did a whole year
> ago, so an average over (say) 2 hour or 12 hour or 24 hour periods
> will suffice. But for recent traffic you want as much detail as
> possible.

It is also useful to have multiple RRAs spanning the same amount
of time but only in a different resolution ...

> Lets consider a case where your step size is one minute. That's 60
> samples every hour, 1440 samples every day, and over half a million
> samples every year. Not only would that take a fair bit of storage (I
> have an rrd with 512 data series in it), but it would take quite a
> bit of processing to reduce it down to a relatively low resolution
> graph for the whole year.

.... to overcome above argument.

E.g. RRA:AVERAGE:.5:1:525600 RRA:AVERAGE:.5:60:8760 RRA:AVERAGE:.5:1440:365

> There isn't actually anything to stop you keeping every one minute
> value for a whole year if your application requires it - it would
> allow you to 'drill down'* as I've seen done. RRD will handle it,
> it'll just mean more storage and more processing required.

Now consider an application which allows you to 'drill down'.

It can start by showing an entire year worth of data, one day on
one pixel column. If available, RRDtool can take this data from
the RRA which stores 86400 seconds per CDP (1440 PDPs of 60 seconds
each).

Zoom in on the beginning of the year, and RRDtool may use the RRA
which stores 3600 seconds per CDP. Zoom in again and RRDtool can
use the RRA storing 60 seconds per CDP.


If there's only one AVERAGE RRA available, the one with 60 seconds
per CDP, then RRDtool has no choice but to take 288 CDPs together,
doing so 365 times (one time for each pixel row). This takes time
and CPU resources.

Sometimes this is okay, if such a query is a very excercise. But
if you expect many of such queries, it is best to have the data
ready when the user asks for it.



Multiple RRAs: RRDtool will have the data available when the user
needs it. This means a fast response, at the expense of more disk
space required and extra resources used each time an update occurs.

One RRA: RRDtool will have to consolidate the data at graph time,
meaning a slower response and a brief moment of high CPU and disk
resources used. It does save disk space.



--
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users

On Mon, May 19, 2008 at 12:46:26PM -0400, R Dicaire wrote:

> > Taking the examples you've given above :
> > RRA:AVERAGE:.5:1:288
> > Gives you one minute values for 288 minutes (4.8 hours)
>
> I see....if the rrd is updated every 5 min, wouldn't that make one
> minute values pointless, or inaccurate?

No, not necessarily.

Consider updates to occur exactly four minutes past the hour.

If you have stepsize 300 (5 minutes) then RRDtool will have to normalize
data to fit into a 5-minute slot.

If you have stepsize 60 (1 minute) then RRDtool will not have to normalize
this data.

Especially for MAX and MIN, this means having a smaller step size is
*more* accurate, not *less*.

In fact, you can have step size 1, and an RRA with 60 (or even 300)
PDPs per CDP as RRA with the best resolution.

Try to figure out how normalization and consolidation work (my site
may help).

Create two RRDs:

Both having MIN, AVERAGE and MAX RRAs.
Both spanning the same amount of time (this may be short for this
excercise).
One RRD has step size 1, and stores 60 PDP per CDP
One RRD has step size 60, and stores 1 PDP per CDP
Both have a high enough heartbeat.

Now update both databases with the same timestamp and rate, and you
should spot a difference unless you happen to update with timestamps
which are a whole multiple of 60.

---------------------------------------------------------------------------
rrdtool create test1.rrd --start 1211234400 \
--step 1 \
DS:x:GAUGE:600:U:U \
RRA:MIN:0:60:5 \
RRA:AVERAGE:0:60:5 \
RRA:MAX:0:60:5
rrdtool create test2.rrd --start 1211234400 \
--step 60 \
DS:y:GAUGE:600:U:U \
RRA:MIN:0:1:5 \
RRA:AVERAGE:0:1:5 \
RRA:MAX:0:1:5

rrdtool update test1.rrd 1211234460:0
rrdtool update test2.rrd 1211234460:0

rrdtool update test1.rrd 1211234519:100
rrdtool update test2.rrd 1211234519:100

rrdtool update test1.rrd 1211234580:50
rrdtool update test2.rrd 1211234580:50

echo
echo Showing MIN rates
rrdtool fetch test1.rrd MIN --start 1211234400 --end 1211234580
rrdtool fetch test2.rrd MIN --start 1211234400 --end 1211234580

echo
echo Showing AVERAGE rates
rrdtool fetch test1.rrd AVERAGE --start 1211234400 --end 1211234580
rrdtool fetch test2.rrd AVERAGE --start 1211234400 --end 1211234580

echo
echo Showing MAX rates
rrdtool fetch test1.rrd MAX --start 1211234400 --end 1211234580
rrdtool fetch test2.rrd MAX --start 1211234400 --end 1211234580
---------------------------------------------------------------------------

--
Alex van den Bogaerdt
http://www.vandenbogaerdt.nl/rrdtool/

_______________________________________________
rrd-users mailing list
rrd-users@lists.oetiker.ch
https://lists.oetiker.ch/cgi-bin/listinfo/rrd-users