NAME
hobbitd_rrd - hobbitd worker module for updating Xymon RRD files
SYNOPSIS
hobbitd_channel --channel=status hobbitd_rrd [options]
hobbitd_channel --channel=data hobbitd_rrd [options]
DESCRIPTION
hobbitd_rrd is a worker module for hobbitd, and as such it is normally
run via the hobbitd_channel(8) program. It receives "status" and "data"
messages from hobbitd via stdin, and updates the RRD databases used to
generate trend-graphs.
Clients can send data to Xymon using both status- and data- messages.
So you will normally run two instances of this module, once for the
"status" channel and once for the "data" channel.
hobbitd_rrd understands data sent by the LARRD 0.43c client-side
scripts (the so-called "bottom-feeder" scripts). So you still want to
install the LARRD bottom-feeders on the clients you monitor.
Note: For certain types of data, the RRD files used by Xymon are
imcompatible with those generated by the Big Brother LARRD add-on. See
the COMPATIBILITY section below.
OPTIONS
--debug
Enable debugging output.
--rrddir=DIRECTORY
Defines the directory where the RRD-files are stored.
hobbitd_rrd will use the location pointed to by the BBRRDS
environment if this option is not present.
--extra-script=FILENAME
Defines the script that is run to get the RRD data for tests
that are not built into hobbitd_rrd. You must also specify which
tests are handled by the external script in the --extra-tests
option. This option can only be given once, so the script must
handle all of the external test-data. See the CUSTOM RRD DATA
section below. Note that this is NOT needed if your custom
graphs are generated by the NCV (Name Colon Value) module
described below, it is only required for data where you have a
custom script to parse the status message and extract the data
that is put into the graph.
--extra-tests=TEST[,TEST]
List of testnames that are handled by the external script. See
the CUSTOM RRD DATA section below. Note that NCV graphs should
NOT be listed here, but in the TEST2RRD environment variable -
see below.
ENVIRONMENT
TEST2RRD
Defines the mapping between a status-log columnname and the
corresponding RRD database format. This is normally defined in
the hobbitserver.cfg(5) file.
BBRRDS Default directory where RRD files are stored.
NCV_testname
Defines the types of data collected by the "ncv" module in
hobbitd_rrd. See below for more information.
SPLITNCV_testname
The same as NCV_testname, but keeps the data into separate
files. That is, it creates one rrd file per "NAME : value" line
found in the status message. It is useful when the list of NCV
lines is varying.
TRACKMAX
Comma-separated list of columnname for which you want to keep
the maximum values along with the default average values. This
only works
for the NCV backend.
COLLECTED DATA
The following RRD-file datasets are generated by hobbitd_rrd:
la Records the CPU load average. Data is collected from the "cpu"
status report. Requires that a Xymon client is running on the
monitored server.
disk Records the disk utilization. Data is collected from the "disk"
status report. Requires that a Xymon-compatible client is
running on the monitored server.
memory Records memory- and swap-utilization. Data is collected from the
"memory" status report. If no "memory" status is reported, it
will use the data from the Win32 client "cpu" status report to
generate this dataset. Requires that a Xymon-compatible client
is running on the monitored server.
netstat
Records TCP and UDP statistics. Data is collected from the
"netstat" status report; however, this data is often sent via
the Xymon "data" protocol, so there need not be a "netstat"
column visible on the Xymon display. To get these data, the
LARRD netstat bottom-feeder script must be running on the
monitored server.
vmstat Records system performance metrics from the "vmstat" command.
Data is collected from the "vmstat" status report; however, this
data is often sent via the Xymon "data" protocol, so there need
not be a "vmstat" column visible on the Xymon display. To get
these data, the LARRD vmstat bottom-feeder script must be
running on the monitored server.
tcp Response-time metrics from all of the Xymon network tests are
recorded in the "tcp" RRD.
apache Apache server performance metrics, taken from the "apache" data
report. See the description of the apache keyword in bb-hosts(5)
for details.
sendmail
Sendmail server performance metrics, taken from the "mailstats"
output. To get these data, the LARRD sendmail bottom-feeder
script must be running on the monitored server.
mailq Mail queue size. To get these data, the LARRD nmailq bottom-
feeder script must be running on the monitored server.
bea BEA Weblogic performance data. This is an experimental set of
data collected from BEA Weblogic servers via SNMP, by the
"beastats" tool included with Xymon.
iishealth
IIS webserver performance data, collected by the "iishealth"
script. This script is a client-side add-on available from the
www.deadcat.net archive.
temperature
Temperature data, collected with the temperature script from
www.deadcat.net. To get these data, the temperature script must
be running on the monitored server.
ntpstat
Tracks the deviation between the local system time and an NTP
server, using the output from the "ntpq -c rv" command. A
simple script to collect these data is included in the Xymon
contrib/ directory.
citrix Tracks the number of active sessions on a Citrix server using
the "query session" command. An extension for the BBNT client
that generates data for this graph is in the Xymon contrib/
directory.
CUSTOM RRD DATA IN NAME-COLON-VALUE (NCV) FORMAT
Many data-collection scripts report data in the form "NAME : value" or
"NAME = value". So a generic module in hobbitd_rrd allows for easy
tracking of this type of data.
The "ncv" module will automatically detect all occurrences of a "NAME :
value" or "NAME = value" string in a status message, and generate an
RRD file holding all of the name/value data found in the message
(unless you use SPLITNCV, see above). The colon- or equal-sign must be
present - if there is only whitespace, this module will fail.
Only the valid letters (A-Z, a-z) and digits (0-9) are used in the
dataset names; whitespace and other characters are stripped off
automatically. Only the first 19 characters of a dataset name are used
(this is an RRD limitation). Underscore ’_’ is not allowed, even though
RRDtool permits this, and will be stripped from the name.
When using the alternative SPLITNCV_testname, the dataset name is not
limited in length, and non-valid characters are changed to underscores
instead of being stripped off. The dataset inside the resulting rrd
file is always "lambda".
Note that each "NAME : value" must be on a line by itself. If you have
a custom script generating the status- or data-message that is fed into
the NCV handler, make sure it inserts a newline before each of the
data-items you want to track.
To enable the ncv module for a status, add a "COLUMNNAME=ncv" to the
TEST2RRD setting and the COLUMNNAME to the GRAPHS setting in
hobbitserver.cfg(5) , then restart Xymon. Xymon will now send all
status-messages for the column COLUMNNAME through the hobbitd_rrd ncv-
handler.
The name of the RRD file will be COLUMNNAME.rrd. When using SPLITNCV,
the name of the RRD file will be COLUMNAME,DATASETNAME.rrd.
By default, all of the datasets are generated as the RRD type "DERIVE"
which works for all types of monotonically increasing counters. If you
have data that are of the type GAUGE, you can override the default via
an environment variable NCV_COLUMNNAME (or SPLITNCV_COLUMNAME).
E.g. if you are using the bb-mysqlstatus script from www.deadcat.net to
collect data about your MySQL server, it generates a report in the
column called "mysql". One data item is the average number of
queries/second, which must be logged in the RRD file as type "GAUGE".
To do that, add the following to hobbitserver.cfg:
NCV_mysql="Queriespersecondavg:GAUGE"
If you have multiple datasets that you myst define, add them to the
environment variable separated by commas, e.g.
NCV_mysql="Uptime:NONE,Queriespersecondavg:GAUGE"
The dataset type "NONE" used above causes hobbitd_rrd to ignore this
data, it is not included in the RRD file.
You can use "*" as the dataset name to match all datasets not listed.
E.g.
NCV_weather="Rain:DERIVE,*:GAUGE"
will cause the "Rainfall" dataset to be of type DERIVE, and all others
of type GAUGE. If you want to track only a few of the variables in your
data, you can use "*:NONE" to drop any dataset not explicitly listed.
For a more detailed "how to" description, see the on-line HTML
documentation of "How to create graph custom data" available in the
Help menu section on your Xymon server.
CUSTOM RRD DATA VIA SCRIPTS
hobbitd_rrd provides a simple mechanism for adding custom graphs to the
set of data collected on your Xymon server. By adding the "--extra-
script" and "--extra-tests" options, data reported to Xymon from
selected tests are passed to an external script, which can define the
RRD data-sets to store in an RRD file.
NOTE: For performance reasons, you should not use this mechanism for
large amounts of data. The overhead involved in storing the received
message to disk and launching the script is significantly larger than
the normal hobbitd_rrd overhead. So if you have a large number of
reports for a given test, you should consider implementing it in C and
including it in the hobbitd_rrd tool.
Apart from writing the script, You must also add a section to
hobbitgraph.cfg(5) so that hobbitgraph.cgi(1) knows how to generate the
graph from the data stored in the RRD file. To make the graphs
actually show up on the status-page and/or the "trends" page, add the
name of the new graph to the TEST2RRD and/or GRAPHS setting in
hobbitserver.cfg(5).
The script is invoked for each message that arrives, where the test-
name matches one of the testnames given in the "--extra-tests" option.
The script receives three commandline parameters:
Hostname
The name of the host reporting the data.
Testname
The name of the test being reported.
Filename
File containing the data that was reported. This file is
generated for you by hobbitd_rrd, and is also deleted
automatically after your script is finished with it.
The script must process the data that is reported, and generate the
following output:
RRD data-set definitions
For each dataset that the RRD file holds, a line beginning with
"DS:" must be output. If multiple data-sets are used, print one
line for each dataset.
Data-set definitions are described in the rrdcreate(1)
documentation, but a common definition for e.g. tracking the
number of users logged on would be "DS:users:GAUGE:600:0:U".
"users" is the name of the dataset, "GAUGE" is the datatype,
"600" is the longest time allowed between updates for the data
to be valid, "0" is the minimum value, and "U" is the maximum
value (a "U" means "unknown").
RRD filename
The name of the RRD file where the data is stored. Note that
Xymon stores all RRD files in host-specific directories, so
unlike LARRD you should not include the hostname in the name of
the RRD file.
RRD values
One line, with all of the data values collected by the script.
Data-items are colon-delimited and must appear in the same
sequence as your data-set definitions, e.g. if your RRD has two
datasets with the values "5" and "0.4" respectively, then the
script must output "5:0.4" as the RRD values.
In some cases it may be useful to define a dataset even though
you will not always have data for it. In that case, use "U"
(unknown) for the value.
If you want to store the data in multiple RRD files, the script
can just print out more sequences of data-set definitions, RRD
filenames and RRD values. If the data-set definitions are
identical to the previous definition, you need not print the
data-set definitions again - just print a new RRD filename and
value.
The following sample script for tracking weather data shows how to use
this mechanism. It assumes the status message include lines like these:
green Weather in Copenhagen is FAIR
Temperature: 21 degrees Celsius
Wind: 4 m/s
Humidity: 72 %
Rainfall: 5 mm since 6:00 AM
A shell-script to track all of these variables could be written like
this:
#!/bin/sh
# Input parameters: Hostname, testname (column), and messagefile
HOSTNAME="$1"
TESTNAME="$2"
FNAME="$3"
if [ "$TESTNAME" = "weather" ]
then
# Analyze the message we got
TEMP=‘grep "^Temperature:" $FNAME | awk ’{print $2}’‘
WIND=‘grep "^Wind:" $FNAME | awk ’{print $2}’‘
HMTY=‘grep "^Humidity:" $FNAME | awk ’{print $2}’‘
RAIN=‘grep "^Rainfall:" $FNAME | awk ’{print $2}’‘
# The RRD dataset definitions
echo "DS:temperature:GAUGE:600:-30:50"
echo "DS:wind:GAUGE:600:0:U"
echo "DS:humidity:GAUGE:600:0:100"
echo "DS:rainfall:DERIVE:600:0:100"
# The filename
echo "weather.rrd"
# The data
echo "$TEMP:$WIND:$HMTY:$RAIN"
fi
exit 0
COMPATIBILITY
Some of the RRD files generated by hobbitd_rrd are incompatible with
the files generated by the Big Brother LARRD add-on:
vmstat The vmstat files with data from Linux based systems are
incompatible due to the addition of a number of new data-items
that LARRD 0.43 do not collect, but hobbitd_rrd does. This is
due to changes in the output from the Linux vmstat command, and
changes in the way e.g. system load metrics are reported.
netstat
All netstat files from LARRD 0.43 are incompatible with
hobbitd_rrd. The netstat data collected by LARRD is quite
confusing: For some types of systems LARRD collects packet-
counts, for others it collects byte- counts. hobbitd_rrd uses a
different RRD file-format with separate counters for packets and
bytes and tracks whatever data the system is reporting.
SEE ALSO
hobbitd_channel(8), hobbitd(8), hobbitserver.cfg(5), xymon(7)