NAME
pmlogger_daily, pmlogger_check, pmlogger_merge - administration of
Performance Co-Pilot archive log files
SYNOPSIS
$PCP_BINADM_DIR/pmlogger_daily [-NoV] [-c control] [-k discard] [-m
addresses] [-s size] [-t want] [-x compress] [-X program] [-Y regex]
$PCP_BINADM_DIR/pmlogger_check [-NTV] [-c control]
$PCP_BINADM_DIR/pmlogger_merge [-fNV] [input-basename ... output-name]
DESCRIPTION
This series of shell scripts and associated control files may be used
to create a customized regime of administration and management for
Performance Co-Pilot (see PCPintro(1)) archive log files.
pmlogger_daily , pmlogger_check , and pmlogger_merge are equivalent to
the scripts cron.pmdaily, cron.pmcheck and cron.pmlogmerge which are
deprecated but preserved on IRIX for backward compatibility.
pmlogger_daily is intended to be run once per day, preferably in the
early morning, as soon after midnight as practicable. Its task is to
aggregate and rotate one or more sets of PCP archives. After some
period, old PCP archives are discarded. This period is 14 days by
default, but may be changed using the -k option. Two special values are
recognized for the period (discard), namely 0 to keep no archives
beyond the current one, and forever to prevent any archives being
discarded.
Archive data files can optionally be compressed after some period
(compress), to conserve disk space. This is particularly useful for
large numbers of pmlogger processes under the control of
pmlogger_check. The -x option specifies the number of days after which
to compress archive data files, and the -X option specifies the program
to use for compression - by default this is bzip2(1). Use of the -Y
option allows a regular expression to be specified causing files in the
set of files matched for compression to be omitted - this allows only
the data file to be compressed, and also prevents the program from
attempting to compress it more than once. The default regex is
".meta$|.index$|.Z$|.gz$|.bz2|.zip$" - such files are filtered using
the -v option to egrep(1).
In addition, if the PCP ‘‘notices’’ file ($PCP_LOG_DIR/NOTICES) is
larger than 20480 bytes, pmlogger_daily will rename the file with a
‘‘.old’’ suffix, and start a new ‘‘notices’’ file. The rotate
threshold may be changed from 20480 to size bytes using the -s option.
Use of the -m option causes pmlogger_daily to construct a summary of
the ‘‘notices’’ file entries which were generated in the last 24 hours,
and e-mail that summary to the set of space-separated addresses. This
daily summary is stored in the file $PCP_LOG_DIR/NOTICES.daily, which
will be empty when no new ‘‘notices’’ entries were made in the previous
24 hour period.
pmlogger_check may be run at any time, and is intended to check that
the desired set of pmlogger(1) processes are running, and if not to re-
launch any failed loggers.
pmlogger_merge is a wrapper script for pmlogmerge(1) that merges all of
the archive logs matching the input-basename arguments, and creates a
new archive using output-name as the base name for the physical files
that constitute an archive log. The input-basename arguments may
contain meta characters in the style of sh(1). If specified, the -f
option causes all of the input files to be removed once the output
archive has been created.
pmlogger_merge is used by pmlogger_daily.
To assist with debugging or diagnosing intermittent failures the -t
option may be used. This will turn on very verbose tracing (-VV) and
capture the trace output in a file named
$PCP_LOG_DIR/pmlogger/daily.datestamp.trace, where datestamp is the
time pmlogger_daily was run in the format YYYYMMDD.HH.MM. In addition,
the want argument will ensure that trace files created with -t will be
kept for want days and then discarded.
Both pmlogger_daily and pmlogger_check are controlled by a PCP logger
control file that specifies the pmlogger instances to be managed. The
default control file is $PCP_PMLOGGERCONTROL_PATH, but an alternate may
be specified using the -c option.
The control file should be customized according to the following rules
that define for the current version (1.1) of the control file format.
1. Lines beginning with a ‘‘#’’ are comments.
2. Lines beginning with a ‘‘$’’ are assumed to be assignments to
environment variables in the style of sh(1), and all text
following the ‘‘$’’ will be eval’ed by the script reading the
control file, and the corresponding variable exported into the
environment. This is particularly useful to set and export
variables into the environment of the administrative scripts,
e.g.
$ PMCD_CONNECT_TIMEOUT=20
Warning: The $PCP_PMLOGGERCONTROL_PATH file is a potential
security risk and should not be writable by any user other than
root.
3. There must be a version line of the form:
$ version=1.1
4. There should be one line in the control file for each pmlogger
instance of the form:
host y|n y|n directory args
5. Fields within a line of the control file are separated by one or
more spaces or tabs.
6. The first field is the name of the host that is the source of
the performance metrics for this pmlogger instance.
7. The second field indicates if this is a primary pmlogger
instance (y) or not (n). Since the primary logger must run on
the local host, and there may be at most one primary logger for
a particular host, this field can be y for at most one pmlogger
instance, in which case the host name must be the name of the
local host.
8. The third field indicates if this pmlogger instance needs to be
started under the control of pmsocks(1) to connect to a pmcd
through a firewall (y or n).
9. The fourth field is a directory name. All files associated with
this pmlogger instance will be created in this directory, and
this will be the current directory for the execution of any
programs required in the maintenance of those archives. A
useful convention is that primary logger archives for the local
host with hostname myhost are maintained in the directory
$PCP_LOG_DIR/pmlogger/myhost (this is where the default pmlogger
start-up script in $PCP_RC_DIR/pcp will create the archives),
while archives for the remote host mumble are maintained in
$PCP_LOG_DIR/pmlogger/mumble.
10. All other fields are interpreted as arguments to be passed to
pmlogger(1) and/or pmnewlog(1). Most typically this would be
the -c option.
The following sample control lines specify a primary logger on the
local host (bozo), and a non-primary logger to collect and log
performance metrics from the host boing.
$version=1.1
bozo y n $PCP_LOG_DIR/pmlogger/bozo -c config.default
boing n n $PCP_LOG_DIR/pmlogger/boing -c ./pmlogger.config
Typical crontab(5) entries for periodic execution of pmlogger_daily and
pmlogger_check are given in $PCP_VAR_DIR/config/pmlogger/crontab and
shown below. Due to the file permissions involved, these should be
added to the crontab for root if automated PCP archive log maintenance
is desired.
# daily processing of archive logs
14 0 * * * $PCP_BINADM_DIR/pmlogger_daily
# every 30 minutes, check pmlogger instances are running
28,58 * * * * $PCP_BINADM_DIR/pmlogger_check
Alternate redirection of the output from the cron(1) execution of the
scripts may be controlled as follows:
(1) The -V option to the scripts will enable verbose tracing of their
activity. By default the scripts generate no output unless some
error or warning condition is encountered.
(2) To redirect the e-mail from cron(1) away from the root login,
+ Instead of using the ‘‘root’’ login, create a special IRIX login
with uid equal to 0, e.g. su_pcp. The password may be locked
and/or the shell invalid to prevent login or su (1), but the home
directory should exist. For example the following entry in
/etc/passwd:
su_pcp:x:0:0:PCP Housekeeping:/usr/people/su_pcp:/dev/null
+ Create a $HOME/.forward for su_pcp, else an entry in
/usr/lib/aliases for su_pcp, redirecting the e-mail to a real
user or user(s).
+ Add the ‘‘crontab’’ entries above to the crontab file for su_pcp
not root, e.g. in the file /usr/spool/cron/crontabs/su_pcp
The -N option enables a ‘‘show me’’ mode, where the actions are echoed,
but not executed, in the style of ‘‘make -n’’. Using -N in conjunction
with -V maximizes the diagnostic capabilities for debugging.
By default all possible archives will be merged. The -o option
reinstates the old behaviour in which only yesterday’s archives will be
considered as merge candidates.
The -T option provides a terser form of output for pmlogger_check that
is most suitable for a pmlogger ‘‘farm’’ where many instances of
pmlogger are expected to be running.
The script $PCP_BINADM_DIR/pmlogger_daily could be copied and modified
to implement a site-specific procedure for end-of-week and/or end-of-
month management for a set of PCP archives.
FILES
$PCP_PMLOGGERCONTROL_PATH
the PCP logger control file
Warning: this file is a potential security risk and should
not be writable by any user other than root.
$PCP_VAR_DIR/config/pmlogger/crontab
sample crontab for automated script execution by root
$PCP_VAR_DIR/config/pmlogger/config.default
default pmlogger configuration file for the local primary
logger
$PCP_VAR_DIR/config/pmlogger/config.*
other pmlogger configuration files suited for particular PCP
monitoring tools, add-on products and application
environments
$PCP_LOG_DIR/pmlogger/hostname
default location for archives of performance information
collected from the host hostname
$PCP_LOG_DIR/pmlogger/hostname/lock
transient lock file to guarantee mutual exclusion during
pmlogger administration for the host hostname - if present,
can be safely removed if neither pmlogger_daily nor
pmlogger_check are running
$PCP_LOG_DIR/pmlogger/hostname/Latest
PCP archive folio created by mkaf(1) for the most recently
launched archive containing performance metrics from the host
hostname
$PCP_LOG_DIR/NOTICES
PCP ‘‘notices’’ file used by pmie(1) and friends
PCP ENVIRONMENT
Environment variables with the prefix PCP_ are used to parameterize the
file and directory names used by PCP. On each installation, the file
/etc/pcp.conf contains the local values for these variables. The
$PCP_CONF variable may be used to specify an alternative configuration
file, as described in pcp.conf(4).
SEE ALSO
bzip2(1), cron(1), egrep(1), PCP(1), pmlc(1), pmlogger(1),
pmlogmerge(1), pmnewlog(1) and pmsocks(1).