NAME
stealth - Stealthy File Integrity Scanner
SYNOPSIS
stealth -dcnoq -i <interval> -r <nr>
[--max-size <size>[BKMG] ]
[--keep-alive pidfile [--repeat <seconds> ] ] policy
stealth [--rerun | --resume | --suppress | --terminate] pidfile
DESCRIPTION
The name of the stealth program is an acronym of:
SSH-based Trust Enforcement Acquired through a Locally Trusted
Host.
stealth is based on an idea by Hans Gankema and Kees Visser, both at
the Center for Information Technology of the University of Groningen.
Hopko Meijering provided valuable suggestions for improvement.
stealth’s main task is to perform file integrity tests. However, the
testing itself will leave no sediments on the tested computer.
Therefore, stealth has stealthy characteristics. This is considered an
important security improving feature.
On the other hand, one should realize that stealth intends to be just
another security tool: other security measures like firewalls,
portscanners, intrusion detection systems, abolishment of unencrypted
protocols, etc. are usually required to improve or promote the security
of a group of computers that are connected to the Internet.
stealth uses a policy file to determine the actions to perform. Each
policy file is uniquely associated with a host to be tested. This
remote host (called the client below) trusts the computer on which
stealth runs (hence: a Locally Trusted Host), called the controller.
The controller performs tasks (normally file integrity tests) that
Enforce the Trust we have in the client computer. Since almost all
integrity tests can be run on the client, one controller can control
many clients, even if the controller itself uses aged hardware
components.
As the controller and the client normally are different computers, the
controller must communicate with the client in a secure fashion. This
is realized using SSH. So, there’s another element of ‘local trust’
involved here: the client should permit the controller to set up a
secure SSH connection allowing the controller to access sensitive files
and private parts of the client’s file system.
It is important to ensure that there is no public access to the
controller. All inbound services should be denied. The only access to
the controller should be via its console and the controller should be
placed in a physically secure location. Sensitive information of
clients are stored in the controller, and passwordless access to
clients can be obtained from the controller by anyone who gains
(root)-access.
The controller itself only needs two kinds of outgoing services: SSH to
reach its clients, and some mail transport agent (e.g., sendmail(1)) to
forward its outgoing mail to some mail-hub.
Here is what happens when stealth is run using the first synopsis:
o First, the policy file is read. This determines the actions to
be performed, and the values of several variables that are used
by stealth.
o If the command-line option --keep-alive pidfile is specified,
stealth will run as a backgrond process, writing its process id
in the file pifile. With --repeat <seconds> the scan will be
rerun every <seconds> seconds. The number of seconds until the
next rerun will be at least 60. However, using the --rerun
pidfile option a background stealth process may always be forced
into its next scan. When --keep-alive is specified the scan will
be performed just once, whereafter PROGRAM will wait until it is
reactivated by another run of stealth, called using the --rerun
pidfile command-line option (note that integrity scans are
suppressed between a --suppress and a --resume command, see
below). Consider specifying --quiet (see below) when
--keep-alive is used.
o Then, the controller opens a command shell on the client using
ssh(1), and a command shell on the controller itself using
sh(1).
o Next, commands defined in the policy file are executed in their
order of appearance. Examples are given below. Normally, return
values of the programs are tested. Non-zero return values will
terminate stealth prematurely. In this case, a message about the
reason why stealth terminated is written to the report file (and
into the mail message sent by stealth). In some cases (e.g.,
when the report file could not be written), the message is
written to the standard error stream.
o In most cases, integrity tests can be controlled by the find(1)
program, calling programs like ls(1), sha1sum(1) or its own
-printf method to produce file-integrity related statistics.
Most of these programs write file names at the end of generated
lines. This characteristic is used by an internal routine of
stealth to detect changes in the generated output, which could
indicate some harmful intent, like an installed root-kit.
o When changes are detected, they are logged on a report file, to
which information is always appended. stealth never reduces or
rewrites the report file. Whenever information is added to the
report file (exceeding a plain time stamp) the appended
information is emailed to a configurable email address for
further (human) processing. Usually this will be the systems
manager of the tested client. stealth follows the ‘dark cockpit’
approach in that no mail is sent when no changes were detected.
o When the --repeat or --rerun options are issued, the report file
should not be rotated by, e.g., a log-rotating process, but the
report file may safely be rotated between a pair of --suppress
and --resume commands.
REPORT FILE ROTATION
Since stealth only appends information to the report file, it will
eventually grow to a large file, and log-rotation may be desirable. It
is of course possible to issue a --terminate command, rotate the
logfiles, and restart stealth, but stealth also offers a facility to
temporarily suppress further scans:
o Starting stealth using the option --suppress pidfile will
suppress a currently active stealth process. If stealth is
actually performing a series of integrity scans when --suppress
is issued, the currently executing command is first completed
before the --suppress command completes. Once --suppress is
active, all scheduled scans are skipped and --rerun is ignored.
However, the --resume and --terminate options are still handled.
o Once ‘stealth --suppress pidfile’ has returned, the report file
may safely be rotated (using, e.g., logrotate(1)), and a new
(empty) report file may optionally be created by the logrotation
process.
o Finally, when the log-rotation has been completed, the
log-rotation process should issue the command ‘stealth --resume
pidfile’. This will resume a suppressed stealth process,
immediately performing the next integrity scan (thus implying
--rerun), whereafter stealth will be back in its normal
integrity scanning mode (so, resuming repeated scans if
originally requested so).
RERUN AND TERMINATE
Here is what happens when stealth is run using other synopses:
o When started using the --rerun pidfile command-line option, the
stealth process associated with process id file pidfile will
perform another scan. This command has no effect following a
--suppress command.
o When started using the --terminate pidfile command-line option,
the stealth process associated with process id file pidfile is
terminated.
OPEN SSH LINK TO CLIENTS
The --keep-alive, --repeat, --rerun, --resume and --suppress options
were implemented in such a way that the ssh link to the client remains
open, thus minimizing the number of sshd entries caused by PROGRAM in
the client’s log files.
THE POLICY FILE
The policy file consists of two sets of data: use directives (starting
with the keyword USE) and commands. Blank lines and information beyond
hash-marks (#) are ignored, while lines following lines terminating in
backslashes (\) will be concatenated (en passant removing the
backslashes). Initial white space on lines of the policy file is
ignored.
DEFINE DIRECTIVES
DEFINE directives may be used to associate longer strings of text with
certain symbols. E.g., after DEFINE FINDARGS -xdev -type f -exec
/usr/bin/sha1sum {} \; the text ${FINDARGS} may be used in USE
DIRECTIVES and commands (see below) to use the text associated with the
FINDARGS symbol.
Note that DEFINE symbols may be used in the definition of other DEFINE
symbols as well. Direct or indirect circular definitions should be
avoided, as they are either not or incompletely expanded.
USE DIRECTIVES
The following USE directives may be specified (directives are written
in capitals, and should appear exactly as written below: letter casing
is preserved). Specifications in angular brackets (like <this>)
represent specifications to be given by users of stealth:
o USE BASE <basedirectory>
BASE defines the directory from where stealth operates. All
relative path specifications are interpreted relative to BASE.
By default this is the directory where stealth was started.
BASE is the only directory that must exist when stealth is
started. All other non-existing paths are created automatically
by stealth.
Example:
USE BASE /root/client
o USE DD <dd>
The DD specification uses /bin/dd as default, and defines the
location of the dd(1) program, both on the server and on the
client. The bin(1) program is used to copy files between the
client and the controller without opening separate
ssh-connections. The program specified here is only used by
stealth for the PUT and GET commands, described below.
Example showing the default:
USE DD /bin/dd
o USE DIFF <diff>
The DIFF specification uses /usr/bin/diff as default, and
defines the location of the diff(1) program on the controller.
The diff(1) program is used to compare a formerly created
logfile of an integrity check with a newly created logfile.
Example showing the default:
USE DIFF /usr/bin/diff
o USE EMAIL <address>
The EMAIL specification defines the email-address to receive the
report of the integrity scan of the client. The ‘dark cockpit’
philosophy is followed here: mail is only sent when a
modification is detected.
Example showing the default (apparently an email address on the
controller):
USE EMAIL root
o USE MAILER <mailer>
The MAILER specification defines the program that is used to
send the mail to the EMAIL-address. Contrary to DIFF and DD and
(see below) SH and SSH, MAILER is run as a /bin/sh command, to
allow shell-scripts to process the mail too. By default MAILER
is defined as /usr/bin/mail(1). MAILER is called with the
following arguments:
----------------------------------------------------------
MAILARGS, see below;
EMAIL, the addressee of the mail.
----------------------------------------------------------
Example showing the default:
USE MAILER /usr/bin/mail
As an alternative, the script stealthmail is provided. It offers
a convenient filter sorting stealth’s output and keeping only
lines containing ADDED, MODIFIED, REMOVED and STEALTH. Usually
these lines are are the ones system managers are interested in.
The report and log files can always be consulted to determine
the actual nature of the changes.
o USE MAILARGS <args>
The MAILARGS specification defines the arguments that are passed
to MAILER, followed by the specification of EMAIL. Example
showing the default:
USE MAILARGS -s "STEALTH scan report"
Note that blanks may be used in the subject specification: use
double or single quotes to define elements containing blanks.
Use \" to use a double quote in a string that is itself delimted
by double quotes, use \’ to use a single quote in a string that
is itself delimted by single quotes.
o USE REPORT <reportfile>
REPORT defines the name of the reportfile. Information is always
appended to this file. For each run of stealth a time marker
line is written to the report file. Only when (in addition to
the marker line) additional information is appended to the
report file the added contents of the report file are mailed to
the mail address specified in the USE EMAIL specification.
Example showing the default:
USE REPORT report
o USE SH <sh>
The SH specification uses /bin/sh as default, and defines the
command shell used by the controller to execute commands on
itself.
Example showing the default:
USE SH /bin/sh
o USE SSH <user>
The SSH specification has no default, and must be specified.
Assuming the client trusts the controller (which is, after all,
what this program is all about; so this should not be a very
strong assumption), preferably the public ssh-identity key of
the controller should be placed in the client’s root
.ssh/authorized_keys file, granting the controller root access
to the client. Root access is normally needed to gain access to
all directories and files of the client’s file system.
In practice, connecting to a account using the sh(1) shell is
preferred. When another shell is already used by that account,
one should make sure that that shell doesn’t setup its own
redirections for standard input and standard output. One way to
accomplish that is for force the execution of /bin/sh in the USE
SSH specification. Examples:
# root’s shell is /bin/sh:
USE SSH root@client -T -q
# root uses another shell
USE SSH root@client -T -q exec /bin/bash
# an alternative:
USE SSH root@client -T -q exec /bin/bash --noprofile
In some installations stealth is used to inspect the computer itself,
even though this is not recommended, as it counters one of the main
reasons for stealth’s existence. In situations where stealth is used to
monitor the integrity of the localhost, /bin/bash could be specified
with the USE SSH directive. For example:
# For stealth inspecting localhost:
USE SSH /bin/bash --noprofile
COMMANDS
Following the USE specifications, commands can be specified. The
commands are executed in their order of appearance in the policy file.
Processing continues until the last command has been processed or until
a tested command (see below) returns a non-zero return value.
LABEL COMMANDS
The following LABEL commands are available:
o LABEL <text>
This defines a text-label which is written to the REPORT file,
in front of the output generated by the next CHECK-command. If
the next CHECK-command generates no output, the text-label is
not written to the REPORT-file. Once a LABEL has been defined,
it is used until it is redefined by the next LABEL. Use an empty
LABEL specification to suppress the printing of labels.
The text may contain \n characters (two characters) which are
transformed to a newline character.
Example:
LABEL Inspecting files in /etc\nIncluding subdirectories
LABEL
(The former LABEL specification clears the latter label text).
LOCAL COMMANDS
The following LOCAL commands are available to be executed on the
controller:
o LOCAL <command>
Execute command on the controller, using the SH command shell.
The command must succeed (i.e., must return a zero exit value).
Example:
LOCAL scp rootsh@client:/usr/bin/sha1sum /tmp
This command will copy the client’s sha1sum(1) program to the
controller.
o LOCAL NOTEST <command>
Execute command on the controller, using the SH command shell.
The command may or may not succeed.
Example:
LOCAL NOTEST mkdir /tmp/subdir
This command will create /tmp/subdir on the controller. The
command will fail if the directory cannot be created, but this
will not terminate stealth.
o LOCAL CHECK [LOG =] <logfile> <command>
Execute command on the controller, using the SH command shell.
The command must succeed. The output of this command is compared
to the output of this command generated during the previous run
of stealth. The phrase LOG = is optional. Any differences are
written to REPORT. If differences were found, the existing
logfile name is renamed to logfile.YYMMDD-HHMMSS, with
YYMMDD-HHMMSS the datetime-stamp at the time stealth was run.
Note that eventually many logfile.YYMMDD-HHMMSS files could be
created: It is up to the controller’s systems manager to decide
what to do with old datetime-stamped logfiles.
The logfile specifications may use relative and absolute paths.
When relative paths are used, these paths are relative to BASE.
When the directories implied by the logfile specifications do
not yet exist, they are created first.
Example:
LOCAL CHECK LOG = local/sha1sum sha1sum /tmp/sha1sum
This command will check the SHA1 sum of the /tmp/sha1sum
program. The resulting output is saved at BASE/local/sha1sum.
The program must succeed (i.e., sha1sum must return a zero
exit-value).
o LOCAL NOTEST CHECK <logfile> <command>
Execute command on the controller, using the SH command shell.
The command may or may not succeed. Otherwise, the program acts
identically as the LOCAL CHECK ... command, discussed
previously.
Example:
LOCAL NOTEST CHECK LOG=local/sha1sum sha1sum /tmp/sha1sum
This command will check the SHA1 sum of the /tmp/sha1sum
program. The resulting output is saved at BASE/local/sha1sum.
The program must succeed (i.e., sha1sum must return a zero
exit-value).
Note that the scp(1) command can be used to copy files between the
client and the controller, using a local command. This, however, is
discouraged, as a separate ssh(1)-connection is required for each
separate scp(1) command. This subtlety was brought to the author’s
attention by Hopko Meijerink (h.meijering@rc.rug.nl). Using scp(1)
results in several additional entries showing sshd(1) connections in
the client’s logfiles, which in turn may provide hints to a hacker that
the client is intensively monitored. In order to copy files between the
client and the controller, the GET and PUT commands (described below)
may be used, which use the existing ssh(1) connection. In general,
LOCAL commands should not be used to establish additional ssh(1)
connections to a client.
REMOTE COMMANDS
Remote commands are commands executed on the client using the SSH
shell. These commands are executed using the standard PATH set for the
SSH shell. However, it is advised to specify the full pathname to the
programs to be executed, to prevent ‘‘trojan approaches’’ where a
trojan horse is installed in an ‘earlier’ directory of the
PATH-specification than the intended program.
Two special remote commands are GET and PUT, which can be used to copy
files between the client and the controller. Internally, GET and PUT
use the DD use-specification. If a non-default specification is used,
one should ensure that the alternate program accepts dd(1)’s if=, of=,
bs= and count= options. With GET the options bs=, count= and of= are
used, with PUT the options bs=, count= and if= are used. Normally there
should be no need to alter the default DD specification.
The GET command may be used as follows:
o GET <client-path> <local-path>
Copy the file indicated by client-path at the client to
local-path at the controller. client-path must be the full path
of an existing file on the client, local-path may either be a
local directory, in which case the client’s file name is used,
or another file name may be specified, in which case the
client’s file is copied to the specified local filename. If the
local file already exists, it is overwritten by the
copy-procedure.
Example:
GET /usr/bin/sha1sum /tmp
The program /usr/bin/sha1sum, available at the client, is copied
to the controller’s /tmp directory. If the copying fails for
some reason, any subsequent commands are skipped, and stealth
terminates.
o GET NOTEST <client-path> <local-path>
Copy the file indicated by client-path at the client to
local-path at the controller. client-path must be the full path
of an existing file on the client, local-path may either be a
local directory, in which case the client’s file name is used,
or another file name may be specified, in which case the
client’s file is copied to the specified local filename. If the
local file already exists, it is overwritten by the
copy-procedure.
Example:
GET NOTEST /usr/bin/sha1sum /tmp
The program /usr/bin/sha1sum, available at the client, is copied
to the controller’s /tmp directory. Remaining commands in the
policy file are executed, even if the copying process wasn’t
successful.
The PUT command may be used as follows:
o PUT <local-path> <remote-path>
Copy the file indicated by local-path at the controller to
remote-path at the client. The argument local-path must be the
full path of an existing file on the controller. The argument
remote-path must be the full path to a file on the client. If
the remote file already exists, it is overwritten by PUT.
Example:
PUT /tmp/sha1sum /usr/bin/sha1sum
The program /tmp/sha1sum, available at the controller, is copied
to the client as usr/bin/sha1sum. If the copying fails for some
reason, any subsequent commands are skipped, and stealth
terminates.
o PUT NOTEST <local-path> <remote-path>
Copy the file indicated by local-path at the controller to
remote-path at the client. The argument local-path must be the
full path of an existing file on the controller. The argument
remote-path must be the full path to a file on the client. If
the remote file already exists, it is overwritten by PUT.
Example:
PUT NOTEST /tmp/sha1sum /usr/bin/sha1sum
Copy the file indicated by local-path at the controller to
remote-path at the client. The argument local-path must be the
full path of an existing file on the controller. The argument
remote-path must be the full path to a file on the client. If
the remote file already exists, it is overwritten by PUT.
Remaining commands in the policy file are executed, even if the
copying process wasn’t successful.
Plain commands can be executed on the client computer by merely
specifying them. Of course, this means that programs on the client
called, e.g., LABEL, LOCAL or USE, cannot be executed, since these
names are interpreted otherwise by stealth. I don’t think that
represents much of a problem, though....
The following commands are available to be executed on the client:
o <command>
Execute command on the client, using the SSH command shell. The
command must succeed (i.e., must return a zero exit value).
However, any output generated by the the command is ignored.
Example:
/usr/bin/find /tmp -type f -exec /bin/rm {} \;
This command will remove all ordinary files in and below the
client’s /tmp directory.
o NOTEST <command>
Execute command on the client, using the SSH command shell. The
command may or may not succeed.
Example:
NOTEST /usr/bin/find /tmp -type f -exec /bin/rm {} \;
Same as the previous command, but this time the exit value of
/usr/bin/find is not interpreted.
o CHECK [LOG =] <logfile> <command>
Execute command on the client, using the SSH command shell. The
phrase LOG = is optional. The command must succeed. The output
of this command is compared to the output of this command
generated during the previous run of stealth. Any differences
are written to REPORT. If differences were found, the existing
logfile name is renamed to logfile.YYMMDD-HHMMSS, with
YYMMDD-HHMMSS the datetime-stamp at the time stealth was run.
Note that the command is executed on the client, but the logfile
is kept on the controller. This command represents the core of
the method implemented by stealth: there will be no residues of
the actions performed by stealth on the client computers.
Several examples (note the use of the backslash as line
continuation characters):
CHECK LOG = remote/ls.root \
/usr/bin/find / \
-xdev -perm +6111 -type f -exec /bin/ls -l {} \;
All suid/gid/executable files on the same device as the
root-directory (/) on the client computer are listed with their
permissions, owner and size information. The resulting listing
is written on the file BASE/remote/ls.root.
CHECK remote/sha1.root \
/usr/bin/find / \
-xdev -perm +6111 -type f -exec /usr/bin/sha1sum {} \;
The SHA1 checksums of all suid/gid/executable files on the same
device as the root-directory (/) on the client computer are
determined. The resulting listing is written on the file
BASE/remote/sha1.root.
o NOTEST CHECK [LOG =] <logfile> <command>
Execute command on the client, using the SSH command shell. The
phrase LOG = is optional. The command may or may not succeed.
Otherwise, the program acts identically as the CHECK ...
command, discussed previously.
Example:
NOTEST CHECK LOG = remote/sha1.root \
/usr/bin/find / \
-xdev -perm +6111 -type f -exec /usr/bin/sha1sum {} \;
The SHA1 checksums of all suid/gid/executable files on the same
device as the root-directory (/) on the client computer are
determined. The resulting listing is written on the file
BASE/remote/sha1.root. stealth will not terminate if the
/usr/bin/find program returns a non-zero exit value.
The maximum download size (using GET or CHECK) can be specified using
the --max-size option, see below. By default it is set at 10M.
OPTIONS
Long options are given immediately following the short-option
equivalents, if available. Either can be used.
o -d --debug: Write debug messages to std error;
o -c --parse-config-file: Process the config file, no further
action,
report the results to std output;
o -e --echo-commands: echo commands to std error when they are
processed (implied by -d);
o -i --random-interval <interval>[m]>: start the scan a random
interval of <interval> seconds (or minutes if an ‘m’ is appended
to <interval>) following the delay specified at --repeat (see
below). This option is ignored unless --repeat is provided as
well.
o -n --no-child-processes: No child processes are executed: child
actions
are faked to be OK.
o -o --only-stdout: Scan report is written to stdout. No mail is
sent.
(implied by -d);
o -q --quiet: Suppress progress messages written to stderr;
o -r --run-command <nr>: Only run command <nr> (natural number).
Command numbers are shown by stealth -c;
o -v --version: Display version information and exit;
o --keep-alive pidfile: Keep running as a daemon, wake up at
interrupts.
o --max-size <size>[BKMG]: the maximum file size that can be
downloaded from the client in bytes (B), Kbytes (K), Mbytes (M),
Gbytes (G). By default download size is 10M. When specified, the
default unit is B.
o --repeat <seconds>: keep running as a daemon, wake up at
interrupts or after <seconds> seconds. The interval will be at
least 60 seconds. To this interval a random delay may be added
(see --random-interval).
o --rerun pidfile: restart the scan of a currently active stealth
process;
o --resume pidfile: resume a suppressed stealth process, implying
--rerun;
o --suppress pidfile: suppress a currently active stealth process.
All scheduled scans following --suppress are skipped, --rerun is
ignored, but --resume and --terminate (see below) may be issued;
o --terminate pidfile: terminate a currently active stealth
process;
o --usage: Display help information and exit;
o --help: Display help information and exit;
o pidfile: file containing the process id of a stealth process;
o policy: path to the policyfile;
DEPLOYMENT SUMMARY
The following summarizes the advised steps to perform when installing
stealth. All these steps are elaborated upon in stealth’s User Guide
(chapter Running stealth):
o Install stealth (e.g., use dpkg(1) to install the .deb file);
o Construct one or more policy files;
o Automate running stealth using cron(1) (possibly calling
stealthcron);
o Set up automated log-file rotation, using, e.g., stealthcleanup
and logrotate(1), defining one or more
/etc/logrotate.d/stealth... configuration files.
FILES
/usr/share/doc/stealth/;
the policy file;
files under the BASE directory as defined in the policy file;
the report file as defined by the policy’s USE REPORT directive.
SEE ALSO
cron(1), dd(1), diff(1), dpkg(1), find(1), logrotate(1), ls(1),
mail(1), sha1sum(1), passwd(5), sendmail(1), sh(1), ssh(1)
DIAGNOSTICS
By default, the executed commands are echoed to stderr. Use -q to
suppress this echoing.
BUGS
None reported
COPYRIGHT
This is free software, distributed under the terms of the ‘GNU General
Public License’. Copyright remains with the author. Stealth is found at
http://stealth.sourceforge.net/.
ORGANIZATION
Center for Information Technology, University of Groningen.
AUTHOR
Frank B. Brokken (f.b.brokken@rug.nl).