Man Linux: Main Page and Category List


       calamaris  -  generate  text  and graphical statistics out of log files
       from Proxy-Cache-Servers


       cat log | calamaris [ --config-file .../calamaris.conf ] [ switches ]


       Calamaris is used to produce statistical output from  Squid,  NetCache,
       Inktomi  Traffic  Server,  Oops!  proxy server, Compaq Tasksmart, Cisco
       Content Engines or related Proxy log files.  The resulting  output  can
       be ascii or html with or without graphic and with or without frames. It
       is possible to cache calculated data in a file to  use  them  in  later

       This  manual  page  describes  the options of Calamaris and gives a few


   Configuration File
       --config-file file

              Not all reports and modification can be  made  through  command-
              line-switches.   To  use all options of Calamaris you´ll have to
              use the configuration  file.   You’ll  find  the  configuration-
              directives  below, always inside of braces.  Examples are in the
              calamaris.conf which should come with this package.

              extracts  all  useful  reports  available,  --all-useful-reports
              equals    --size-distribution-report   10   --domain-report   20
              --performance-report 60  --requester-report  20  --status-report
              --type-report     20     --response-time-report     --errorcode-

       --domain-report|-d n ($domain_report)
              switches the top level and the second level report on.  The data
              is  derived  from  the URL.  The output is limited by n.  (-1 is

       --domain-report-limit n ($domain_report_limit)
              limit the domain-report to lines which have n or more  requests.

       --domain-report-n-level|-N n ($domain_report_n_level)
              All URL-Host reports will be switched from 2nd-level to n-level-
              reports. (-1 shows a full urlhost-report)

              Note: This option is only useful with activated domain-report.

       --errorcode-distribution-report ($errorcode_distribution_report)
              shows the Response code distribution over all objects

              shows the freshness of objects in your cache.   Calamaris  looks
              for  freshness tags like ’TCP_HIT’, ’TCP_REFRESH_MISS’, ...  and
              make statistics on it. With this information  you  can  optimize
              the  caching  behaviour  of  your cache depending on the objects
              content type.  E.g. squid admins could use this  information  to
              configure   the   refresh_pattern.   This   option   needs  more
              configuration in the configuration-file.

       --peak-report|-p type ($peak_report)
              Measures the peaks of the Proxy usage in  requests  per  second,
              per  minute and per hour.  It reports peaks for TCP, UDP and ALL
              requests. If set to old these values were calculated with  the
              old slow method, if set to new the new faster (but still slow)
              method is used.

       --performance-report|-P n ($performance_report)
              Shows the throughput of TCP requests for every n minutes.

       --performance-report-adjust|-T n ($performance_report_adjust)
              Time: Adjust the  Performancereport  in  minutes  for  non  GMT-

       --requester-report|-r n ($requester_report)
              Switches  the  UDP  and TCP requester reports on.  The output is
              limited by n.  (-1 is unlimited)

       --requester-report-no-dns-lookup|-n ($requester_report_no_dns_lookup)
              Switches the IP lookup for the requesters off.

       --requester-report-use-user-info|-u ($requester_report_use_user_info)
              Switches the usage of eventually available ident information for
              requester reports on.

              Warning:  This  breaks  the privacy of your users! (see PRIVACY-
              Section below)

       --requester-report-with-targets|-R n ($requester_report_with_targets)
              adds to each line of the requester report  the  requested  URLs.
              The output is limited by n.  (-1 is unlimited, and can result in
              very very long reports.)

              Warning: Using this option breaks the  privacy  of  your  users!
              (see PRIVACY-Section below)

       --response-time-report ($response_time_report)
              sums up the time distribution over all objects

              This  array  defines the time steps, which should be reported in
              the response-time-report.

       --size-distribution-report|-D n ($size_distribution_report)
              shows size-based  distribution  of  requested  objects,  smaller
              numbers result in more verbose reports. (choose 2, 10 or 100 for
              useful output.)

       --status-report|-s ($status_report)
              alters the default behaviour of Calamaris and makes  the  status
              reports more verbose.

       --type-report|-t n ($type_report)
              switches the content type and the file extension report on.  The
              output is limited by n.  (-1 is unlimited)

       --type-report-ignore-case|-c ($type_report_ignore_case)
              Switch to case-insensitive. This is useful  for  the  ’Requested
              extensions’ report.

       --input-format|-f type ($input_format)
              sets the type of input logfiles. If set to

              auto  Calamaris tries to guess the input file format.  This is
              the Default.

              Note: If the  first  line  of  your  input  file  is  corrupted,
              Calamaris will stop with an error.

              squid  Calamaris  expects  native  logfile  derived from Squid
              V1.1.beta26-V2.x or OOPS.

              squid-extended Calamaris expects native logfile  derived  from
              Squid  V1.1.alpha1-V2.x with log_mime_hdrs enabled or Squid with
              Smartfilter-Patch or squid-style logfiles out of  Cisco  Content
              Engines.   (This  only enables parsing of these kind of logfile,
              the additional data will be ignored.)

              (Logging of MIME-Headers breaks the privacy of your users!  (see
              PRIVACY-Section below)

              squid-old  Calamaris expects native logfile derived from Squid

              nc Calamaris  expects  Logfiles  from  NetCache  up  to  V4.x.
              (Please see the README on this.)

              its Calamaris expects Logfiles from Inktomi Traffic Server.

              elff  Calamaris  expects  Logfiles  in Extended Logfile Format
              (i.e. from Compaq Tasksmart, Novell Internet Caching  System  or
              NetCache V5.x)

              nse  Calamaris  expects  Logfiles  in  Netscape  Extended-1 or
              Netscape Extended-2 Logfile Format (from Netscape/iPlanet/SunOne
              Proxy-Server )

       --ipfilter-exclude IP/range ($ipfilter_exclude)
              all    IPs    are    analyzed,    except    IP/range.    Format:

              IP  list  separated  by  ’:’  This  switch needs the perl Module

              Warning: This breaks the privacy of your  users!  (see  PRIVACY-
              Section below)

       --ipfilter-include IP/range ($ipfilter_include)
              no  IPs  are  analyzed, except IP/range. Format: see --ipfilter-

              Warning: This breaks the privacy of your  users!  (see  PRIVACY-
              Section below)

       --no-input|-z ($no_input)
              Switches  reading  from standard input off.  You can use this to
              merge many  cache  files  to  one  (see  --cache-input-file  and
              --cache-output-file) or to generate a report out of cache files.

       --time-interval|-I t-t ($time_interval)
              defines which time-interval should be parsed.  t has to  be  the
              format yyyymmddhhmmss (localtime)

              Note: omitting the beginning or ending date is allowed.

       Standard output format is plain ascii with 80 chars width.

              ($column2_color) defines the colors for the columns in graphics.
              (only useful with --output-format graph)

              Through the config-file you are able to modify the width of  the
              report  and alter the culomns that are displayed in the reports.
              n is the number of the report, as displayed  by  --help  in  the

       --hostname|-H name ($hostname)
              The  name  for  the  title  or subject of the output.  If set to
              lookup Calamaris looks up the host name of the system its been
              run on.

       --image-type ($image_type)
              Sets  the  image  type to gif, png, jpeg, gd or gd2. Only useful
              when --output-format graph is set. The  available  images  types
              are  dependend on your GD::Graph installation. Default is ’png’.

       --logo|-l string ($logo)
              add a custom string to a HTML-Report. It’ll be added to a  table
              on  the  top  of the output.  -l <A HREF=""><IMG
              BORDER=0          SRC=""
              ALT="Cord"></A>’ will add my logo with a link to the Report.

              Note: --logo works only in combination with --output-format html
              or html-frame

       --meta|-M string ($meta)
              Meta: adds a custom string or the content of  a  file  into  the
              <HEAD>  of  a HTML-Report. Useful if you want to add Stylesheets
              or something to the Report.

              Note: --meta works only in combination with --output-format html
              or html-frame

       --output-format|-F type[,type[,type[,...]]] ($output_format)
              Format: sets the format of the output-report. If set to

              mail adds a subject header to the beginning of the report.

              html all output is given in html with tables.  Can be combined
              with mail to send html mails.

              html-frame all output is given in html frames with tables.

              html-embed all output is given in  html  with  tables  without
              HTML-Headers. Useful for Server-Side-Includes.

              graph enables graphics for html, html-embed or html-frame.

              unformatted  gives  out  the  raw numbers separated by spaces.
              Useful for re-using the output in other scripts. If you use this
              along with -U, the byte values are calculated in the given Unit,
              and displayed without indication along  with  the  numbers.  the
              indication moves up to the header of the report.

       --output-path ($output_path)
              output  calamaris  statistics to /path. In case of graph output,
              the  graphics  destination  is  /path  and   the   filename   is
              index.html,  else  it is calamaris.txt.  If --output-path is not
              given, all graphics are written to the working directory.

       --output-file ($output_file)
              alter the filename of --output-path.

       --output-file-prefix ($output_file_prefix)
              adds a prefix to --output-file %t is replaced by  the  timerange
              of the report, %h by the hostname (see --hostname )

       --show-reports|-S n[,n[,n[,...]]] ($show_reports)
              Show:  Shows  only  the  defined reports in the specified order.
              Default is to display the reports as they  are  defined  through
              the report-switches above. The following numbers are defined:

              0  Summary
              1  Incoming request peak per protocol
              2  Incoming transfer volume peak per protocol
              3  Incoming requests by method
              4  Incoming UDP-requests by status
              5  Incoming TCP-requests by status
              6  Outgoing requests by status
              7  Outgoing requests by destination
              8  Request-destinations by 2ndlevel-domain
              9  Request-destinations by toplevel-domain
              10 TCP-Request-protocol
              11 Requested content-type
              12 Requested extensions
              13 Incoming UDP-requests by host
              14 Incoming TCP-requests by host
              15 Size Distribution Diagram
              16 Performance in n minute steps
              17 UDP-Request duration distribution in msec
              18 TCP-Request duration distribution in msec
              19 UDP Response code distribution
              20 TCP Response code distribution

       Note:  Using  this  doesn’t  make  Calamaris  any  faster, the internal
       calculations will be done as the report-switches were set (see  above).

       --sort-order|-O ($sort_order)
              Changes  the  sort order in the reports to request size, default
              is sorting by number of requests.

              defines the  colors  for  text/axis/legend/labels  in  graphics.
              (only useful with --output-format graph )

       --unit|-U string ($unit)
              You  can define this to K(ilo), M(ega), G(iga) or T(era) for the

              defines the width of the graphics.  height  is  calculated  from
              this with a 3:2-ratio. (only useful with --output-format graph )

              defines how many datasets should be drawn on the graph. 30 is  a
              good value, but you can play with this. if $x_scale gets to big,
              you’re on your own ;-)

       --generate-index ($generate_index)
              generates an index for all  reports  that  match  --output-file-

       --cache-input-file|-i file ($cache_input_file)
              You  can  reuse  a cache file generated with --cache-output-file
              file to add old data to a new  report.   Several  files  can  be
              separated with a ’:’.

              Note:  if  you  use more than one cache file, make sure they are
              chronologicaly ordered (oldest first).

              Note: if you reuse cache-files, which were not created  with  -d
              -1  -r  -1  -t  -1  -R  -1 the number of ’others’ would be wrong
              everywhere. In this case the number of ’others’ are omitted.

       --cache-output-file|-o file ($cache_output_file)
              Calamaris stores a summary of the computed information  in  file
              and you can reuse it at a later time with --cache-input-file

              Note:  The  output file can be the same as the input file: it is
              simply  overwritten  after  reading  the  data.    It   is   not
              recommended  to change the options between different runs if you
              include older data as this can result in strange measurements.

       --benchmark|-b n ($benchmark)
              benchmark: A switch for the impatient as it  prints  a  ’#’  for
              every n parsed lines.

              Prints the copyright information of Calamaris

              Prints a brief description of the command line options.

              Prints out the Version-Number.

              prints the internal loop to STDERR. (for Debugging)

              activates some small tests for the programmer.

       --verbose|-v ($verbose)
              print  more  information  about  what  is Calamaris is doing and


       This example mails the daily statistics to root:
              cat /var/log/squid/access.log | nice -39 calamaris --all-useful-
              reports --hostname --output-format mail | mail root

       This one only caches a summary for later use:
              cat  /var/log/squid/access.log  | calamaris --all-useful-reports
              --cache-output-file daily.‘date +"%w"‘ > /dev/null

       You can then use the caches to have weekly statistics:

              if [ $DAYOFWEEK = "0"  ];  then  calamaris  --all-useful-reports
              daily.1:daily.2:daily.3:daily.4:daily.5:daily.6:daily.0    --no-
              input  --output-format mail --hostname "weekly worf" | mail root
              ; fi


       If you have a problem with Calamaris , please make sure  that  you  use
       the  recent  version  of Calamaris (see VERSION below). Also check that
       your proxy works correctly and doesn’t produce invalid  Logfiles.  (see
       the README for buglist and pointers.)

       If you’re sure that you’ve encountered a bug in Calamaris please report
       it to This also applies if Calamaris itself says
       ’please report this’.


       Calamaris can be (mis-)used to track what users are requesting.

       So please read the following and think about it, before using Calamaris
       to be the Big Brother.

       -      If you don’t trust your users than there is something more wrong
              than the loss of productivity.

       -      Squid has some nice acl-mechanisms. If you think that your users
              don’t use the net properly, don’t let them use it. (You can also
              open  the  net  at  specific  times or to specific sites, if you

       -      If  you  still  want  to  use  Calamaris  that  way,  let   your
              vict^Wusers  know,  that  they’ll  be monitored. (in Germany you
              have to let them know!)




       Cord     Beermann      <>,      Michael      Pophal
       <>.   There  are  also  a  lot  of  people who
       contributed code, gave ideas or requested features. Look them up in the

       This     man    page    was    written    by    Philipp    Frauenfelder
       <>, maintainer of the Debian  package.   Maintenance
       is now taken over by Cord Beermann.


       Version  of  this  manpage:  $Id: calamaris.1,v 3.1 2006-03-19 17:52:48
       cord Exp $

       It describes the usage of Calamaris V3.0 and later.

       Information about new releases, mailing lists, and other related issues
       can be found from the Calamaris home page at


       Calamaris comes with "absolutely no warranty".


       Copyright  (C)  1997,  1998,  1999,  2000,  2001, 2002, 2003, 2004 Cord

       This program is free software; you can redistribute it and/or modify it
       under  the  terms of the GNU General Public License as published by the
       Free Software Foundation; either version 2 of the License, or (at  your
       option) any later version.

       (If you modify and want to publish it under the name Calamaris , please
       ask me.  I don’t want to confuse the  ’audience’  with  many  different
       versions  of the same name and/or Version number.  (This is not part of
       the license, it is only a favour i asked of you.))

       This program is distributed in the hope that it  will  be  useful,  but
       WITHOUT   ANY   WARRANTY;   without   even   the  implied  warranty  of
       General Public License for more details.

       You should have received a copy of the GNU General Public License along
       with this program; if not, write to the Free Software Foundation, Inc.,
       59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.