Man Linux: Main Page and Category List


       dnshistory  -  processes  various log file formats doing dns IP Address
       lookups. Store these pairs in a database for later retrieval.


       dnshistory [OPTION]... [--file=FILE]


       dnshistory is Copyright © 2004, 2005, 2006, 2007 Stephen McInerney


       dnshistory currently processes Apache  CLF  and  Combined  logs,  Squid
       access logs, FTP xferlog files and iptables based logs.  The log format
       is auto-detected.

       dnshistory has five modes of operation:

       ·      Do Lookups. The default mode. Given a web log  file,  dnshistory
              will  perform  DNS reverse lookups on each unique IP Address and
              store the results in a history database.

       ·      Do Translations. Given a raw web log file, dnshistory will  make
              use  of a previously created history database and send to STDOUT
              the same web log  but  with  addresses  replaced  by  the  Fully
              Qualified Domain Name as previously looked up.

       ·      Do  Recombining.  Given  two  web  log  files,  one  raw and one
              previously translated (eg. by using dnstran): Create  a  history
              database from the values in these separate log files.

       ·      Do Dump. Dump a given history database to STDOUT.

       ·      Do  Import.  Import  a previously dumped history database from a
              given file.

       ·      Show History. Given one or more  IP  Addresses  on  the  command
              line, show the history of those addresses.

       The lookups make use of threads for maximum speed, and use the standard
       resolution libraries on a system. Thus hosts files, NIS, LDAP and other
       name resolution methods should work transparently.

       It  is  strongly  recommended  that for massive lookups a DNS server is
       "nearby". Preferably not a forwarding server.

       dnshistory can read .gz files. STDIN is assumed to not be gz encoded.


       -L --dolookups
              The default mode. Given a log file,  either  via  STDIN  or  via
              ’-f’, do the lookups and store the results.

       -T --dotranslate
              Given  a  log file, either via STDIN or via ’-f’, lookup each IP
              Address from the history database; replace the IP  Address  with
              the FQDN and send the newly updated log line to STDOUT.

       -R --dorecombine=FILE
              Given  a  previously  translated file (eg. via dnstran) via this
              option for the names, do the lookups for a file given via  STDIN
              or ’-f’ and store the results. This file can be gz encoded.

              The  date/time of each stored entry is taken as being the actual
              time for the lookup stored in the recombine log  file.  This  is
              probably incorrect, but "Good Enough".

       -D --dodump
              Dump the history database to STDOUT.

       -I --doimport=FILE
              Given  a  previously  dumped  database,  import  that into a new
              database. Will fail and exit  if  the  chosen  database  already

       -S --showhistory
              Given  one  or more IP Addresses show their history. Address are
              the last item(s) on the command line.  Addresses with no as  yet
              discovered  FQDN will display ’NONAME’.  The Date/Time displayed
              is formatted  as  YYYY-MM-DD:hh:mm:ss,  vs  the  ’seconds  since
              epoch’ for "--dodump"

              By  default  dnshistory  will attempt to autodetect what type of
              logfile  is  being  processed.  By  using   this   option,   the
              autodetection  is overridden. The choices are: auto, clf or www,
              squid, ftp or iptables.

       -c --cache=SIZE
              Set the size of the  memory  cache  to  use.  Value  is  in  Mb.
              Default is 20Mb.

       -d --database=FILE
              Change  the default database file to use to store stateful data.

       -f --file=FILE
              Web Log File to process. This file can be gz encoded.
              Will use STDIN if not set

       -h --help
              Help screen. Very brief.

       -l --maxlookups=NUMBER
              The maximum number of lookups to attempt. The default is 1. This
              has not shown to be at all useful in testing...

       -m --maxthreads=NUMBER
              How  many  name lookup threads to spawn off. The default is 100.
              Setting this too high can do evil things to  bandwidth  and  the
              CPU usage of any queried DNS server(s).

              If  doing  lots of DNS queries, setting this too high can have a
              very negative impact on  the  ability  to  successfully  resolve

       -t --timeout=VALUE
              The  time  in seconds before a stored DNS value is deemed "old".
              The default is 7 days.

       -v --verbose
              Verboseness of a run.  More  v’s  will  increase  the  level  of
              verbosity,  up  to  a maximum of 5. All of the higher levels are
              only of value for debugging purposes.

       -V --version
              Display the version information and exit

       -w --wait
              Delay time between query retries within a single run


       At  verbose  level  1  (-v)  some  success/failure  counters  will   be
       displayed. As well as any problematic log lines to STDERR.
       At  verbose  level  2  (-vv)  lines  that  may not match up (eg. Due to
       dnstran modifying referrers or URL’s) will be sent to STDERR.


       A typical run, using a database in /tmp/ (/tmp/c.db), and a log file in
       the current directory (test.log). First, do the lookups:

           dnshistory -d /tmp/c.db -f test.log

       Then  the  translation  run  for  input  into,  for  example, a web log

           dnshistory -T -d /tmp/c.db -f test.log | webalizer ....

       Do three attempts on failed queries, with  a  2  second  delay  between

           dnshistory -l 3 -w 2 -d /tmp/c.db -f test.log

       Lookup    and    Display   the   history   of   three   IP   Addresses:,,

           dnshistory -S -d /tmp/c.db

       Import a previously dumped database via dnsdb.dump

           dnshistory -I dnsdb.dump -d /tmp/d.db


           The default history database file.


       ·      Ignores IP Addresses located in the URL and Referrer fields.


       Stephen McInerney <>