Man Linux: Main Page and Category List

NAME

       httrack - offline browser : copy websites to a local directory

SYNOPSIS

       httrack  [ url ]... [ -filter ]... [ +filter ]... [ -O, --path ] [ -%O,
       --chroot ] [ -w, --mirror ] [ -W, --mirror-wizard ] [ -g, --get-files ]
       [  -i,  --continue  ]  [  -Y,  --mirrorlinks  ]  [ -P, --proxy ] [ -%f,
       --httpproxy-ftp[=N] ] [ -%b, --bind ] [  -rN,  --depth[=N]  ]  [  -%eN,
       --ext-depth[=N]  ]  [  -mN, --max-files[=N] ] [ -MN, --max-size[=N] ] [
       -EN,   --max-time[=N]   ]   [   -AN,   --max-rate[=N]   ]    [    -%cN,
       --connection-per-second[=N]   ]   [  -GN,  --max-pause[=N]  ]  [  -%mN,
       --max-mms-time[=N] ] [ -cN, --sockets[=N] ] [ -TN, --timeout ]  [  -RN,
       --retries[=N]  ]  [ -JN, --min-rate[=N] ] [ -HN, --host-control[=N] ] [
       -%P, --extended-parsing[=N] ] [ -n, --near ] [  -t,  --test  ]  [  -%L,
       --list   ]  [  -%S,  --urllist  ]  [  -NN,  --structure[=N]  ]  [  -%D,
       --cached-delayed-type-check   ]   [   -%M,   --mime-html   ]   [   -LN,
       --long-names[=N] ] [ -KN, --keep-links[=N] ] [ -x, --replace-external ]
       [ -%x, --disable-passwords ] [  -%q,  --include-query-string  ]  [  -o,
       --generate-errors  ] [ -X, --purge-old[=N] ] [ -%p, --preserve ] [ -bN,
       --cookies[=N] ] [ -u, --check-type[=N] ] [  -j,  --parse-java[=N]  ]  [
       -sN,  --robots[=N]  ]  [  -%h, --http-10 ] [ -%k, --keep-alive ] [ -%B,
       --tolerant ] [ -%s, --updatehack ] [ -%u, --urlhack ] [ -%A, --assume ]
       [ -@iN, --protocol[=N] ] [ -%w, --disable-module ] [ -F, --user-agent ]
       [ -%R, --referer ] [ -%E, --from ] [ -%F, --footer ] [ -%l,  --language
       ]   [   -C,   --cache[=N]   ]  [  -k,  --store-all-in-cache  ]  [  -%n,
       --do-not-recatch ] [ -%v, --display  ]  [  -Q,  --do-not-log  ]  [  -q,
       --quiet  ]  [ -z, --extra-log ] [ -Z, --debug-log ] [ -v, --verbose ] [
       -f, --file-log ]  [  -f2,  --single-log  ]  [  -I,  --index  ]  [  -%i,
       --build-top-index  ]  [ -%I, --search-index ] [ -pN, --priority[=N] ] [
       -S, --stay-on-same-dir ] [ -D, --can-go-down ] [ -U,  --can-go-up  ]  [
       -B,  --can-go-up-and-down  ]  [  -a,  --stay-on-same-address  ]  [  -d,
       --stay-on-same-domain   ]   [   -l,   --stay-on-same-tld   ]   [    -e,
       --go-everywhere     ]    [    -%H,    --debug-headers    ]    [    -%!,
       --disable-security-limits ] [ -V, --userdef-cmd ] [  -%U,  --user  ]  [
       -%W, --callback ] [ -K, --keep-links[=N] ] [

DESCRIPTION

       httrack  allows you to download a World Wide Web site from the Internet
       to a local directory, building  recursively  all  directories,  getting
       HTML, images, and other files from the server to your computer. HTTrack
       arranges the original site’s relative  link-structure.  Simply  open  a
       page  of the "mirrored" website in your browser, and you can browse the
       site from link to link, as if you were viewing it online.  HTTrack  can
       also   update   an  existing  mirrored  site,  and  resume  interrupted
       downloads.

EXAMPLES

       httrack www.someweb.com/bob/
               mirror site www.someweb.com/bob/ and only this site

       httrack  www.someweb.com/bob/  www.anothertest.com/mike/   +*.com/*.jpg
       -mime:application/*
               mirror the two sites together (with shared  links)  and  accept
              any .jpg files on .com sites

       httrack www.someweb.com/bob/bobby.html +* -r6
              means get all files starting from bobby.html, with 6 link-depth,
              and possibility of going everywhere on the web

       httrack        www.someweb.com/bob/bobby.html        --spider        -P
       proxy.myhost.com:8080
              runs the spider on www.someweb.com/bob/bobby.html using a proxy

       httrack --update
              updates a mirror in the current folder

       httrack
              will bring you to the interactive mode

       httrack --continue
              continues a mirror in the current folder

OPTIONS

   General options:
       -O     path for mirror/logfiles+cache (-O path mirror[,path  cache  and
              logfiles]) (--path <param>)

       -%O    chroot path to, must be r00t (-%O root path) (--chroot <param>)

   Action options:
       -w     *mirror web sites (--mirror)

       -W     mirror    web    sites,    semi-automatic    (asks    questions)
              (--mirror-wizard)

       -g     just get files (saved in the current directory) (--get-files)

       -i     continue an interrupted mirror using the cache (--continue)

       -Y     mirror ALL links located in the first level pages (mirror links)
              (--mirrorlinks)

   Proxy options:
       -P     proxy  use  (-P  proxy:port or -P user:pass@proxy:port) (--proxy
              <param>)

       -%f    *use proxy for ftp (f0 don t use) (--httpproxy-ftp[=N])

       -%b    use this local hostname to  make/send  requests  (-%b  hostname)
              (--bind <param>)

   Limits options:
       -rN    set the mirror depth to N (* r9999) (--depth[=N])

       -%eN   set the external links depth to N (* %e0) (--ext-depth[=N])

       -mN    maximum file length for a non-html file (--max-files[=N])

       -mN,N2 maximum file length for non html (N) and html (N2)

       -MN    maximum    overall    size    that   can   be   uploaded/scanned
              (--max-size[=N])

       -EN    maximum mirror  time  in  seconds  (60=1  minute,  3600=1  hour)
              (--max-time[=N])

       -AN    maximum   transfer   rate   in  bytes/seconds  (1000=1KB/s  max)
              (--max-rate[=N])

       -%cN   maximum     number      of      connections/seconds      (*%c10)
              (--connection-per-second[=N])

       -GN    pause  transfer  if N bytes reached, and wait until lock file is
              deleted (--max-pause[=N])

       -%mN   maximum mms stream download time in seconds (60=1 minute, 3600=1
              hour) (--max-mms-time[=N])

   Flow control:
       -cN    number of multiple connections (*c8) (--sockets[=N])

       -TN    timeout,  number  of  seconds  after  a  non-responding  link is
              shutdown (--timeout)

       -RN    number of retries, in case of timeout or non-fatal errors  (*R1)
              (--retries[=N])

       -JN    traffic  jam  control,  minimum  transfert  rate (bytes/seconds)
              tolerated for a link (--min-rate[=N])

       -HN    host is abandonned if: 0=never, 1=timeout, 2=slow, 3=timeout  or
              slow (--host-control[=N])

   Links options:
       -%P    *extended  parsing,  attempt to parse all links, even in unknown
              tags or Javascript (%P0 don t use) (--extended-parsing[=N])

       -n     get non-html files  near  an html file  (ex:  an  image  located
              outside) (--near)

       -t     test all URLs (even forbidden ones) (--test)

       -%L    <file>  add all URL located in this text file (one URL per line)
              (--list <param>)

       -%S    <file> add all scan rules located in this text  file  (one  scan
              rule per line) (--urllist <param>)

   Build options:
       -NN    structure   type   (0   *original   structure,  1+:  see  below)
              (--structure[=N])

       -or    user defined structure (-N "%h%p/%n%q.%t")

       -%N    delayed type check, don t make any link test but wait for  files
              download to start instead (experimental) (%N0 don t use, %N1 use
              for unknown extensions, * %N2 always use)

       -%D    cached delayed type check, don t wait  for  remote  type  during
              updates,   to  speedup  them  (%D0  wait,  *  %D1  don  t  wait)
              (--cached-delayed-type-check)

       -%M    generate   a   RFC   MIME-encapsulated    full-archive    (.mht)
              (--mime-html)

       -LN    long  names  (L1  *long  names  / L0 8-3 conversion / L2 ISO9660
              compatible) (--long-names[=N])

       -KN    keep original links  (e.g.  http://www.adr/link)  (K0  *relative
              link,  K  absolute  links,  K4  original  links, K3 absolute URI
              links) (--keep-links[=N])

       -x     replace external html links by error pages (--replace-external)

       -%x    do not include any  password  for  external  password  protected
              websites (%x0 include) (--disable-passwords)

       -%q    *include  query string for local files (useless, for information
              purpose only) (%q0 don t include) (--include-query-string)

       -o     *generate output html file in case of error (404..)  (o0  don  t
              generate) (--generate-errors)

       -X     *purge old files after update (X0 keep delete) (--purge-old[=N])

       -%p    preserve html  files   as  is   (identical  to   -K4  -%F  ""  )
              (--preserve)

   Spider options:
       -bN    accept  cookies  in  cookies.txt  (0=do  not  accept,* 1=accept)
              (--cookies[=N])

       -u     check document type if unknown (cgi,asp..) (u0 don t check, * u1
              check but /, u2 check always) (--check-type[=N])

       -j     *parse  Java Classes (j0 don t parse, bitmask: |1 parse default,
              |2 don t parse .class |4 don t parse .js |8 don t be aggressive)
              (--parse-java[=N])

       -sN    follow  robots.txt  and  meta robots tags (0=never,1=sometimes,*
              2=always, 3=always (even strict rules)) (--robots[=N])

       -%h    force HTTP/1.0 requests (reduce update features,  only  for  old
              servers or proxies) (--http-10)

       -%k    use  keep-alive if possible, greately reducing latency for small
              files and test requests (%k0 don t use) (--keep-alive)

       -%B    tolerant requests (accept bogus responses on some  servers,  but
              not standard!) (--tolerant)

       -%s    update  hacks: various hacks to limit re-transfers when updating
              (identical size, bogus response..) (--updatehack)

       -%u    url hacks: various hacks to  limit  duplicate  URLs  (strip  //,
              www.foo.com==foo.com..) (--urlhack)

       -%A    assume that a type (cgi,asp..) is always linked with a mime type
              (-%A   php3,cgi=text/html;dat,bin=application/x-zip)   (--assume
              <param>)

       -can   also   be   used   to  force  a  specific  file  type:  --assume
              foo.cgi=text/html

       -@iN   internet protocol (0=both ipv6+ipv4, 4=ipv4 only,  6=ipv6  only)
              (--protocol[=N])

       -%w    disable a specific external mime module (-%w htsswf -%w htsjava)
              (--disable-module <param>)

   Browser ID:
       -F     user-agent field sent in HTTP  headers  (-F  "user-agent  name")
              (--user-agent <param>)

       -%R    default referer field sent in HTTP headers (--referer <param>)

       -%E    from email address sent in HTTP headers (--from <param>)

       -%F    footer string in Html code (-%F "Mirrored [from host %s [file %s
              [at %s]]]" (--footer <param>)

       -%l    preffered language (-%l "fr, en, jp, *" (--language <param>)

   Log, index, cache
       -C     create/use a cache for updates and retries (C0 no cache,C1 cache
              is prioritary,* C2 test update before) (--cache[=N])

       -k     store  all  files  in  cache  (not  useful  if  files  on  disk)
              (--store-all-in-cache)

       -%n    do not re-download locally erased files (--do-not-recatch)

       -%v    display on screen filenames downloaded (in  realtime)  -  *  %v1
              short version - %v2 full animation (--display)

       -Q     no log - quiet mode (--do-not-log)

       -q     no questions - quiet mode (--quiet)

       -z     log - extra infos (--extra-log)

       -Z     log - debug (--debug-log)

       -v     log on screen (--verbose)

       -f     *log in files (--file-log)

       -f2    one single log file (--single-log)

       -I     *make an index (I0 don t make) (--index)

       -%i    make  a  top  index  for  a  project  folder  (* %i0 don t make)
              (--build-top-index)

       -%I    make an searchable index for this mirror  (*  %I0  don  t  make)
              (--search-index)

   Expert options:
       -pN    priority mode: (* p3) (--priority[=N])

       -p0    just scan, don t save anything (for checking links)

       -p1    save only html files

       -p2    save only non html files

       -*p3   save all files

       -p7    get html files before, then treat other files

       -S     stay on the same directory (--stay-on-same-dir)

       -D     *can only go down into subdirs (--can-go-down)

       -U     can only go to upper directories (--can-go-up)

       -B     can    both    go   up&down   into   the   directory   structure
              (--can-go-up-and-down)

       -a     *stay on the same address (--stay-on-same-address)

       -d     stay on the same principal domain (--stay-on-same-domain)

       -l     stay on the same TLD (eg: .com) (--stay-on-same-tld)

       -e     go everywhere on the web (--go-everywhere)

       -%H    debug HTTP headers in logfile (--debug-headers)

   Guru options: (do NOT use if possible)
       -#X    *use  optimized  engine   (limited   memory   boundary   checks)
              (--fast-engine)

       -#0    filter     test     (-#0     *.gif      www.bar.com/foo.gif    )
              (--debug-testfilters <param>)

       -#1    simplify test (-#1 ./foo/bar/../foobar)

       -#2    type test (-#2 /foo/bar.php)

       -#C    cache list (-#C  *.com/spider*.gif  (--debug-cache <param>)

       -#R    cache repair (damaged cache) (--repair-cache)

       -#d    debug parser (--debug-parsing)

       -#E    extract new.zip cache meta-data in meta.zip

       -#f    always flush log files (--advanced-flushlogs)

       -#FN   maximum number of filters (--advanced-maxfilters[=N])

       -#h    version info (--version)

       -#K    scan stdin (debug) (--debug-scanstdin)

       -#L    maximum number of links (-#L1000000) (--advanced-maxlinks)

       -#p    display ugly progress information (--advanced-progressinfo)

       -#P    catch URL (--catch-url)

       -#R    old FTP routines (debug) (--repair-cache)

       -#T    generate transfer ops. log every minutes (--debug-xfrstats)

       -#u    wait time (--advanced-wait)

       -#Z    generate    transfer    rate    statictics     every     minutes
              (--debug-ratestats)

       -#!    execute a shell command (-#! "echo hello") (--exec <param>)

   Dangerous options: (do NOT use unless you exactly know what you are doing)
       -%!    bypass  built-in  security limits aimed to avoid bandwith abuses
              (bandwidth,              simultaneous               connections)
              (--disable-security-limits)

       -IMPORTANT
              NOTE: DANGEROUS OPTION, ONLY SUITABLE FOR EXPERTS

       -USE   IT WITH EXTREME CARE

   Command-line specific options:
       -V     execute  system command after each files ($0 is the filename: -V
              "rm ") (--userdef-cmd <param>)

       -%U    run the engine with another id when called as root  (-%U  smith)
              (--user <param>)

       -%W    use   an   external   library   function   as   a  wrapper  (-%W
              myfoo.so[,myparameters]) (--callback <param>)

   Details: Option N
       -N0    Site-structure (default)

       -N1    HTML in web/, images/other files in web/images/

       -N2    HTML in web/HTML, images/other in web/images

       -N3    HTML in web/,  images/other in web/

       -N4    HTML in web/, images/other in web/xxx, where  xxx  is  the  file
              extension (all gif will be placed onto web/gif, for example)

       -N5    Images/other in web/xxx and HTML in web/HTML

       -N99   All files in web/, with random names (gadget !)

       -N100  Site-structure, without www.domain.xxx/

       -N101  Identical to N1 exept that "web" is replaced by the site s name

       -N102  Identical to N2 exept that "web" is replaced by the site s name

       -N103  Identical to N3 exept that "web" is replaced by the site s name

       -N104  Identical to N4 exept that "web" is replaced by the site s name

       -N105  Identical to N5 exept that "web" is replaced by the site s name

       -N199  Identical to N99 exept that "web" is replaced by the site s name

       -N1001 Identical to N1 exept that there is no "web" directory

       -N1002 Identical to N2 exept that there is no "web" directory

       -N1003 Identical to N3 exept that there is no "web"  directory  (option
              set for g option)

       -N1004 Identical to N4 exept that there is no "web" directory

       -N1005 Identical to N5 exept that there is no "web" directory

       -N1099 Identical to N99 exept that there is no "web" directory

   Details: User-defined option N
          %n  Name of file without file type (ex: image)
          %N  Name of file, including file type (ex: image.gif)
          %t  File type (ex: gif)
          %p  Path [without ending /] (ex: /someimages)
          %h  Host name (ex: www.someweb.com)
          %M  URL MD5 (128 bits, 32 ascii bytes)
          %Q  query string MD5 (128 bits, 32 ascii bytes)
          %r  protocol name (ex: http)
          %q  small query string MD5 (16 bits, 4 ascii bytes)
             %s?  Short name version (ex: %sN)
          %[param]  param variable in query string
          %[param:before:after:empty:notfound]  advanced variable extraction

   Details: User-defined option N and advanced variable extraction
          %[param:before:after:empty:notfound]

       -param : parameter name

       -before
              : string to prepend if the parameter was found

       -after : string to append if the parameter was found

       -notfound
              : string replacement if the parameter could not be found

       -empty : string replacement if the parameter was empty

       -all   fields, except the first one (the parameter name), can be empty

   Details: Option K
       -K0    foo.cgi?q=45  ->  foo4B.html?q=45 (relative URI, default)

       -K     ->    http://www.foobar.com/folder/foo.cgi?q=45  (absolute  URL)
              (--keep-links[=N])

       -K4    ->  foo.cgi?q=45 (original URL)

       -K3    ->  /folder/foo.cgi?q=45 (absolute URI)

   Shortcuts:
       --mirror
                   <URLs> *make a mirror of site(s) (default)

       --get          <URLs>  get the files indicated, do not seek other  URLs
              (-qg)

       --list   <text file>  add all URL located in this text file (-%L)

       --mirrorlinks
              <URLs>  mirror all links in 1st level pages (-Y)

       --testlinks
                <URLs>  test links in pages (-r1p0C0I0t)

       --spider
                   <URLs>   spider  site(s),  to  test links: reports Errors &
              Warnings (-p0C0I0t)

       --testsite
                 <URLs>  identical to --spider

       --skeleton
                 <URLs>  make a mirror, but gets only html files (-p1)

       --update
                           update a mirror, without confirmation (-iC2)

       --continue
                         continue a mirror, without confirmation (-iC1)

       --catchurl
                         create a temporary proxy to capture an URL or a  form
              post URL

       --clean
                            erase cache & log files

       --http10
                           force http/1.0 requests (-%h)

   Details: Option %W: External callbacks prototypes
   see htsdefines.h

FILES

       /etc/httrack.conf
              The system wide configuration file.

ENVIRONMENT

       HOME   Is  being used if you defined in /etc/httrack.conf the line path
              ~/websites/#

DIAGNOSTICS

       Errors/Warnings are reported to hts-log.txt by default, or to stderr if
       the -v option was specified.

LIMITS

       These  are  the principals limits of HTTrack for that moment. Note that
       we did not heard about any other utility that would have solved them.

       - Several scripts generating complex filenames may not find  them  (ex:
       img.src=’image’+a+Mobj.dst+’.gif’)

       - Some java classes may not find some files on them (class included)

       -  Cgi-bin  links  may  not  work  properly  in  some cases (parameters
       needed). To avoid them: use filters like -*cgi-bin*

BUGS

       Please reports bugs to <bugs@httrack.com>.  Include a  complete,  self-
       contained  example  that  will  allow the bug to be reproduced, and say
       which version of httrack you are using. Do not forget to detail options
       used, OS version, and any other information you deem necessary.

COPYRIGHT

       Copyright (C) Xavier Roche and other contributors

       This program is free software; you can redistribute it and/or modify it
       under the terms of the GNU General Public License as published  by  the
       Free Software Foundation; either version 2 of the License, or any later
       version.

       This program is distributed in the hope that it  will  be  useful,  but
       WITHOUT   ANY   WARRANTY;   without   even   the  implied  warranty  of
       MERCHANTABILITY or FITNESS FOR  A  PARTICULAR  PURPOSE.   See  the  GNU
       General Public License for more details.

       You should have received a copy of the GNU General Public License along
       with this program; if not, write to the Free Software Foundation, Inc.,
       59 Temple Place - Suite 330, Boston, MA  02111-1307, USA.

AVAILABILITY

       The   most   recent  released  version  of  httrack  can  be  found at:
       http://www.httrack.com

AUTHOR

       Xavier Roche <roche@httrack.com>

SEE ALSO

       The      HTML      documentation       (available       online       at
       http://www.httrack.com/html/  )  contains  more  detailed  information.
       Please  also  refer  to  the   httrack   FAQ   (available   online   at
       http://www.httrack.com/html/faq.html )

httrack website cHTTrack version 3.43-8 (compiled Jan  1 2010)      httrack(1)