Man Linux: Main Page and Category List

NAME

       debmirror - Debian partial mirror script, with ftp, http, hftp or rsync
       and package pool support

SYNOPSIS

       debmirror [options] <mirrordir>

DESCRIPTION

       This program downloads and maintains a partial local Debian mirror. It
       can mirror any combination of architectures, distributions, and
       sections. Files are transferred by ftp, and package pools are fully
       supported. It also does locking and updates trace files.

       To support package pools, this program mirrors in three steps.

       1. download Packages and Sources files
           First it downloads all Packages and Sources files for the subset of
           Debian it was instructed to get.

       2. clean up unknown files
           Any files and directories on the local mirror that are not in the
           list are removed.

       3. download everything else
           The Packages and Sources files are scanned, to build up a list of
           all the files they refer to. A few other miscellaneous files are
           added to the list.  Then the program makes sure that each file in
           the list is present on the local mirror and is up-to-date, using
           file size (and optionally md5sum) checks.  Any necessary files are
           downloaded.

OPTIONS

       <mirrordir>
           This required (unless defined in a configuration file) parameter
           specifies where the local mirror directory is. If the directory
           does not exist, it will be created. Be careful; telling this
           program that your home directory is the mirrordir is guaranteed to
           replace your home directory with a Debian mirror!

       --progress -p
           Displays progress bars as files are downloaded.

       --verbose -v
           Displays progress between file downloads.

       --debug
           Enables verbose debug output, including ftp protocol dump.

       --dry-run
           Simulate a mirror run. This will still download the meta files to
           the ./.temp working directory, but won’t replace the old meta
           files, won’t download debs and source files and only simulates
           cleanup.

       --skip-installer foo[,bar,..]
           Don’t download debian-installer files for the specified
           distribution.

       --help
           Display a usage summary.

       --host=remotehost -h
           Specify the remote host to mirror from. Defaults to
           ’ftp.debian.org’, you are strongly encouraged to find a closer
           mirror.

       --root=directory -r directory
           Specifies the directory on the remote host that is the root of the
           Debian archive. Defaults to "debian", which will work for most
           mirrors. The root directory has a ./dists subdirectory.

       --method=ftp|hftp|http|rsync -e
           Specify the method to download files. Currently, supported methods
           are ftp, hftp (ftp over http proxy), http or rsync.

           Note: starting with version 1.1 it is no longer needed to add a ’:’
           prefix for the root directory.

       --passive
           Download in passive mode.

       --user=remoteusername -u
           Specify the remote user name to use to log to the remote host.
           Helpful when dealing with brain damaged proxy servers. Defaults to
           anonymous.

       --passwd=remoteuserpassword
           Specify the remote user password to use to log into the remote ftp
           host.  It is used with --user and defaults to anonymous@.

       --proxy=http://user:pass@url:port/
           Specifies the http proxy (like Squid) to use for http and hftp
           method.

       --dist=foo[,bar,..] -d foo
           Specify the distribution (etch, lenny, squeeze, sid) of Debian to
           mirror. This switch may be used multiple times, and multiple
           distributions may be specified at once, separated by commas. Using
           the links (stable, testing, unstable) does not have the expected
           results but you may add those links manually. Defaults to mirroring
           sid.

       --omit-suite-symlinks
           With this option set, debmirror will not create the
           ’suite -> codename’ symlink. This is needed for example when
           mirroring archived Debian releases as they will all have either
           ’stable’ or ’oldstable’ as suite in their Release files.

       --section=foo[,bar,..] -s foo
           Specify the section of Debian to mirror. Defaults to
           main,contrib,non-free,main/debian-installer.

       --arch=foo[,bar,..] -a foo
           Specify the architectures to mirror. The default is --arch=i386.
           Specifying --arch=none will mirror no archs.

       --adddir directory
           Also download Packages and Sources files from the specified
           directory on the remote host (the directory is relative to the root
           of the Debian archive). This feature is now obsolete and may be
           removed in a future release.

       --rsync-extra=foo[,bar,..]
           Allows to also mirror files from a number of directories that are
           not part of the package archive itself. Debmirror will always use
           rsync for the transfer of these files, irrespective of what
           transfer method is specified in the --method option.

           This option can therefore not be used if your remote mirror does
           not support rsync, or if the mirror needs a different --root option
           for rsync than for the main transfer method specified with
           --method. Excluding individual files in the directories is not
           supported.

           The following values are supported.

           doc
             Download all files and subdirectories in ./doc directory, and all
             README files in the root directory of the archive.

           indices
             Download all files and subdirectories in ./indices directory.
             Note that this directory can contain some rather large files;
             don’t include this type unless you know you need these files.

           tools
             Download all files and subdirectories in ./tools directory.

           trace
             Download the remote mirror’s trace files for the archive
             (./project/trace/*).

           If specified, the update of trace files will be done at the
           beginning of the mirror run; the other types are done near the end.

           This switch may be used multiple times, and multiple values may be
           specified at once, separated by comma’s; unknown values are
           ignored.

       --di-dist=dists | foo[,bar,..]
           Mirror "current" Debian Installer images for the specified dists.
           See further the section "Mirroring Debian Installer images" below.

       --di-arch=arches | foo[,bar,..]
           Mirror "current" Debian Installer images for the specified
           architectures.  See further the section "Mirroring Debian Installer
           images" below.

       --source
           Include source in the mirror (default).

       --nosource
           Do not include source.

       --i18n
           Additionally download Translation-<lang>.bz2 files, which contain
           translations of package descriptions. Selection of specific
           translations is possible using the --include and --exclude options.

       --getcontents
           Additionally download Contents.<arch>.gz files. Note that these
           files can be relatively big and can change frequently, especially
           for the testing and unstable suites. Use of the available diff
           files is strongly recommended (see the --diff option).

       --md5sums -m
           Use md5sums to determine if files on the local mirror that are the
           correct size actually have the correct content. Not enabled by
           default, because it is too paranoid, and too slow.

           When the state cache is used, debmirror will only check md5sums
           during runs where the cache has expired or been invalidated, so it
           is worth considering to use these two options together.

       --ignore-missing-release
           Don’t fail if the Release file is missing.

       --ignore-release-gpg
           Don’t fail if the Release.gpg file is missing. If the file does
           exist, it is mirrored and verified, but any errors are ignored.

       --ignore=regex
           Never delete any files whose filenames match the regex. May be used
           multiple times.

       --exclude=regex
           Never download any files whose filenames match the regex. May be
           used multiple times.

       --include=regex
           Don’t exclude any files whose filenames match the regex. May be
           used multiple times.

       --exclude-deb-section=regex
           Never download any files whose Debian Section (games, doc, oldlibs,
           science, ...) match the regex. May be used multiple times.

       --limit-priority=regex
           Limit download to files whose Debian Priority (required, extra,
           optional, ...) match the regex. May be used multiple times.

       --timeout=seconds -t
           Specifies the timeout to use for network operations (either FTP or
           rsync).  Set this to a higher value if you experience failed
           downloads. Defaults to 300 seconds.

       --max-batch=number
           Download at most max-batch number of files (and ignore rest).

       --rsync-batch=number
           Download at most number of files with each rsync call and then
           loop.

       --rsync-options=options
           Specify alternative rsync options to be used. Default options are
           "-aL --partial". Care must be taken when specifying alternative
           options not to disrupt operations, it’s best to only add to those
           options.

           The most likely option to add is "--bwlimit=x" to avoid saturating
           the bandwidth of your link.

       --postcleanup
           Clean up the local mirror but only after mirroring is complete and
           only if there was no error. This is the default.

       --cleanup
           Do clean up any unknown files and directories on the local mirror
           (see step 2 above).

       --nocleanup
           Do not clean up the local mirror after mirroring is complete.

       --skippackages
           Don’t re-download Packages and Sources files. Useful if you know
           they are up-to-date.

       --diff=use|mirror|none
           If --diff=use is specified and the Release file contains entries
           for diff files, then debmirror will attempt to use them to update
           Packages, Sources and Contents files (which can significantly
           reduce the download size for meta files), but will not include them
           in the mirror. This is the default behavior and avoids having time
           consuming diff files for a fast local mirror.

           Specifying --diff=mirror does the same as ’use’, but will also
           include the downloaded diff files in the local mirror. Specify
           --diff=none to completely ignore diff files.

           Note that if rsync is used as method to download files and the
           archive being mirrored has "rsyncable" gzipped meta files, then
           using --diff=none may be the most efficient way to download them.
           See the gzip(1) man page for information about its rsyncable
           option.

       --gzip-options=options
           Specify alternative options to be used when calling gzip to
           compress meta files after applying diffs. The default options are
           "-9 -n --rsyncable" which corresponds with the options used to gzip
           meta files for the main Debian archive.

           These options may need to be modified if the md5sum of the file as
           gzipped by debmirror does not match the md5sum listed in the
           Release file (which will result in the gzipped file being
           downloaded unnecessarily after diffs were successfully applied).

       --state-cache-days=number
           Save the state of the mirror in a cache file between runs. The
           cache will expire after the specified number of days, at which time
           a full check and cleanup of the mirror will be done. While the
           cache is valid, debmirror will trust that the mirror is consistent
           with this cache.

           The cache is only used for files that have a unique name, i.e.
           binary packages and source files. If a mirror update fails for any
           reason, the cache will be invalidated and the next run will include
           a full check.

           Main advantage of using the state cache is that it avoids a large
           amount of disk access while checking which files need to be
           fetched. It may also reduce the time required for mirror updates.

       --ignore-small-errors
           Normally debmirror will report an error if any deb files or sources
           fail to download and refuse to update the meta data to an
           inconsistent mirror. Normally this is a good things as it indicates
           something went wrong during download and should be retried. But
           sometimes the upstream mirror actually is broken. Specifying
           --ignore-small-errors causes debmirror to ignore missing or broken
           deb and source files but still be pedantic about checking meta
           files.

       --allow-dist-rename
           The directory name for a dist should be equal to its Codename and
           not to a Suite. If the local mirror currently has directories named
           after Suites, debmirror can rename them automatically.  An existing
           symlink codename -> suite will be removed, but debmirror will
           automatically create a new symlink suite -> codename (immediately
           after moving meta files in place). This conversion should only be
           needed once.

USING DEBMIRROR

   Using regular expressions in options
       Various options accept regular expressions that can be used to tune
       what is included in the mirror. They can be any regular expression
       valid in perl, which also means that extended syntax is standard. Make
       sure to anchor regular expressions appropriately: this is not done by
       debmirror.

       The --include and --exclude options can be combined. This combination
       for example will, if the --i18n option is used, exclude all Translation
       files, except for the ones for Portuguese (pt) and Brazillian (pt_BR):

         --exclude='/Translation-.*\.bz2$' --include='/Translation-pt.*\.bz2$'

   Mirroring Debian Installer images
       Debmirror will only mirror the "current" images that are on the remote
       mirror. At least one of the options --di-dist or --di-arch must be
       passed to enable mirroring of the images.

       The special values "dists" resp. "arches" can be used to tell debmirror
       to use the same dists and architectures for D-I images as for the
       archive, but it is also possible to specify different values. If either
       option is not set, it will default to the same values as for the
       archive.

       If you wish to create custom CD images using for example debian-cd, you
       will probably also want add the option "--rsync-extra=doc,tools".

       Limitations

       There are no progress updates displayed for D-I images.

   Archive size
       The tables in the file /usr/share/doc/debmirror/archive_size give an
       indication of the space needed to mirror the Debian archive. They are
       particularly useful if you wish to set up a partial mirror.  Only the
       size of source and binary packages is included. You should allow for
       around 1-4 GB of meta data (in ./dists/<dist>) per suite (depending in
       your settings). Plus whatever space is needed for extra directories
       (e.g. tools, doc) you wish to mirror.

       The tables also show how much additional space is required if you add a
       release on top of its predecessor. Note that the additional space
       needed for testing and (to a lesser extend) unstable varies during the
       development cycle of a release. The additional space needed for testing
       is zero immediately after a stable release and grows from that time
       onwards.

       Note Debmirror keeps an extra copy of all meta data. This is necessary
       to guarantee that the local mirror stays consistent while debmirror is
       running.

EXAMPLES

       Simply make a mirror in /srv/mirror/debian, using all defaults (or the
       settings defined in debmirror.conf):

         debmirror /srv/mirror/debian

       Make a mirror of i386 and sparc binaries, main only, and include both
       unstable and testing versions of Debian; download from
       ’ftp.kernel.org’:

         debmirror -a i386,sparc -d sid -d etch -s main --nosource \
                   -h ftp.nl.debian.org --progress $HOME/mirror/debian

       Make a mirror using rsync (rsync server is ’ftp.debian.org::debian’),
       excluding the section ’debug’ and the package ’foo-doc’:

         debmirror -e rsync $HOME/mirror/debian --exclude='/foo-doc_' \
                   --exclude-deb-section='^debug$'

FILES

         /etc/debmirror.conf
         ~/.debmirror.conf

           Debmirror will look for the presence of these files and load them
           in the indicated order if they exist.
           See the example in /usr/share/doc/debmirror/examples for syntax.

         ~/.gnupg/trustedkeys.gpg

           Debmirror uses gpgv to verify Release and Release.gpg using the
           default keying ~/.gnupg/trustedkeys.gpg. This can be changed by
           exporting GNUPGHOME resulting in $GNUPGHOME/trustedkeys.gpg being
           used.

           To add the right key to this keyring you can import it from the
           debian keyring (in case of the debian archive) using:

             gpg --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export \
                 | gpg --no-default-keyring --keyring trustedkeys.gpg --import

           or download the key from a keyserver:

             gpg --no-default-keyring --keyring trustedkeys.gpg \
                 --keyserver keyring.debian.org --recv-keys <key ID>

           The <key ID> can be found in the gpgv error message in debmirror:
           gpgv: Signature made Tue Jan 23 09:07:53 2007 CET using DSA key ID 2D230C5F

COPYRIGHT

       This program is copyright 2001 by Joey Hess <joeyh@debian.org>, under
       the terms of the GNU GPL (either version 2 of the licence or, at your
       option, any later version), copyright 2001-2002 by Joerg Wendland
       <joergland@debian.org>, copyright 2003-2007 by Goswin von Brederlow
       <goswin-v-b@web.de> and copyright 2009 by Frans Pop <fjp@debian.org>.

       The author disclaims any responsibility for any mangling of your
       system, unexpected bandwidth usage bills, meltdown of the Debian mirror
       network, etc, that this script may cause. See NO WARRANTY section of
       GPL.

AUTHOR

        Current maintainer:
          Frans Pop <fjp@debian.org>

        Previous authors:
          Joey Hess <joeyh@debian.org> (original author)
          Joerg Wendland <joergland@debian.org>
          Goswin von Brederlow <goswin-v-b@web.de>

MOTTO

       Waste bandwith -- put a partial mirror on your laptop today!