Man Linux: Main Page and Category List


       dphys-config - daily auto-install/update and/or remove config files


       dphys-config [-f filter] [-cqvD]

       dphys-config -h


       dphys-config  installs/updates  and/or  removes  config  files. It also
       triggers commands after an new/updated  config  file  is  available  or
       before  an  existing config file will disappear. It can be run by hand,
       from cron and/or from init.d.

       Get an list of config files from an configuration server. For each file
       in  the  list retrieve that file from the same server, and only install
       it if it is new or changed relative to what is already here. If a  file
       is  newly  installed (or changed) then run an postinstall script, which
       may trigger actions which are wanted to process the new config (such as
       inserting  data  from  an  config  file  into an database). Also remove
       unwanted files. If doing so first run an preremove script  to  tidy  up

       This  is  part  of  the  D-PHYS  (ETH  Zuerich, Departement of Physics)
       automatic system operation and maintenance setup.


       -c     configname: Use this set of config  files  instead  of  hostname
              set. Useful for chroot or vhost installs, or for tests.

       -f  filter
              filter: Only process lines which match the filter spec.

       -q     quiet: Don’t produce an running report of activities.

       -v     verbose: Give large volume output, where sensible.

       -D     Debug: Activate an debug option. See source for how to use this.

       -h     help: Output help text, and then abort operation.


       The  config  files  /etc/dphys-config  (sitewide)  and  ~/.dphys-config
       (personal)  allow the admin and users to set up the working environment
       for dphys-config.

       These config files are sh script fragments full of  assignments,  which
       are  sourced,  in  above row, later config files assignments overriding
       earlier ones.  Standard sh syntax rules apply. Assignments are:

              Sets the base directory in which all temporary files are stored.
              It  defaults  to  /var/tmp (for enough size and safe operation).
              Some users may like to use /tmp  for  higher  speed  (tmpfs)  or
              automatic  deletion  at  boot  time.   Standard  sh syntax rules
              apply. Assignments are:

              Sets  the  base  URL  to   which   all   */<hostname>/<filename>
              combinations  are  added when wget-ing config files. This can be
              an http: or ftp: or  whatever  other  type  of  URL  which  wget
              understands  and  can fetch an file from. Additionally it can be
              an file: (this may be from an NFS server)  URL,  in  which  case
              wget  is  bypassed  and  the files fetched directly using cp. It
              defaults to the error message generating  and  aborting  invalid
              setting                                                       of
              as  there  is no sensible default possible. You must set this to
              where ever your config files should be taken from.

              Selects the name for which set of configuration files  shall  be
              used for this host. Defaults to hostname.

              Sets  an  regexp  which selects which lines from the config file
              list are processed. Defaults to .* (all).

              Log to syslog that dphys-config has run. Good  to  see  if  cron
              and/or init.d have done their job. Defaults to yes.

       The  config  file  list  dphys-config.list,  which  is  found via above
       settings,   and   is   downloaded    to    /etc/dphys-config.list    or
       ~/.dphys-config.list,  then  allows the admin to list what config files
       are to be fetched and installed/updated or removed, and what scripts to
       run for them. These can be each given for the entire site (= all hosts)
       and/or group and/or each host,  or  even  merged  from  site+group+host


              site admin config

              users personal config

              roots config file list gets stored here

              users config file list gets stored here

              site-global (all hosts) common (usually, or group-global or host
              specific) config file list

              facultative   host-specific    (usually)    or    group-specific
              include-able subsection(s) to be added to above config file list
              dphys-config.list. We often use *.group (one per group of users)
              and  *.host  (per  host), sometimes also *.base (all host types)
              and   *.workstation   (only   workstation   hosts)   subsections
              $CONF_BASEURL/hostname/<file-name>    actual    config   files
              referred to in config file list,  common  section  (usually  the
              only       section)       $CONF_BASEURL/hostname/<file-name>.*
              facultative   host-specific    (usually)    or    group-specific
              include-able  subsection(s)  to  be  added  to above config file


       The config file list to be used for checking  what  files  need  to  be
       installed/updated  or removed and its subsections included by #@include
       lines are merged to one list file, analog to cpp #include.

       These are all fetched via wget (or cp for file:), adding their names to
       the  user-defined  base  URL  in  CONF_BASEURL,  and  then  merged.  So
       CONF_BASEURL can be any URL that wget  understands  http:  or  ftp:  or
       whatever else, or file:.

       The  format  of  the resulting concatenated file must consist of lines,
       one per config file, of following format:


       Where the 3 fields have following meanings:

                     Name of the config file to be installed/updated. Must  be
                     only the base part of the filename on the server, without
                     URL and hostname before it, and without any .* subsection
                     endings  after  it,  as these are all auto-added whenever
                     they are needed. If this is set to - the  line  specifies
                     an config file to be removed

                     Full  directory  or full filename (directory+filename) of
                     where the file is to be placed on the target  system.  If
                     only  an  directory  is  given (any name that ends in /),
                     then the above file-name (inclusive  any  directories  in
                     it)  will be automatically added to it. For removing this
                     must be the full filename (or an directory name  (without
                     an  /)  if  an  entire  directory  and its contents shall
                     disapper).  An  directory  name  ending  with  /  is  not
                     processed, to prevent incomplete edits (filename replaced
                     by -, but not added to  directory)  from  killing  entire
                     directories (such as say all of /etc/ :-))

                     Full  command (directory+filename, with parameters) of an
                     command to be run, after this config file has been  newly
                     installed  or changed/updated, or before this config file
                     is removed. This can also be multiple commands  separated
                     by  ;  separators.  Useful for doing chown/chmod to files
                     that need it. If the marker {} appears  in  the  command,
                     this  will be substituted by the filename the config file
                     is going to be installed as. This is analog to find -exec
                     filename substitution

       Lines  which  begin  with an # are regarded as comments, and don’t have
       any effect anything (Lines extended with one are chopped  off  at  that
       point). The same applies for empty lines.


       If  the  first line of the config file list, or any config file fetched
       on  its  behalf,  has  the  special  format   #@dphys-config-preprocess
       [action...]  then  this line will be stripped, and the rest of the file
       will be preprocessed. Depending on the  list  of  actions  present  and
       their  order  (repeats  are  allowed)  the file will be procesed. Valid
       actions are:

                     Anything inside backticks (‘‘)  will  be  executed  as  a
                     command,  and its stdout will then be substituted for the
                     ‘‘ expression. This is analog to sh backtick substitution

              if     For  any  line  beginning with #@if the stuff between the
                     #@if and the  first  ;  character  will  be  executed  as
                     command,  and  if it returns true, everything after the ;
                     will be left, else the entire line will be removed.  This
                     is analog to shell if ... ; conditional execution

                     For  any  line  beginning  with #@include the rest of the
                     line is regarded as an subsection  name,  which  will  be
                     added  to  the  base  filename,  and  then  the resulting
                     subsection  file  fetched  (also  by  wget  or  cp)   and
                     substituted  for  the  line.  This  is  analog  to  an  C
                     preprocessor #include oder an shell .  include


       The following allows you to fetch all your config file  lists  from  an
       HTTP  VirtualHost  called under its subdirectory

       In file /etc/dphys-config, on every host, so it  can  find  the  config
       file server:

       # system will use ${CONF_BASEURL}/‘hostname‘/<file-name>*

       We    advise    using    an    subdirectory    here,    because   other
       /* directories may already  contain  other
       admin stuff you put on the same VirtualHost. Such as software packages,
       site news, etc.

       For dphys-config to be useful you then need to make config  file  lists
       for  it.   And  provide  the actual config files that can be installed,
       driven by the lists.  This is the largest job, as it basically  amounts
       to  extracting all your relevant config work from your site. Also known
       as reengineering your site.

       Assuming  your  VirtualHost  on   has   as   its
       DocumentRoot /vhost/www.admin, you would then begin with an pseudo-host
       Directory         for         site-global         common         stuff:

       If  your hosts are organised in groups with group-global common configs
       (such as professors, students, staff), make  an  pseudo-host  for  each
       group,  such  as:  /vhost/www.admin/dphys-config/PROFS/ and */STUDENTS/
       and */STAFF/.

       Then for host specific stuff, assuming systems called
       to,  stud1  to stud20, staff1 to staff5, server1 and
       server2,      make      for      each      its      own      directory:
       /vhost/www.admin/dphys-config/prof1/ (and so on).

       Note  that we suggest using CAPITALS for pseudo-hosts and lowercase for
       actual hosts. This avoids name space collisions. You can also use loops
       like  for  host  in  [a-z]*  ;  do  ...  ; done to work (say generating
       symlinks to an new config file in all hosts). Well at least you can  do
       this  so long no one goes and sets LANGUAGE= or similar junk, then bash
       (or libc?) will hapily screw up case  sensitivity  and  produce  random
       lossage (yes, it was painfull).

       After  this  add  to  /vhost/www.admin/dphys-config/SITE/,  the  actual
       config files as far as they are not host specific, or at least have  an
       common  section to all hosts. Example this would be /etc/hosts for all,
       an common section for /etc/motd, common or all for,  common
       for inetd.conf, nothing for the ssh hostkeys.

       Then  add,  to  an  group, say /vhost/www.admin/dphys-config/STUDENTS/,
       whatever is specific to that group.  Example  this  may  be  an  entire
       special  motd  for  the  many  changing users, or just an to
       #@include into the common one.

       Then for each host in its /vhost/www.admin/dphys-config/prof1/ (or  so)
       add  all that is specific to it. Such as its ssh key files. And its own, it it needs one. Same its if it is going  to
       offer special stuff. An configs for services only this host has such as

       Then for each host add symlinks to the SITE or group versions  that  it
       is       to      use      for      common      stuff,      like      on

        .../dphys-config/stud1/dphys-config.list -> ../SITE/dphys-config.list
        .../dphys-config/stud1/hosts -> ../SITE/hosts
        .../dphys-config/stud1/inetd.conf -> ../SITE/inetd.conf
        .../dphys-config/stud1/motd -> ../SITE/motd
        .../dphys-config/stud1/ -> GROUP/motd
        .../dphys-config/stud1/GROUP -> ../STUDENTS
        .../dphys-config/stud1/ -> ../SITE/

       In  the   /vhost/www.admin/dphys-config/SITE/   directory   place   the
       site-global  common  dphys-config.list  for  all your hosts, containing
       stuff like this:

       # SITE dphys-config.list - just example stuff, for our exemplaric site
       # basics
       hosts:/etc/                      # simply works, no command
       motd:/etc/                       # this will be assembled group specific
       inetd.conf:/etc/:/etc/init.d/inetd restart  # needs an command to reload restart  # not in /etc
       # ssh restart only after last file, and ensure file modes for each file
       ssh_host_key:/etc/ssh/:chown root:root {}; chmod 600 {}
       ssh_host_rsa_key:/etc/ssh/:chown root:root {}; chmod 600 {}
       ssh_host_dsa_key:/etc/ssh/:chown root:root {}; chmod 600 {}; /etc/init.d/sshd restart
       # load stuff into an existing database file
       seed.debconf:/etc/:debconf-set-selections {}
       # other stuff
       daemon1-conf:/etc/daemon1/conf   # rename so names can differ on server
       daemon1/conf:/etc/               # same as above, but with directories on server
       testing:/etc/                    # put something in there for an test
       # delete some stuff
       -:/etc/testing                   # change to above test to get rid of it again
       -:/etc/                          # you will get a warning if you leave this
       #-:/etc                          # you would reinstall your system after the resulting  rm -rf /etc  :-)
       # and some errors
       #only-an-name                    # you would get an error: no place on target
       #only-an-name:                   # you would get an error: no place on target
       #:only-an-place                  # you would get an error: no file to install

       For special services add an on  each  host  that
       has   special   config   files  not  present  on  others,  such  as  on

       # server2 - only used on our web server
       httpd.conf:/etc/apache/httpd.conf:/etc/init.d/apache restart

       You can also use  dphys-config  to  run  arbitrary  commands,  whenever
       config  files  are  installed/updated  or  removed,  to modify existing
       config files, or more likely modify complex config databases which  can
       not be provided as files, but where one can provide edit info as files.

       dphys-config can even install scripts to use as above commands (or even
       just to run scripts while installing), such as into /usr/local/sbin/.

       For  this  make  an ../SITE/local/sbin/ directory, place the scripts in
       there (such as ../SITE/local/sbin/dphys-config-<whatever>), and symlink
       local  to ../SITE/local on each host, and then add config lines for the
       scripts, with the command to trigger them, giving something like this:

       local/sbin/dphys-config-<whatever>:/usr/:chmod 755 {}; {}  # chmod and run

       It this script processes an config file your will want it to be run  if
       either  the  script or the config file is updated, so add the script to
       the laters line as well:

       dphys-config-<whatever>:/etc/:if [ -x /usr/local/sbin/dphys-config-<whatever> ] ; then /usr/local/sbin/dphys-config-<whatever> ; fi  # run also here

       Finally, new hosts can then later simply be added, by  making  the  new
       hosts  directory  and  copying  all files and symlinks from an existing
       host of the same group. Such as by doing:

       mkdir student21
       tar -cf - -C student1 . | tar xpf - -C student21

       To then run dphys-config by hand (say for tests), type on the host:


       But usually you will want  to  run  dphys-config  automatically,  every
       night  (or  if  a  machine was/is switched off, at every boot), to keep
       your configs up to date.

       For nightly updates the best thing is to use an cron job on every host.
       03:00  to  03:59  is  most likely idle time. Use an line like this one,
       with the cron option to avoid an load peak on the config  file  server,
       by  random  delaying  the  run  by 0..3599 seconds, and with stdout and
       strerr thrown away to avoid getting an mail from every host,  as  error
       messages are also allways sent to syslog:

       0 3 * * * root dphys-config cron > /dev/null 2>&1

       To  catch  machines  switched off over night, with no cron run on them,
       also run an init.d script. Use an  script  like  this  one,  also  with
       stdout  and  stderr  thrown  away to avoid cluttering your boot console

       # /etc/init.d/dphys-config - boot time automatic config updates, if no cron
       case "$1" in
           dphys-config init > /dev/null 2>&1
       exit 0


       If dphys-config is to be used to distribute all config files, this will
       also  include  files  which  are security relevant, such as ssh private
       keys (host key or  (root)  user  authentification),  SSL  certificates,
       passwd and shadow, lilo.conf, software license keys, etc.

       As  all  files are most likely fetched from an http: URL, measures must
       be taken to secure the config website  from  other  people  downloading
       them.  We  here  use  an restriction to only IP addresses registered as
       hosts in our NIS server, and additionally run  identd  on  all  allowed
       hosts,  and  require the wget process opening the HTTP connection to be
       running by user root, and so also require dphys-config to run as  root.

       To avoid sniffing it is recommended to give wget an https: URL.


       Config  files  are  read  by wget from an webserver, so they lose their
       owners and modes. So the commands triggered on their lines must be used
       to chown/chmod them to proper values.

       When  used  together with dphys-admin, dphys-config should run as first
       (earlier cron and init.d entries). This is needed  to  provide  configs
       before  new packages are installed, so dphys-admin can pretend that the
       packages were already once installed (and then non-purge removed),  and
       so  prevent  questions  on  install,  which  is required for unattended
       installs. [Note that this pretending does not  go  as  far  as  setting
       debconf  up.  Broken packages that ignore config files and only look at
       debconf will still ask questions.]

       As result of this, when installing for the first time on an new  system
       (such  as  installing  Debian  by  the  dphys3 end2stage feature, which
       installs  first  dphys-config  and  then  dphys-admin),   any   scripts
       installed  by  packages  by  dphys-admin,  to  be called on config file
       install/update will still be  missing,  and  so  not  runnable.  Either
       ignore the warnings, or better call the scripts by something like this:

       file:place:if [ -x script ]; then script; fi

       Note that in this case, trying to run dphys-config for  a  second  time
       after   dphys-admin  has  installed  packages  and  scripts,  will  not
       automatically mend this, as the config files have not changed,  and  so
       dphys-config  will  not  (re-)run  their  scripts.  Therefore  packages
       containing such scripts must also, as part of their postinst (or init.d
       which  is called by postinst), check for existing config files and then
       run their scripts. This is the normal behaviour of quite a few packages
       anyway.  Of course this requires the scripts to be idempotent, which is
       official Debian policy anyway.