NAME
amanda-devices - Configuring and Using Amanda Devices
DESCRIPTION
The Device API specifies a generic interface between Amanda and storage
devices such as tapes or disks. This manual page describes the device
drivers included with Amanda.
This is a user-level description of the API, and does not address
details that are only of concern to developers. For that purpose,
consult the Amanda source code and http://wiki.zmanda.com.
The term "device driver" describes the software that can communicate
with some kind of backend storage, e.g., a tape driver. A "device" is
the storage element itself, usually a piece of hardware. When
discussing a device and its driver as a unit, the term "device" is
sometimes also used to refer to the combination of device and driver.
SPECIFYING DEVICES
Device names take the form TYPE:NODE, where TYPE selects a device
driver, and NODE provides further information to that driver. The
syntax for each device driver is given in the corresponding section
below.
Devices can be described in amanda.conf(5) with "device" sections,
e.g.,
define device top_drive {
tapedev "tape:/dev/nst0"
device_property "BLOCK_SIZE" "131072"
}
Such a device defininition creates a device "alias", in this case named
top_drive, which can then be named in the global tapedev or tpchanger
parameter:
tapedev "top_drive"
The global tapedev parameter can also specify a literal device name.
For example,
tapedev "file:/amdisks"
is equivalent to
tapedev "default"
define device default {
tapedev "file:/amdisks"
}
Note that, in both cases, the specified devices are actually accessed
through the chg-single changer driver; see amanda-changers(7) for more
information.
Device properties specified outside of any device definition apply to
all devices. This syntax is provided mainly for backward compatibility,
and for simple Amanda configurations. Note that there is no way to
provide properties specific to a device without defining a device
alias.
See amanda-changers(7) for details on how devices are configured, and
in particular on how device properties are specified. See
amanda.conf(5) for more information on Amanda configuration in general.
Note
There is no way to reset a device property to its default value.
PROPERTIES
Device drivers use properties as a generic means to interact with other
parts of Amanda. Some properties are set by the device driver and used
by Amanda to determine how its devices should be used. Other properties
can be set by Amanda or by the user to influence the driver´s behavior.
Properties are set for a particular device, so that if you have two
tape devices, they will not share property values.
Properties are specified in amanda.conf with the device-property
parameter. The syntax looks like this:
device_property "FROBNICATOR_PATH" "/var/frobd/state"
device_property "BYTES_PER_FORTNIGHT" "128k"
device_property "USE_QUBITS" "no"
Both the property name and the property value are always quoted.
Property names, like Amanda configuration parameters, are not
case-sensitive, and - (dash) and _ (underscore) may be used
interchangeably. String values are given as simple strings, like
FROBNICATOR_PATH in the example above. Integer values can be specified
with any of the suffixes given in the "VALUE SUFFIXES" section of
amanda.conf(5), like BYTES_PER_FORTNIGHT, above. Boolean values can be
specified using the same names as in amanda.conf(5), like USE_QUBITS,
above. Some properties have special formats, as described below.
Some properties are set based on other configuration values, such as
tapetype parameters. These special cases are detailed under the
appropriate property, below.
The order in which device properties are set is as follows:
1. Tapetype parameters (including length, blocksize, and
readblocksize) are translated into device properties and set
accordingly.
2. Device properties from any device_property configuration parameters
are set, in the order they appear in the configuration file.
Properties described as read-only are not accessible to users. They are
listed here for completeness.
COMMON PROPERTIES
Note that some of these properties are currently unused, and present
only for future expansion. Not all devices implement all of these
properties.
APPENDABLE
(read-only) This boolean property indicates whether this device
supports appending data to volumes.
BLOCK_SIZE
(read-write) This property gives the block size, in bytes, that
will be used to write to the device. The usual suffixes ("kbytes",
etc.) are allowed. The tapetype parameter blocksize sets this
property.
CANONICAL_NAME
(read-only) This property contains the full canonical name for
this device. This name may not be the same as the user-supplied
name, but is a valid name by which to access this device.
COMMENT
(read-write) This string property is entirely for the user´s
convenience. It is supported by all devices, but no device
interprets its value in any way.
COMPRESSION
(read-write) This boolean property represents the compression
status of the device, and can be used to enable and disable such
compression. This applies mostly to tape devices, although many
tape devices do not support setting compression from software.
COMPRESSION_RATE
(read-only) This property gives the compression rate, as a decimal
ratio. It may be a measured value over some unspecified period or
a simple estimate.
CONCURRENCY
(read-only) This property indicates the level of concurrent access
that this device supports.
FREE_SPACE
(read-only) This property gives the amount of free space available
on the current volume, if known. This is often an estimate; for
example, tape devices can only estimate the amount of tape left on
a spool.
FULL_DELETION
(read-only) This property indicates whether the device supports
erasing the entire volume. Aside from S3 and VFS, most devices
cannot support this feature.
MAX_BLOCK_SIZE
(read-only) This property gives the maximum block size this device
can support. See BLOCK SIZES, below.
MEDIUM_ACCESS_TYPE
(read-only) This property gives the type of the media in the
device: read only, WORM (Write Once, Read Many), read/write, or
write only. Write-only devices do not support recovery, but the
data are not necessarily thrown out.
MIN_BLOCK_SIZE
(read-write) This property gives the minimum block size this
device can support. See BLOCK SIZES, below.
MAX_VOLUME_USAGE
(read-write) On devices that support it, this property will limit
the total amount of data written to a volume; attempts to write
beyond this point will cause the device to simulate "out of space."
Zero means no limit. The tapetype parameter length sets this
property.
PARTIAL_DELETION
(read-only) This property indicates whether the device supports
deletion of specific files. Aside from linear tapes and S3, most
devices can support this feature. It is currently unused by
Amanda.
STREAMING
(read-only) This property gives the streaming requirement for this
device. For example, tape drives often require a steady supply of
data to avoid shoe-shining, while disk devices have no such
requirement.
VERBOSE
(read-write) If this boolean property is set, then the device will
produce verbose debugging output. This property is not recognized
by most devices.
BLOCK SIZES
Amanda writes device data in blocks. On most devices the block
boundaries are embedded in the media along with the data itself, so
subsequent reads must use the same block sizes. On tape devices, the
block size is dictated by the capabilities of the hardware -- buffer
sizes, physical format, and so on.
Amanda has historically supported a single, fixed block size -- usually
32k. The Device API adds the ability to specify a block size at
runtime, using the BLOCK_SIZE property. Devices provide MIN_BLOCK_SIZE
and MAX_BLOCK_SIZE as a guide to the range of acceptable block sizes.
Note that this does not imply that all sizes in the range
MIN_BLOCK_SIZE - MAX_BLOCK_SIZE are available -- the device may require
that block sizes are even multiples of some power of two, for example.
Consult the documentation for your hardware and operating system for
more information.
Most devices are flexible enough to read a volume using a different
block size than that with which it was written. This can be useful when
handling old volumes written with a smaller blocksize, or volumes of
unknown blocksize. Unfortunately, some tape devices do not detect
oversized blocks correctly, and may lose data if the configured block
size is smaller than the volume´s block size. The tape device driver
has a READ_BLOCK_SIZE property which specifies the minimum buffer size
that will be allocated for reads from tape. If the hardware supports
it, setting this property allows Amanda to correctly read from tapes
written with any blocksize less than or equal to READ_BLOCK_SIZE.
Note
The RAIT device does not support flexible block sizes, as its
parity algorithm requires that all child devices have the same,
fixed block size.
DEVICES
This section lists the device drivers included with Amanda, and basic
instructions for using them. For complete How-To information, consult
the Amanda wiki at http://wiki.zmanda.com.
Null Device
tapedev "null:"
The null device driver only supports writing, and discards all data. It
is generally only useful for testing purposes.
RAIT Device
tapedev "rait:tape:/dev/rmt/tps0d{4,5,6}n"
The RAIT device driver mirrors or stripes data over multiple "child"
devices. The child devices are specified using a shell-like syntax,
where alternatives are enclosed in braces and separated by commas.
Braces and commas can be escaped with a backslash. Note that the
backslash itself must be escaped in most contexts. For example:
tapedev "rait:{commandev:foo\\,bar,bracedev:foo\\}bar}"
With two child devices, the RAIT device driver mirrors data such that
the two devices contain identical data and can be used singly for
recovery. With more than two devices, the RAIT device "stripes" data
across all but one device and writes a parity block to the final
device, usable for data recovery in the event of a device or volume
failure. The RAIT device scales its blocksize as necessary to match the
number of children that will be used to store data.
When a child device is known to have failed, the RAIT device should be
reconfigured to replace that device with the text "ERROR", e.g.,
tapedev "rait:{tape:/dev/st0,ERROR,tape:/dev/st2}"
This will cause the RAIT device to start up in degraded mode,
reconstructing the data from the missing device.
Like ordinary RAID drivers, the RAIT device driver can automatically
enter degraded mode when one of its child devices fails. However, the
RAIT device cannot automatically recover from any write error nor write
any data in degraded mode. When reading, certain errors may be fatal
(rather than causing degraded mode). And in any case, labels on all
volumes must initially match (labeled or otherwise). If you have lost
one volume from a set, explicitly start the device in degraded mode as
described above.
Child Device Block Sizes
The RAIT device driver requires that all of its child devices use the
same block size. If no block sizes are specified, the driver selects
the block size closest to 32k that is within the MIN_BLOCK_SIZE -
MAX_BLOCK_SIZE range of all child devices, and calculates its own
blocksize according to the formula rait_blocksize = child_blocksize *
(num_children - 1). If a block size is specified for the RAIT device,
then it calculates its child block sizes according to the formula
child_blocksize = rait_blocksize / (num_children - 1). Either way, it
sets the BLOCK_SIZE property of each child device accordingly.
S3 Device
tapedev "s3:foocorp-backups/DailySet1-"
device_property "S3_ACCESS_KEY" "MYACCESSKEY"
device_property "S3_SECRET_KEY" "MYSECRETKEY"
The S3 device driver uploads data to the Amazon S3 "storage cloud". Its
device name is a slash-sparated combination of bucket name and prefix:
"s3:BUCKET/PREFIX". Since buckets must be unique across all Amazon S3
users, and since the number of buckets allowed to each user is limited,
the driver can store multiple Amanda volumes in a single S3 bucket,
distinguished by prefix. The prefix and slash can be omitted if they
are not needed: "s3:BUCKET".
The access and secret keys used to authenticate to Amazon S3 are
provided as properties.
The S3 device driver stores each block in a distinct S3 object. Due to
high HTTP overhead for each request, use of larger than normal block
sizes (> 1 megabyte) is reccomended with the S3 device.
Amanda automatically creates a bucket when writing, if the bucket
doesn´t already exist. At that time, it specifies where Amazon should
store the data based on the S3_BUCKET_LOCATION property. Currently,
there are two valid settings: "*" (any location, probably US) and "EU"
(Europe). If this property is not set, Amazon´s default value of "*" is
used. The bucket location has both billing and legal concerns, so you
are encouraged to consult Amazon´s documentation for details.
Amazon does not permit changes to bucket locations, so this is a
permanent specification. If the bucket already exists and the property
is set, then Amanda checks the property against the location of the
bucket, and produces an error if they do not match.
Note
If a location constraint is set, the bucket name must consist only
of lower-case letters, numbers, dashes, and dots.
This driver supports the VERBOSE property, but use it carefully -- it
produces a great deal of output, and may cause spurious failures by
filling your debug log partition. Its logging is generally only useful
for developers chasing down a problem in communications with Amazon´s
servers.
Device-Specific Properties
In addition to the common properties, the S3 device supports the
properties listed in this section.
Most Amanda devices work just fine without any properties, but not the
S3 device. A typical S3 configuration will have an access key and
secret key specified:
device_property "S3_ACCESS_KEY" "27D3B8C6C4E7AA423C2B37C72A0D22C8"
device_property "S3_SECRET_KEY" "agphc2Q7Zmxragphc2RmO2xragpzZGY7a2xqCgr"
MAX_RECV_SPEED
(read-write) Maximum speed, in bytes per second, that this device
will receive data from S3. If the average speed exceeds this
value, the device will stop reading long enough to bring the
average below this value.
MAX_SEND_SPEED
(read-write) Maximum speed, in bytes per second, that this device
will send data to S3. If the average speed exceeds this value, the
device will stop writing long enough to bring the average below
this value.
S3_ACCESS_KEY
(read-write) This property gives the Amazon S3 access key used to
access the service.
S3_BUCKET_LOCATION
(read-write) Location constraint for buckets on Amazon S3.
Currently, it can be set to "", for no constraint (i.e. store data
in the US), or "EU" (i.e. store data in the EU). See Amazon´s
documentation for details and latest information
SSL_CA_INFO
(read-write) Path to CA certificate to use to verify the identity
of the S3 server. Only applicable when SSL/TLS is in use. The
certificate should be in PEM format if OpenSSL or GnuTLS is being
used with libcurl. Multiple certificates can be bundled together
simply by concatenating them. If NSS is being used, then it is the
directory that the database resides in. The value is passed to
curl_easy_setopt(3) as CURLOPT_CAINFO.
S3_SECRET_KEY
(read-write) This property gives the Amazon S3 secret key used to
access the service.
S3_SSL
(read-write) Whether or not to use SSL/TLS to secure
communications with Amazon S3.
S3_USER_TOKEN
(read-write) This property specifies the user token for Amanda
Enterprise Edition customers.
VERBOSE
(read-write) If true, verbose data about each HTTP transaction is
sent to the debug log.
Tape Device
tapedev "tape:/dev/nst0"
The tape device driver interacts with a tape drive. The device uses the
operating system´s built-in tape support, which is generally similar to
that available via the command-line utilities dd(1) and mt(1).
The tape device name should specify a path to the operating system´s
device file.
Device-Specific Properties
Most of these properties are automatically detected, but can be
overridden in the configuration file if the autodetection fails. Note
that tape drives are required to at least support the MTREW (rewind)
operation; all other operations can be emulated with the MTREW and read
data operations.
BROKEN_GMT_ONLINE
(read-write) Set this boolean property if the system´s GMT_ONLINE
macro gives incorrect results. This is currently true for the
Linux IDE-TAPE driver.
BSF
(read-write) This boolean property specifies whether the device
driver may execute the MTBSF operation (backward seek file).
BSF_AFTER_EOM
(read-write) This boolean property specifies whether the device
driver should execute an MTBSF (backward seek file) operation
after
MTEOM (seek to end of recorded data) in order to append.
BSR
(read-write) This boolean property specifies whether the device
driver may use the MTBSR operation (backward seek record).
EOM
(read-write) This boolean property specifies whether the device
driver may use the MTEOM command (seek to end of recorded data).
FINAL_FILEMARKS
(read-write) This integer property gives the number of filemarks
that should be written at EOD. It is usually 1 or 2.
FSF
(read-write) This boolean property specifies whether the device
driver may use the MTFSF operation (forward seek file).
FSF_AFTER_FILEMARK
(read-write) This boolean property specifies whether the device
driver needs a FSF to go the next file after the filemark is read.
Default to "TRUE" on Solaris and "FALSE" on all others machines.
FSR
(read-write) This boolean property specifies whether the device
driver may use the MTFSR operation (forward seek record).
NONBLOCKING_OPEN
(read-write) Set this boolean property to "true" if O_NONBLOCK
must be used on the open call. Default to "true" on Linux and
"false" on all others machines. Witout it, Linux wait for a few
seconds if no tape are loaded. Solaris have strange error it is set
to "yes".
READ_BLOCK_SIZE
(read-write) This property (previously known as READ_BUFFER_SIZE)
specifies the block size that will be used for reads; this should
be large enough to contain any block that may be read from the
device (for example, from a tape containing variable-sized blocks),
and must be larger than BLOCK_SIZE. This property is most often
used when overwriting tapes using a new, smaller block size.
The tapetype parameter READBLOCKSIZE sets this property. See
BLOCK SIZES, above.
NDMP Device
tapedev "ndmp:my.filer.com:10000@st1"
device_property "NDMP_USERNAME" "jimmy"
device_property "NDMP_PASSWORD" "thelock"
This device enables Amanda to communicate with a tape service on an
NDMP server. The device name specifies the hostname and optionally the
TCP port of the NDMP server, followed by the name of the tape device on
the server (st1 in the example above).
Device-Specific Properties
The properties NDMP_USERNAME and NDMP_PASSWORD set the username and
password with which to access the NDMP server. The default for both is
"ndmp".
NDMP_AUTH
(read-write) Authentication method to use to connect to the NDMP
server. One of "md5" (default), "text", "none" (for an empty
authentication attempt) or "void" (for no authentication attempt at
all).
NDMP_PASSWORD
(read-write) Password for md5 or text authentications.
NDMP_USERNAME
(read-write) Username for md5 or text authentications.
VFS Device
tapedev "file:/path/to/vtape"
The VFS device driver stores data on a UNIX filesystem. Note that
although one typically uses the VFS device driver to store data on hard
disks, the driver does not interface with any hardware on a block
level.
The device name specifies a path to a directory which must exist and
contain a "data/" subdirectory. Each tape file is stored as a distinct
file in this directory, the name of which reflects the Amanda header in
the tape file. Block boundaries are not maintained: the driver supports
reads of arbitrary size, regardless of the blocksize used to write the
data.
DVD-RW Device
tapedev "dvdrw:/var/cache/amanda/dvd-cache:/dev/scd0"
device_property "DVDRW_MOUNT_POINT" "/media/dvd"
device_property "DVDRW_KEEP_CACHE" "false"
device_property "DVDRW_UNLABELLED_WHEN_UNMOUNTABLE" "true"
The DVD-RW device driver reads and writes optical media such as DVDs
and CDs. The device name must specify a cache directory for data to be
temporarily stored, followed by the operating system name for the
optical drive. The cache directory must contain a "data/" subdirectory.
The DVDRW_MOUNT_POINT property is required, and specifies a directory
where the optical media can be mounted. This directory must be
configured to enable non-root users to mount the optical media. On
Linux, that means a line similar to the following in /etc/fstab:
/dev/scd0 /media/dvd auto ro,user,noauto 0 0
Note the "user" option.
When writing data, the device acts as a VFS device using the given
cache directory. On completion of writing the tape, the cache directory
is written to optical media. The DVDRW_KEEP_CACHE property controls
whether the cache contents are immediately deleted. When reading, the
optical media is first mounted and read as a VFS device.
Attempting to mount unformatted media or media that is formatted but
contains no filesystem will usually result in an error. The boolean
DVDRW_UNLABELLED_WHEN_UNMOUNTABLE property specifies whether media that
cannot be mounted should be treated as an empty, unlabelled volume when
attempting to read the volume label. It is necessary to set this
property to "true" when labelling such media.
Device-Specific Properties
The properties DVDRW_GROWISOFS_COMMAND, DVDRW_MOUNT_COMMAND and
DVDRW_UMOUNT_COMMAND specify alternative commands for writing, mounting
and unmounting optical media. The default is to find the programs using
the PATH environment variable.
The DVDRW_MOUNT_POINT property is required. Other properties are
optional.
DVDRW_KEEP_CACHE
(read-write) Set this boolean property to "true" if the disk cache
directory should be kept after successfully writing tape data to
optical media. The default is false, which causes the cache
contents to be deleted immediately after a successful write
operation.
DVDRW_MOUNT_POINT
(read-write) This property specifies the filesystem mount point
for the optical media. Non-root users must be able to mount optical
media by invoking "mount" and specifying this mount point.
DVDRW_UNLABELLED_WHEN_UNMOUNTABLE
(read-write) Treat unmountable media as empty, unlabelled media.
This is necessary when attempting to label freshly formatted media.
DVDRW_GROWISOFS_COMMAND
(read-write) The command to invoke to burn the DVD.
DVDRW_MOUNT_COMMAND
(read-write) The command to invoke to mount the DVD.
DVDRW_UMOUNT_COMMAND
(read-write) The command to invoke to unmount the DVD.
SEE ALSO
amanda(8), amanda.conf(5)
The Amanda Wiki: : http://wiki.zmanda.com/
AUTHORS
Ian Turner <ian@zmanda.com>
Zmanda, Inc. (http://www.zmanda.com)
Dustin J. Mitchell <dustin@zmanda.com>
Zmanda, Inc. (http://www.zmanda.com)