NAME
zshexpn - zsh expansion and substitution
DESCRIPTION
The following types of expansions are performed in the indicated order
in five steps:
History Expansion
This is performed only in interactive shells.
Alias Expansion
Aliases are expanded immediately before the command line is
parsed as explained under Aliasing in zshmisc(1).
Process Substitution
Parameter Expansion
Command Substitution
Arithmetic Expansion
Brace Expansion
These five are performed in one step in left-to-right fashion.
After these expansions, all unquoted occurrences of the
characters `\', `'' and `"' are removed.
Filename Expansion
If the SH_FILE_EXPANSION option is set, the order of expansion
is modified for compatibility with sh and ksh. In that case
filename expansion is performed immediately after alias
expansion, preceding the set of five expansions mentioned above.
Filename Generation
This expansion, commonly referred to as globbing, is always done
last.
The following sections explain the types of expansion in detail.
HISTORY EXPANSION
History expansion allows you to use words from previous command lines
in the command line you are typing. This simplifies spelling
corrections and the repetition of complicated commands or arguments.
Immediately before execution, each command is saved in the history
list, the size of which is controlled by the HISTSIZE parameter. The
one most recent command is always retained in any case. Each saved
command in the history list is called a history event and is assigned a
number, beginning with 1 (one) when the shell starts up. The history
number that you may see in your prompt (see EXPANSION OF PROMPT
SEQUENCES in zshmisc(1)) is the number that is to be assigned to the
next command.
Overview
A history expansion begins with the first character of the histchars
parameter, which is `!' by default, and may occur anywhere on the
command line; history expansions do not nest. The `!' can be escaped
with `\' or can be enclosed between a pair of single quotes ('') to
suppress its special meaning. Double quotes will not work for this.
Following this history character is an optional event designator (see
the section `Event Designators') and then an optional word designator
(the section `Word Designators'); if neither of these designators is
present, no history expansion occurs.
Input lines containing history expansions are echoed after being
expanded, but before any other expansions take place and before the
command is executed. It is this expanded form that is recorded as the
history event for later references.
By default, a history reference with no event designator refers to the
same event as any preceding history reference on that command line; if
it is the only history reference in a command, it refers to the
previous command. However, if the option CSH_JUNKIE_HISTORY is set,
then every history reference with no event specification always refers
to the previous command.
For example, `!' is the event designator for the previous command, so
`!!:1' always refers to the first word of the previous command, and
`!!$' always refers to the last word of the previous command. With
CSH_JUNKIE_HISTORY set, then `!:1' and `!$' function in the same manner
as `!!:1' and `!!$', respectively. Conversely, if CSH_JUNKIE_HISTORY
is unset, then `!:1' and `!$' refer to the first and last words,
respectively, of the same event referenced by the nearest other history
reference preceding them on the current command line, or to the
previous command if there is no preceding reference.
The character sequence `^foo^bar' (where `^' is actually the second
character of the histchars parameter) repeats the last command,
replacing the string foo with bar. More precisely, the sequence
`^foo^bar^' is synonymous with `!!:s^foo^bar^', hence other modifiers
(see the section `Modifiers') may follow the final `^'. In particular,
`^foo^bar:G' performs a global substitution.
If the shell encounters the character sequence `!"' in the input, the
history mechanism is temporarily disabled until the current list (see
zshmisc(1)) is fully parsed. The `!"' is removed from the input, and
any subsequent `!' characters have no special significance.
A less convenient but more comprehensible form of command history
support is provided by the fc builtin.
Event Designators
An event designator is a reference to a command-line entry in the
history list. In the list below, remember that the initial `!' in each
item may be changed to another character by setting the histchars
parameter.
! Start a history expansion, except when followed by a blank,
newline, `=' or `('. If followed immediately by a word
designator (see the section `Word Designators'), this forms a
history reference with no event designator (see the section
`Overview').
!! Refer to the previous command. By itself, this expansion
repeats the previous command.
!n Refer to command-line n.
!-n Refer to the current command-line minus n.
!str Refer to the most recent command starting with str.
!?str[?]
Refer to the most recent command containing str. The trailing
`?' is necessary if this reference is to be followed by a
modifier or followed by any text that is not to be considered
part of str.
!# Refer to the current command line typed in so far. The line is
treated as if it were complete up to and including the word
before the one with the `!#' reference.
!{...} Insulate a history reference from adjacent characters (if
necessary).
Word Designators
A word designator indicates which word or words of a given command line
are to be included in a history reference. A `:' usually separates the
event specification from the word designator. It may be omitted only
if the word designator begins with a `^', `$', `*', `-' or `%'. Word
designators include:
0 The first input word (command).
n The nth argument.
^ The first argument. That is, 1.
$ The last argument.
% The word matched by (the most recent) ?str search.
x-y A range of words; x defaults to 0.
* All the arguments, or a null value if there are none.
x* Abbreviates `x-$'.
x- Like `x*' but omitting word $.
Note that a `%' word designator works only when used in one of `!%',
`!:%' or `!?str?:%', and only when used after a !? expansion (possibly
in an earlier command). Anything else results in an error, although
the error may not be the most obvious one.
Modifiers
After the optional word designator, you can add a sequence of one or
more of the following modifiers, each preceded by a `:'. These
modifiers also work on the result of filename generation and parameter
expansion, except where noted.
a Turn a file name into an absolute path: prepends the current
directory, if necessary, and resolves any use of `..' and `.' in
the path. Note that the transformation takes place even if the
file or any intervening directories do not exist.
A As `a', but also resolve use of symbolic links where possible.
Note that resolution of `..' occurs before resolution of
symbolic links. This call is equivalent to a unless your system
has the realpath system call (modern systems do).
c Resolve a command name into an absolute path by searching the
command path given by the PATH variable. This does not work for
commands containing directory parts. Note also that this does
not usually work as a glob qualifier unless a file of the same
name is found in the current directory.
e Remove all but the extension.
h Remove a trailing pathname component, leaving the head. This
works like `dirname'.
l Convert the words to all lowercase.
p Print the new command but do not execute it. Only works with
history expansion.
q Quote the substituted words, escaping further substitutions.
Works with history expansion and parameter expansion, though for
parameters it is only useful if the resulting text is to be
re-evaluated such as by eval.
Q Remove one level of quotes from the substituted words.
r Remove a filename extension of the form `.xxx', leaving the root
name.
s/l/r[/]
Substitute r for l as described below. The substitution is done
only for the first string that matches l. For arrays and for
filename generation, this applies to each word of the expanded
text. See below for further notes on substitutions.
The forms `gs/l/r' and `s/l/r/:G' perform global substitution,
i.e. substitute every occurrence of r for l. Note that the g or
:G must appear in exactly the position shown.
& Repeat the previous s substitution. Like s, may be preceded
immediately by a g. In parameter expansion the & must appear
inside braces, and in filename generation it must be quoted with
a backslash.
t Remove all leading pathname components, leaving the tail. This
works like `basename'.
u Convert the words to all uppercase.
x Like q, but break into words at whitespace. Does not work with
parameter expansion.
The s/l/r/ substitution works as follows. By default the left-hand
side of substitutions are not patterns, but character strings. Any
character can be used as the delimiter in place of `/'. A backslash
quotes the delimiter character. The character `&', in the
right-hand-side r, is replaced by the text from the left-hand-side l.
The `&' can be quoted with a backslash. A null l uses the previous
string either from the previous l or from the contextual scan string s
from `!?s'. You can omit the rightmost delimiter if a newline
immediately follows r; the rightmost `?' in a context scan can
similarly be omitted. Note the same record of the last l and r is
maintained across all forms of expansion.
If the option HIST_SUBST_PATTERN is set, l is treated as a pattern of
the usual form described in the section FILENAME GENERATION below.
This can be used in all the places where modifiers are available; note,
however, that in globbing qualifiers parameter substitution has already
taken place, so parameters in the replacement string should be quoted
to ensure they are replaced at the correct time. Note also that
complicated patterns used in globbing qualifiers may need the extended
glob qualifier notation (#q:s/.../.../) in order for the shell to
recognize the expression as a glob qualifier. Further, note that bad
patterns in the substitution are not subject to the NO_BAD_PATTERN
option so will cause an error.
When HIST_SUBST_PATTERN is set, l may start with a # to indicate that
the pattern must match at the start of the string to be substituted,
and a % may appear at the start or after an # to indicate that the
pattern must match at the end of the string to be substituted. The %
or # may be quoted with two backslashes.
For example, the following piece of filename generation code with the
EXTENDED_GLOB option:
print *.c(#q:s/#%(#b)s(*).c/'S${match[1]}.C'/)
takes the expansion of *.c and applies the glob qualifiers in the
(#q...) expression, which consists of a substitution modifier anchored
to the start and end of each word (#%). This turns on backreferences
((#b)), so that the parenthesised subexpression is available in the
replacement string as ${match[1]}. The replacement string is quoted so
that the parameter is not substituted before the start of filename
generation.
The following f, F, w and W modifiers work only with parameter
expansion and filename generation. They are listed here to provide a
single point of reference for all modifiers.
f Repeats the immediately (without a colon) following modifier
until the resulting word doesn't change any more.
F:expr:
Like f, but repeats only n times if the expression expr
evaluates to n. Any character can be used instead of the `:';
if `(', `[', or `{' is used as the opening delimiter, the
closing delimiter should be ')', `]', or `}', respectively.
w Makes the immediately following modifier work on each word in
the string.
W:sep: Like w but words are considered to be the parts of the string
that are separated by sep. Any character can be used instead of
the `:'; opening parentheses are handled specially, see above.
PROCESS SUBSTITUTION
Each part of a command argument that takes the form `<(list)',
`>(list)' or `=(list)' is subject to process substitution. The
expression may be preceeded or followed by other strings except that,
to prevent clashes with commonly occurring strings and patterns, the
last form must occur at the start of a command argument, and the forms
are only expanded when first parsing command or assignment arguments.
Process substitutions may be used following redirection operators; in
this case, the substitution must appear with no trailing string.
In the case of the < or > forms, the shell runs the commands in list
asynchronously. If the system supports the /dev/fd mechanism, the
command argument is the name of the device file corresponding to a file
descriptor; otherwise, if the system supports named pipes (FIFOs), the
command argument will be a named pipe. If the form with > is selected
then writing on this special file will provide input for list. If < is
used, then the file passed as an argument will be connected to the
output of the list process. For example,
paste <(cut -f1 file1) <(cut -f3 file2) |
tee >(process1) >(process2) >/dev/null
cuts fields 1 and 3 from the files file1 and file2 respectively, pastes
the results together, and sends it to the processes process1 and
process2.
If =(...) is used instead of <(...), then the file passed as an
argument will be the name of a temporary file containing the output of
the list process. This may be used instead of the < form for a program
that expects to lseek (see lseek(2)) on the input file.
There is an optimisation for substitutions of the form =(<<<arg), where
arg is a single-word argument to the here-string redirection <<<. This
form produces a file name containing the value of arg after any
substitutions have been performed. This is handled entirely within the
current shell. This is effectively the reverse of the special form
$(<arg) which treats arg as a file name and replaces it with the file's
contents.
The = form is useful as both the /dev/fd and the named pipe
implementation of <(...) have drawbacks. In the former case, some
programmes may automatically close the file descriptor in question
before examining the file on the command line, particularly if this is
necessary for security reasons such as when the programme is running
setuid. In the second case, if the programme does not actually open
the file, the subshell attempting to read from or write to the pipe
will (in a typical implementation, different operating systems may have
different behaviour) block for ever and have to be killed explicitly.
In both cases, the shell actually supplies the information using a
pipe, so that programmes that expect to lseek (see lseek(2)) on the
file will not work.
Also note that the previous example can be more compactly and
efficiently written (provided the MULTIOS option is set) as:
paste <(cut -f1 file1) <(cut -f3 file2) \
> >(process1) > >(process2)
The shell uses pipes instead of FIFOs to implement the latter two
process substitutions in the above example.
There is an additional problem with >(process); when this is attached
to an external command, the parent shell does not wait for process to
finish and hence an immediately following command cannot rely on the
results being complete. The problem and solution are the same as
described in the section MULTIOS in zshmisc(1). Hence in a simplified
version of the example above:
paste <(cut -f1 file1) <(cut -f3 file2) > >(process)
(note that no MULTIOS are involved), process will be run
asynchronously. The workaround is:
{ paste <(cut -f1 file1) <(cut -f3 file2) } > >(process)
The extra processes here are spawned from the parent shell which will
wait for their completion.
PARAMETER EXPANSION
The character `$' is used to introduce parameter expansions. See
zshparam(1) for a description of parameters, including arrays,
associative arrays, and subscript notation to access individual array
elements.
Note in particular the fact that words of unquoted parameters are not
automatically split on whitespace unless the option SH_WORD_SPLIT is
set; see references to this option below for more details. This is an
important difference from other shells.
In the expansions discussed below that require a pattern, the form of
the pattern is the same as that used for filename generation; see the
section `Filename Generation'. Note that these patterns, along with
the replacement text of any substitutions, are themselves subject to
parameter expansion, command substitution, and arithmetic expansion.
In addition to the following operations, the colon modifiers described
in the section `Modifiers' in the section `History Expansion' can be
applied: for example, ${i:s/foo/bar/} performs string substitution on
the expansion of parameter $i.
${name}
The value, if any, of the parameter name is substituted. The
braces are required if the expansion is to be followed by a
letter, digit, or underscore that is not to be interpreted as
part of name. In addition, more complicated forms of
substitution usually require the braces to be present;
exceptions, which only apply if the option KSH_ARRAYS is not
set, are a single subscript or any colon modifiers appearing
after the name, or any of the characters `^', `=', `~', `#' or
`+' appearing before the name, all of which work with or without
braces.
If name is an array parameter, and the KSH_ARRAYS option is not
set, then the value of each element of name is substituted, one
element per word. Otherwise, the expansion results in one word
only; with KSH_ARRAYS, this is the first element of an array.
No field splitting is done on the result unless the
SH_WORD_SPLIT option is set. See also the flags = and
s:string:.
${+name}
If name is the name of a set parameter `1' is substituted,
otherwise `0' is substituted.
${name-word}
${name:-word}
If name is set, or in the second form is non-null, then
substitute its value; otherwise substitute word. In the second
form name may be omitted, in which case word is always
substituted.
${name+word}
${name:+word}
If name is set, or in the second form is non-null, then
substitute word; otherwise substitute nothing.
${name=word}
${name:=word}
${name::=word}
In the first form, if name is unset then set it to word; in the
second form, if name is unset or null then set it to word; and
in the third form, unconditionally set name to word. In all
forms, the value of the parameter is then substituted.
${name?word}
${name:?word}
In the first form, if name is set, or in the second form if name
is both set and non-null, then substitute its value; otherwise,
print word and exit from the shell. Interactive shells instead
return to the prompt. If word is omitted, then a standard
message is printed.
In any of the above expressions that test a variable and substitute an
alternate word, note that you can use standard shell quoting in the
word value to selectively override the splitting done by the
SH_WORD_SPLIT option and the = flag, but not splitting by the s:string:
flag.
In the following expressions, when name is an array and the
substitution is not quoted, or if the `(@)' flag or the name[@] syntax
is used, matching and replacement is performed on each array element
separately.
${name#pattern}
${name##pattern}
If the pattern matches the beginning of the value of name, then
substitute the value of name with the matched portion deleted;
otherwise, just substitute the value of name. In the first
form, the smallest matching pattern is preferred; in the second
form, the largest matching pattern is preferred.
${name%pattern}
${name%%pattern}
If the pattern matches the end of the value of name, then
substitute the value of name with the matched portion deleted;
otherwise, just substitute the value of name. In the first
form, the smallest matching pattern is preferred; in the second
form, the largest matching pattern is preferred.
${name:#pattern}
If the pattern matches the value of name, then substitute the
empty string; otherwise, just substitute the value of name. If
name is an array the matching array elements are removed (use
the `(M)' flag to remove the non-matched elements).
${name/pattern/repl}
${name//pattern/repl}
Replace the longest possible match of pattern in the expansion
of parameter name by string repl. The first form replaces just
the first occurrence, the second form all occurrences. Both
pattern and repl are subject to double-quoted substitution, so
that expressions like ${name/$opat/$npat} will work, but note
the usual rule that pattern characters in $opat are not treated
specially unless either the option GLOB_SUBST is set, or $opat
is instead substituted as ${~opat}.
The pattern may begin with a `#', in which case the pattern must
match at the start of the string, or `%', in which case it must
match at the end of the string, or `#%' in which case the
pattern must match the entire string. The repl may be an empty
string, in which case the final `/' may also be omitted. To
quote the final `/' in other cases it should be preceded by a
single backslash; this is not necessary if the `/' occurs inside
a substituted parameter. Note also that the `#', `%' and `#%
are not active if they occur inside a substituted parameter,
even at the start.
The first `/' may be preceded by a `:', in which case the match
will only succeed if it matches the entire word. Note also the
effect of the I and S parameter expansion flags below; however,
the flags M, R, B, E and N are not useful.
For example,
foo="twinkle twinkle little star" sub="t*e" rep="spy"
print ${foo//${~sub}/$rep}
print ${(S)foo//${~sub}/$rep}
Here, the `~' ensures that the text of $sub is treated as a
pattern rather than a plain string. In the first case, the
longest match for t*e is substituted and the result is `spy
star', while in the second case, the shortest matches are taken
and the result is `spy spy lispy star'.
${#spec}
If spec is one of the above substitutions, substitute the length
in characters of the result instead of the result itself. If
spec is an array expression, substitute the number of elements
of the result. Note that `^', `=', and `~', below, must appear
to the left of `#' when these forms are combined.
${^spec}
Turn on the RC_EXPAND_PARAM option for the evaluation of spec;
if the `^' is doubled, turn it off. When this option is set,
array expansions of the form foo${xx}bar, where the parameter xx
is set to (a b c), are substituted with `fooabar foobbar
foocbar' instead of the default `fooa b cbar'. Note that an
empty array will therefore cause all arguments to be removed.
Internally, each such expansion is converted into the equivalent
list for brace expansion. E.g., ${^var} becomes
{$var[1],$var[2],...}, and is processed as described in the
section `Brace Expansion' below. If word splitting is also in
effect the $var[N] may themselves be split into different list
elements.
${=spec}
Perform word splitting using the rules for SH_WORD_SPLIT during
the evaluation of spec, but regardless of whether the parameter
appears in double quotes; if the `=' is doubled, turn it off.
This forces parameter expansions to be split into separate words
before substitution, using IFS as a delimiter. This is done by
default in most other shells.
Note that splitting is applied to word in the assignment forms
of spec before the assignment to name is performed. This
affects the result of array assignments with the A flag.
${~spec}
Turn on the GLOB_SUBST option for the evaluation of spec; if the
`~' is doubled, turn it off. When this option is set, the
string resulting from the expansion will be interpreted as a
pattern anywhere that is possible, such as in filename expansion
and filename generation and pattern-matching contexts like the
right hand side of the `=' and `!=' operators in conditions.
In nested substitutions, note that the effect of the ~ applies
to the result of the current level of substitution. A
surrounding pattern operation on the result may cancel it.
Hence, for example, if the parameter foo is set to *,
${~foo//\*/*.c} is substituted by the pattern *.c, which may be
expanded by filename generation, but ${${~foo}//\*/*.c}
substitutes to the string *.c, which will not be further
expanded.
If a ${...} type parameter expression or a $(...) type command
substitution is used in place of name above, it is expanded first and
the result is used as if it were the value of name. Thus it is
possible to perform nested operations: ${${foo#head}%tail} substitutes
the value of $foo with both `head' and `tail' deleted. The form with
$(...) is often useful in combination with the flags described next;
see the examples below. Each name or nested ${...} in a parameter
expansion may also be followed by a subscript expression as described
in Array Parameters in zshparam(1).
Note that double quotes may appear around nested expressions, in which
case only the part inside is treated as quoted; for example,
${(f)"$(foo)"} quotes the result of $(foo), but the flag `(f)' (see
below) is applied using the rules for unquoted expansions. Note
further that quotes are themselves nested in this context; for example,
in "${(@f)"$(foo)"}", there are two sets of quotes, one surrounding the
whole expression, the other (redundant) surrounding the $(foo) as
before.
Parameter Expansion Flags
If the opening brace is directly followed by an opening parenthesis,
the string up to the matching closing parenthesis will be taken as a
list of flags. In cases where repeating a flag is meaningful, the
repetitions need not be consecutive; for example, `(q%q%q)' means the
same thing as the more readable `(%%qqq)'. The following flags are
supported:
# Evaluate the resulting words as numeric expressions and output
the characters corresponding to the resulting integer. Note
that this form is entirely distinct from use of the # without
parentheses.
If the MULTIBYTE option is set and the number is greater than
127 (i.e. not an ASCII character) it is treated as a Unicode
character.
% Expand all % escapes in the resulting words in the same way as
in prompts (see EXPANSION OF PROMPT SEQUENCES in zshmisc(1)). If
this flag is given twice, full prompt expansion is done on the
resulting words, depending on the setting of the PROMPT_PERCENT,
PROMPT_SUBST and PROMPT_BANG options.
@ In double quotes, array elements are put into separate words.
E.g., `"${(@)foo}"' is equivalent to `"${foo[@]}"' and
`"${(@)foo[1,2]}"' is the same as `"$foo[1]" "$foo[2]"'. This
is distinct from field splitting by the the f, s or z flags,
which still applies within each array element.
A Create an array parameter with `${...=...}', `${...:=...}' or
`${...::=...}'. If this flag is repeated (as in `AA'), create
an associative array parameter. Assignment is made before
sorting or padding. The name part may be a subscripted range
for ordinary arrays; the word part must be converted to an
array, for example by using `${(AA)=name=...}' to activate field
splitting, when creating an associative array.
a Sort in array index order; when combined with `O' sort in
reverse array index order. Note that `a' is therefore
equivalent to the default but `Oa' is useful for obtaining an
array's elements in reverse order.
c With ${#name}, count the total number of characters in an array,
as if the elements were concatenated with spaces between them.
C Capitalize the resulting words. `Words' in this case refers to
sequences of alphanumeric characters separated by
non-alphanumerics, not to words that result from field
splitting.
e Perform parameter expansion, command substitution and arithmetic
expansion on the result. Such expansions can be nested but too
deep recursion may have unpredictable effects.
f Split the result of the expansion to lines. This is a shorthand
for `ps:\n:'.
F Join the words of arrays together using newline as a separator.
This is a shorthand for `pj:\n:'.
i Sort case-insensitively. May be combined with `n' or `O'.
k If name refers to an associative array, substitute the keys
(element names) rather than the values of the elements. Used
with subscripts (including ordinary arrays), force indices or
keys to be substituted even if the subscript form refers to
values. However, this flag may not be combined with subscript
ranges.
L Convert all letters in the result to lower case.
n Sort decimal integers numerically; if the first differing
characters of two test strings are not digits, sorting is
lexical. Integers with more initial zeroes are sorted before
those with fewer or none. Hence the array `foo1 foo02 foo2 foo3
foo20 foo23' is sorted into the order shown. May be combined
with `i' or `O'.
o Sort the resulting words in ascending order; if this appears on
its own the sorting is lexical and case-sensitive (unless the
locale renders it case-insensitive). Sorting in ascending order
is the default for other forms of sorting, so this is ignored if
combined with `a', `i' or `n'.
O Sort the resulting words in descending order; `O' without `a',
`i' or `n' sorts in reverse lexical order. May be combined with
`a', `i' or `n' to reverse the order of sorting.
P This forces the value of the parameter name to be interpreted as
a further parameter name, whose value will be used where
appropriate. Note that flags set with one of the typeset family
of commands (in particular case transformations) are not applied
to the value of name used in this fashion.
If used with a nested parameter or command substitution, the
result of that will be taken as a parameter name in the same
way. For example, if you have `foo=bar' and `bar=baz', the
strings ${(P)foo}, ${(P)${foo}}, and ${(P)$(echo bar)} will be
expanded to `baz'.
q Quote the resulting words with backslashes; unprintable or
invalid characters are quoted using the $'\NNN' form, with
separate quotes for each octet. If this flag is given twice,
the resulting words are quoted in single quotes and if it is
given three times, the words are quoted in double quotes; in
these forms no special handling of unprintable or invalid
characters is attempted. If the flag is given four times, the
words are quoted in single quotes preceded by a $.
Q Remove one level of quotes from the resulting words.
t Use a string describing the type of the parameter where the
value of the parameter would usually appear. This string
consists of keywords separated by hyphens (`-'). The first
keyword in the string describes the main type, it can be one of
`scalar', `array', `integer', `float' or `association'. The
other keywords describe the type in more detail:
local for local parameters
left for left justified parameters
right_blanks
for right justified parameters with leading blanks
right_zeros
for right justified parameters with leading zeros
lower for parameters whose value is converted to all lower case
when it is expanded
upper for parameters whose value is converted to all upper case
when it is expanded
readonly
for readonly parameters
tag for tagged parameters
export for exported parameters
unique for arrays which keep only the first occurrence of
duplicated values
hide for parameters with the `hide' flag
special
for special parameters defined by the shell
u Expand only the first occurrence of each unique word.
U Convert all letters in the result to upper case.
v Used with k, substitute (as two consecutive words) both the key
and the value of each associative array element. Used with
subscripts, force values to be substituted even if the subscript
form refers to indices or keys.
V Make any special characters in the resulting words visible.
w With ${#name}, count words in arrays or strings; the s flag may
be used to set a word delimiter.
W Similar to w with the difference that empty words between
repeated delimiters are also counted.
X With this flag, parsing errors occurring with the Q, e and #
flags or the pattern matching forms such as `${name#pattern}'
are reported. Without the flag, errors are silently ignored.
z Split the result of the expansion into words using shell parsing
to find the words, i.e. taking into account any quoting in the
value.
Note that this is done very late, as for the `(s)' flag. So to
access single words in the result, one has to use nested
expansions as in `${${(z)foo}[2]}'. Likewise, to remove the
quotes in the resulting words one would do: `${(Q)${(z)foo}}'.
0 Split the result of the expansion on null bytes. This is a
shorthand for `ps:\0:'.
The following flags (except p) are followed by one or more arguments as
shown. Any character, or the matching pairs `(...)', `{...}', `[...]',
or `<...>', may be used in place of a colon as delimiters, but note
that when a flag takes more than one argument, a matched pair of
delimiters must surround each argument.
p Recognize the same escape sequences as the print builtin in
string arguments to any of the flags described below that follow
this argument.
~ Force string arguments to any of the flags below that follow
within the parentheses to be treated as patterns. Compare with
a ~ outside parentheses, which forces the entire substituted
string to be treated as a pattern. Hence, for example,
[[ "?" = ${(~j.|.)array} ]]
with the EXTENDED_GLOB option set succeeds if and only if $array
contains the string `?' as an element. The argument may be repeated to
toggle the behaviour; its effect only lasts to the end of the
parenthesised group.
j:string:
Join the words of arrays together using string as a separator.
Note that this occurs before field splitting by the s:string:
flag or the SH_WORD_SPLIT option.
l:expr::string1::string2:
Pad the resulting words on the left. Each word will be
truncated if required and placed in a field expr characters
wide.
The arguments :string1: and :string2: are optional; neither, the
first, or both may be given. Note that the same pairs of
delimiters must be used for each of the three arguments. The
space to the left will be filled with string1 (concatenated as
often as needed) or spaces if string1 is not given. If both
string1 and string2 are given, string2 is inserted once directly
to the left of each word, truncated if necessary, before string1
is used to produce any remaining padding.
If the MULTIBYTE option is in effect, the flag m may also be
given, in which case widths will be used for the calculation of
padding; otherwise individual multibyte characters are treated
as occupying one unit of width.
IF the MULTIBYTE option is not in effect, each byte in the
string is treated as occupying one unit of width.
Control characters are always assumed to be one unit wide; this
allows the mechanism to be used for generating repetitions of
control characters.
m Only useful together with one of the flags l or r or with the #
length operator when the MULTIBYTE option is in effect. Use the
character width reported by the system in calculating the how
much of the string it occupies or the overall length of the
string. Most printable characters have a width of one unit,
however certain Asian character sets and certain special effects
use wider characters; combining characters have zero width.
r:expr::string1::string2:
As l, but pad the words on the right and insert string2
immediately to the right of the string to be padded.
Left and right padding may be used together. In this case the
strategy is to apply left padding to the first half width of
each of the resulting words, and right padding to the second
half. If the string to be padded has odd width the extra
padding is applied on the left.
s:string:
Force field splitting at the separator string. Note that a
string of two or more characters means that all of them must
match in sequence; this differs from the treatment of two or
more characters in the IFS parameter. See also the = flag and
the SH_WORD_SPLIT option.
For historical reasons, the usual behaviour that empty array
elements are retained inside double quotes is disabled for
arrays generated by splitting; hence the following:
line="one::three"
print -l "${(s.:.)line}"
produces two lines of output for one and three and elides the
empty field. To override this behaviour, supply the "(@)" flag
as well, i.e. "${(@s.:.)line}".
The following flags are meaningful with the ${...#...} or ${...%...}
forms. The S and I flags may also be used with the ${.../...} forms.
S Search substrings as well as beginnings or ends; with # start
from the beginning and with % start from the end of the string.
With substitution via ${.../...} or ${...//...}, specifies
non-greedy matching, i.e. that the shortest instead of the
longest match should be replaced.
I:expr:
Search the exprth match (where expr evaluates to a number).
This only applies when searching for substrings, either with the
S flag, or with ${.../...} (only the exprth match is
substituted) or ${...//...} (all matches from the exprth on are
substituted). The default is to take the first match.
The exprth match is counted such that there is either one or
zero matches from each starting position in the string, although
for global substitution matches overlapping previous
replacements are ignored. With the ${...%...} and ${...%%...}
forms, the starting position for the match moves backwards from
the end as the index increases, while with the other forms it
moves forward from the start.
Hence with the string
which switch is the right switch for Ipswich?
substitutions of the form ${(SI:N:)string#w*ch} as N increases
from 1 will match and remove `which', `witch', `witch' and
`wich'; the form using `##' will match and remove `which switch
is the right switch for Ipswich', `witch is the right switch for
Ipswich', `witch for Ipswich' and `wich'. The form using `%'
will remove the same matches as for `#', but in reverse order,
and the form using `%%' will remove the same matches as for `##'
in reverse order.
B Include the index of the beginning of the match in the result.
E Include the index of the end of the match in the result.
M Include the matched portion in the result.
N Include the length of the match in the result.
R Include the unmatched portion in the result (the Rest).
Rules
Here is a summary of the rules for substitution; this assumes that
braces are present around the substitution, i.e. ${...}. Some
particular examples are given below. Note that the Zsh Development
Group accepts no responsibility for any brain damage which may occur
during the reading of the following rules.
1. Nested Substitution
If multiple nested ${...} forms are present, substitution is
performed from the inside outwards. At each level, the
substitution takes account of whether the current value is a
scalar or an array, whether the whole substitution is in double
quotes, and what flags are supplied to the current level of
substitution, just as if the nested substitution were the
outermost. The flags are not propagated up to enclosing
substitutions; the nested substitution will return either a
scalar or an array as determined by the flags, possibly adjusted
for quoting. All the following steps take place where
applicable at all levels of substitution. Note that, unless the
`(P)' flag is present, the flags and any subscripts apply
directly to the value of the nested substitution; for example,
the expansion ${${foo}} behaves exactly the same as ${foo}.
At each nested level of substitution, the substituted words
undergo all forms of single-word substitution (i.e. not filename
generation), including command substitution, arithmetic
expansion and filename expansion (i.e. leading ~ and =). Thus,
for example, ${${:-=cat}:h} expands to the directory where the
cat program resides. (Explanation: the internal substitution
has no parameter but a default value =cat, which is expanded by
filename expansion to a full path; the outer substitution then
applies the modifier :h and takes the directory part of the
path.)
2. Internal Parameter Flags
Any parameter flags set by one of the typeset family of
commands, in particular the L, R, Z, u and l flags for padding
and capitalization, are applied directly to the parameter value.
3. Parameter Subscripting
If the value is a raw parameter reference with a subscript, such
as ${var[3]}, the effect of subscripting is applied directly to
the parameter. Subscripts are evaluated left to right;
subsequent subscripts apply to the scalar or array value yielded
by the previous subscript. Thus if var is an array,
${var[1][2]} is the second character of the first word, but
${var[2,4][2]} is the entire third word (the second word of the
range of words two through four of the original array). Any
number of subscripts may appear.
4. Parameter Name Replacement
The effect of any (P) flag, which treats the value so far as a
parameter name and replaces it with the corresponding value, is
applied.
5. Double-Quoted Joining
If the value after this process is an array, and the
substitution appears in double quotes, and no (@) flag is
present at the current level, the words of the value are joined
with the first character of the parameter $IFS, by default a
space, between each word (single word arrays are not modified).
If the (j) flag is present, that is used for joining instead of
$IFS.
6. Nested Subscripting
Any remaining subscripts (i.e. of a nested substitution) are
evaluated at this point, based on whether the value is an array
or a scalar. As with 2., multiple subscripts can appear. Note
that ${foo[2,4][2]} is thus equivalent to ${${foo[2,4]}[2]} and
also to "${${(@)foo[2,4]}[2]}" (the nested substitution returns
an array in both cases), but not to "${${foo[2,4]}[2]}" (the
nested substitution returns a scalar because of the quotes).
7. Modifiers
Any modifiers, as specified by a trailing `#', `%', `/'
(possibly doubled) or by a set of modifiers of the form :...
(see the section `Modifiers' in the section `History
Expansion'), are applied to the words of the value at this
level.
8. Forced Joining
If the `(j)' flag is present, or no `(j)' flag is present but
the string is to be split as given by rules 8. or 9., and
joining did not take place at step 4., any words in the value
are joined together using the given string or the first
character of $IFS if none. Note that the `(F)' flag implicitly
supplies a string for joining in this manner.
9. Forced Splitting
If one of the `(s)', `(f)' or `(z)' flags are present, or the
`=' specifier was present (e.g. ${=var}), the word is split on
occurrences of the specified string, or (for = with neither of
the two flags present) any of the characters in $IFS.
10. Shell Word Splitting
If no `(s)', `(f)' or `=' was given, but the word is not quoted
and the option SH_WORD_SPLIT is set, the word is split on
occurrences of any of the characters in $IFS. Note this step,
too, takes place at all levels of a nested substitution.
11. Uniqueness
If the result is an array and the `(u)' flag was present,
duplicate elements are removed from the array.
12. Ordering
If the result is still an array and one of the `(o)' or `(O)'
flags was present, the array is reordered.
13. Re-Evaluation
Any `(e)' flag is applied to the value, forcing it to be
re-examined for new parameter substitutions, but also for
command and arithmetic substitutions.
14. Padding
Any padding of the value by the `(l.fill.)' or `(r.fill.)' flags
is applied.
15. Semantic Joining
In contexts where expansion semantics requires a single word to
result, all words are rejoined with the first character of IFS
between. So in `${(P)${(f)lines}}' the value of ${lines} is
split at newlines, but then must be joined again before the P
flag can be applied.
If a single word is not required, this rule is skipped.
Examples
The flag f is useful to split a double-quoted substitution line by
line. For example, ${(f)"$(<file)"} substitutes the contents of file
divided so that each line is an element of the resulting array.
Compare this with the effect of $(<file) alone, which divides the file
up by words, or the same inside double quotes, which makes the entire
content of the file a single string.
The following illustrates the rules for nested parameter expansions.
Suppose that $foo contains the array (bar baz):
"${(@)${foo}[1]}"
This produces the result b. First, the inner substitution
"${foo}", which has no array (@) flag, produces a single word
result "bar baz". The outer substitution "${(@)...[1]}" detects
that this is a scalar, so that (despite the `(@)' flag) the
subscript picks the first character.
"${${(@)foo}[1]}"
This produces the result `bar'. In this case, the inner
substitution "${(@)foo}" produces the array `(bar baz)'. The
outer substitution "${...[1]}" detects that this is an array and
picks the first word. This is similar to the simple case
"${foo[1]}".
As an example of the rules for word splitting and joining, suppose $foo
contains the array `(ax1 bx1)'. Then
${(s/x/)foo}
produces the words `a', `1 b' and `1'.
${(j/x/s/x/)foo}
produces `a', `1', `b' and `1'.
${(s/x/)foo%%1*}
produces `a' and ` b' (note the extra space). As substitution
occurs before either joining or splitting, the operation first
generates the modified array (ax bx), which is joined to give
"ax bx", and then split to give `a', ` b' and `'. The final
empty string will then be elided, as it is not in double quotes.
COMMAND SUBSTITUTION
A command enclosed in parentheses preceded by a dollar sign, like
`$(...)', or quoted with grave accents, like ``...`', is replaced with
its standard output, with any trailing newlines deleted. If the
substitution is not enclosed in double quotes, the output is broken
into words using the IFS parameter. The substitution `$(cat foo)' may
be replaced by the equivalent but faster `$(<foo)'. In either case, if
the option GLOB_SUBST is set, the output is eligible for filename
generation.
ARITHMETIC EXPANSION
A string of the form `$[exp]' or `$((exp))' is substituted with the
value of the arithmetic expression exp. exp is subjected to parameter
expansion, command substitution and arithmetic expansion before it is
evaluated. See the section `Arithmetic Evaluation'.
BRACE EXPANSION
A string of the form `foo{xx,yy,zz}bar' is expanded to the individual
words `fooxxbar', `fooyybar' and `foozzbar'. Left-to-right order is
preserved. This construct may be nested. Commas may be quoted in
order to include them literally in a word.
An expression of the form `{n1..n2}', where n1 and n2 are integers, is
expanded to every number between n1 and n2 inclusive. If either number
begins with a zero, all the resulting numbers will be padded with
leading zeroes to that minimum width. If the numbers are in decreasing
order the resulting sequence will also be in decreasing order.
If a brace expression matches none of the above forms, it is left
unchanged, unless the option BRACE_CCL (an abbreviation for `brace
character class') is set. In that case, it is expanded to a list of
the individual characters between the braces sorted into the order of
the characters in the ASCII character set (multibyte characters are not
currently handled). The syntax is similar to a [...] expression in
filename generation: `-' is treated specially to denote a range of
characters, but `^' or `!' as the first character is treated normally.
For example, `{abcdef0-9}' expands to 16 words 0 1 2 3 4 5 6 7 8 9 a b
c d e f.
Note that brace expansion is not part of filename generation
(globbing); an expression such as */{foo,bar} is split into two
separate words */foo and */bar before filename generation takes place.
In particular, note that this is liable to produce a `no match' error
if either of the two expressions does not match; this is to be
contrasted with */(foo|bar), which is treated as a single pattern but
otherwise has similar effects.
To combine brace expansion with array expansion, see the ${^spec} form
described in the section Parameter Expansion above.
FILENAME EXPANSION
Each word is checked to see if it begins with an unquoted `~'. If it
does, then the word up to a `/', or the end of the word if there is no
`/', is checked to see if it can be substituted in one of the ways
described here. If so, then the `~' and the checked portion are
replaced with the appropriate substitute value.
A `~' by itself is replaced by the value of $HOME. A `~' followed by a
`+' or a `-' is replaced by the value of $PWD or $OLDPWD, respectively.
A `~' followed by a number is replaced by the directory at that
position in the directory stack. `~0' is equivalent to `~+', and `~1'
is the top of the stack. `~+' followed by a number is replaced by the
directory at that position in the directory stack. `~+0' is equivalent
to `~+', and `~+1' is the top of the stack. `~-' followed by a number
is replaced by the directory that many positions from the bottom of the
stack. `~-0' is the bottom of the stack. The PUSHD_MINUS option
exchanges the effects of `~+' and `~-' where they are followed by a
number.
Dynamic named directories
The feature described here is only available if the shell function
zsh_directory_name exists.
A `~' followed by a string namstr in unquoted square brackets is
treated specially as a dynamic directory name. Note that the first
unquoted closing square bracket always terminates namstr. The shell
function is passed two arguments: the string n (for name) and namstr.
It should either set the array reply to a single element which is the
directory corresponding to the name and return status zero (executing
an assignment as the last statement is usually sufficient), or it
should return status non-zero. In the former case the element of reply
is used as the directory; in the latter case the substitution is deemed
to have failed and NOMATCH handling is applied if the option is set.
The function zsh_directory_name is also used to see if a directory can
be turned into a name, for example when printing the directory stack or
when expanding %~ in prompts. In this case the function is passed two
arguments: the string d (for directory) and the candidate for dynamic
naming. The function should either return non-zero status, if the
directory cannot be named by the function, or it should set the array
reply to consist of two elements: the first is the dynamic name for the
directory (as would appear within `~[...]'), and the second is the
prefix length of the directory to be replaced. For example, if the
trial directory is /home/myname/src/zsh and the dynamic name for
/home/myname/src (which has 16 characters) is s, then the function sets
reply=(s 16)
The directory name so returned is compared with possible static names
for parts of the directory path, as described below; it is used if the
prefix length matched (16 in the example) is longer than that matched
by any static name.
As a working example, here is a function that expands any dynamic names
beginning with the string p: to directories below /home/pws/perforce.
In this simple case a static name for the directory would be just as
effective.
zsh_directory_name() {
emulate -L zsh
setopt extendedglob
local -a match mbegin mend
if [[ $1 = d ]]; then
if [[ $2 = (#b)(/home/pws/perforce/)([^/]##)* ]]; then
typeset -ga reply
reply=(p:$match[2] $(( ${#match[1]} + ${#match[2]} )) )
else
return 1
fi
else
[[ $2 != (#b)p:(?*) ]] && return 1
typeset -ga reply
reply=(/home/pws/perforce/$match[1])
fi
return 0
}
Static named directories
A `~' followed by anything not already covered consisting of any number
of alphanumeric characters or underscore (`_'), hyphen (`-'), or dot
(`.') is looked up as a named directory, and replaced by the value of
that named directory if found. Named directories are typically home
directories for users on the system. They may also be defined if the
text after the `~' is the name of a string shell parameter whose value
begins with a `/'. Note that trailing slashes will be removed from the
path to the directory (though the original parameter is not modified).
It is also possible to define directory names using the -d option to
the hash builtin.
In certain circumstances (in prompts, for instance), when the shell
prints a path, the path is checked to see if it has a named directory
as its prefix. If so, then the prefix portion is replaced with a `~'
followed by the name of the directory. The shortest way of referring
to the directory is used, with ties broken in favour of using a named
directory, except when the directory is / itself. The parameters $PWD
and $OLDPWD are never abbreviated in this fashion.
`=' expansion
If a word begins with an unquoted `=' and the EQUALS option is set, the
remainder of the word is taken as the name of a command. If a command
exists by that name, the word is replaced by the full pathname of the
command.
Notes
Filename expansion is performed on the right hand side of a parameter
assignment, including those appearing after commands of the typeset
family. In this case, the right hand side will be treated as a
colon-separated list in the manner of the PATH parameter, so that a `~'
or an `=' following a `:' is eligible for expansion. All such
behaviour can be disabled by quoting the `~', the `=', or the whole
expression (but not simply the colon); the EQUALS option is also
respected.
If the option MAGIC_EQUAL_SUBST is set, any unquoted shell argument in
the form `identifier=expression' becomes eligible for file expansion as
described in the previous paragraph. Quoting the first `=' also
inhibits this.
FILENAME GENERATION
If a word contains an unquoted instance of one of the characters `*',
`(', `|', `<', `[', or `?', it is regarded as a pattern for filename
generation, unless the GLOB option is unset. If the EXTENDED_GLOB
option is set, the `^' and `#' characters also denote a pattern;
otherwise they are not treated specially by the shell.
The word is replaced with a list of sorted filenames that match the
pattern. If no matching pattern is found, the shell gives an error
message, unless the NULL_GLOB option is set, in which case the word is
deleted; or unless the NOMATCH option is unset, in which case the word
is left unchanged.
In filename generation, the character `/' must be matched explicitly;
also, a `.' must be matched explicitly at the beginning of a pattern or
after a `/', unless the GLOB_DOTS option is set. No filename
generation pattern matches the files `.' or `..'. In other instances
of pattern matching, the `/' and `.' are not treated specially.
Glob Operators
* Matches any string, including the null string.
? Matches any character.
[...] Matches any of the enclosed characters. Ranges of characters
can be specified by separating two characters by a `-'. A `-'
or `]' may be matched by including it as the first character in
the list. There are also several named classes of characters,
in the form `[:name:]' with the following meanings. The first
set use the macros provided by the operating system to test for
the given character combinations, including any modifications
due to local language settings, see ctype(3):
[:alnum:]
The character is alphanumeric
[:alpha:]
The character is alphabetic
[:ascii:]
The character is 7-bit, i.e. is a single-byte character
without the top bit set.
[:blank:]
The character is either space or tab
[:cntrl:]
The character is a control character
[:digit:]
The character is a decimal digit
[:graph:]
The character is a printable character other than
whitespace
[:lower:]
The character is a lowercase letter
[:print:]
The character is printable
[:punct:]
The character is printable but neither alphanumeric nor
whitespace
[:space:]
The character is whitespace
[:upper:]
The character is an uppercase letter
[:xdigit:]
The character is a hexadecimal digit
Another set of named classes is handled internally by the shell
and is not sensitive to the locale:
[:IDENT:]
The character is allowed to form part of a shell
identifier, such as a parameter name
[:IFS:]
The character is used as an input field separator, i.e.
is contained in the IFS parameter
[:IFSSPACE:]
The character is an IFS white space character; see the
documentation for IFS in the zshparam(1) manual page.
[:WORD:]
The character is treated as part of a word; this test is
sensitive to the value of the WORDCHARS parameter
Note that the square brackets are additional to those enclosing
the whole set of characters, so to test for a single
alphanumeric character you need `[[:alnum:]]'. Named character
sets can be used alongside other types, e.g. `[[:alpha:]0-9]'.
[^...]
[!...] Like [...], except that it matches any character which is not in
the given set.
<[x]-[y]>
Matches any number in the range x to y, inclusive. Either of
the numbers may be omitted to make the range open-ended; hence
`<->' matches any number. To match individual digits, the [...]
form is more efficient.
Be careful when using other wildcards adjacent to patterns of
this form; for example, <0-9>* will actually match any number
whatsoever at the start of the string, since the `<0-9>' will
match the first digit, and the `*' will match any others. This
is a trap for the unwary, but is in fact an inevitable
consequence of the rule that the longest possible match always
succeeds. Expressions such as `<0-9>[^[:digit:]]*' can be used
instead.
(...) Matches the enclosed pattern. This is used for grouping. If
the KSH_GLOB option is set, then a `@', `*', `+', `?' or `!'
immediately preceding the `(' is treated specially, as detailed
below. The option SH_GLOB prevents bare parentheses from being
used in this way, though the KSH_GLOB option is still available.
Note that grouping cannot extend over multiple directories: it
is an error to have a `/' within a group (this only applies for
patterns used in filename generation). There is one exception:
a group of the form (pat/)# appearing as a complete path segment
can match a sequence of directories. For example, foo/(a*/)#bar
matches foo/bar, foo/any/bar, foo/any/anyother/bar, and so on.
x|y Matches either x or y. This operator has lower precedence than
any other. The `|' character must be within parentheses, to
avoid interpretation as a pipeline.
^x (Requires EXTENDED_GLOB to be set.) Matches anything except the
pattern x. This has a higher precedence than `/', so `^foo/bar'
will search directories in `.' except `./foo' for a file named
`bar'.
x~y (Requires EXTENDED_GLOB to be set.) Match anything that matches
the pattern x but does not match y. This has lower precedence
than any operator except `|', so `*/*~foo/bar' will search for
all files in all directories in `.' and then exclude `foo/bar'
if there was such a match. Multiple patterns can be excluded by
`foo~bar~baz'. In the exclusion pattern (y), `/' and `.' are
not treated specially the way they usually are in globbing.
x# (Requires EXTENDED_GLOB to be set.) Matches zero or more
occurrences of the pattern x. This operator has high
precedence; `12#' is equivalent to `1(2#)', rather than `(12)#'.
It is an error for an unquoted `#' to follow something which
cannot be repeated; this includes an empty string, a pattern
already followed by `##', or parentheses when part of a KSH_GLOB
pattern (for example, `!(foo)#' is invalid and must be replaced
by `*(!(foo))').
x## (Requires EXTENDED_GLOB to be set.) Matches one or more
occurrences of the pattern x. This operator has high
precedence; `12##' is equivalent to `1(2##)', rather than
`(12)##'. No more than two active `#' characters may appear
together. (Note the potential clash with glob qualifiers in the
form `1(2##)' which should therefore be avoided.)
ksh-like Glob Operators
If the KSH_GLOB option is set, the effects of parentheses can be
modified by a preceding `@', `*', `+', `?' or `!'. This character need
not be unquoted to have special effects, but the `(' must be.
@(...) Match the pattern in the parentheses. (Like `(...)'.)
*(...) Match any number of occurrences. (Like `(...)#'.)
+(...) Match at least one occurrence. (Like `(...)##'.)
?(...) Match zero or one occurrence. (Like `(|...)'.)
!(...) Match anything but the expression in parentheses. (Like
`(^(...))'.)
Precedence
The precedence of the operators given above is (highest) `^', `/', `~',
`|' (lowest); the remaining operators are simply treated from left to
right as part of a string, with `#' and `##' applying to the shortest
possible preceding unit (i.e. a character, `?', `[...]', `<...>', or a
parenthesised expression). As mentioned above, a `/' used as a
directory separator may not appear inside parentheses, while a `|' must
do so; in patterns used in other contexts than filename generation (for
example, in case statements and tests within `[[...]]'), a `/' is not
special; and `/' is also not special after a `~' appearing outside
parentheses in a filename pattern.
Globbing Flags
There are various flags which affect any text to their right up to the
end of the enclosing group or to the end of the pattern; they require
the EXTENDED_GLOB option. All take the form (#X) where X may have one
of the following forms:
i Case insensitive: upper or lower case characters in the pattern
match upper or lower case characters.
l Lower case characters in the pattern match upper or lower case
characters; upper case characters in the pattern still only
match upper case characters.
I Case sensitive: locally negates the effect of i or l from that
point on.
b Activate backreferences for parenthesised groups in the pattern;
this does not work in filename generation. When a pattern with
a set of active parentheses is matched, the strings matched by
the groups are stored in the array $match, the indices of the
beginning of the matched parentheses in the array $mbegin, and
the indices of the end in the array $mend, with the first
element of each array corresponding to the first parenthesised
group, and so on. These arrays are not otherwise special to the
shell. The indices use the same convention as does parameter
substitution, so that elements of $mend and $mbegin may be used
in subscripts; the KSH_ARRAYS option is respected. Sets of
globbing flags are not considered parenthesised groups; only the
first nine active parentheses can be referenced.
For example,
foo="a string with a message"
if [[ $foo = (a|an)' '(#b)(*)' '* ]]; then
print ${foo[$mbegin[1],$mend[1]]}
fi
prints `string with a'. Note that the first parenthesis is
before the (#b) and does not create a backreference.
Backreferences work with all forms of pattern matching other
than filename generation, but note that when performing matches
on an entire array, such as ${array#pattern}, or a global
substitution, such as ${param//pat/repl}, only the data for the
last match remains available. In the case of global
replacements this may still be useful. See the example for the
m flag below.
The numbering of backreferences strictly follows the order of
the opening parentheses from left to right in the pattern
string, although sets of parentheses may be nested. There are
special rules for parentheses followed by `#' or `##'. Only the
last match of the parenthesis is remembered: for example, in `[[
abab = (#b)([ab])# ]]', only the final `b' is stored in
match[1]. Thus extra parentheses may be necessary to match the
complete segment: for example, use `X((ab|cd)#)Y' to match a
whole string of either `ab' or `cd' between `X' and `Y', using
the value of $match[1] rather than $match[2].
If the match fails none of the parameters is altered, so in some
cases it may be necessary to initialise them beforehand. If
some of the backreferences fail to match -- which happens if
they are in an alternate branch which fails to match, or if they
are followed by # and matched zero times -- then the matched
string is set to the empty string, and the start and end indices
are set to -1.
Pattern matching with backreferences is slightly slower than
without.
B Deactivate backreferences, negating the effect of the b flag
from that point on.
cN,M The flag (#cN,M) can be used anywhere that the # or ## operators
can be used; it cannot be combined with other globbing flags and
a bad pattern error occurs if it is misplaced. It is equivalent
to the form {N,M} in regular expressions. The previous
character or group is required to match between N and M times,
inclusive. The form (#cN) requires exactly N matches; (#c,M) is
equivalent to specifying N as 0; (#cN,) specifies that there is
no maximum limit on the number of matches.
m Set references to the match data for the entire string matched;
this is similar to backreferencing and does not work in filename
generation. The flag must be in effect at the end of the
pattern, i.e. not local to a group. The parameters $MATCH,
$MBEGIN and $MEND will be set to the string matched and to the
indices of the beginning and end of the string, respectively.
This is most useful in parameter substitutions, as otherwise the
string matched is obvious.
For example,
arr=(veldt jynx grimps waqf zho buck)
print ${arr//(#m)[aeiou]/${(U)MATCH}}
forces all the matches (i.e. all vowels) into uppercase,
printing `vEldt jynx grImps wAqf zhO bUck'.
Unlike backreferences, there is no speed penalty for using match
references, other than the extra substitutions required for the
replacement strings in cases such as the example shown.
M Deactivate the m flag, hence no references to match data will be
created.
anum Approximate matching: num errors are allowed in the string
matched by the pattern. The rules for this are described in the
next subsection.
s, e Unlike the other flags, these have only a local effect, and each
must appear on its own: `(#s)' and `(#e)' are the only valid
forms. The `(#s)' flag succeeds only at the start of the test
string, and the `(#e)' flag succeeds only at the end of the test
string; they correspond to `^' and `$' in standard regular
expressions. They are useful for matching path segments in
patterns other than those in filename generation (where path
segments are in any case treated separately). For example,
`*((#s)|/)test((#e)|/)*' matches a path segment `test' in any of
the following strings: test, test/at/start, at/end/test,
in/test/middle.
Another use is in parameter substitution; for example
`${array/(#s)A*Z(#e)}' will remove only elements of an array
which match the complete pattern `A*Z'. There are other ways of
performing many operations of this type, however the combination
of the substitution operations `/' and `//' with the `(#s)' and
`(#e)' flags provides a single simple and memorable method.
Note that assertions of the form `(^(#s))' also work, i.e. match
anywhere except at the start of the string, although this
actually means `anything except a zero-length portion at the
start of the string'; you need to use `(""~(#s))' to match a
zero-length portion of the string not at the start.
q A `q' and everything up to the closing parenthesis of the
globbing flags are ignored by the pattern matching code. This
is intended to support the use of glob qualifiers, see below.
The result is that the pattern `(#b)(*).c(#q.)' can be used both
for globbing and for matching against a string. In the former
case, the `(#q.)' will be treated as a glob qualifier and the
`(#b)' will not be useful, while in the latter case the `(#b)'
is useful for backreferences and the `(#q.)' will be ignored.
Note that colon modifiers in the glob qualifiers are also not
applied in ordinary pattern matching.
u Respect the current locale in determining the presence of
multibyte characters in a pattern, provided the shell was
compiled with MULTIBYTE_SUPPORT. This overrides the MULTIBYTE
option; the default behaviour is taken from the option. Compare
U. (Mnemonic: typically multibyte characters are from Unicode
in the UTF-8 encoding, although any extension of ASCII supported
by the system library may be used.)
U All characters are considered to be a single byte long. The
opposite of u. This overrides the MULTIBYTE option.
For example, the test string fooxx can be matched by the pattern
(#i)FOOXX, but not by (#l)FOOXX, (#i)FOO(#I)XX or ((#i)FOOX)X. The
string (#ia2)readme specifies case-insensitive matching of readme with
up to two errors.
When using the ksh syntax for grouping both KSH_GLOB and EXTENDED_GLOB
must be set and the left parenthesis should be preceded by @. Note
also that the flags do not affect letters inside [...] groups, in other
words (#i)[a-z] still matches only lowercase letters. Finally, note
that when examining whole paths case-insensitively every directory must
be searched for all files which match, so that a pattern of the form
(#i)/foo/bar/... is potentially slow.
Approximate Matching
When matching approximately, the shell keeps a count of the errors
found, which cannot exceed the number specified in the (#anum) flags.
Four types of error are recognised:
1. Different characters, as in fooxbar and fooybar.
2. Transposition of characters, as in banana and abnana.
3. A character missing in the target string, as with the pattern
road and target string rod.
4. An extra character appearing in the target string, as with stove
and strove.
Thus, the pattern (#a3)abcd matches dcba, with the errors occurring by
using the first rule twice and the second once, grouping the string as
[d][cb][a] and [a][bc][d].
Non-literal parts of the pattern must match exactly, including
characters in character ranges: hence (#a1)??? matches strings of
length four, by applying rule 4 to an empty part of the pattern, but
not strings of length two, since all the ? must match. Other
characters which must match exactly are initial dots in filenames
(unless the GLOB_DOTS option is set), and all slashes in filenames, so
that a/bc is two errors from ab/c (the slash cannot be transposed with
another character). Similarly, errors are counted separately for
non-contiguous strings in the pattern, so that (ab|cd)ef is two errors
from aebf.
When using exclusion via the ~ operator, approximate matching is
treated entirely separately for the excluded part and must be activated
separately. Thus, (#a1)README~READ_ME matches READ.ME but not READ_ME,
as the trailing READ_ME is matched without approximation. However,
(#a1)README~(#a1)READ_ME does not match any pattern of the form READ?ME
as all such forms are now excluded.
Apart from exclusions, there is only one overall error count; however,
the maximum errors allowed may be altered locally, and this can be
delimited by grouping. For example, (#a1)cat((#a0)dog)fox allows one
error in total, which may not occur in the dog section, and the pattern
(#a1)cat(#a0)dog(#a1)fox is equivalent. Note that the point at which
an error is first found is the crucial one for establishing whether to
use approximation; for example, (#a1)abc(#a0)xyz will not match
abcdxyz, because the error occurs at the `x', where approximation is
turned off.
Entire path segments may be matched approximately, so that
`(#a1)/foo/d/is/available/at/the/bar' allows one error in any path
segment. This is much less efficient than without the (#a1), however,
since every directory in the path must be scanned for a possible
approximate match. It is best to place the (#a1) after any path
segments which are known to be correct.
Recursive Globbing
A pathname component of the form `(foo/)#' matches a path consisting of
zero or more directories matching the pattern foo.
As a shorthand, `**/' is equivalent to `(*/)#'; note that this
therefore matches files in the current directory as well as
subdirectories. Thus:
ls (*/)#bar
or
ls **/bar
does a recursive directory search for files named `bar' (potentially
including the file `bar' in the current directory). This form does not
follow symbolic links; the alternative form `***/' does, but is
otherwise identical. Neither of these can be combined with other forms
of globbing within the same path segment; in that case, the `*'
operators revert to their usual effect.
Glob Qualifiers
Patterns used for filename generation may end in a list of qualifiers
enclosed in parentheses. The qualifiers specify which filenames that
otherwise match the given pattern will be inserted in the argument
list.
If the option BARE_GLOB_QUAL is set, then a trailing set of parentheses
containing no `|' or `(' characters (or `~' if it is special) is taken
as a set of glob qualifiers. A glob subexpression that would normally
be taken as glob qualifiers, for example `(^x)', can be forced to be
treated as part of the glob pattern by doubling the parentheses, in
this case producing `((^x))'.
If the option EXTENDED_GLOB is set, a different syntax for glob
qualifiers is available, namely `(#qx)' where x is any of the same glob
qualifiers used in the other format. The qualifiers must still appear
at the end of the pattern. However, with this syntax multiple glob
qualifiers may be chained together. They are treated as a logical AND
of the individual sets of flags. Also, as the syntax is unambiguous,
the expression will be treated as glob qualifiers just as long any
parentheses contained within it are balanced; appearance of `|', `(' or
`~' does not negate the effect. Note that qualifiers will be
recognised in this form even if a bare glob qualifier exists at the end
of the pattern, for example `*(#q*)(.)' will recognise executable
regular files if both options are set; however, mixed syntax should
probably be avoided for the sake of clarity.
A qualifier may be any one of the following:
/ directories
F `full' (i.e. non-empty) directories. Note that the opposite
sense (^F) expands to empty directories and all non-directories.
Use (/^F) for empty directories
. plain files
@ symbolic links
= sockets
p named pipes (FIFOs)
* executable plain files (0100)
% device files (character or block special)
%b block special files
%c character special files
r owner-readable files (0400)
w owner-writable files (0200)
x owner-executable files (0100)
A group-readable files (0040)
I group-writable files (0020)
E group-executable files (0010)
R world-readable files (0004)
W world-writable files (0002)
X world-executable files (0001)
s setuid files (04000)
S setgid files (02000)
t files with the sticky bit (01000)
fspec files with access rights matching spec. This spec may be a octal
number optionally preceded by a `=', a `+', or a `-'. If none of
these characters is given, the behavior is the same as for `='.
The octal number describes the mode bits to be expected, if
combined with a `=', the value given must match the file-modes
exactly, with a `+', at least the bits in the given number must
be set in the file-modes, and with a `-', the bits in the number
must not be set. Giving a `?' instead of a octal digit anywhere
in the number ensures that the corresponding bits in the
file-modes are not checked, this is only useful in combination
with `='.
If the qualifier `f' is followed by any other character anything
up to the next matching character (`[', `{', and `<' match `]',
`}', and `>' respectively, any other character matches itself)
is taken as a list of comma-separated sub-specs. Each sub-spec
may be either an octal number as described above or a list of
any of the characters `u', `g', `o', and `a', followed by a `=',
a `+', or a `-', followed by a list of any of the characters
`r', `w', `x', `s', and `t', or an octal digit. The first list
of characters specify which access rights are to be checked. If
a `u' is given, those for the owner of the file are used, if a
`g' is given, those of the group are checked, a `o' means to
test those of other users, and the `a' says to test all three
groups. The `=', `+', and `-' again says how the modes are to be
checked and have the same meaning as described for the first
form above. The second list of characters finally says which
access rights are to be expected: `r' for read access, `w' for
write access, `x' for the right to execute the file (or to
search a directory), `s' for the setuid and setgid bits, and `t'
for the sticky bit.
Thus, `*(f70?)' gives the files for which the owner has read,
write, and execute permission, and for which other group members
have no rights, independent of the permissions for other users.
The pattern `*(f-100)' gives all files for which the owner does
not have execute permission, and `*(f:gu+w,o-rx:)' gives the
files for which the owner and the other members of the group
have at least write permission, and for which other users don't
have read or execute permission.
estring
+cmd The string will be executed as shell code. The filename will be
included in the list if and only if the code returns a zero
status (usually the status of the last command). The first
character after the `e' will be used as a separator and anything
up to the next matching separator will be taken as the string;
`[', `{', and `<' match `]', `}', and `>', respectively, while
any other character matches itself. Note that expansions must be
quoted in the string to prevent them from being expanded before
globbing is done.
During the execution of string the filename currently being
tested is available in the parameter REPLY; the parameter may be
altered to a string to be inserted into the list instead of the
original filename. In addition, the parameter reply may be set
to an array or a string, which overrides the value of REPLY. If
set to an array, the latter is inserted into the command line
word by word.
For example, suppose a directory contains a single file
`lonely'. Then the expression `*(e:'reply=(${REPLY}{1,2})':)'
will cause the words `lonely1 lonely2' to be inserted into the
command line. Note the quotation marks.
The form +cmd has the same effect, but no delimiters appear
around cmd. Instead, cmd is taken as the longest sequence of
characters following the + that are alphanumeric or underscore.
Typically cmd will be the name of a shell function that contains
the appropriate test. For example,
nt() { [[ $REPLY -nt $NTREF ]] }
NTREF=reffile
ls -l *(+nt)
lists all files in the directory that have been modified more
recently than reffile.
ddev files on the device dev
l[-|+]ct
files having a link count less than ct (-), greater than ct (+),
or equal to ct
U files owned by the effective user ID
G files owned by the effective group ID
uid files owned by user ID id if that is a number. Otherwise, id
specifies a user name: the character after the `u' will be taken
as a separator and the string between it and the next matching
separator will be taken as a user name. The starting separators
`[', `{', and `<' match the final separators `]', `}', and `>',
respectively; any other character matches itself. The selected
files are those owned by this user. For example, `u:foo:' or
`u[foo]' selects files owned by user `foo'.
gid like uid but with group IDs or names
a[Mwhms][-|+]n
files accessed exactly n days ago. Files accessed within the
last n days are selected using a negative value for n (-n).
Files accessed more than n days ago are selected by a positive n
value (+n). Optional unit specifiers `M', `w', `h', `m' or `s'
(e.g. `ah5') cause the check to be performed with months (of 30
days), weeks, hours, minutes or seconds instead of days,
respectively.
Any fractional part of the difference between the access time
and the current part in the appropriate units is ignored in the
comparison. For instance, `echo *(ah-5)' would echo files
accessed within the last five hours, while `echo *(ah+5)' would
echo files accessed at least six hours ago, as times strictly
between five and six hours are treated as five hours.
m[Mwhms][-|+]n
like the file access qualifier, except that it uses the file
modification time.
c[Mwhms][-|+]n
like the file access qualifier, except that it uses the file
inode change time.
L[+|-]n
files less than n bytes (-), more than n bytes (+), or exactly n
bytes in length. If this flag is directly followed by a `k'
(`K'), `m' (`M'), or `p' (`P') (e.g. `Lk-50') the check is
performed with kilobytes, megabytes, or blocks (of 512 bytes)
instead.
^ negates all qualifiers following it
- toggles between making the qualifiers work on symbolic links
(the default) and the files they point to
M sets the MARK_DIRS option for the current pattern
T appends a trailing qualifier mark to the filenames, analogous to
the LIST_TYPES option, for the current pattern (overrides M)
N sets the NULL_GLOB option for the current pattern
D sets the GLOB_DOTS option for the current pattern
n sets the NUMERIC_GLOB_SORT option for the current pattern
oc specifies how the names of the files should be sorted. If c is n
they are sorted by name (the default); if it is L they are
sorted depending on the size (length) of the files; if l they
are sorted by the number of links; if a, m, or c they are sorted
by the time of the last access, modification, or inode change
respectively; if d, files in subdirectories appear before those
in the current directory at each level of the search -- this is
best combined with other criteria, for example `odon' to sort on
names for files within the same directory; if N, no sorting is
performed. Note that a, m, and c compare the age against the
current time, hence the first name in the list is the youngest
file. Also note that the modifiers ^ and - are used, so
`*(^-oL)' gives a list of all files sorted by file size in
descending order, following any symbolic links. Unless oN is
used, multiple order specifiers may occur to resolve ties.
oe and o+ are special cases; they are each followed by shell
code, delimited as for the e glob qualifier and the + glob
qualifier respectively (see above). The code is executed for
each matched file with the parameter REPLY set to the name of
the file on entry. The code should modify the parameter REPLY
in some fashion. On return, the value of the parameter is used
instead of the file name as the string on which to sort. Unlike
other sort operators, oe and o+ may be repeated, but note that
the maximum number of sort operators of any kind that may appear
in any glob expression is 12.
Oc like `o', but sorts in descending order; i.e. `*(^oc)' is the
same as `*(Oc)' and `*(^Oc)' is the same as `*(oc)'; `Od' puts
files in the current directory before those in subdirectories at
each level of the search.
[beg[,end]]
specifies which of the matched filenames should be included in
the returned list. The syntax is the same as for array
subscripts. beg and the optional end may be mathematical
expressions. As in parameter subscripting they may be negative
to make them count from the last match backward. E.g.:
`*(-OL[1,3])' gives a list of the names of the three largest
files.
More than one of these lists can be combined, separated by commas. The
whole list matches if at least one of the sublists matches (they are
`or'ed, the qualifiers in the sublists are `and'ed). Some qualifiers,
however, affect all matches generated, independent of the sublist in
which they are given. These are the qualifiers `M', `T', `N', `D',
`n', `o', `O' and the subscripts given in brackets (`[...]').
If a `:' appears in a qualifier list, the remainder of the expression
in parenthesis is interpreted as a modifier (see the section
`Modifiers' in the section `History Expansion'). Each modifier must be
introduced by a separate `:'. Note also that the result after
modification does not have to be an existing file. The name of any
existing file can be followed by a modifier of the form `(:..)' even if
no actual filename generation is performed, although note that the
presence of the parentheses causes the entire expression to be
subjected to any global pattern matching options such as NULL_GLOB.
Thus:
ls *(-/)
lists all directories and symbolic links that point to directories, and
ls *(%W)
lists all world-writable device files in the current directory, and
ls *(W,X)
lists all files in the current directory that are world-writable or
world-executable, and
echo /tmp/foo*(u0^@:t)
outputs the basename of all root-owned files beginning with the string
`foo' in /tmp, ignoring symlinks, and
ls *.*~(lex|parse).[ch](^D^l1)
lists all files having a link count of one whose names contain a dot
(but not those starting with a dot, since GLOB_DOTS is explicitly
switched off) except for lex.c, lex.h, parse.c and parse.h.
print b*.pro(#q:s/pro/shmo/)(#q.:s/builtin/shmiltin/)
demonstrates how colon modifiers and other qualifiers may be chained
together. The ordinary qualifier `.' is applied first, then the colon
modifiers in order from left to right. So if EXTENDED_GLOB is set and
the base pattern matches the regular file builtin.pro, the shell will
print `shmiltin.shmo'.