NAME
shape - identify and build program configurations from versions of
source objects
SYNOPSIS
shape
[ -f <description file> ]
[ -R <version selection rule> ] [ -V <variant name> ]
[ -echo <macro name> ] [ -force <target> ] [ -rebuild <target> ]
[ - dDehiknprs ]
[ -bct ] [ -help ] [ -nomsg ] [ -novclass ] [ -version ]
[ -xpoff ] [ -xpon ]
[ target1 target2 ... ] [ macro=value ... ] [ macro+=value ... ]
DESCRIPTION
Shape allows to transparently compile source objects that are either
regular files, or source object versions in the ShapeTools version
object base. More generally, shape produces a set of derived objects
(‘‘targets’’) from appropriately selected versions of corresponding
source objects according to a description of the dependencies between
the objects. Shape keeps track of the relevant parameters for
compilations (source versions, compiler versions, compiler switches
etc.) and thus provides a safe and efficient build machinery.
When shape compiles source objects, it stores the resulting derived
objects together with the effective compile parameters in its derived
object cache. Before the derivation process for a requested object is
actually started, shape attempts to find an existing derived object
that matches the requirements for the target, in the derived object
cache. Caching and restoring of objects that are derived from
immutable versions allows developers to profit from previous builds by
other team members. Overall compile costs in projects are substantially
reduced.
When serving a build request, shape considers a possibly large number
of versions that are stored for an object. Which particular version is
bound to an object’s name in the description file, is determined by
version selection rules.
Shape can manage builds of different variants of a system in parallel.
Shape uses the combined potential of dynamic version selection, dynamic
macro redefinition, and derived object management to handle variant
builds. As most - if not all - of the objects and parameters involved
in a build are defined as macros, shape provides a great deal of
flexibility by allowing to alter some or all of the macros dynamically,
depending on which variant shall be built. The concept of variant
definition in shape’s description file offers a clear focal point for
all definitions that are relevant for a certain variant.
OPTIONS
-f <description file>
shape uses the supplied argument as name of the description file
to be used for the build. If no -f option is specified, shape
tries to find a description file under one of the names
‘‘Shapefile’’, ‘‘shapefile’’, ‘‘Makefile’’, and ‘‘makefile’’
(from left to right). If no regular file with one of these names
can be found, but versions of respective files are available in
the version object base, shape will use the most recent version.
When more than one -f <description file> argument pair appears,
shape reads each description file in turn. If the name of the
description file is specified as ‘‘-’’, shape will read the
system description from standard input. It is possible to
specify the description file in bound version notation, e.g.
Shapefile[2.8] or Shapefile[Release4] (see vbind(1) for details
about bound version notation).
-R <selection rule name>
activates the specified selection rule as initial version
binding for source objects. If the -R option is present, and a
selection rule is defined as the first dependency of the first
target, shape will use the selection rule passed via the command
line, and ignore the first (and only the first) selection rule
activation within the description file. The option is useful to
override initial default selection rules, specified within the
description file, from the command line.
-V <variant name>
activates the variant specified by <variant name>. Several
variants can be activated simultaneously from the command line
by specifying the -V option multiple times. All variant specific
definitions will be in effect as soon as shape reads the
corresponding variant definition in the description file.
-force <target>
forces shape to build the specified target unconditionally, i.e.
even if a suitable, previously build object exists.
-echo <macro name>
the value of the macro <macro name> is written to standard
output. This option is useful to extract information from the
system description file (e.g. shape -echo SOURCES, or shape
-echo SUBSYSTEMS), or to control the effect of variant
activations.
-rebuild <target>
attempt a precise rebuild of target according to a bound
configuration thread, supplied in a file named <target>.bct (see
description of -bct switch).
-d run shape in debug mode. Print out detailed information about
object dependencies and attributes.
-D print detailed information about the version binding process,
and shape’s reasoning regarding (re)builds of targets, or
retrievals from the derived object cache. This switch is useful
to find out about the exact reasons, why shape rederives a
target (or not).
-e macro definitions that are imported from the environment (see
description of special macro IMPORT, below) override macro
definitions in the description file (by default, macro
definitions in the description file have precedence over imports
from the environment).
-h print usage information on standard output (this is an
abbreviation for the -help switch, see below).
-i ignore error codes returned by commands.
-k when a nonzero error status is returned by an invoked command,
the work on the current target is abandoned but shape continues
with other branches that do not depend on the failed target.
-n no execution mode. Shape prints out commands, but does not
execute them. Even command lines beginning with @ are printed.
If a command contains the $(MAKE) macro reference, however, that
line is always executed in order to allow tracing of recursive
build processes.
-p print out the complete set of macro definitions, target
descriptions, and rule definitions, respectively.
-r do not use shape’s built-in implicit rules. Implicit rules
defined in the description file remain in effect.
-s run in silent mode. Shape does not print out the commands before
executing them.
-bct record the build in a bound configuration thread file. A shape
configuration thread contains precise definitions of all source
versions, their dependencies, the involved tools, and related
options that were in effect for a build. The configuration
thread for a produced toplevel target (the first target in the
description file, or a target requested from the command line)
is stored in a file named <target>.bct. Bound configuration
threads can be used as input for rebuilds (see option -rebuild,
above). If the source version context of a bct-build is unsafe,
shape will record that fact in the bct, and issue a warning
message.
-help print usage information on standard output.
-nomsg turn off the trace facility msg in version selection rules.
-novclass
disable checking for incompatibility of activated variants.
-version
print the version identification of the shape program.
-xpoff turn off attribute expansion in source versions retrieved from
the object base. By default, attribute expansion is turned on
for all source objects that are directly retrieved from the
object base, and turned off for source objects that are regular
files (see retrv(1) for details about attribute expansion).
-xpon turn on attribute expansion for all source objects, even in
regular files. By default, attribute expansion is turned off for
source objects that are regular files, and turned on for all
source objects that are directly retrieved from the object base.
target ...
A list of target names can be passed to shape via the command
line. If no target is given on the command line, and the special
target .DEFAULT is not defined within the description file,
shape tries to produce the first target defined in the
description file.
<macro definition>
It is possible to define or modify macros in the description
file from the command line. Macros that are defined this way
take precedence over all other definitions. Command line macro
definitions have either of two forms:
NAME=VALUE
and
NAME+=VALUE
with NAME being a word and VALUE an arbitrary string. If VALUE contains
white space, make sure to quote it. The first form of command line
macro definitions sets NAME to the substitution VALUE. If VALUE is
empty, the macro is reset. The second form appends VALUE with a
leading space character to the current substitution of NAME. The
current substitution may be defined in the description file, or by a
previous setting on the command line. For details about the semantics
of macro definitions and substitutions, see the respective sections
below.
DESCRIPTION FILES
The operation of shape is controlled by a system description file
(usually a Makefile) that provides structural information about the
system to be managed. Other than make(1), shape works on top of AtFS
(Attributed File System), a repository of versioned objects, rather
than plain files. Thus, genuine shape description files (usually called
Shapefile) feature version selection rules, and variant definitions in
addition to standard Makefile dependency rules. Shape’s description
file is an upward compatible extension of make(1)’s description file,
the Makefile. A useful structuring convention for shape description
files is to maintain a Makefile, and a Shapefile in parallel. Only
genuine shape constructs (such as version selection rules, or variant
definitions) are kept in Shapefile, while the bulk of target rule- and
macro definitions is kept in Makefile. The Makefile shall be included
in Shapefile (see description of include directive, below). This
structuring convention has the advantage that programs that were
developed with the support of the ShapeTools system can be shipped as
source distribution to sites that don’t use ShapeTools.
Although shape is largely downward compatible with the original make
program, it should be noted that several popular extensions of the
original make program, such as GNU Make or Sun Make, provide features
not present in shape. See the section on known incompatibilities
below.
The description file provides an ideal central information base for all
sorts of product related definitions. Shape encourages the development
of a set of (project- or organization-specific) conventions for system
description, and provides a simple way to extract this information for
use by other tools (see -echo option, above). The description file
syntax not only serves to specify component dependencies that are
relevant for build processes, but allows a general, hierarchical
definition of product oriented tasks. The concept of recursive
dependencies maps directly to a stepwise refinement of task
definitions. Such tasks can be fully, partly, or not at all automated
as appropriate. Thus, certain activities may be automated and
standardized, while other activities are just informally described in
order to document them or to reason about them (see shape_rms(1) for
examples).
Syntactical Structure
The basic syntactical structure of shape’s description file is made up
of:
Comments
Comments begin with a ‘‘#’’ character and extend to the end of
the line. In Shapefiles, the end of a line is defined as an
unescaped newline (‘‘\<newline>’’), or the end of the file. The
comment character can’t be escaped, but can be quoted in single-
or double quotes. Comment characters in command lines of target
rules are ignored by shape.
Directives
Directives are special keywords, known to shape. Directives
begin at column 0 of a line and extend to the line end.
Currently, the only directive recognized by shape is
include <list of file names>
Macro Definitions
Macro definitions have the general form:
NAME <macro definition symbol> VALUE
NAME must be a single word consisting of a sequence of name characters.
Name characters are all printable characters except the following:
$ # : = ; <space> \t \n
The macro definition symbol is either of ‘‘=’’, ‘‘+=’’, or ‘‘:=’’.
VALUE is an arbitrary string terminated by the end of the line, or a
comment. Macro definitions usually begin in the first column of a line,
but may be preceded by leading <space> characters. Macro definitions
must not contain leading <tab> characters (see section on Macro
Definitions, below, for more details).
Macro References
Macro references have one of the following forms:
$(<macro name>)
${<macro name>}
$<single character name>
The macro substitution operator (‘‘$’’) can’t be escaped, but can be
represented by the substitution ‘‘$$’’. Macro substitution occurs
anywhere in the description file, except in comments, macro names, left
hand sides of version selection rule- and variant definition headers
(see next section), and variant class definitions (see section on Macro
Substitutions, below, for more details).
Rules Rules are made up from a rule header, and an optional rule body.
The rule header consists of a left hand side, a rule definition
symbol, and an optional right hand side. The left hand side
usually begins in column 0 of a line, and may be preceded by
leading <space> characters. Left hand sides of rule headers must
not contain leading <tab> characters. The optional right hand
side of a rule header extends to the end of the line, or the
beginning of the rule body. A rule body consists of consecutive
lines beginning with a <tab> character. The body of a rule is
terminated by the next line not beginning with a <tab>
character, or the end of the file.
Shape recognizes three different kinds of rules, distinguished
by their respective rule definition symbols:
·
target rules. Target rules have a single colon character
(‘‘:’’) as rule definition symbol. The left hand side of
target rule headers is a space-separated list of names. The
optional right hand side consists of a space-separated list of
names, followed by an optional list of production ingredients
(see section on Target Rules, below).
·
version selection rules. Version selection rules have the rule
definition symbol ‘‘:-’’. The rule header of version selection
rules has a single word on its left hand side, and no right
hand side (see section on Version Selection Rules, below).
·
variant definitions. Although variant definitions are - as the
name suggests - definitions, not rules (from a semantical view
point), their syntactical representation is that of a rule.
Variant definitions have the rule definition symbol ‘‘:+’’.
The rule header of a variant definition has a single word on
its left hand side, and no right hand side (see section on
Variant Definitions, below).
Variant Class Definitions
Variant class definitions have the form
vclass <name> ::= (variant1, variant2, ...)
(see section on Variants, below).
Line Continuations
If the end of an input line is escaped by a backslash (‘‘\’’)
the next line is considered as a continuation line. The
backslash newline character sequence is replaced by a space.
Macro Definitions
Macro definitions associate names with strings that will be substituted
wherever the name of the macro is referenced (see next section). Macros
are useful for writing maintainable, and somewhat generic description
files. Even moderately large projects will find it extremely rewarding
to define conventions for naming and usage of certain macros throughout
the product description file.
There are three different kinds of macro definitions:
Simple Macro Definitions
A simple macro definition looks like
NAME = <any string>
The string that is associated with the macro name can contain macro
references. If a macro is defined multiple times within a description
file, the last definition will be effective. Macros defined on the
command line take precedence over definitions of the same macro in the
description file.
Additive Macro Definitions
This type of macro definition looks like
NAME += <any string>
The string on the right hand side of the definition is appended to any
existing value associated with NAME, separated by a space character.
Multiple additive macro definitions are concatenated in the order in
which they appear in the description file. If an additive macro
definition occurs on the command line, the last string value defined in
the description file is prepended to the string value defined on the
command line. Additive macro definitions in the description file are
appended to string values defined on the command line.
Evaluative Macro Definitions
Evaluative macros are defined in the following way:
NAME := <any string>
First, the string value is associated to NAME in the same way as for
simple macro definitions. When NAME is substituted for the first time,
the right hand side of the definition is evaluated, and the result of
this evaluation replaces the original string value associated with
NAME. Thus, evaluation of the right hand side occurs exactly once. This
is particularly useful if the defining string is a command substitution
(see next section).
Macro Substitutions
Macro substitution is the process of substituting a macro reference by
the string value associated with a macro name. References to undefined
macros are substituted by an empty string. Macro references have either
of the forms:
$(NAME)
${NAME}
$<any single character>
The following are valid macro references:
$(CFLAGS)
$7
${SOURCE-FILES}
$(X)
$X
The last two references have identical substitutions. The macro
reference
$$
will substitute a single dollar sign.
Before a macro reference is substituted, the associated string will be
evaluated. Evaluation of a string value includes
-
substitution of all macro references in the string value
-
command substitution. Any substring of the string value enclosed in
backquotes (‘‘‘’’) will be passed as command to the shell, and be
replaced by the command’s standard output.
-
string substitution. If a macro reference has the form
$(NAME:<old>=<new>)
the reference will be substituted by the evaluated value of NAME, with
all occurrences of the string <old> replaced by the string <new>. This
is particularly useful to maintain related lists, such as CSOURCES and
OBJECTS for example, automatically:
CSOURCES := ‘echo *.c‘
OBJECTS := $(CSOURCES:.c=.o)
Shape substitutes macro references as late as possible. Macro
references occurring in a macro definition are only substituted when
the defined macro itself is substituted. Macro references on the
dependencies side of target rules are substituted when the rule is
evaluated. Macro references on the target side of target rules are
substituted immediately after shape has read the description file, i.e.
before any production is started. Macro references in include
directives are substituted when the directive is executed while shape
reads the description file.
Built-in and Special Purpose Macros
In order to provide parametrization of shape’s built-in implicit rules,
a number of predefined macros is supplied by convention. These macros
have meaningful initial values that can be altered by the user. There
are also several macros that have special meaning for shape.
Macro Purpose Initial value Remark
@ full name of the current <dynamic>special
target
? list of target dependencies <dynamic>special
< name of the first target <dynamic>special
dependency
* prefix shared by target <dynamic>special
and the dependent filenames
# bound version id of the current <dynamic>special
dependency
$ the character ‘‘$’’ $<special>
+ name of object to be bound <dynamic>special
to a version (selection rules
only!)
AS Program for doing assembly asconventional
ASFLAGS Flags for the assembler <none>conventional
CC Program for compiling C ccconventional
programs
CFLAGS Flags for the C compiler <none>conventional
FC Program for compiling Fortran f77conventional
programs
FFLAGS Flags for the Fortran compiler <none>conventional
HOSTTYPE Host architecture of the <none>special
computer that runs shape.
The value of this macro is
used by shape to construct
the derivation key attribute
for derived objects
IMPORT List of environment variables <none>special
that shall be imported as
macro definitions
LD Program to link programs ldconventional
LDFLAGS Flags for the linker <none>conventional
LEX Program to turn Lex grammars lexconventional
into C or Ratfor programs
LFLAGS Flags for the lexical analyzer <none>conventional
lex
LOGNAME The name or network-id under <dynamic>special
which the user who owns the
shape process is logged on
M2C Program for compiling Modula2 m2cconventional
programs
M2FLAGS Flags for the Modula2 compiler <none>conventional
MAKE The command line with which shape $(MAKEFLAGS)special
shape has been invoked.
This macro is used for
recursive calls to shape
MAKEFLAGS Command line flags relevant <defined fromspecial
for recursive calls to shape command line>
PC Program for compiling Pascal pcconventional
programs
PFLAGS Flags for the Pascal compiler <none>conventional
RFLAGS Flags for the Fortran compiler <none>conventional
for Ratfor programs
SHAPEPID The process id of the <dynamic>special
running shape program
SHAPEVERSION The version id of theshape_CM-4.4special
shape program (or above)
SHELL The command processor for /bin/shspecial
the target rule command
lines. The referenced command
processor must be able to
take its commands from
standard input (see section
on Command execution,
below)
VPATH Search path extension for <none>special
localizing source components
YACC Program to turn Yacc grammars yaccconventional
into C programs
YFLAGS Flags for yacc <none> conventional
vpath Dynamic search path extension <none>special
for variants of source components
The function of the special purpose macros HOSTTYPE, IMPORT, MAKE,
VPATH, and vpath are described in the sections on OPERATION, and
Variants below.
Target Rules
A target rule defines how, and under what conditions a target is
derived from a set of source objects and/or other targets. A target is
a name that can refer to a file but need not to do so. Target rules
have the following format:
<target>... : [<version binding>] [+<variant>...] [<dependency>...] \
[: <ingredient>...] [; <command>]
\t[<command>]
...
The header of a target rule (see Syntactical Structure, above) consists
of a list of targets, terminated by a colon, followed by an optional
list of dependencies, and an optional list of production ingredients,
beginning after a second colon character. The rule header is
terminated by a newline or a semicolon, and followed by the optional
rule body. The rule body consists of command lines that are executed
when a target needs to be rederived. The first command line may
immediately follow the semicolon that terminates the rule header.
Subsequent command lines must begin with a <tab> character. The target
rule body is terminated by the first line that doesn’t begin with a
<tab>, or by the end of the file.
Targets
When multiple targets appear on the left hand side of a rule header,
and the derivation process needs to be started, shape will derive all
of the targets in a single run.
Dependencies
Shape checks a target’s dependencies from left to right. The first
dependency is examined whether it is the name of a version selection
rule. If it is, shape sets the selection rule active (eclipsing all
previous selection rule activations), and proceeds to the next
dependency. Next, shape checks whether the dependency is a variant
activation. If the dependency starts with a ‘‘+’’ character followed by
the name of a variant, the variant is activated (see the section on
Variants, below). Shape proceeds to check for variant activations until
the first dependency that isn’t a variant activation is found. Next,
shape proceeds through the list of remaining dependencies, and binds
(or derives) each of them as necessary, performing a depth first
traversal of the dependency graph (see the section on OPERATION,
below).
Production Ingredients
After all dependencies have been bound, shape constructs the derivation
key for the target. The derivation key is an attribute that defines the
complete set of parameters that determine whether a target needs to be
rebuild. Besides all bound dependencies, the derivation key contains
the production ingredients that were specified in the target rule
header. Production ingredients are typically complete definitions of
the macros that are referenced in the command lines of the rule’s body.
Thus, tool versions and switches affecting the operation of a tool can
be made part of the derivation parameters of a target. In order to
include macro definitions into the derivation key of a target, the
special reference
+(NAME1) +(NAME2) ...
must occur in place of the production ingredients.
Command Lines
When shape concludes that a target needs to be (re-)derived, the
commands in the target rule body are executed. The rule body consists
of consecutive lines that are treated as separate commands. Each
command line is evaluated as described in the section on Macro
Substitution, above, and passed to the command interpreter defined by
the macro SHELL. Each command line is executed as a separate process.
If complex commands are needed that don’t fit on a single line, or if
the overhead of repeated process invocations shall be avoided, a
logical command line can be extended by escaping the newline with a
backslash character (\<newline>), and continuing it on the next
physical line.
Command lines may be preceded by one or two special characters:
- shape ignores any nonzero error code returned by a command line
for which the first character is a minus sign. The minus sign is
not passed to the shell. When a command returns a nonzero return
status, shape usually considers the derivation process for the
target as failure and terminates, unless the -i or -k switches,
or the .IGNORE special target is in effect.
@ If the first character of a command is a ‘‘@’’, shape does not
print the command before executing it. The ‘‘@’’ is not passed
to the shell.
@- If the first two non-<tab> characters are ‘‘@-’’, shape ignores
nonzero return codes, and suppresses the printing of the command
line.
If shape is invoked in no execution mode (-n), the evaluated command
lines are printed on standard output, showing what shape would do if
invoked without -n. Command lines that contain the macro reference
$(MAKE) are always executed, even if -n is set. This is done to allow
simulation of recursive builds that may span over subdirectories. The
reference $(MAKE) is substituted by a shape command invocation with all
relevant command line switches set.
Within command lines of the rule body, some parts of the target rule
header can be dynamically referenced. When a command line is evaluated,
the following substitutions are possible
Reference Substitution
$@ full name of the current target
$? list of dependencies
$< name of the first dependency
$* prefix shared by current and the dependent filenames
$# bound version id of the current dependency
(implicit rules only)
Implicit Rules
Shape’s target rules come in two different flavors: explicit, and
implicit. Implicit rules can be seen as templates that define
dependency patterns which apply to most targets of a given kind. For
this reason, implicit rules are sometimes called pattern rules. Shape
converts make’s old-style implicit rules (e.g. .c.o:) to pattern rules
while it reads the description file. A typical dependency pattern is,
for example, the dependency of files containing linkable object code,
e.g. module.o to corresponding files containing source code, e.g.
module.c. The derivation process for most of these source/derived pairs
is identical. Rather than writing separate rules for all
source/derived dependencies of a system, it is possible to write a
single, generic rule, called implicit rule. An implicit rule has the
following format:
%[.<suff1>] %[.<suff2>] ... : %[.<suff3>] %[.<suff4>]... \
[: <ingredient>...] [; <command>]
\t[<command>]
\t[<command>]
...
While the structure of implicit rules is the same as described above,
the names of targets and dependencies are replaced by target patterns,
and dependency templates respectively. The percent character in a
target pattern acts as wildcard that is matched against all of a
target’s name up to the optional trailing suffix. For shape, an object
name suffix is the sequence of characters enclosed by the last period
character (‘‘.’’) within the name, and the <space> character
terminating the name. The following example illustrates shape’s concept
of suffixes:
Name Suffix
sample.cde cde
sample.x.y.cc
sample_c
.sample.c c
The following is an example for an implicit rule that derives linkable
object code from corresponding C source files:
%.o : %.c : +(CC) +(CFLAGS)
\t@echo shape - executing: $(CC) -c $(CFLAGS) $#;
\t@$(CC) $(CFLAGS) -E %.c |
sed ’s;^\(# [0-9][0-9]* \"\)%.c\(\".*\)$$;e1$#\2;’ > %.i;
\t@$(CC) -c $(CFLAGS) %.i;
\t@rm %.i;
NOTE: This rule is shape’s built-in implicit rule to compile C source
files. The cryptic command sequence has the purpose to encode the
complete file version id into the object code (e.g. sample.c[3.4]
rather than sample.c). This is extremely useful in conjunction with
with the -g switch of most C compilers, and version sensitive
debuggers, such as vgdb.
If a target is derived using implicit rules, the name of the target is
matched against the target patterns of the implicit rules. If a rule
matches, the matching portion of the target name (the stem, referred to
by the ‘‘%’’) is consistently substituted for all other occurrences of
the wildcard character throughout the rule. Once this is done, the
implicit rule is treated like an explicit target rule.
Explicit Rules
Explicit rules associate explicit target names with explicit
dependencies. Explicit rules are most typically used to specify
dependencies that cannot be covered by implicit rules, such as deriving
an executable program by linking many object code modules. In many
cases, explicit rules are used to specify only those target
dependencies that are not implied by an implicit rule (such as include
dependencies for object files), while the ‘‘natural’’ dependencies are
assumed as being present. If a description file contains only this sort
of explicit dependencies, the omitted implicit dependencies (and an
applicable rule body) are automatically added by shape to the total
list of target dependencies.
Built-in Implicit Rules
Shape provides a number of predefined implicit target rules that cover
many common source/target derivations. The following table lists target
patterns, and dependency templates for shape’s built-in implicit rules.
Target Dependency Derivation
%.a %.c Compile archive library from C source
%.c %.l Generate C programs from Lex grammar
%.c %.y Generate C programs from Yacc grammar
%.o %.l Compile object code from Lex grammar
%.o %.y Compile object code from Yacc grammar
%.o %.s Translate assembler program to object code
%.o %.r Compile Ratfor source
%.o %.F Compile Fortran source
%.o %.f Compile Fortran source
%.sym %.def Compile Modula definition modules
%.o %.mod Compile Modula implementation modules
%.o %.p Compile Pascal source
%.o %.c Compile C source
% %.sh Make executable program from shell-script
% %.r Build executable program from Ratfor source
% %.F Build executable program from Fortran source
% %.f Build executable program from Fortran source
% %.p Build executable program from Pascal source
% %.mod Build executable program from Modula source
% %.c Build executable program from C source
For a complete definition of shape’s built-in implicit rules, run shape
-p.
Special Purpose Targets
Several aspects of shape’s operation are controlled by special purpose
targets that can be put into the description file. Special purpose
targets by convention begin with a period character, and have no
associated commands.
Target Purpose
.DEFAULT: commands in the rule body of the .DEFAULT target rule are
executed for all targets that cannot be derived by explicit
or implicit target rules. If no commands at all shall be
executed for a rule but .DEFAULT is needed for other
targets, that rule can be given an empty command (either a
‘‘;’’ at the end of the rule header, or an empty line
beginning with <tab>). If .DEFAULT has dependencies, and no
targets are requested from the command line, these
dependencies are treated as if they were targets requested
from the command line.
.IGNORE: causes shape to ignore non zero return codes of invoked
commands. Equivalent to the -i switch
.SILENT: silent command execution. The command lines are not printed
before execution. Equivalent to the -s switch
.BPOOL: only the dependencies associated with this target are
stored in the derived object cache
.NOBPOOL: dependencies associated with this target are not stored in
the derived object cache.
If both, .BPOOL, and .NOBPOOL are defined, only the difference set of
both dependency lists will be stored in the derived object cache.
Version Selection Rules
When shape builds a target, it uses version selection rules to bind a
unique version to each name of the prerequisite source objects. Version
selection rules consist of a name, and an associated set of predicate
lists in the rule body. The format of version selection rules is:
<name> [( <arg1>, <arg2>,...)] :-
\t[<pattern1>,] <pred1> (...), <pred2> (...);
\t[<pattern2>,] <pred1> (...), <pred2> (...);
...
\t.
The body of a version selection rule consists of a sequence of
alternatives, separated by semicolons. Each of the alternatives is an
optional pattern, followed by a comma-separated list of predicates. The
selection rule is terminated by a period character. The semicolon-
separated sequence of alternatives in a version selection rule
constitutes a logical OR expression. The comma-separated list of
predicates in an alternative constitutes a logical AND expression.
Version Binding
Version binding is the process of determining exactly one version of a
given source object from the set of all available versions. Version
binding is said to succeed if one of the rule alternatives succeeds. An
alternative succeeds, if it leads to the identification of exactly one
version. It is said to fail otherwise. When shape binds a version to
the name of a source object, it tries each alternative with a matching
pattern, until the name is unambiguously bound to a version. If the
pattern is omitted, the alternative will be tried unconditionally.
The functioning of version selection rules is one of shape’s most
important, yet most subtile aspects. In order to provide a basis for an
intuitive understanding of the selection rule mechanism, an example is
described. The rule most_recent, below, binds:
- files that were checked out for modification by the shape-invoking
user
- versions of files that were recently modified (→ status saved) by the
same user
- the most recently proposed version (→ status proposed) of files
modified by other users,
- or the file version from the last release.
LASTRELEASE := ‘lastrelease‘# "lastrelease" returns the name
# of the last release
most_recent :-
eq (status, busy), exists ($+[locked_by($(LOGNAME)):]);
ge (status, saved), max (mtime),
max (version), eq (author, $(LOGNAME));
ge (status, proposed), max (mtime),
max (version);
eq (__SymbolicName__, $(LASTRELEASE));
cut ($_rule$: couldn’t bind $+ as requested!).
locked_by (user_id) :-
max (version), eq (locker, $_user_id$).
For a more detailed description of version selection rule syntax,
semantics, and the list of built-in predicates, see BindRules(7).
Activation of Version Selection Rules
A version selection for a certain target is invoked by specifying the
name of the selection rule as first dependency of a target, or by
supplying a selection rule name as argument to the -R option. If no
selection rule is specified explicitly, shape uses its built-in version
selection rule that tries to bind a regular file, or the most recent
version to the name of an object.
Variants
The term variant refers to the intention to manage a product that must
comply with different sets of varying external constraints as a unit.
Independently from particular semantics that might be associated with
the variant notion, there exists a small number of techniques to
implement software variation on a technical level. These techniques
are:
physical separation of variant components. This is achieved by
maintaining separate copies of components in different directories, or
by maintaining variant specific branches in version control systems;
source preprocessing of variant components. With this technique,
multiple logical variants of a source component are maintained in a
single file that contains preprocessor instructions. Before a
particular variant can be accessed, a preprocessor must extract it from
the common source. A popular example of this technique is conditional
compilation, controlled by the #if, and #ifdef instructions within the
domain of C/C++ programming;
composition variation of complex product variants. This technique
addresses the case when different variants of a complex product (such
as a program) are composed from different sets of components;
derivation variation (or variation of the process) that produces
different variants of derived objects from the same set of sources by
modifying parameters of the derivation process. A typical example for
this case is cross compilation of the same sources for different target
platforms, or code instrumentation for various purposes, such as
debugging, testing, profiling, or optimization.
Depending on the particular needs of a project, all of these techniques
may be in simultaneous use, and can occur intermixed as appropriate.
Shape allows to associate logical variant names with a set of
definitions that control all of the above mentioned techniques, making
it possible to request builds of particular system variants (and
combinations of compatible variants) without the need to worry about
how these variants are realized technically.
Variant Definitions
Shape derives its flexibility from using macro substitution in the
description file wherever possible. Shape variant definitions are
basically groups of macro definitions that take effect when the variant
is activated for a build. A variant definition has the following
format:
<variant-name> :+
\t<Macro name1>=<Value>
...
When a variant is activated, the macro definitions associated with the
variant become effective. Any previous definition of a macro made in
the description file, or on the command line is replaced by the variant
macro substitution. If a macro is defined in several variants that are
activated together, the respective values are concatenated.
Locating physically separate Variant Source Objects
Shape provides a special macro, vpath, that is intended to be used in
variant definitions. The vpath macro defines shape’s search precedence
when source version archives are located. If vpath is non-empty, shape
tries to find any referenced source object in the vpath directories
first. If several activated variants define vpath, the variant search
path is concatenated and searched from right to left, i.e. the last
variant that has been activated has precedence. Only if a referenced
source component cannot be found in any of the vpath directories, the
current directory is searched. If a source object has been found, it
will be bound by the current version selection rule, and be temporarily
installed in the build directory. This means that components which are
maintained in a vpath subdirectory are temporarily moved up to the main
directory. Thus, it is not necessary to make any reference to a vpath
subdirectory path in the target rules.
Variant Activation
When a product is configured and built, variants are typically
activated by supplying a variant name as argument to the -V options.
Variants can also be activated for a given target by specifying
respective, ‘‘+’’-prefixed variant names as dependencies (see section
on Target Rules, above). Variant activations for a target must occur
before any real object dependency on the dependency line, and after the
optional version selection rule activation.
Variant Class Definitions
With Variant class definitions, shape offers a construct that allows to
define incompatible variants, i.e. variants that cannot be activated
simultaneously. Shape variant class definitions have the following
format:
vclass <variant-class-name> ::= ( <var1>, <var2> ...)
The same variant name can occur in multiple variant class definitions.
If a combination of variants is requested with any two variant names
that are member of the same variant class, shape will issue an error
message, and terminate. Checking of variant classes can be disabled by
specifying the -novclass switch on the command line.
NOTE: variant class definitions must occur in the description file
before any variant definition referenced in a variant class. Variant
classes that are defined after referenced variants cannot enforce
mutual exclusion of incompatible variants.
An Example
The following example shall illustrate the use of variant definitions,
and variant classes:
vclass compiler ::= (gnu, prop)
gnu:+
CC = gcc -Wall
OPTIMIZE = -O2 -inline-functions
DEBUG = -g -g3
PROFILE = -pg -a
STDC = -ansi
prop:+
CC = cc
OPTIMIZE = +O3
DEBUG = -g -z +Y
PROFILE = -G
STDC = -Aa
vclass quality ::= (debug, profile, optimize)
debug:+
VARCFLAGS = $(DEBUG)
profile:+
VARCFLAGS = $(PROFILE)
optimize:+
VARCFLAGS = $(OPTIMIZE)
CFLAGS += $(VARCFLAGS)
If a variant requires the modification of macros with predefined
meaning, it is sometimes a good idea not to redefine the macro itself
in the variant section. In such a case it is possible to augment an
existing macro value by using shape’s additive macro definition
facility, and a macro from the variant definition defined for this
purpose (e.g. VARCFLAGS in the example above).
OPERATION
When invoked, shape first parses the command line. Shape records the
names of the variants to be activated from the command line via the -V
option. Next, shape initializes the built-in, and special macros. Also,
shape’s built-in derivation rules are initialized.
Reading the Description File
After that, all macro definitions made on the command line are made
effective. Shape then locates and opens its description file. If no
description file is specified as argument to the -f option, shape tries
to find one of the files Shapefile, shapefile, Makefile, or makefile.
For each of these names, shape tries to find a regular file first, and,
if no such file exists, to find the most recent version of that file in
a version control archive. If no such version can be found, shape tries
the next name.
When shape reads the description file, it collects all macro
definitions, and makes them immediately effective, unless a macro of
the same name has been defined on the command line. If the special
macro IMPORT is encountered, the listed environment variables are
defined as macros. If macros with the same name as an imported
environment variable occurs in the description file, it has precedence
over the definition from the environment, unless the -e switch is in
effect.
When shape reads an include directive, it evaluates the rest of the
line (i.e. the characters that immediately follow the directive), and
interprets each word as the name of a file to be read. Each of the file
names is bound to either a regular file, or the most recent version of
the file. Shape opens each of the included files, suspends reading the
current description file, and continues to read the contents of the
included file(s), before it resumes reading of the original control
file. If multiple file names are specified in an include directive,
shape reads each of the files in turn, starting with the leftmost, and
ending with the rightmost file name. If an included file could not be
opened, shape issues a warning.
While shape reads its description files, version selection rules, and
target rules are collected. They are defined only after shape has
finished reading the description file. Macro-, variant-, and variant
class definitions are made effective as soon as they have been
recognized.
The Build Process
After the description file has been read, shape determines which
targets have been requested. If targets have been requested from the
command line, shape will attempt to build each of them, starting with
the leftmost target and proceeding towards the rightmost. If no target
has been requested from the command line, shape searches the
description file for a target named .DEFAULT. If such a target exists,
and there are any dependencies associated with it, shape will attempt
to build each of these dependencies, from left to right. If no .DEFAULT
target rule has been defined in the description file, shape will
attempt to build the first target defined in the description file.
When shape builds a target, it proceeds as follows:
1)
determine the names of the source objects for a given target by
traversing the dependency graph, using built-in and user supplied
target rules. The dependency graph is traversed depth first. The ids
of all applied rules are recorded.
2)
for each required source object, locate the source version archive
in the repository. Locating of source version archives takes the
current vpath into account.
3)
bind each of the source object’s names to an appropriate version as
implied by the currently active version selection rule. Record the
id of each bound dependency. If a dependency is itself a derived
object, use its cache key as id.
4)
construct the derivation key for the current target from the target
name and the records resulting from steps 1) and 3).
5)
search the derived object cache for an object that has a derivation
key identical to the key constructed in step 4).
6a)
if an appropriate derived object was found, a copy of it is
installed in the build directory, rather than deriving it from its
sources.
6b)
if no appropriate derived object was found, it is created by
deriving it from its parts. The resulting derived object is put into
the derived object cache, and associated with the derivation key
resulting from step 4).
Targets with an empty list of dependencies - and thus an empty
derivation key - are always (re-) derived.
When shape determines the dependencies of a requested target, it does
so by evaluating either explicit target rules, or by applying -
possibly built-in - implicit rules. If explicit target rules specify
object dependencies but no derivation script in the rule body, shape
will attempt to supply an appropriate default derivation script. When
searching for such a default derivation script, shape tries to find an
applicable implicit rule for the current target. An implicit rule is
considered applicable, if it has the current target in its list of
targets (after pattern substitution), and all - explicit, and implied -
dependencies exist. If no implicit rule is found to be applicable,
shape looks for the .DEFAULT target rule. If such a rule exists, and if
it has an associated derivation script in its rule body, this script
will be supplied as default derivation script. If neither of the two
possibilities leads to a default derivation script, shape gives up.
Derived Object Caching
Before the derivation process for a requested target is started, it is
attempted to find a suitable derived object in the derived object cache
that matches the required properties. Shape is based on the derivation
key concept for target objects. The derivation key is constructed
according to the algorithm described above. Relevant parameters that go
into the derivation key are the list of dependency ids, the target rule
id, the list of production ingredients, the build platform (usually
defined by the macro HOSTTYPE; if this macro is not defined, shape
takes the host id as build platform), and the attribute expansion
status of each source object. When an object has been derived, shape
stores it in the derived object cache, and marks it with the derivation
key attribute. For a detailed trace of shape’s derived object cache
handling, and the use of derivation keys, run shape with the -D switch.
Command Execution
When a target needs to be (re-) derived, shape executes the commands
associated with the target. Before the commands are executed, shape
sets up the command execution context. The version objects of the
target’s dependencies are installed as regular files in the file
system. If necessary, shape retrieves source objects from the version
control archive. If a file with the object’s name already exists in the
place where a version is to be installed, shape will temporarily move
it to the AtFS subdirectory. After the command script has completed,
shape will restore the original state of all affected directories.
Shape executes a command line by starting the program referenced in the
$(SHELL) macro, and opening a pipe to the resulting process. The
command line is written to the pipe, and thus sent to the $(SHELL)
process’ standard input.
Each of the command lines in a rule body are executed by a separate
process. Thus, the execution status of separate commands is not
preserved. If multiple commands are needed that rely on the execution
status of previous commands, all these commands must occur in a single
command line. This is possible with line continuations (see section on
Syntactical Structure, above).
NOTE: many command interpreters use the ‘‘$’’ character as special
symbol (typically as variable reference). Make sure to pass ‘‘$’’
characters in commands to the $(SHELL) process by using the ‘‘$$’’
special macro (see section on Macro References, above).
INCOMPATIBILITIES
In order to facilitate migration from make(1), shape was designed to be
upward compatible with Makefiles. Although most of make’s description
file features are present in shape, there is a number of
incompatibilities that may need to be taken care of. There exists also
a number of popular extensions of the original make program (e.g. Sun’s
Make, HP’s Make, GNU Make, nmake etc.) that offer various special
features that aren’t supported by other make extensions, or by shape.
When a migration from make to shape is planned, it should be checked
whether special extensions or incompatible features are used.
Features not supported by shape
Double colon rules
Double colon rules associate the same target with different
derivation scripts. This type of rule is useful to support
different derivations for a target depending on which
dependencies are out of date. Because shape bases its decision
whether to derive on the derivation key, rather than mere
modification time stamps of files, this sort of rule makes no
sense in shape.
Archive member targets
Archive member targets are objects that live in an archive file
(see ar(1)) rather than the file system. Within these archives,
make bases its decisions on the modification time stamps of
source files, and archive entry dates. There is no way for shape
to simulate the concept of derivation keys for archive members.
Maintenance of archives, however, is easy with shape, because
all data for compiled object files is maintained in the derived
object cache. If the source for an object that is stored in an
archive is modified, shape can rederive this object, and
selectively replace the entry in the archive.
SCCS stuff
In order to provide basic support for team oriented development
processes, make allows to retrieve the most recent version of
source files from SCCS archives. Because of the awkward naming
convention for SCCS version archive files, special support for
dealing with these archives had to be built into make. Because
shape is tightly integrated with the AtFS version object
repository, there is no need for any special SCCS support.
Special targets
Shape does not recognize the special targets .PRECIOUS:, and
.SUFFIXES:. The .PRECIOUS target in Makefiles has the purpose to
prevent deletion of expensively derived intermediate targets (by
default, make deletes intermediate targets). Because shape
stores intermediate targets in the derived object cache, there
is no need for the .PRECIOUS feature. To prevent caching of
possibly large, useless intermediate targets, use the .NOBPOOL:
special target (see section on Special Targets, above). The
.SUFFIXES target in Makefiles has the purpose to introduce new
suffix types into make’s derivation engine, and to determine the
order in which implicit rules (suffix rules in make terminology)
are applied. In shape, new suffix types can be added
dynamically, simply by introducing new implicit rules. Moreover,
shape has an intelligent algorithm the determines the applicable
implicit rule.
Features with different semantics
Environment Variables
Many make programs import the entire set of environment
variables as macro definitions into the build process. This can
sometimes produce surprising results. In shape, environment
variables are explicitly imported with the IMPORT special macro.
? Macro
In make’s target rules, the special macro reference $? is
substituted by the names of those dependency file names that
have been updated since the current target has been derived.
Because shape bases its decision whether to derive on the
concept of derivation key, rather than mere file modification
time stamps, the ? macro cannot be correctly defined. Instead,
shape substitutes the entire list of dependency names - updated
or not.
FILES
Shapefile, shapefile, Makefile, makefile, /tmp/shapeXXXXXX, <target
name>.bct
SEE ALSO
make(1), save(1), retrv(1), vadm(1), vl(1), vgdb(1), vbind(1),
afintro(3), atfstkintro(3), sttkintro(3), bindrules(7)
CAVEATS AND BUGS
Macro references containing string substitutions cause a syntax error
if used in place of target dependencies. Workaround: use indirect macro
substitution.
There are probably more bugs in shape. Please report any bug findings
to shape-cr@cs.tu-berlin.de.
FURTHER READING
Axel Mahler:
‘‘Using the Shape Toolkit for Cooperative Software
Development - A Tutorial’’, in the toolkit distribution.
Axel Mahler and Andreas Lampen:
‘‘An Integrated Toolset for Engineering Software
Configurations’’, Sigplan Notices, Vol. 24, No. 2, or
Software Engineering Notes, Vol. 13, No. 5, November 1988.
Andreas Lampen and Axel Mahler:
‘‘An Object Base for Attributed Software Objects’’,
Proceedings of the Fall 1988 EUUG Conference.
These and other papers are available via anonymous ftp from coma.cs.tu-
berlin.de (pub/shapeTools/papers).
AUTHOR
Shape was designed by the shape project team at Technical University
Berlin. The program was originally implemented by Wolfgang Obst (1988).
Extensive fixes and modifications were introduced by Axel Mahler
(1992). Valuable contributions came from Steve Emmerson. In 1993 most
parts of shape were re-implemented by Axel Mahler. The version binding
was re-implemented by Andreas Lampen. A complete re-implementation of
the Shapefile parser was done by Juergen Nickelsen.
Contact:
SHAPE
Technical University Berlin
Sekr. FR 5-6
Franklinstr. 28/29
10587 Berlin
General correspondence: shape@cs.tu-berlin.de
Bug reports and modification requests: shape-cr@cs.tu-berlin.de