The FCM make system provides a common environment for running the extract
system, build system, and other utilities. Simply put, it controls a chain of
steps. It is NOT to be confused with the standard Unix utility
make.
See Annex: FCM Configuration File > FCM Make Configuration for a full list of declarations in an FCM Make configuration file.
The FCM make system can be invoked using the command line interface:
fcm make [OPTIONS] [DECLARAION ...]
The FCM make system relies on a configuration file to tell it what to do. It looks for configurations with the following logic:
--config-file=PATH option in the order they are
specified.--config-file=PATH option is not specified, it will
attempt to read fcm-make.cfg if it exists.--directory=PATH option for configuration files
specified as relative paths.--config-file-path=PATH option is
specified, it also searches for configuration files under the specified
values, in the order the options are specified.For details of the other command line options please see FCM Command Reference > fcm make.
The FCM make configuration file is expected to be an FCM configuration file. (See Annex: FCM Configuration File.) A typical FCM make configuration file may look like:
steps = extract build # 1 extract.ns = egg ham bacon # 2 # ... more extract configuration build.target = egg.exe ham.exe bacon # 3 # ... more build configuration
At point 1, the steps
declaration tells the system to invoke two steps. Their IDs are
extract and build, both are IDs for built-in
systems. (The other 2 being mirror and preprocess.)
Each step has its own declarations, which are prefixed with the step ID and a
full stop. The declarations at point 2 are used by the logic for running the
extract step and the declarations at point 3 are used by the
logic for running the build step.
There are times when you may need to invoke the same system (e.g. the
build system) in slightly different configurations. To do this you need to
use the step.class
declaration to define custom step IDs for each build. E.g.:
step.class[build-this build-that] = build # 1
steps = extract build-this build-that # 2
extract.ns = egg ham bacon # 3
# ... more extract configuration
build-this.prop{fc.defs} = DEFS TO BUILD THIS # 4
build-this.target = this.exe
# ... more build configuration for "build-this"
build-that.prop{fc.defs} = DEFS TO BUILD THAT # 5
build-that.target = that.exe
# ... more build configuration for "build-that"
At point 1, we define the IDs build-this and build-that to be an instance of the build class. At point 2, we tell the system to run extract, then build-this, then build-that. At point 4 and 5, we define the configurations for build-this and build-that respectively.
The FCM Make system allows inheritance of configuration and files from
succeeded makes, so the current make does not have to re-do everything that
may already be available else where. To do this, specify the inheritance with
a use declaration,
e.g.:
use = /path/to/succeeded/make/
For more information on how an inheritance behaves, see:
When you run fcm make, it will create a directory
structure that may look like:
.fcm-make/... extract/... build/... ...
Each normal sub-directory, such as extract/ and
build/, contains the result of each step. The hidden
sub-directory .fcm-make/ is used by fcm make as a
working area. It may contain the following items:
fcm
make is invoked with the --archive option, contents in
this directory will be compressed into TAR-GZIP files, e.g.
.fcm-make/cache/extract/ will become
.fcm-make/cache/extract.tar.gz.fcm make
command at this destination.fcm make command at this destination.fcm make command at this destination. It is a
gzip file containing data in the Perl Storable format.fcm make command
at this destination. The content should be equivalent to the diagnostic
output in STDOUT and STDERR at -vv verbosity.You may sometimes need to run fcm make in the same location of
the file system. Perhaps you are on a network with a shared file system, but
only certain tools are available on different hosts, e.g. you can only extract
on one host and build on a different host, you should name the makes so that
they don't end up overwriting each other's context, logs and other
outputs. To do so, you can use the --name=NAME option on the
command line, and/or the name
declaration in the configuration file. Using --name=2 as an
example, you will get the following:
fcm-make2.cfg instead of
fcm-make.cfg.The extract system provides an interface between the version control system (currently Subversion) and the build system. Where appropriate, it extracts code and combines changes from the repositories and other user-defined locations to a directory tree suitable for feeding into the build system.
The extract system supports the --jobs=N option of the
fcm make command. It uses N child processes to get
source trees information and to export source trees from their repositories in
parallel.
The following is an example of how to extract the source trees from a file system path and the trunks of 2 known projects (version controlled in Subversion repositories with a standard FCM layout):
steps = extract # line 1 extract.ns = ops var um # line 2 extract.location[ops] = $HOME/ops # line 3
Here is an explanation of what each line does:
steps declaration tells the make
system to invoke the extract system.extract.ns declaration is used
to specify a space delimited list of names of the projects to extract.extract.location
declaration is used to specify the base source tree of a project
(ops in this case) to extract. The value of the declaration
can be a path in the file system or a location in a version control system.
In this case, the declaration specifies the base source tree of the
ops project to be a path in the file system at
$HOME/ops. If the base source tree of a specified project
name-space is not declared, the system will make an assumption. For a
project hosted in a Subversion repository, the system will assume
trunk@HEAD if the project URL is registered in the keyword
configuration file or with the extract.location{primary}
declaration (see below).If you save this file as fcm-make.cfg and invoke the
fcm make command you should end up with a source tree in the
current working directory that looks like (hidden path not shown):
extract/ops/... extract/um/... extract/var/...
The result of the extract can be found in the extract/ sub-directory. Note: The generated source tree will not contain any symbolic links or hidden files (e.g. file names beginning with a .), because the extract system ignores them.
Note - project name-space and location
Whoever installed FCM at your site would have defined the name-spaces
and the locations of the common projects at your site using the keyword
configuration file at $FCM/etc/fcm-keyword.cfg (where
$FCM/bin is the path at which FCM is installed).
Alternatively, users can define their own set of project name-spaces and
their locations using $HOME/.metomi/fcm/keyword.cfg. For
information on keyword configuration files, please refer to System Administration > FCM
keywords. If a project name-space is not defined in a keyword
configuration, it can be defined in the FCM make configuration using the
extract.location{primary}
declaration. E.g. to define the location of the foo project,
you can do: extract.location{primary}[foo] =
svn://server/repos/foo.
Note - incremental mode
Suppose you have already invoked fcm make using the above
configuration file. At a later time, you have made some changes to some of
the files in the source trees. Re-running fcm make on the same
configuration will trigger the incremental mode. In the incremental mode,
the extract system will update only those files that are modified. If the
last modified time (or last commit revision) of a source file in the
current extract differs from that in the previous extract, the system will
attempt a content comparison. The system updates the destination only if
the content and/or file access permission of the source differs from that
of the destination. To avoid the incremental mode and start afresh, invoke
fcm make with the --new option.
The extract system currently supports the following location types:
~/my-project, /home/lily/my-project, etc.ssh and rsync, specified in the form
[USER@]HOST:PATH. The ssh access must be
without a password, and you must be able to run the GNU coreutils version of the
find and stat on the remote host. E.g.
mylinuxbox:my-project,
holly@hpc:/data/holly/my-project.file, http,
https, svn and svn+ssh. E.g.
svn://mysvnhost/my-project/trunk@40778,
~/my-project@32734.See also extract.location.
Consider a project called foo with a source tree that looks like:
doc/... src/bar/... src/baz/...
Suppose you are only interested in the contents of the src/
sub-tree. You can specify the root of the extract using the extract.path-root
declaration. E.g.:
steps = extract # line 1 extract.ns = foo # line 2 extract.path-root[foo] = src # line 3
Running fcm make with this configuration should give a source
tree that looks like (hidden path not shown):
extract/foo/bar/... extract/foo/baz/...
Going back to the above source tree in the foo project, imagine the src/bar/ sub-directory contains:
src/bar/wine/red.c src/bar/wine/rose.c src/bar/wine/white.c src/bar/...
If you do not want src/bar/wine/rose.c in the extract, you
can ask for it to be excluded using the extract.path-excl
declaration. E.g.:
steps = extract # line 1 extract.ns = foo extract.path-root[foo] = src # line 3 extract.path-excl[foo] = bar/wine/rose.c # line 4
Note: Because the root is redefined in line 3, the path in the
extract.path-excl
declaration in line 4 is declared from the new root level.
Running fcm make with this configuration should give a source
tree that looks like (hidden path not shown):
extract/foo/bar/wine/red.c extract/foo/bar/wine/white.c extract/foo/bar/... extract/foo/baz/...
On the other hand, if you only want src/bar/wine/rose.c in
the extract, you can ask the system to exclude everything in
src/bar/wine/ but include src/bar/wine/rose.c using
the extract.path-incl
declaration. E.g.:
steps = extract # line 1 extract.ns = foo # line 2 extract.path-root[foo] = src # line 3 extract.path-excl[foo] = bar/wine # line 4 extract.path-incl[foo] = bar/wine/rose.c # line 5
Running fcm make with this configuration should give a source
tree that looks like (hidden path not shown):
extract/foo/bar/wine/rose.c extract/foo/bar/... extract/foo/baz/...
We have so far only dealt with extracts from a single base source tree in each project. The extract system can also be used to combine changes from different source trees (against a base source tree) of a project.
E.g. consider a project called food. In the latest release (trunk@3739), the source tree looks like this:
doc/... src/egg/boiled.y src/egg/microwave.x src/egg/omelette.c src/egg/poached.f90 src/...
Jamie, Gordon and Rick are all developing changes against the latest release of the project.
Jamie has made the following changes (displayed using the notation of
svn status) in his branch at
branches/dev/jamie/r3739_t381@3984:
D src/egg/boiled.y A src/egg/fried.pl M src/egg/omelette.c
Gordon has made the following changes in his branch at branches/dev/gordon/r3739_t376@3993:
M src/egg/omelette.c M src/egg/poached.f90
Rick has made the following changes in his working copy at ~rick/food, but has yet to commit his changes back:
M src/egg/boiled.y A src/egg/scrambled.bash
To combine their changes in an extract, the FCM make configuration file should look like:
steps = extract
extract.ns = food
extract.location[food] = trunk@3739
extract.location{diff}[food] = \
branches/dev/jamie/r3739_t381@3984 \
branches/dev/gordon/r3739_t376@3993 \
~rick/food
Here we have an extract.location
declaration like before. This time it is pointing to the latest release of
the food project. The changes against the base source tree are
declared using the extract.location{diff}
declaration.
Invoking fcm make with this configuration file will result in
a source tree that looks like (hidden file not shown):
extract/doc/... extract/src/egg/fried.pl extract/src/egg/microwave.x extract/src/egg/omelette.c extract/src/egg/poached.f90 extract/src/egg/scrambled.bash extract/src/...
Note:
If a previous extract with a similar configuration exists in another location, it can be more efficient to inherit from this previous extract in your current extract. This works like a normal incremental extract, except that your extract will only contain the changes you have specified (compared with the inherited extract) instead of the full directory tree in the destination. This type of incremental extract is useful in several ways. For instance:
Consider the previous example. Imagine an
extract already exists for the latest release for the food
project at /home/food/latest/, and now you want to test the
changes introduced by Jamie, Gordon and Rick. You can use the original extract with the
changes:
use = /home/food/latest
extract.location{diff}[food] = \
branches/dev/jamie/r3739_t381@3984 \
branches/dev/gordon/r3739_t376@3993 \
~rick/food
Invoking fcm make with this configuration file will result in
a source tree that looks like (hidden file not shown):
extract/src/egg/fried.pl extract/src/egg/omelette.c extract/src/egg/poached.f90 extract/src/egg/scrambled.bash
The extract will work in an incremental-like mode. The only difference is that the original extract at /home/food/latest/ will be left untouched, and the new extract will contain only the changes introduced by the diff locations. Note that, although extract/src/egg/boiled.y remains in the original extract, it will not be used in any subsequent build step.
Extract inheritance allows you to add more diff locations to a project, but you should not include any other declarations relating to the extract. Doing so is not safe and should trigger an exception.
In some situations this implies that it will not be possible to use inherited extracts. You should use a new extract if, for instance, a new diff location contains a change which requires the use of source files in a previously excluded name-space.
The amount of diagnostic messages generated by the extract system is
dependent on the diagnostic verbosity level that can be modified by the
-v and -q options to the fcm make
command.
The following is a list of diagnostic output at each verbosity level:
[info] location um: 0: svn://fcm2/UM_svn/UM/trunk@11732 [info] location um: 1: svn://fcm2/UM_svn/UM/branches/dev/Share/VN7.3_hg3_dust_443@11858 [info] location um: 2: svn://fcm2/UM_svn/UM/branches/dev/Share/VN7.3_hg3_ccw_precip@11857 [info] location um: 3: svn://fcm2/UM_svn/UM/branches/dev/hadco/VN7.3_HG3_porting_lsp_fixes@12029 ...
[info] dest: 3 [A added] [info] dest: 134 [a added, overriding inherited] [info] dest: 6 [d deleted, overriding inherited] [info] dest: 2818 [U unchanged] [info] source: 3 [A added by a diff source tree] [info] source: 6 [D deleted by a diff source tree] [info] source: 16 [G merged from 2+ diff source trees] [info] source: 118 [M modified by a diff source tree] [info] source: 2818 [U from base]
... [info] aM um:0,13 atmosphere/short_wave_radiation/r2_lwrad3c.F90 [info] aG um:0,6,13 control/top_level/scm_main.F90 [info] AA um:-,8 include/constant/cnv_parc_lim.h ...
The 2 letters following [info] is the destination status and the source status of the target. This is followed by the name-space of the project, a colon and the source tree number(s) providing the source for the target (in a comma separated list). This is then followed by the name-space of the target path. The source tree number 0 denotes the base source tree, a dash - in place of a 0 means that the source only exists in a diff source tree.
Here is an explanation of each target destination status:
Here is an explanation of each target source status:
The mirror system provides a way to mirror the results of the make steps to another location, where the FCM make may need to continue. It is typically used after an extract to set up the build on an alternate platform.
Consider the following example:
steps = extract mirror # 1
# ... some extract declarations
mirror.target = user@somewhere:/path/in/somewhere # 2
mirror.prop{config-file.name} = -at-somewhere # 3
mirror.prop{config-file.steps} = preprocess build # 4
# ... some preprocess declarations
# ... some build declarations
When the system runs with this configuration, the system will mirror the
result of the extract to the mirror.target, i.e.
somewhere:/path/in/somewhere using ssh and
rsync.
With the config-file.name property set at point 3 and the config-file.steps property set at point 4, it will write a configuration file called fcm-make-at-somewhere.cfg with the make name set as -at-somewhere in the mirror target so that the preprocess and build steps can continue there.
The amount of diagnostic messages generated by the mirror system is
dependent on the diagnostic verbosity level that can be modified by the
-v and -q options to the fcm make
command.
The following is a list of diagnostic output at each verbosity level:
The build system performs actions on a set of source files using its predefined logic and the properties specified in the configuration. For example, it will attempt to create a binary executable for a source file containing a Fortran program.
The build system supports the --jobs=N option of the fcm
make command. It uses N child processes to analyse the
source files and to update the targets in parallel.
Consider a source tree at $HOME/my-source-tree/ containing some Fortran source files including at least one with a main program (and maybe other supported types of source files), you can set up an FCM make configuration file to build an executable. E.g.:
steps = build
build.target{task} = link
build.source = $HOME/my-source-tree
In this simple 3 line configuration, the steps declaration tells the make
system to invoke the build system, the build.target declaration tells
the system to build any targets which require linking (i.e. any main
programs), and the build.source declaration
specifies the location of the source tree.
If you save this file as fcm-make.cfg and invoke the
fcm make command it should attempt to build the source tree in
the current working directory, using the default properties. If the default
Fortran compiler gfortran is installed, and nothing goes wrong,
you will end up with a directory tree that looks like (hidden path not
shown):
build/bin/... build/include/... build/o/...
The result of the build can be found in the sub-directories of the build/ sub-directory. Each build/*/ sub-directory contains a category of targets:
Sub-directories are only created as necessary, so you may not find all of the above in your destination tree.
If fcm make is invoked with the --archive option,
sub-directories in categories containing intermediate build files (or any
category specified in the
archive-ok-target-category
property) will be put into TAR-GZIP files. E.g. with the default setting,
include/ and o/ will become
include.tar.gz and o.tar.gz respectively.
To use a different compiler and/or compiler options for Fortran/C/C++, you use the build.prop declaration to redefine the build properties. E.g.:
steps = build
build.target{task} = link
build.source = $HOME/my-source-tree
# Set Fortran compiler/linker
build.prop{fc} = ifort
# Set Fortran compiler options
build.prop{fc.flags} = -i8 -r8 -O3
# Add include paths to Fortran compiler
build.prop{fc.include-paths} = /a/path/to/include /more/path/to/include
# Set link libraries for Fortran executables
build.prop{fc.lib-paths} = /path/to/my-lib
build.prop{fc.libs} = mine
# Set C compiler/linker
build.prop{cc} = icc
# Set C compiler options
build.prop{cc.flags} = -O3
# Set C++ compiler options
build.prop{cxx.flags} = -O2
# Set link libraries for C executables
build.prop{cc.lib-paths} = /path/to/my-lib /path/to/your-lib
build.prop{cc.libs} = mine yours
# Set linker, if compiler cannot be used as linker
#build.prop{ld} = ld
The build system locates its source files from various places, including:
build.prop{no-step-source}
declaration can be used to switch off this behaviour.build.source declaration.
Note: If you assign a relative path to the build.source declaration, the
system will assume the path to be relative to the make destination (not the
current working directory, unless they happen to be the same).There are situations when it may not be desirable to consider every source
file in a build. You can apply filters by source file name-spaces using the
build.ns-excl
and build.ns-incl declarations.
E.g.:
# To include items in "foo" and "bar/baz" only build.ns-excl = / # exclude everything ... build.ns-incl = foo bar/baz # but include items from these name-spaces
Each source file is assigned a name-space (which is used to fine tune the
build properties, such as the compiler flags). If the source file is a target
of an extract, the name-space will be the same as the extract target (the
relative path of the extract/ sub-directory in the make
destination). If the source file is a file in a source tree specified by
build.source, the
name-space is the relative path to the specified value. The build.source declaration also
accepts an optional name-space, in which case the name-space of each source
file in the tree will be prefixed with the specified name-space. Suppose you
have a source tree in $HOME/food:
$HOME/food/egg.c $HOME/food/ham.f90
If you specify the source tree with build.source = $HOME/food, then $HOME/food/egg.c will be given the name-space egg.c and $HOME/food/ham.f90 will be given the name-space ham.f90.
On the other hand, if you specify the source tree with build.source[food] = $HOME, then $HOME/food/egg.c will be given the name-space food/egg.c and $HOME/food/ham.f90 will be given the name-space food/ham.f90.
The name-space is organised in a simple hierarchy. For instance, the foo/bar/egg name-space belongs to foo/bar, which belongs to foo, which belongs to the root name-space. (The root name-space is either an empty string or a /.)
For instance, you can set the flags of the Fortran compiler using the
build.prop{fc.flags}
declaration at different name-space levels:
# The global Fortran compiler flags
build.prop{fc.flags} = -O3
# The Fortran compiler flags for the "food" name-space
build.prop{fc.flags}[food] = -O2 -i8 -r8
# The Fortran compiler flags for a source file
build.prop{fc.flags}[food/bacon.f90] = -O0 -g -C
Before the build system can do anything with its source files, it needs to
know what they are. It determines the type of each source file by looking at
its file name extension, then the file name itself, and then the
#! line for a text file. Source files without a type are treated
as data files.
The following types are associated with file extensions:
The prop{file-ext.type} = extensions declaration can be used
to modify the extensions associated with a type. E.g. if you need to add
.fort as a file extension for a Fortran source file, you can
do:
build.prop{file-ext.fortran} = .F .FOR .FORT .FTN .F90 .F95 \
.f .for .fort .ftn .f90 .f95 .inc
You can associate file names to some file types using a
prop{file-pat.type} = regular-expression declaration. E.g. if
you have executable scripts in the source tree with no #! lines
but are recognised by a *Scr_* pattern of their file names, you
can specify a regular expression to match their file names using the file-pat.script
property:
build.prop{file-pat.script} = (?msx-i:\w+Scr_\w+)
All other text files with a #! line are recognised as scripts
by the build system.
Each source file with a known type (that is not ignored) is analysed by the build system for dependencies and other information. Here is a list of what the system looks for in each type of file:
main program: e.g. int main().
dependency on include: e.g. #include "name.h".
dependency on object: e.g. /* depends on: name.o */ (for legacy support).
main program: e.g. program name.
list of symbols: i.e. names of top level program units including blockdata, function, module, program and subroutine.
dependency on include: e.g. #include "name.h" and include 'name.f90'.
dependency on module: e.g. use name.
dependency on object: e.g. ! depends on: name.o (for legacy support).
dependency on include: e.g. #include "file-name" (and include 'file-name' for legacy support).
dependency on executable: e.g. # calls: name (for legacy support).
Note: The following features are for legacy support.
DEPENDS ON: x directives in C/Fortran source filesDEPENDS ON: x directive can be used to identify
dependencies on other compiled objects. However, it is much better to
specify this kind of dependency information in the configuration for the
build where necessary. In any case, in modern Fortran code almost all
dependencies should be identified automatically via the use of modules
and/or interface files.calls: x directives in scriptscalls: x directive can be used to identify a
dependency on another executable. However, it is much better to specify
this kind of dependency information in the configuration for the build, and
leave the source code to concentrate on the run time logic.Note: Dependency Analysis and Fortran OpenMP Sentinels.
The build system recognises statements with Fortran OpenMP sentinels that affect build dependencies. E.g.:
!$ USE my_omp_mod, ONLY: my_omp_sub ! ... !$ INCLUDE 'my_omp_logic'
These dependencies are normally ignored. However, if a relevant
build.prop{fc.flag-omp} property is specified, the build system
will treat these statements as normal dependency statements.
There are some situations when it is not possible for the system to
identify a dependency. E.g. a Fortran source file may depend on external
objects that are not detected by the automatic analysis. Therefore, the
system allows you to specify manual dependencies in the configuration file
using the build.prop{dep.type} and
build.prop{ns-dep.type} declarations. E.g.:
# Tell the system that (the object of) food/egg.c depends on chicken.o
build.prop{dep.o}[food/egg.c] = chicken.o
# Tell the system that (the object of) meal/big.c depends on all objects in the
# "food" and "drink" name-spaces
build.prop{ns-dep.o}[meal/big.c] = food drink
Like all declarations that accept name-spaces, if you specify a name-space in this declaration, the property will apply to all source files in the name-space. If you do not specify a name-space, it applies to the root name-space (i.e. globally to all relevant source files).
The following manual dependency declarations are recognised:
There are times when you know that your source tree does not contain a
particular type of dependency, in which case you can switch off the automatic
analysis by using the build.prop{no-dep.type} declaration. E.g.
if you know that all include files in the food name-space are
provided outside of the source tree, you can do:
# Do not check for "include" dependencies
build.prop{no-dep.include}[food] = *
All the types supported by the build.prop{dep.type}
declarations are supported by the build.prop{no-dep.type},
except that there is no build.prop{no-dep.o.special}
(because this type of dependency is never automatic).
The system derives the build targets from the source files. E.g. a C source file egg.c is turned into a compile target to generate egg.o.
The following is a list of what targets are available for each type of file. The title of each item in the list is in the format source type -> target key. The description of each target describes what the target is, and where appropriate, explains how the target keys are named. The task is the action the target needs to perform to get up to date. The category and destination is the sub-directory and destination of the target. The properties are the list of properties that may be used by the task to update the target. The dependencies list the types of dependencies the target may have. The update if is the condition when the target is considered out of date. The pass on information is a list of dependeny types which a target can pass on the status, (see Build Targets Update in Incremental Mode for an explanation of what this means.)
description: source file as an include file.
task: install.
category and destination: include and include/name
dependencies: include and o (object).
update if: source file is modified.
pass on: include, o and o.special.
description: object file. The file is named by mapping the
base name of the original source file in lower case characters, with the
file extension replaced by the first value of the file-ext.o
property.
task: compile (cc).
category and destination: o and o/name.o
properties: cc, cc.flags, cc.defs, cc.flag-compile, cc.flag-define, cc.flag-include, cc.include-paths, cc.flag-omp, cc.flag-output
dependencies: include and o (object).
update if: source file or any of the required properties are modified, or if any include dependencies are updated.
pass on: o and o.special.
description: binary executable. The file is named after the
base name of the original source file, with the file extension replaced by
the first value of the file-ext.bin
property.
task: link (cc).
category and destination: bin and bin/name.exe
properties: ar, ar.flags, file-ext.a, cc, cc.flags-ld, cc.flag-lib, cc.flag-lib-path, cc.libs, cc.lib-paths, cc.flag-omp, cc.flag-output
dependencies: name.o and other objects (o and o.special).
update if: source file or any of the required properties are modified, or if any dependencies are updated.
description: source file as an include file.
task: install.
category and destination: include and include/name
dependencies: include and o (object). A source's f.module dependency on a module called xyz is turned into an include dependency on the xyz.mod.
update if: source file is modified.
pass on: include, o and o.special.
description: object file. The file is named by concatenating
the lower case characters of the name of the first program unit in the
source file and the first value of the file-ext.o
property.
task: compile (fc).
category and destination: o and o/unit.o
properties: fc, fc.flags, fc.defs, fc.flag-compile, fc.flag-define, fc.flag-include, fc.include-paths, fc.flag-module, fc.flag-omp, fc.flag-output
dependencies: include and o (object). A source's f.module dependency on a module called xyz is turned into an include dependency on the xyz.mod.
update if: source file or any of the required properties are modified, or if any include dependencies are updated.
pass on: o and o.special.
remark: trigger unit.mod targets if source file contains a Fortran module.
description: Fortran interface file. The file is named by
concatenating the base name of the source file with the file extension
replaced by the file-ext.f90-interface
property.
task: ext-iface
category and destination: include and include/name.interface
dependencies: unit.o
update if: source file or unit.o is modified.
pass on: include, o and o.special.
description: Fortran module definition file. The file is
named by concatenating the lower case characters of the name of the
module and the first value of the file-ext.f90-mod
property.
task: compile+.
category and destination: include and include/unit.mod
dependencies: unit.o
update if: source file or unit.o is modified.
pass on: o.
description: binary executable. The object file is named
after the base name of the original source file, with the file extension
replaced by the first value of the file-ext.bin
property.
task: link (fc).
category and destination: bin and bin/name.exe
properties: ar, ar.flags, fc, fc.flags-ld, fc.flag-lib, fc.flag-lib-path, fc.libs, fc.lib-paths, fc.flag-omp, fc.flag-output
dependencies: unit.o and other objects (o and o.special).
update if: source file or any of the required properties are modified, or if any dependencies are updated.
description: a header (include) file.
task: install.
category and destination: include and include/name
dependencies: include and o (object).
update if: source file is modified.
pass on: include, o and o.special.
description: an executable script.
task: install.
category and destination: bin and bin/name
dependencies: bin (executable).
update if: source file is modified.
pass on: bin.
description: a data file.
task: install.
category and destination: etc and etc/name-space
update if: source file is modified.
Here is an explanation of what each build system task does:
If you need to specify a property for a specific target, you can either use
their source file namespace or the target key. E.g. If the
sausage.o target is generated from the source file in the
src/food/sausage.f90 namespace, you can specify its Fortran
compiler flags build.prop{fc.flags} by
doing either:
build.prop{fc.flags}[sausage.o] = -O4
# would be the same as:
build.prop{fc.flags}[src/food/sausage.f90] = -O4
This works with most property modifiers, even for dependency related
modifiers such as no-dep.o. E.g.:
build.prop{no-dep.include}[sausage.o] = enum.f90
build.prop{include-paths}[sausage.o] = /path/to/additives
However, the following will not work:
build.prop{no-dep.f.module}[sausage.o] = pork
This is because an object file target is never dependent on a Fortran module by its name. The following will work, however:
build.prop{no-dep.include}[sausage.o] = pork.mod
build.prop{no-dep.o}[sausage.mod] = pork.o
# would be the same as:
build.prop{no-dep.f.module}[src/food/sausage.f90] = pork
To ensure that a Fortran application is built automatically, its source code should be designed with the following considerations:
The name of each compilable program unit should be unique in the source tree, bearing in mind that Fortran is NOT case sensitive.
Always supply an interface for functions and subroutines, i.e.:
PUBLIC attribute. Import them with the USE
<module> statement. We recommend adding the ONLY
clause in a USE <module> when importing symbols from a
module. This makes it easier to locate the source of each symbol, and avoids
unintentional access to other PUBLIC symbols within the
MODULE. If you are importing from an intrinsic module, you
should add the INTRINSIC clause to the USE
<module> statement to tell the build system not to look for the
module from your source tree.CONTAINS section of
a standalone program unit. There are two advantages for this approach.
Firstly, the sub-programs will get an automatic interface when the container
program unit is compiled. Secondly, it should be easier for the compiler to
provide optimisation when the sub-programs are internal to the caller. The
disadvantage of this approach is that the sub-programs are local to the
caller, and so they cannot be called by other program units. Therefore, this
approach is only suitable for small sub-programs local to a particular
program unit.For each free format Fortran source file, e.g. name.f90, with 1 or more top level function and/or subroutine, the system creates a target with the ext-iface task in the include category, e.g. name.interface, to extract the calling interfaces of all functions and subroutines into an interface block. Another Fortran source file, e.g. caller.f90 that relies on the functions and/or subroutines in name.f90 can have an INCLUDE 'name.interface' statement in its specification section, which serves 2 purposes:
In an incremental build, if you modify name.f90, the system will only regenerate name.interface if only the calling interfaces of the functions and/or subroutines in name.f90 have changed. Consequently, non-interface changes in name.f90 will not trigger the re-compile of caller.o, but will only trigger a re-link of the executable.
You need to tell the build system what targets to build or it will do
nothing. The build.target declaration allows
you to select targets according to their categories, source name-spaces,
tasks and keys. The logic is demonstrated by the following example:
# Select targets matching these keys
build.target = egg.bin ham.o bacon.sh
# Select all targets doing tasks "install" or "link"
build.target{task} = install link
# Select targets in name-space "foo" or "bar" doing tasks "link"
build.target{task}[foo bar] = link
# Select all targets in the "bin" category
build.target{category} = bin
# Select targets in name-space "foo" in the "etc" category
build.target{category}[foo] = etc
There are times when an automatic target name is not what you want. In
which case, you can rename a target using the build.target-rename
declaration to specify an alternate name. E.g. if the target
bacon.sh should be called streaky, you can do:
build.target-rename = bacon.sh:streaky
In order for a target to build, all its dependencies must be satisfied. If
a target has a dependency that is not available in the list of targets, the
build will fail. Normally, you can avoid this by using one of the
build.prop{no-dep.*} declarations to switch off a non-existent
dependency, as described in the Build Source
Analysis section. However, there may be times when this is inefficient or
insufficient, in which case you can use the property ignore-missing-dep-ns
to specify a list of source name-spaces, in which targets can ignore missing
dependencies. E.g.:
# Allows targets in the "foo" and "bar/baz"
# name-spaces to ignore missing dependencies.
build.prop{ignore-missing-dep-ns} = foo bar/baz
You can rename the file name extension of the targets using
build.prop{file-ext.type} declaration (provided that the file
name extension is supported by your compiler, etc). E.g. if you want your
binary executables to have .bin extension rather than the
default .exe, you can do:
build.prop{file-ext.bin} = .bin
The following file extensions are currently used by the system:
Apart from source file targets, the build system also generates targets for each (directory-level) name-space. One target is for creating an object archive to contain all object files in the name-space. The other target is a convenient shorthand to allow all data files in the name-space to be installed. The following is the full description:
description: object archive.
task: archive.
category and destination: lib and lib/name-space/libo.a
dependencies: all o (object) targets in the name-space.
update if: any dependencies or properties are modified.
description: dummy file.
task: install.
category and destination: etc and etc/name-space/.etc
dependencies: all data files in the name-space.
update if: any dependencies are modified.
In incremental mode, a target is only updated if it is marked out of date. A target is considered out of date if:
modifiedstatus for a dependency type, which cannot be passed on by the target.
If, after an update, the target's checksum is the same as before, the target will be considered unchanged and up to date. In an incremental build, the use of checksum ensures that any targets manually modified by the user after the previous build is rebuilt accordingly. It also prevents unnecessary updates of targets in incremental and inherited builds.
E.g. Consider an incremental build where the only change is the content of a Fortran module my_mod.f90. The content change should trigger an update of the my_mod.o and my_mod.mod targets, and everything depending on them. However, if the source content is modified in such a way that it does not affect the module's public interface, most compilers will generate an identical my_mod.mod. The system can detect this by comparing the checksums. If my_mod.mod is unchanged, the build system will not need to trigger the re-compile of all targets depending on my_mod.mod, and it will only need to re-link the executable. This allows incremental builds to be more efficient.
Note - checksum algorithm
By default, the MD5 algorithm is used to calculate the checksum. This is normally good enough to detect whether a file is modified or not. If this is insufficient for whatever reasons, you can tell the build system to use one of the SHA algorithms supported by the Perl module Digest::SHA, by setting the value of build.prop{checksum-method}.
If a previous build with a similar configuration exists in another location, it can be more efficient to inherit from this previous build in your current build. This works like a normal incremental build, except that your build will only contain the changes you have specified (compared with the inherited build) instead of the full set of targets.
The current build inherits all properties and target settings, as well as
sources and targets from the inherited build. While properties and target
settings can be overridden with a corresponding declaration, source
inheritance can only be prevented by using a build.prop{no-inherit-source}
declaration. E.g.:
# Prevents inheritance from some name-spaces:
build.prop{no-inherit-source} = food/mint drink/soft/cola.c
For multiple inheritance, the last one takes precedence, and any search for source files or targets are recursive and depth first. For instance, if we have the following declarations in the current FCM make configuration:
use = /path/to/a /path/to/b /path/to/c
and the following in the FCM make configuration of /path/to/b:
use /path/to/d
The relationship looks like:
/path/to/current
/path/to/c
/path/to/b
/path/to/d
/path/to/a
Therefore, we would expect the search path to follow the order:
/path/to/current /path/to/c /path/to/b /path/to/d /path/to/a
In its normal setting, the system does not inherit targets in the
bin, etc and lib categories. A target
in one of these categories is rebuilt in the current destination, whether the
inherited target is up to date or not. This allows someone to use the
executables of the build by setting the PATH environment variable
to point only to $DEST/build/bin/ (where $DEST is the
destination of the current make). If this behaviour is undesirable for
whatever reason, it can be altered using the build.prop{no-inherit-target-category}
declaration.
The build system uses the compiler's -I option to specify
the search path for include files. E.g. it uses this option to specify
the inc/ sub-directories of the current build and its
inherited build.
However, some compilers (e.g. cpp) search for include
files from the container directory of the source file before searching
for the paths specified by the -I options. This behaviour
may cause the build to behave incorrectly.
Consider a source file egg/hen.c that includes fried.h. If the directory structure looks like:
# Sources in inherited build: egg/hen.c egg/fried.h # Sources in current build: egg/fried.h
The system will correctly identify that fried.h is out of date, and trigger a re-compilation of egg/hen.c. However, if the compiler searches for the include files from the container directory of the source file first, it will wrongly use the include file in the inherited build instead of the current one.
If your directory structure does not have any include files in the same
directory as the source files that include them then you do not need to
worry. If it does then you need to check whether you are affected by this
problem before using an inherited build. The situation will vary
according to whether the affected code uses Fortran or preprocessor
include statements and also whether you are using the
preprocess system. Some compilers (e.g.
gfortran) work fine for Fortran includes but not preprocessor
includes. Others (e.g. ifort) have options which can be used
(e.g. -assume nosource_include) to get the desired behaviour.
The FCM distribution includes some simple test code to help you test how
your chosen compilers behave. If you cannot ensure the correct behaviour
then it is safer not to use inherited builds.
The amount of diagnostic messages generated by the build system is
dependent on the diagnostic verbosity level that can be modified by the
-v and -q options to the fcm make
command.
The following is a list of diagnostic output at each verbosity level:
[info] compile targets: modified=5, unchanged=0, total-time=0.5s [info] compile+ targets: modified=1, unchanged=0, total-time=0.0s [info] ext-iface targets: modified=2, unchanged=0, total-time=0.0s [info] install targets: modified=1, unchanged=0, total-time=0.0s [info] link targets: modified=1, unchanged=0, total-time=0.1s [info] TOTAL targets: modified=10, unchanged=0, elapsed-time=0.7s
[info] compile 0.0 M hello_func.o <- lib/function/hello_func.f90 [info] ext-iface 0.0 M hello_func.interface <- lib/function/hello_func.f90 [info] install 0.0 M hello_inc.f90 <- include/hello_inc.f90 [info] compile 0.0 M hello_1.o <- bin/hello_1.f90 [info] link 0.1 M hello_1.exe <- bin/hello_1.f90
[info] analyse 0.0 bin/hello_1.f90 [info] -> ( include) hello_inc.f90 [info] -> ( o) hello_void.o [info] -> ( f.module) hello_mod
[info] source->target / -> (archive) lib/ libo.a [info] source->target hello.f90 -> (link) bin/ hello.exe [info] source->target hello.f90 -> (install) include/ hello.f90 [info] source->target hello.f90 -> (compile) o/ hello.o [info] source->target world.f90 -> (install) include/ world.f90 [info] source->target world.f90 -> (compile+) include/ world.mod [info] source->target world.f90 -> (compile) o/ world.o
[info] required-target: link bin hello_1.exe
[info] target hello_1.exe [info] target - hello_void.o [info] target - hello_1.o [info] target - - hello_mod.mod [info] target - - - hello_mod.o [info] target - - hello_void.o [info] target - - hello_inc.f90 [info] target - - - hello_sub.interface [info] target - - - - hello_sub.o [info] target - - - - - hello_func.interface [info] target - - - - - - hello_func.o [info] target - hello_2.o [info] target - - hello_mod.mod (n-deps=1)
As most modern compilers can handle preprocessing, you should normally leave preprocessing to the compiler. However, it is recognised that some code is written with preprocessor directives that can alter the calling interface of the procedure and/or their dependencies. If a source file requires preprocessing in such a way, we have to preprocess it before feeding it to the build system. The preprocess system can be used to do this. It is typically run as a step before build.
However, using a separate preprocess step is not the best way of working, as it adds an overhead to the build process. If your code requires preprocessing, you should try to design it to avoid changes in the above.
In practice, the only reasonable use of a preprocessor with Fortran is for code selection. For example, preprocessing is useful for isolating machine specific libraries or instructions, where it may be appropriate to use inline alternatives for small sections of code. Another example is when multiple versions of the same procedure exist in the source tree and you need to use the preprocessor to select the correct version for your build.
Avoid using the a preprocessor for code inclusion, as you should be able
to do the same via the Fortran INCLUDE statement. You should
also avoid embedding preprocessor macros within the continuations of a
Fortran statement, as it can make your code very confusing.
The preprocess system works using the same logic as the build system, but is configured primarily to preprocess C/C++ and Fortran source files. We shall document only the main differences to the build system in the remainder of this section.
A typical usage of the preprocess system may look like:
steps = extract preprocess build
# ... some extract configuration
# Switch off preprocessing for all
preprocess.target{task} =
# Only preprocess source files in these name-spaces
preprocess.target{task}[foo/bar egg/fried.F90] = process
# Specifies the macro definitions for the Fortran preprocessor
preprocess.prop{fpp.defs} = THING=stuff HIGH=tall
# Specifies the macro definitions for the C preprocessor
preprocess.prop{cpp.defs} = LOWER=lower UNDER LINUX
# ... some build configuration
The result of the preprocess can be found in the sub-directories of the preprocess/ sub-directory. There are only 2 target categories:
Only files in the following types (with the given file extensions) are recognised by the preprocess system:
The preprocess system only looks for include dependencies using the pattern #include "name.h". Macros using the angle brackets syntax (e.g. #include <name.h>) are ignored.
The preprocess system only generates targets from source files, (i.e. targets are not generated by name-spaces). Targets of the preprocess system perform one of the following tasks:
By default, it attempts to build all targets with a process task, i.e.:
preprocess.target =
preprocess.target{task} = process
preprocess.target{category} =
Here is a list of what targets are available for each type of file:
description: the preprocessed version of the original file.
task: process (cpp).
category and destination: src and src/name-space
properties: cpp, cpp.flags, cpp.defs, cpp.flag-define, cpp.flag-include, cpp.include-paths
dependencies: include.
update if: source file or any of the required properties are modified, or if any include dependencies are updated.
description: the preprocessed version of the original file.
task: process (fpp).
category and destination: src and src/name-space
properties: fpp, fpp.flags, fpp.defs, fpp.flag-define, fpp.flag-include, fpp.include-paths
dependencies: include.
update if: source file or any of the required properties are modified, or if any include dependencies are updated.
description: a header (include) file.
task: install.
category and destination: include and include/name
dependencies: include.
update if: source file is modified.