Difference between revisions of "Compiling the Einstein Toolkit"

From Einstein Toolkit Documentation
Jump to: navigation, search
(→‎Building the Einstein Toolkit)
Line 56: Line 56:
 
===Unsupported machines===
 
===Unsupported machines===
  
−
If there is no existing optionlist or runscript (for a cluster, also a submit script) for your machine, you will need to write them yourself.  It is recommended to start from one of the already-written files for another machine which is as close as possible to the machine you are trying to use.  See the sections below for help with this. Once the files are written, run
+
If there is no existing optionlist or runscript (for a cluster, also a submit script) for your machine, you will need to write them yourself.  It is recommended to start from one of the already-written files for another machine which is as close as possible to the machine you are trying to use.  See [[Configuring a new machine]] for help with this. For a workstation or laptop, you only need to provide the option list and run script, and then run
  
−
     simfactory/bin/sim setup --optionlist=<optionlist> --runscript <runscript> --submitscript <submitscript>
+
     simfactory/bin/sim setup --optionlist=<optionlist> --runscript <runscript>
  
−
with the new configuration files.  For a cluster, it is unlikely that this will be enough, as SimFactory needs a lot of information about the details of the cluster, provided in a "machine definition file" in simfactory/mdb/machines.  Copy an example file, and adapt it to your machine.  Getting this right is nontrivial.
+
with the new configuration files.  For a cluster, it is unlikely that this will be enough, as SimFactory needs a lot of information about the details of the cluster, provided in a "machine definition file" in simfactory/mdb/machines.  Copy an example file, and adapt it to your machine.  Getting this right is nontrivial.  In that case, the machine definition file contains the name of the option list and run script to us, so once the machine definition is complete, you only need to run
 +
 
 +
    simfactory/bin/sim setup
 +
 
 +
to configure SimFactory.
  
 
==Building the Einstein Toolkit==
 
==Building the Einstein Toolkit==
Line 67: Line 71:
  
 
     simfactory/bin/sim build --thornlist path/to/thornlist.th
 
     simfactory/bin/sim build --thornlist path/to/thornlist.th
−
 
−
==Configuring a new machine==
 
−
 
−
If your machine is not supported by SimFactory already, you will need to write your own option list, run script and (for a cluster) submit script.
 
−
 
−
===Writing your own optionlist===
 
−
 
−
The options provided by Cactus are described in the [http://cactuscode.org/documentation/usersguide/UsersGuidech6.html#x9-19000B2.1 Cactus documentation].  This page provides additional information and recommendations.
 
−
 
−
The following is based on the ubuntu.cfg optionlist.
 
−
 
−
<pre>
 
−
VERSION = 2012-09-28
 
−
</pre>
 
−
 
−
Cactus will reconfigure when the VERSION string changes.
 
−
 
−
====Compilers====
 
−
 
−
<pre>
 
−
CPP = cpp
 
−
FPP = cpp
 
−
CC  = gcc
 
−
CXX = g++
 
−
F77 = gfortran
 
−
F90 = gfortran
 
−
</pre>
 
−
 
−
The C and Fortran preprocessors, and the C, C++, Fortran 77 and Fortran 90 compilers, are specified by these options.  You can specify a full path if the compiler you want to use is not available on your default path.  Note that it is strongly recommended to use compilers from the same family; e.g. don't mix the Intel C Compiler with the GNU Fortran Compiler.
 
−
 
−
====Compilation and linking flags====
 
−
 
−
<pre>
 
−
CPPFLAGS = -DMPICH_IGNORE_CXX_SEEK
 
−
FPPFLAGS = -traditional
 
−
CFLAGS  = -g3 -march=native -std=gnu99
 
−
CXXFLAGS = -g3 -march=native -std=gnu++0x
 
−
F77FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
 
−
F90FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
 
−
LDFLAGS  = -rdynamic
 
−
</pre>
 
−
 
−
Cactus thorns can be written in C or C++.  Cactus supports the C99 and C++0x standards respectively.  Additionally, the Einstein Toolkit requires the GNU extensions provided by the options gnu99 / gnu++0x.  If these extensions are not available, some Einstein Toolkit thorns will not compile.
 
−
 
−
-g3 ensures that debugging symbols are included in the object files.  It is not necessary to set DEBUG = yes to get debugging symbols.
 
−
 
−
The rdynamic linker flag ensures that additional information is available in the executable for producing backtraces at runtime in the event of an internal error.
 
−
 
−
<pre>
 
−
LIBDIRS =
 
−
 
−
C_LINE_DIRECTIVES = yes
 
−
F_LINE_DIRECTIVES = yes
 
−
</pre>
 
−
 
−
==== Debugging ====
 
−
 
−
<pre>
 
−
DEBUG          = no
 
−
CPP_DEBUG_FLAGS = -DCARPET_DEBUG
 
−
FPP_DEBUG_FLAGS = -DCARPET_DEBUG
 
−
C_DEBUG_FLAGS  = -O0
 
−
CXX_DEBUG_FLAGS = -O0
 
−
F77_DEBUG_FLAGS = -O0
 
−
F90_DEBUG_FLAGS = -O0
 
−
</pre>
 
−
 
−
When DEBUG = yes is set (e.g. on the make command line or with SimFactory's --debug option), these debug flags are used.  The intention here is to disable optimisation and enable additional code which may slow down execution but makes the code easier to debug.
 
−
 
−
==== Optimisation ====
 
−
 
−
<pre>
 
−
OPTIMISE          = yes
 
−
CPP_OPTIMISE_FLAGS = -DKRANC_VECTORS # -DCARPET_OPTIMISE -DNDEBUG
 
−
FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG
 
−
C_OPTIMISE_FLAGS  = -O2 -ffast-math
 
−
CXX_OPTIMISE_FLAGS = -O2 -ffast-math
 
−
F77_OPTIMISE_FLAGS = -O2 -ffast-math
 
−
F90_OPTIMISE_FLAGS = -O2 -ffast-math
 
−
</pre>
 
−
 
−
==== Profiling ====
 
−
 
−
<pre>
 
−
PROFILE          = no
 
−
CPP_PROFILE_FLAGS =
 
−
FPP_PROFILE_FLAGS =
 
−
C_PROFILE_FLAGS  = -pg
 
−
CXX_PROFILE_FLAGS = -pg
 
−
F77_PROFILE_FLAGS = -pg
 
−
F90_PROFILE_FLAGS = -pg
 
−
</pre>
 
−
==== OpenMP ====
 
−
 
−
<pre>
 
−
OPENMP          = yes
 
−
CPP_OPENMP_FLAGS = -fopenmp
 
−
FPP_OPENMP_FLAGS = -fopenmp
 
−
C_OPENMP_FLAGS  = -fopenmp
 
−
CXX_OPENMP_FLAGS = -fopenmp
 
−
F77_OPENMP_FLAGS = -fopenmp
 
−
F90_OPENMP_FLAGS = -fopenmp
 
−
</pre>
 
−
 
−
==== Warnings ====
 
−
 
−
<pre>
 
−
WARN          = yes
 
−
CPP_WARN_FLAGS = -Wall
 
−
FPP_WARN_FLAGS = -Wall
 
−
C_WARN_FLAGS  = -Wall
 
−
CXX_WARN_FLAGS = -Wall
 
−
F77_WARN_FLAGS = -Wall
 
−
F90_WARN_FLAGS = -Wall
 
−
</pre>
 
−
 
−
====ExternalLibraries====
 
−
The Einstein toolkit thorns use a variety of third-party libraries like MPI or HDF5. These are usually provided by helper thorns in the <nowiki>ExternalLibaries</nowiki> arrangement. As a general rule, to enable a capability FOO add
 
−
<pre>
 
−
ExternalLibraries/FOO
 
−
</pre>
 
−
to your <nowiki>ThornList</nowiki> and set <tt>FOO_DIR</tt> to the directory where the <tt>include</tt> and <tt>lib</tt> directories are found.
 
−
=====HDF5=====
 
−
 
−
If no HDF5 options are given, then HDF5 will be used if it can be automatically detected from standard locations, and will be built from a source package in the HDF5 thorn if not.  Alternatively you can specify HDF5_DIR to point to an HF5 installation, for example
 
−
 
−
<pre>
 
−
HDF5_DIR = /usr/local/hdf5-1.9.1
 
−
</pre>
 
−
 
−
The following options disable support for Fortran and C++ when building HDF5, as it is not required by the Einstein Toolkit.
 
−
 
−
<pre>
 
−
HDF5_ENABLE_FORTRAN = no
 
−
HDF5_ENABLE_CXX    = no
 
−
</pre>
 
−
 
−
=====MPI=====
 
−
 
−
<pre>
 
−
MPI_DIR      = /usr
 
−
MPI_INC_DIRS = /usr/include/mpich2
 
−
MPI_LIB_DIRS = /usr/lib
 
−
MPI_LIBS    = mpich fmpich mpl
 
−
</pre>
 
−
 
−
=====Others=====
 
−
 
−
<pre>
 
−
PTHREADS_DIR = NO_BUILD
 
−
</pre>
 
−
 
−
===Submission script===
 
−
 
−
The submission script is used to submit a job to the queueing system.  See the examples in simfactory/mdb/submitscripts, and create a new one for your cluster that uses the same queueing system.
 
−
 
−
===Run script===
 
−
 
−
The most important part of the run script is usually the set of modules that need to be loaded, and the mpirun command to use on the machine.  See the examples in simfactory/mdb/runscripts, and create a new one for your cluster that is similar to one that already exists.
 

Revision as of 07:24, 13 June 2016

The recommended way to compile the Einstein Toolkit is to use the Simulation Factory ("SimFactory").

Configuring SimFactory for your machine

The ET depends on various libraries, and needs to interact with machine-specific queueing systems and MPI implementations. As such, it needs to be configured for a given machine. For this, it uses SimFactory. Generally, configuring SimFactory means providing an optionlist, for specifying library locations and build options, a submit script for using the batch queueing system, and a runscript, for specifying how Cactus should be run, e.g. which mpirun command to use.

Supported clusters

SimFactory comes with support for many of the large supercomputers; run "simfactory/bin/sim list-machines" from your Cactus directory to see the currently-supported list of machines. If you are compiling on one of these, it should be sufficient to run 

   simfactory/bin/sim setup

from your Cactus directory and answer the questions, and SimFactory will automatically detect the correct option list, submit script and run script.

Supported laptop/workstation operating systems

SimFactory also contains general support for specific operating systems commonly used on workstations or laptops, including Mac OS, Ubuntu, Cent OS and Scientific Linux. To configure SimFactory for one of these, you need to find the suitable files in simfactory/mdb/optionlists and simfactory/mdb/runscripts and specify their names on the sim setup command line. The following table lists the files that you need for each supported operating system:

Configuration files
Operating system Option list Run script Notes
Debian debian.cfg debian.sh
Ubuntu/Linux Mint ubuntu.cfg debian.sh
Fedora fedora.cfg debian.sh Run "module load mpi" before compiling or running, and you may need to log out and back in if you have just installed mpich to make the module command work.
OS X (with MacPorts) osx-macports.cfg osx-macports.run
OS X (with HomeBrew) osx-homebrew.cfg generic-mpi.run Run 'export CPATH=/usr/local/include LIBRARY_PATH=/usr/local/lib" before building or running.

You do not need to specify a submit script, as there is no queueing system on a workstation or laptop. Often the optionlist will contain a comment near the top explaining which system packages need to be installed to use the optionlist. For example, for Debian, after installing the packages listed in the comment at the top of debian.cfg, 

   simfactory/bin/sim setup --optionlist=debian.cfg --runscript debian.sh

will configure SimFactory for your machine. For some systems, the runscript can be the general "generic-mpi.sh", or one from a similar OS, for example for Ubuntu you can use debian.sh as a run script.

Unsupported machines

If there is no existing optionlist or runscript (for a cluster, also a submit script) for your machine, you will need to write them yourself. It is recommended to start from one of the already-written files for another machine which is as close as possible to the machine you are trying to use. See Configuring a new machine for help with this. For a workstation or laptop, you only need to provide the option list and run script, and then run 

   simfactory/bin/sim setup --optionlist=<optionlist> --runscript <runscript>

with the new configuration files. For a cluster, it is unlikely that this will be enough, as SimFactory needs a lot of information about the details of the cluster, provided in a "machine definition file" in simfactory/mdb/machines. Copy an example file, and adapt it to your machine. Getting this right is nontrivial. In that case, the machine definition file contains the name of the option list and run script to us, so once the machine definition is complete, you only need to run 

   simfactory/bin/sim setup

to configure SimFactory.

Building the Einstein Toolkit

Assuming that SimFactory has been successfully set up on your machine, you should be able to build the Einstein Toolkit with 

   simfactory/bin/sim build --thornlist path/to/thornlist.th
Retrieved from "https://docs.einsteintoolkit.org/et-docs/index.php?title=Compiling_the_Einstein_Toolkit&oldid=4172"