Compiling the Einstein Toolkit

The recommended way to compile the Einstein Toolkit is to use the Simulation Factory ("SimFactory").

Supercomputers

SimFactory comes with support for many of the large supercomputers. If you are compiling on one of these, it should be sufficient to run

   sim build --thornlist path/to/thornlist.th

Generic operating systems

SimFactory also contains general support for specific operating systems, including Mac OS, Ubuntu, and Scientific Linux. To build a configuration, you need to determine the correct optionlist to use from simfactory/mdb/optionlists. You can then choose the optionlist to use on the "sim build" command line:

   sim build --thornlist path/to/thornlist.th --optionlist <optionlist-name>

Writing your own optionlist

If your operating system is not supported, you will need to write your own optionlist. The options provided by Cactus are described in the Cactus documentation. This page provides additional information and recommendations.

The following is based on the ubuntu.cfg optionlist.

VERSION = 2012-09-28

Cactus will reconfigure when the VERSION string changes.

Compilers

CPP = cpp
FPP = cpp
CC  = gcc
CXX = g++
F77 = gfortran
F90 = gfortran

The C and Fortran preprocessors, and the C, C++, Fortran 77 and Fortran 90 compilers, are specified by these options. You can specify a full path if the compiler you want to use is not available on your default path. Note that it is strongly recommended to use compilers from the same family; e.g. don't mix the Intel C Compiler with the GNU Fortran Compiler.

Compilation and linking flags

CPPFLAGS = -DMPICH_IGNORE_CXX_SEEK
FPPFLAGS = -traditional
CFLAGS   = -g3 -march=native -std=gnu99
CXXFLAGS = -g3 -march=native -std=gnu++0x
F77FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
F90FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
LDFLAGS  = -rdynamic

Cactus thorns can be written in C or C++. Cactus supports the C99 and C++0x standards respectively. Additionally, the Einstein Toolkit requires the GNU extensions provided by the options gnu99 / gnu++0x. If these extensions are not available, some Einstein Toolkit thorns will not compile.

-g3 ensures that debugging symbols are included in the object files. It is not necessary to set DEBUG = yes to get debugging symbols.

The rdynamic linker flag ensures that additional information is available in the executable for producing backtraces at runtime in the event of an internal error.

LIBDIRS =

C_LINE_DIRECTIVES = yes
F_LINE_DIRECTIVES = yes

Debugging

DEBUG           = no
CPP_DEBUG_FLAGS = -DCARPET_DEBUG
FPP_DEBUG_FLAGS = -DCARPET_DEBUG
C_DEBUG_FLAGS   = -O0 
CXX_DEBUG_FLAGS = -O0 
F77_DEBUG_FLAGS = -O0
F90_DEBUG_FLAGS = -O0

When DEBUG = yes is set (e.g. on the make command line or with SimFactory's --debug option), these debug flags are used. The intention here is to disable optimisation and enable additional code which may slow down execution but makes the code easier to debug.

Optimisation

OPTIMISE           = yes
CPP_OPTIMISE_FLAGS = -DKRANC_VECTORS # -DCARPET_OPTIMISE -DNDEBUG
FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG
C_OPTIMISE_FLAGS   = -O2 -ffast-math
CXX_OPTIMISE_FLAGS = -O2 -ffast-math
F77_OPTIMISE_FLAGS = -O2 -ffast-math
F90_OPTIMISE_FLAGS = -O2 -ffast-math

Profiling

PROFILE           = no
CPP_PROFILE_FLAGS =
FPP_PROFILE_FLAGS =
C_PROFILE_FLAGS   = -pg
CXX_PROFILE_FLAGS = -pg
F77_PROFILE_FLAGS = -pg
F90_PROFILE_FLAGS = -pg

OpenMP

OPENMP           = yes
CPP_OPENMP_FLAGS = -fopenmp
FPP_OPENMP_FLAGS = -fopenmp
C_OPENMP_FLAGS   = -fopenmp
CXX_OPENMP_FLAGS = -fopenmp
F77_OPENMP_FLAGS = -fopenmp
F90_OPENMP_FLAGS = -fopenmp

Warnings

WARN           = yes
CPP_WARN_FLAGS = -Wall
FPP_WARN_FLAGS = -Wall
C_WARN_FLAGS   = -Wall
CXX_WARN_FLAGS = -Wall
F77_WARN_FLAGS = -Wall
F90_WARN_FLAGS = -Wall

HDF5

If no HDF5 options are given, then HDF5 will be used if it can be automatically detected from standard locations, and will be built from a source package in the HDF5 thorn if not. Alternatively you can specify HDF5_DIR to point to an HF5 installation, for example

HDF5_DIR = /usr/local/hdf5-1.9.1

The following options disable support for Fortran and C++ when building HDF5, as it is not required by the Einstein Toolkit.

HDF5_ENABLE_FORTRAN = no
HDF5_ENABLE_CXX     = no

MPI

MPI_DIR      = /usr
MPI_INC_DIRS = /usr/include/mpich2
MPI_LIB_DIRS = /usr/lib
MPI_LIBS     = mpich fmpich mpl

Others

PTHREADS = yes