Setup Tutorial

Introduction

Here you will find a step by step guide to downloading, configuring, and running the Einstein Toolkit. You may use this tutorial on a workstation or laptop, or on a supported cluster. Configuring the Einstein Toolkit on an unsupported cluster is beyond the scope of this tutorial. If you find something that does not work, please feel free to edit the wiki or mail users@einsteintoolkit.org.

These instructions were last tested on XX-XXX-XXXX with the ET_XXXX_XX release of the Einstein Toolkit.

Download

A script called GetComponents is used to fetch the components of the Einstein Toolkit. GetComponents serves as convenient wrapper around lower level tools like git and svn to download the codes that make up the Einstein toolkit from their individual repositories. You may download and make it executable as follows:

curl -O -L https://raw.githubusercontent.com/gridaphobe/CRL/ET_2017_06/GetComponents
chmod a+x GetComponents

GetComponents accepts a thorn list as an argument. To check out the needed components:

./GetComponents --parallel https://bitbucket.org/einsteintoolkit/manifest/raw/ET_2017_06/einsteintoolkit.th

This thornlist checks out the entire Einstein Toolkit, including Cactus, the Simulation Factory, the Einstein Toolkit thorns, and Kranc.

Configure and build

The recommended way to compile the Einstein Toolkit is to use the Simulation Factory ("SimFactory").

Configuring SimFactory for your machine

The ET depends on various libraries, and needs to interact with machine-specific queueing systems and MPI implementations. As such, it needs to be configured for a given machine. For this, it uses SimFactory. Generally, configuring SimFactory means providing an optionlist, for specifying library locations and build options, a submit script for using the batch queueing system, and a runscript, for specifying how Cactus should be run, e.g. which mpirun command to use. For a cluster, a "machine definition file" is also required.

Laptop/workstation operating systems

SimFactory contains general support for specific operating systems commonly used on workstations or laptops. The following table lists the supported operating systems, and the information needed to configure SimFactory on each one.

Configuration files

Operating system	Option list	Run script	Notes
Debian	debian.cfg	debian.sh
Ubuntu/Linux Mint	ubuntu.cfg	debian.sh
Fedora	fedora.cfg	debian.sh	Run "module load mpi" before compiling or running, and you may need to log out and back in if you have just installed mpich to make the module command work.
OS X (with MacPorts)	osx-macports.cfg	osx-macports.run
OS X (with HomeBrew)	osx-homebrew.cfg	generic-mpi.run

Find your operating system in the list above, and look at the start of the optionlist for instructions on installing any needed libraries for that operating system.

Then, run

   simfactory/bin/sim setup --optionlist=<optionlist> --runscript <runscript>

where <optionlist> and <runscript> are taken from the table above.

For some systems, the runscript can be the general "generic-mpi.sh", or one from a similar OS, for example for Ubuntu you can use debian.sh as a run script.

Supported clusters

SimFactory comes with support for many of the large supercomputers, and the ET is tested on these at the time of each release. To find out if the cluster you are running on is supported by SimFactory, run

   sim whoami

If this returns

   Current machine: None

then unfortunately SimFactory does not support your cluster, and you will need to configure it manually (see Unsupported Machines below).

If you see

   Current machine: <name of your cluster>

then SimFactory supports your cluster. Run

   simfactory/bin/sim setup

from your Cactus directory and answer the questions, and SimFactory will automatically detect the correct option list, submit script and run script.

Unsupported machines

If there is no existing optionlist or runscript (for a cluster, also a submit script) for your machine, you will need to write them yourself. It is recommended to start from one of the already-written files for another machine which is as close as possible to the machine you are trying to use. See Configuring a new machine for help with this. First, write the option list and run script, and run

   simfactory/bin/sim setup --optionlist=<optionlist> --runscript <runscript>

with the new configuration files. This will create a basic machine definition file in simfactory/mdb/machines named after your hostname. For a cluster, it is unlikely that this basic file will be enough, as SimFactory needs a lot of information about the details of the cluster.

Building the Einstein Toolkit

Assuming that SimFactory has been successfully set up on your machine, you should be able to build the Einstein Toolkit with

   simfactory/bin/sim build --thornlist path/to/thornlist.th

Running

You can now run the Einstein Toolkit with a simple test parameter file.

   simfactory/bin/sim submit helloworld --parfile arrangements/CactusExamples/HelloWorld/par/HelloWorld.par

This will submit the simulation to the queue, if you are running on a cluster, or run it immediately in the background if not. You can check the status of the simulation with

   simfactory/bin/sim list-simulations helloworld

You should see

   helloworld              [ACTIVE (FINISHED), restart 0000, job id 17456]

once the simulation has finished. You can look at the output with

   simfactory/bin/sim show-output helloworld

in which you should see the lines

   INFO (HelloWorld): Hello World!

If you see these line, then congratulations, you have successfully downloaded, compiled and run the Einstein Toolkit! You may now want to try some of the other tutorials to explore some interesting physics examples.