Configuring a new machine - Revision history

Rhaas at 16:34, 9 March 2021

2021-03-09T16:34:18Z

Rhaas: /* MPI */

2021-03-09T16:31:53Z

‎MPI

Rhaas: /* MPI */

2021-03-09T16:31:34Z

‎MPI

Rhaas: /* MPI */

2021-03-09T16:30:52Z

‎MPI

Rhaas: remove F77 flags, add NO_OPTIMISE_FALGS

2020-12-11T00:08:38Z

remove F77 flags, add NO_OPTIMISE_FALGS

Rhaas: /* Configuring a new machine */

2020-10-29T14:38:38Z

‎Configuring a new machine

Rhaas: fix typo in machine ini example

2020-07-23T18:45:25Z

fix typo in machine ini example

Rhaas: remove KRANC_VECTORS

2019-10-09T12:48:49Z

remove KRANC_VECTORS

Rhaas: require gnu++11

2019-10-09T12:38:41Z

require gnu++11

Rhaas: provide link to simfactory option lists

2019-10-07T17:14:32Z

provide link to simfactory option lists

@@ Line 7: / Line 7: @@
 When using SimFactory on a cluster, it needs to know a lot of information about the details of the cluster, provided in a "machine definition file" in simfactory/mdb/machines.  For example, it needs to know the number of cores on each node.  Copy one of the provided files, and adapt it to your machine.  Getting this right is nontrivial.
-When using SimFactory on a laptop or workstation (i.e. a machine for which SimFactory has no machine definition file), the "sim setup" command will write a suitable machine definition file automatically.
+When using SimFactory on your ''laptop or workstation'' (for which SimFactory naturally has no machine definition file), the "sim setup" command will write a suitable machine definition file automatically. "sim setup" does ''not support clusters'' and you have to manually configure things.
 The following is based in stampede.ini:

@@ Line 322: / Line 322: @@
 The correct values to use can be difficult to find out since MPI comes with its own compiler wrappers mpicc etc. that are expected to be used.
-Note that the thorn <code>ExternalLibraries/MPI</code> will often be able to determine the correct options if you set <code>MPI_DIR = NO_BUILD</code>, ensure that <code>mpicxx/code> (or similar) is found in <code>$PATH</code> and leave the other variables unset.
+Note that the thorn <code>ExternalLibraries/MPI</code> will often be able to determine the correct options if you set <code>MPI_DIR = NO_BUILD</code>, ensure that <code>mpicxx</code> (or similar) is found in <code>$PATH</code> and leave the other variables unset.
 In cases where auto-detection fails (not unlikely on clusters), often, the <code>mpicxx</code> wrapper will come with options to query the values to use here.

@@ Line 324: / Line 324: @@
 Note that the thorn <code>ExternalLibraries/MPI</code> will often be able to determine the correct options if you set <code>MPI_DIR = NO_BUILD</code>, ensure that <code>mpicxx/code> (or similar) is found in <code>$PATH</code> and leave the other variables unset.
-In cases where auto-detection fails (not unlikely on clusters), often, the <code>mpicxx/code> wrapper will come with options to query the values to use here.
+In cases where auto-detection fails (not unlikely on clusters), often, the <code>mpicxx</code> wrapper will come with options to query the values to use here.
 {| class="wikitable"

@@ Line 322: / Line 322: @@
 The correct values to use can be difficult to find out since MPI comes with its own compiler wrappers mpicc etc. that are expected to be used.
-Note that the thorn <code>ExternalLibraries/MPI</code> will often be able to determine the correct options if you set <code>MPI_DIR = NO_BUILD</code>, ensure that <code>mpicc</code> (or similar) is found in <code>$PATH</code> and leave the other variables unset.
+Note that the thorn <code>ExternalLibraries/MPI</code> will often be able to determine the correct options if you set <code>MPI_DIR = NO_BUILD</code>, ensure that <code>mpicxx/code> (or similar) is found in <code>$PATH</code> and leave the other variables unset.
-In cases where auto-detection fails (not unlikely on clusters), often, the mpicc wrapper will come with options to query the values to use here.
+In cases where auto-detection fails (not unlikely on clusters), often, the <code>mpicxx/code> wrapper will come with options to query the values to use here.
 {| class="wikitable"
@@ Line 330: / Line 330: @@
 ! MPI stack !! Options !! Comments
 |-
-| OpenMPI || <code>mpicc -showme:compile</code> and <code>mpicc -showme:link</code> ||
+| OpenMPI || <code>mpicxx -showme:compile</code> and <code>mpicc -showme:link</code> ||
 |-
-| mpich (and mvapich) || <code>mpicc -compile_info</code> and <code>mpicc -link_info</code> ||
+| mpich (and mvapich) || <code>mpicxx -compile_info</code> and <code>mpicc -link_info</code> ||
 |-
-| Intel || <code>mpiicc -compile_info</code> and <code>mpiicc -link_info</code> || impi is a derivative of mpich. Note the name of the wrapper is: '''mpiicc''' (with an extra "i").
+| Intel || <code>mpiicxx -compile_info</code> and <code>mpiicxx -link_info</code> || impi is a derivative of mpich. Note the name of the wrapper is: mpi'''i'''cxx (with an extra "i").
 |-
 | Cray ||  || please use the cc, CC, ftn wrappers and load the correct modules

@@ Line 199: / Line 199: @@
 CC  = gcc
 CXX = g++
 F90 = gfortran
 </pre>
@@ Line 212: / Line 211: @@
 CFLAGS   = -g3 -march=native -std=gnu99
 CXXFLAGS = -g3 -march=native -std=gnu+11
 F90FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
 LDFLAGS  = -rdynamic
@@ Line 238: / Line 236: @@
 C_DEBUG_FLAGS   = -O0
 CXX_DEBUG_FLAGS = -O0
 F90_DEBUG_FLAGS = -O0
 </pre>
@@ Line 252: / Line 249: @@
 C_OPTIMISE_FLAGS   = -O2 -ffast-math
 CXX_OPTIMISE_FLAGS = -O2 -ffast-math
 F90_OPTIMISE_FLAGS = -O2 -ffast-math
 </pre>
@@ Line 264: / Line 267: @@
 C_PROFILE_FLAGS   = -pg
 CXX_PROFILE_FLAGS = -pg
 F90_PROFILE_FLAGS = -pg
 </pre>
@@ Line 275: / Line 277: @@
 C_OPENMP_FLAGS   = -fopenmp
 CXX_OPENMP_FLAGS = -fopenmp
 F90_OPENMP_FLAGS = -fopenmp
 </pre>
@@ Line 287: / Line 288: @@
 C_WARN_FLAGS   = -Wall
 CXX_WARN_FLAGS = -Wall
 F90_WARN_FLAGS = -Wall
 </pre>

@@ Line 114: / Line 114: @@
 created, this should live on a fast, parallel file system. <code>cpu</code>,
 <code>cpufreq</code>, <code>flop/cycle</code>, <code>spn</code> and <code>mpn</code>
-are currently unused by simfactoqy. <code>spn</code> is the number of CPU
+are currently unused by simfactory. <code>spn</code> is the number of CPU
 sockets per node, <code>mpn</code> is the number of NUMA domains per node
 (memory sockets). <code>ppn</code> is the nubmer of cores per

@@ Line 54: / Line 54: @@
        ExternalLibraries/OpenSSL
    EOT
-   enabled-thorns = <<EOT:b
+   enabled-thorns = <<EOT
        ExternalLibraries/OpenCL
            CactusExamples/HelloWorldOpenCL

@@ Line 248: / Line 248: @@
 <pre>
 OPTIMISE           = yes
-CPP_OPTIMISE_FLAGS = -DKRANC_VECTORS # -DCARPET_OPTIMISE -DNDEBUG
+CPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG
 FPP_OPTIMISE_FLAGS = # -DCARPET_OPTIMISE -DNDEBUG
 C_OPTIMISE_FLAGS   = -O2 -ffast-math

@@ Line 211: / Line 211: @@
 FPPFLAGS = -traditional
 CFLAGS   = -g3 -march=native -std=gnu99
-CXXFLAGS = -g3 -march=native -std=gnu++0x
+CXXFLAGS = -g3 -march=native -std=gnu+11
 F77FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
 F90FLAGS = -g3 -march=native -fcray-pointer -m128bit-long-double -ffixed-line-length-none
@@ Line 217: / Line 217: @@
 </pre>
-Cactus thorns can be written in C or C++.  Cactus supports the C99 and C++11 standards respectively.  Additionally, the Einstein Toolkit requires the GNU extensions provided by the options gnu99 / gnu++11.  If these extensions are not available, some Einstein Toolkit thorns will not compile.
+Cactus thorns can be written in C or C++.  Cactus supports the C99 and C++11 standards respectively.  Additionally, the Einstein Toolkit requires some of the GNU extensions provided by the options gnu99 / gnu++11.  Without these options each feature required needs to be enabled manually, which is quite error prone. Both GNU and Intel compilers support these options.
 -g3 ensures that debugging symbols are included in the object files.  It is not necessary to set DEBUG = yes to get debugging symbols.

@@ Line 184: / Line 184: @@
 The options provided by Cactus are described in the [https://einsteintoolkit.org/usersguide/UsersGuidech6.html#x9-19000B2.1 Cactus documentation].  This page provides additional information and recommendations.
-The following is based on the ubuntu.cfg optionlist which can be found in  simfactory/mdb/optionslists .
+The following is based on the ubuntu.cfg optionlist which can be found in [https://bitbucket.org/simfactory/simfactory2/src/master/mdb/optionlists/ simfactory/mdb/optionslists]. Usually it is best to start from files describing a cluster that is similar, e.g. uses the same MPI stack and compilers, to the machine you would like to set up.
 <pre>