Difference between revisions of "Testsuite Machines"

From Einstein Toolkit Documentation
Jump to: navigation, search
(use pushable URL for testsuite_results repo)
 
(22 intermediate revisions by 7 users not shown)
Line 3: Line 3:
 
=General=
 
=General=
  
−
To check out the ET:
+
To check out the ET, follow the instructions at http://einsteintoolkit.org/download/ under "Current development version".
  
−
  mkdir etrelease
+
Now configure simfactory:
−
  cd etrelease
 
−
  curl -O https://github.com/gridaphobe/CRL/raw/master/GetComponents
 
−
  chmod a+x GetComponents
 
−
  ./GetComponents --root=. -a https://svn.einsteintoolkit.org/manifest/trunk/einsteintoolkit.th
 
−
  cp udb.example.pm udb.pm
 
  
−
Replace "redshift" with the name of your local machine, unless you are running on one of the supported machines.
+
  cd Cactus
 +
  cp simfactory/etc/defs.local.ini.simple simfactory/etc/defs.local.ini
 +
  nano simfactory/etc/defs.local.ini
  
−
Replace YOUR_LOGIN, YOUR_EMAIL_ADDRESS and YOUR_ALLOCATION as appropriate.
+
Edit defs.local.ini and replace
  
−
Set sourcebasedir for your local machine to be the directory containing etrelease (do not set it to etrelease itself).
+
  YOUR_LOGIN with your username
−
 
+
  YOUR@EMAIL.ADDRESS with your usual email address
−
Set your username on each machine:
+
  YOUR_ALLOCATION with your project allocation
−
 
+
  YOUR_THORNLIST with manifest/einsteintoolkit.th
−
    set_option 'MACHINE'  , 'user', 'USERNAME';
 
  
 
See the machine-specific notes below for any additional steps for each machine.
 
See the machine-specific notes below for any additional steps for each machine.
  
−
  touch par/testsuite.par
 
 
   sim sync <machine>
 
   sim sync <machine>
−
   sim build --thornlist manifest/einsteintoolkit.th --scriptfile <machine>-testsuite.sh
+
  sim login <machine>
 +
   sim build
 +
  sim create-submit ettests_1proc --testsuite --procs <N> --num-threads <N> --walltime 3:00:00
 +
  sim create-submit ettests_2proc --testsuite --procs <N> --num-threads <N/2> --walltime 3:00:00
  
−
  sim remote <machine> create-submit par/testsuite.par NSLOTS 6:00:00
+
Replace <N> and <N/2> with the number of cores on each node, and half of this, respectively, for the machine you are using.  Remember that "procs" here means "cores" and "num-threads" means "number of threads per process". The idea is to use a full node, i.e. all the cores, and then either one or two MPI processes.
  
−
This is a dummy parameter file.
+
When the jobs have finished, you should have the summary.log files in
  
−
When the job is finished, you should have the required <machine>_i_j.log files in machine:etrelease/*.log.  The testsuite output will also be stored in machine:TEST_XXX, where XXX is constructed from the date at which the job was run.
+
  <simulations>/ettests_1proc/output-0000/TEST/sim/summary.log
 +
  <simulations>/ettests_2proc/output-0000/TEST/sim/summary.log
  
 
Update the testsuite status page by adding the log files to the release-info repository:
 
Update the testsuite status page by adding the log files to the release-info repository:
  
−
   svn co https://svn.einsteintoolkit.org/www/release-info
+
   git clone git@bitbucket.org:einsteintoolkit/testsuite_results
 
   cd release-info
 
   cd release-info
−
   scp machine:etrelease/*.log .
+
   scp machine:<simulations>/ettests_1proc/output-0000/TEST/sim/summary.log <machine>__1_<N>.log
−
 
+
  scp machine:<simulations>/ettests_2proc/output-0000/TEST/sim/summary.log <machine>__2_<N/2>.log
−
Then commit the new/updated files.
+
   git add <machine>*.log
−
 
+
  git commit
−
=Machines=
+
  git push
−
 
 
−
==Kraken==
 
−
 
 
−
Edit simfactory/mdb.pm
 
−
 
 
−
  -        myproxy-logon -p 7514 -s myproxy.teragrid.org -T -l @USER@ -o @SOURCEDIR@/.globus/proxy-teragrid
 
−
  +        myproxy-logon -p 7514 -s myproxy.teragrid.org -T -l <username> -o @SOURCEDIR@/.globus/proxy-teragrid
 
−
 
 
−
where <username> is your user name on Kraken.  This is related to [https://trac.einsteintoolkit.org/ticket/381 Ticket #381]
 
−
 
 
−
Change the sourcebasedir in simfactory/udb.pm to be under /lustre/scratch.  This is necessary because the Cactus directory must be visible from the compute node when running the tests (to see the parameter files, testsuite reference output and test.ccl files).
 
−
 
 
−
    set_option 'kraken', 'sourcebasedir', '/lustre/scratch/USERNAME';
 
−
 
 
−
Copy kraken.sh as kraken-testsuite.sh in simfactory/scriptfiles.  Replace the aprun command with the following:
 
−
 
 
−
<pre>
 
−
cd @SOURCEDIR@
 
−
 
 
−
export CCTK_TESTSUITE_RUN_COMMAND="aprun -n \$nprocs -d 1 \$exe \$parfile"
 
−
 
 
−
test_dir=TEST_$(date +%Y-%m-%d-%H%M%S)
 
−
mkdir $test_dir
 
−
 
 
−
CONFIGNAME=$(ls configs|tail) # SimFactory should provide the configuration name
 
−
 
 
−
for i in 1 2 3; do
 
−
  case $i in
 
−
    1 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=1
 
−
      export OMP_NUM_THREADS=1;;
 
−
    2 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=1;;
 
−
    3 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=2;;
 
−
  esac
 
−
  make $CONFIGNAME-testsuite PROMPT=no
 
−
  cp TEST/$CONFIGNAME/summary.log kraken__${CCTK_TESTSUITE_RUN_PROCESSORS}_${OMP_NUM_THREADS}.log
 
−
   mv TEST $test_dir/TEST.$i
 
−
done
 
−
</pre>
 
−
 
 
−
==Datura==
 
−
 
 
−
Copy datura.sh as datura-testsuite.sh in simfactory/scriptfiles.  Replace the mpirun command with the following:
 
−
 
 
−
<pre>
 
−
cd @SOURCEDIR@
 
−
 
 
−
test_dir=TEST_$(date +%Y-%m-%d-%H%M%S)
 
−
mkdir $test_dir
 
−
 
 
−
CONFIGNAME=$(ls configs|tail) # SimFactory should provide the configuration name
 
−
 
 
−
export CACTUS_STARTTIME=$(date +%s)
 
  
−
for i in 1 2 3; do
+
To re-run the tests with an updated checkout, run the GetComponents command above with the --update flag, rebuild, delete the "ettests_*" simulations, and resubmit the simulations.
−
  case $i in
 
−
    1 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=1
 
−
      export OMP_NUM_THREADS=1;;
 
−
    2 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=1;;
 
−
    3 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=2;;
 
−
  esac
 
−
  export CCTK_TESTSUITE_RUN_COMMAND="${MPIDIR}/bin/mpirun -v --mca btl openib,self --mca mpi_leave_pinned 0 -np \$nprocs -npernode $CCTK_TESTSUITE_RUN_PROCESSORS \$exe -L 3 \$parfile"
 
−
  make $CONFIGNAME-testsuite PROMPT=no
 
−
  cp TEST/$CONFIGNAME/summary.log datura__${CCTK_TESTSUITE_RUN_PROCESSORS}_${OMP_NUM_THREADS}.log
 
−
  mv TEST $test_dir/TEST.$i
 
−
done
 
−
</pre>
 

Latest revision as of 18:29, 3 November 2022

This page contains notes and instructions for people running the ET testsuites on various different machines. If you have experience running testsuites on a machine which is not listed here, please consider adding some information which might help others (or yourself!) in future.

General

To check out the ET, follow the instructions at http://einsteintoolkit.org/download/ under "Current development version".

Now configure simfactory: 

 cd Cactus
 cp simfactory/etc/defs.local.ini.simple simfactory/etc/defs.local.ini
 nano simfactory/etc/defs.local.ini

Edit defs.local.ini and replace 

 YOUR_LOGIN with your username
 YOUR@EMAIL.ADDRESS with your usual email address
 YOUR_ALLOCATION with your project allocation
 YOUR_THORNLIST with manifest/einsteintoolkit.th

See the machine-specific notes below for any additional steps for each machine. 

 sim sync <machine>
 sim login <machine>
 sim build
 sim create-submit ettests_1proc --testsuite --procs <N> --num-threads <N> --walltime 3:00:00
 sim create-submit ettests_2proc --testsuite --procs <N> --num-threads <N/2> --walltime 3:00:00

Replace <N> and <N/2> with the number of cores on each node, and half of this, respectively, for the machine you are using. Remember that "procs" here means "cores" and "num-threads" means "number of threads per process". The idea is to use a full node, i.e. all the cores, and then either one or two MPI processes.

When the jobs have finished, you should have the summary.log files in 

 <simulations>/ettests_1proc/output-0000/TEST/sim/summary.log
 <simulations>/ettests_2proc/output-0000/TEST/sim/summary.log

Update the testsuite status page by adding the log files to the release-info repository: 

 git clone git@bitbucket.org:einsteintoolkit/testsuite_results
 cd release-info
 scp machine:<simulations>/ettests_1proc/output-0000/TEST/sim/summary.log <machine>__1_<N>.log
 scp machine:<simulations>/ettests_2proc/output-0000/TEST/sim/summary.log <machine>__2_<N/2>.log
 git add <machine>*.log
 git commit
 git push

To re-run the tests with an updated checkout, run the GetComponents command above with the --update flag, rebuild, delete the "ettests_*" simulations, and resubmit the simulations.

Retrieved from "https://docs.einsteintoolkit.org/et-docs/index.php?title=Testsuite_Machines&oldid=7957"