Difference between revisions of "Testsuite Machines"

From Einstein Toolkit Documentation
Jump to: navigation, search
(use pushable URL for testsuite_results repo)
 
(23 intermediate revisions by 7 users not shown)
Line 1: Line 1:
−
This page contains notes and instructions for people running the ET testsuites on various different machines.  If you have experience running testsuites on a machine which is not listed here, please consider adding some information which might help others (or yourself!) in future.
+
This page contains notes and instructions for people running the ET testsuites on various different machines.  If you have experience running testsuites on a machine which is not listed here, please consider adding some information which might help others (or yourself!) in future.
  
 
=General=
 
=General=
  
−
To check out the ET:
+
To check out the ET, follow the instructions at http://einsteintoolkit.org/download/ under "Current development version".
  
−
  mkdir etrelease
+
Now configure simfactory:
−
  cd etrelease
 
−
  curl -O https://github.com/gridaphobe/CRL/raw/master/GetComponents
 
−
  chmod a+x GetComponents
 
−
  ./GetComponents --root=. -a https://svn.einsteintoolkit.org/manifest/trunk/einsteintoolkit.th
 
  
−
   cp udb.example.pm udb.pm
+
  cd Cactus
 +
   cp simfactory/etc/defs.local.ini.simple simfactory/etc/defs.local.ini
 +
  nano simfactory/etc/defs.local.ini
  
−
  Replace "redshift" with the name of your local machine, unless you are running on one of the supported machines.
+
Edit defs.local.ini and replace
  
−
   Replace YOUR_LOGIN, YOUR_EMAIL_ADDRESS and YOUR_ALLOCATION as appropriate.
+
   YOUR_LOGIN with your username
 +
  YOUR@EMAIL.ADDRESS with your usual email address
 +
  YOUR_ALLOCATION with your project allocation
 +
  YOUR_THORNLIST with manifest/einsteintoolkit.th
  
−
  Set sourcebasedir for your local machine to be the directory containing etrelease (do not set it to etrelease itself).
+
See the machine-specific notes below for any additional steps for each machine.
  
−
   Set your username on each machine:
+
   sim sync <machine>
 +
  sim login <machine>
 +
  sim build
 +
  sim create-submit ettests_1proc --testsuite --procs <N> --num-threads <N> --walltime 3:00:00
 +
  sim create-submit ettests_2proc --testsuite --procs <N> --num-threads <N/2> --walltime 3:00:00
  
−
    set_option 'MACHINE' , 'user', 'USERNAME';
+
Replace <N> and <N/2> with the number of cores on each node, and half of this, respectively, for the machine you are using. Remember that "procs" here means "cores" and "num-threads" means "number of threads per process". The idea is to use a full node, i.e. all the cores, and then either one or two MPI processes.
  
−
=Machines=
+
When the jobs have finished, you should have the summary.log files in
  
−
==Kraken==
+
  <simulations>/ettests_1proc/output-0000/TEST/sim/summary.log
 +
  <simulations>/ettests_2proc/output-0000/TEST/sim/summary.log
  
−
Edit simfactory/mdb.pm
+
Update the testsuite status page by adding the log files to the release-info repository:
  
−
   -        myproxy-logon -p 7514 -s myproxy.teragrid.org -T -l @USER@ -o @SOURCEDIR@/.globus/proxy-teragrid
+
   git clone git@bitbucket.org:einsteintoolkit/testsuite_results
−
   +        myproxy-logon -p 7514 -s myproxy.teragrid.org -T -l <username> -o @SOURCEDIR@/.globus/proxy-teragrid
+
  cd release-info
 +
   scp machine:<simulations>/ettests_1proc/output-0000/TEST/sim/summary.log <machine>__1_<N>.log
 +
  scp machine:<simulations>/ettests_2proc/output-0000/TEST/sim/summary.log <machine>__2_<N/2>.log
 +
  git add <machine>*.log
 +
  git commit
 +
  git push
  
−
where <username> is your user name on Kraken.  This is related to [https://trac.einsteintoolkit.org/ticket/381 Ticket #381]
+
To re-run the tests with an updated checkout, run the GetComponents command above with the --update flag, rebuild, delete the "ettests_*" simulations, and resubmit the simulations.
−
 
 
−
Change the sourcebasedir in simfactory/udb.pm to be under /lustre/scratch.  This is necessary because the Cactus directory must be visible from the compute node when running the tests (to see the parameter files, testsuite reference output and test.ccl files).
 
−
 
 
−
    set_option 'kraken', 'sourcebasedir', '/lustre/scratch/USERNAME';
 
−
 
 
−
Copy kraken.sh as kraken-testsuite.sh in simfactory/scriptfiles.  Replace the aprun command with the following:
 
−
 
 
−
cd @SOURCEDIR@
 
−
 
 
−
export CCTK_TESTSUITE_RUN_COMMAND="aprun -n \$nprocs -d 1 \$exe \$parfile"
 
−
 
 
−
test_dir=TEST_$(date +%Y-%m-%d-%H%M%S)
 
−
mkdir $test_dir
 
−
 
 
−
CONFIGNAME=$(ls configs|tail) # SimFactory should provide the configuration name
 
−
 
 
−
for i in 1 2 3; do
 
−
  case $i in
 
−
    1 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=1
 
−
      export OMP_NUM_THREADS=1;;
 
−
    2 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=1;;
 
−
    3 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=2;;
 
−
  esac
 
−
  make $CONFIGNAME-testsuite PROMPT=no
 
−
  cp TEST/$CONFIGNAME/summary.log kraken__${CCTK_TESTSUITE_RUN_PROCESSORS}_${OMP_NUM_THREADS}.log
 
−
  mv TEST $test_dir/TEST.$i
 
−
done
 
−
 
 
−
 
 
−
 
 
−
==Datura==
 
−
 
 
−
Copy datura.sh as datura-testsuite.sh in simfactory/scriptfiles.  Replace the mpirun command with the following:
 
−
 
 
−
 
 
−
cd @SOURCEDIR@
 
−
 
 
−
test_dir=TEST_$(date +%Y-%m-%d-%H%M%S)
 
−
mkdir $test_dir
 
−
 
 
−
CONFIGNAME=$(ls configs|tail) # SimFactory should provide the configuration name
 
−
 
 
−
export CACTUS_STARTTIME=$(date +%s)
 
−
 
 
−
for i in 1 2 3; do
 
−
  case $i in
 
−
    1 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=1
 
−
      export OMP_NUM_THREADS=1;;
 
−
    2 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=1;;
 
−
    3 )
 
−
      export CCTK_TESTSUITE_RUN_PROCESSORS=2
 
−
      export OMP_NUM_THREADS=2;;
 
−
  esac
 
−
  export CCTK_TESTSUITE_RUN_COMMAND="${MPIDIR}/bin/mpirun -v --mca btl openib,self --mca mpi_leave_pinned 0 -np \$nprocs -npernode $CCTK_TESTSUITE_RUN_PROCESSORS \$exe -L 3 \$parfile"
 
−
  make $CONFIGNAME-testsuite PROMPT=no
 
−
  cp TEST/$CONFIGNAME/summary.log datura__${CCTK_TESTSUITE_RUN_PROCESSORS}_${OMP_NUM_THREADS}.log
 
−
  mv TEST $test_dir/TEST.$i
 
−
done
 

Latest revision as of 18:29, 3 November 2022

This page contains notes and instructions for people running the ET testsuites on various different machines. If you have experience running testsuites on a machine which is not listed here, please consider adding some information which might help others (or yourself!) in future.

General

To check out the ET, follow the instructions at http://einsteintoolkit.org/download/ under "Current development version".

Now configure simfactory: 

 cd Cactus
 cp simfactory/etc/defs.local.ini.simple simfactory/etc/defs.local.ini
 nano simfactory/etc/defs.local.ini

Edit defs.local.ini and replace 

 YOUR_LOGIN with your username
 YOUR@EMAIL.ADDRESS with your usual email address
 YOUR_ALLOCATION with your project allocation
 YOUR_THORNLIST with manifest/einsteintoolkit.th

See the machine-specific notes below for any additional steps for each machine. 

 sim sync <machine>
 sim login <machine>
 sim build
 sim create-submit ettests_1proc --testsuite --procs <N> --num-threads <N> --walltime 3:00:00
 sim create-submit ettests_2proc --testsuite --procs <N> --num-threads <N/2> --walltime 3:00:00

Replace <N> and <N/2> with the number of cores on each node, and half of this, respectively, for the machine you are using. Remember that "procs" here means "cores" and "num-threads" means "number of threads per process". The idea is to use a full node, i.e. all the cores, and then either one or two MPI processes.

When the jobs have finished, you should have the summary.log files in 

 <simulations>/ettests_1proc/output-0000/TEST/sim/summary.log
 <simulations>/ettests_2proc/output-0000/TEST/sim/summary.log

Update the testsuite status page by adding the log files to the release-info repository: 

 git clone git@bitbucket.org:einsteintoolkit/testsuite_results
 cd release-info
 scp machine:<simulations>/ettests_1proc/output-0000/TEST/sim/summary.log <machine>__1_<N>.log
 scp machine:<simulations>/ettests_2proc/output-0000/TEST/sim/summary.log <machine>__2_<N/2>.log
 git add <machine>*.log
 git commit
 git push

To re-run the tests with an updated checkout, run the GetComponents command above with the --update flag, rebuild, delete the "ettests_*" simulations, and resubmit the simulations.

Retrieved from "https://docs.einsteintoolkit.org/et-docs/index.php?title=Testsuite_Machines&oldid=7957"