Test Failures

As of 13-Mar-2017, several tests fail on the Jenkins build system, but pass on other systems. All tests here were run with code checked out on or after 13-Mar-2017.

Some details are in ticket 2019.

System	/proc/cpuinfo	/etc/issue	gcc --version
Jenkins build node	Intel Core Processor (Haswell)	Ubuntu 16.04 LTS	gcc (Ubuntu 5.3.1-14ubuntu2) 5.3.1 20160413
login.barrywardell.net	Intel Core Processor (Haswell)	Ubuntu 16.04.2 LTS	gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Archimedes	Intel(R) Xeon(R) CPU E5-2630 v4	Ubuntu 14.04.5 LTS	gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

Test	Jenkins build node	login (ubuntu.cfg)	login (build.cfg)	login (ubuntu.cfg without march=native	login (ubuntu.cfg without march=native, without fast-math	Archimedes (ubuntu.cfg)
CT_MultiLevel.boostedpuncture/2procs	Fail	Fail	Fail		Fail	Fail
SphericalHarmonicReconGen.SpEC-dat-test/1procs	Fail
SphericalHarmonicReconGen.SpEC-h5-test/1procs	Fail
SphericalHarmonicRecon.regression_test/2procs	Fail
SphericalHarmonicReconGen.SpEC-dat-test/2procs	Fail
SphericalHarmonicReconGen.SpEC-h5-test/2procs	Fail
GRHydro.GRHydro_test_shock_weno/1procs	Fail	Fail	Pass	Fail	Pass	Pass
GRHydro.GRHydro_test_shock_weno/2procs	Fail

SphericalHarmonicReconGen

• one failure was due to a bug in NaNChecker introduced in 9fc4866
• the other uses HDF5 incorrectly which however does not seem to be the root cause of the failure

WENO Test

• Roland tried multiple different resolutions (up to 8 times more and 16 times the time resolution, as well as 2 times bigger and offset from 0 grids) none of which prevented NaNs to from on my laptop
• next thing to try is to find out which is the exact optimization options that causes the NaN
• the option responsible for test failure is -fassociative-math