Difference between revisions of "Test Failures"
(update GRHydro_test_shock_weno status) |
|||
| (6 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
As of 13-Mar-2017, several tests fail on the Jenkins build system, but pass on other systems. All tests here were run with code checked out on or after 13-Mar-2017. | As of 13-Mar-2017, several tests fail on the Jenkins build system, but pass on other systems. All tests here were run with code checked out on or after 13-Mar-2017. | ||
| + | |||
| + | Some details are in ticket [https://trac.einsteintoolkit.org/ticket/2019 2019]. | ||
{| {{PrettyTable}} | {| {{PrettyTable}} | ||
| Line 14: | Line 16: | ||
! Test !! Jenkins<br>build node || login (ubuntu.cfg) || login (build.cfg) || login (ubuntu.cfg<br> without march=native || login (ubuntu.cfg<br> without march=native, <br>without fast-math || Archimedes (ubuntu.cfg) | ! Test !! Jenkins<br>build node || login (ubuntu.cfg) || login (build.cfg) || login (ubuntu.cfg<br> without march=native || login (ubuntu.cfg<br> without march=native, <br>without fast-math || Archimedes (ubuntu.cfg) | ||
|- | |- | ||
| − | | CT_MultiLevel.boostedpuncture/2procs || Fail || Fail || Fail || || Fail | + | | CT_MultiLevel.boostedpuncture/2procs || Fail || Fail || Fail || || Fail || Fail |
|- | |- | ||
| SphericalHarmonicReconGen.SpEC-dat-test/1procs || Fail | | SphericalHarmonicReconGen.SpEC-dat-test/1procs || Fail | ||
| Line 30: | Line 32: | ||
| GRHydro.GRHydro_test_shock_weno/2procs || Fail | | GRHydro.GRHydro_test_shock_weno/2procs || Fail | ||
|} | |} | ||
| + | |||
| + | |||
| + | == SphericalHarmonicReconGen == | ||
| + | |||
| + | * one failure was due to a bug in NaNChecker introduced in 9fc4866 | ||
| + | * the other uses HDF5 incorrectly <strike>which however does not seem to be the root cause of the failure</strike> | ||
| + | |||
| + | == WENO Test == | ||
| + | * Roland tried multiple different resolutions (up to 8 times more and 16 times the time resolution, as well as 2 times bigger and offset from 0 grids) none of which prevented NaNs to from on my laptop | ||
| + | * <strike>next thing to try is to find out which is the exact optimization options that causes the NaN</strike> | ||
| + | * the option responsible for test failure is <code>-fassociative-math</code> | ||
| + | * the affected file is GRHydro_WENOReconstruct.cc and in particular the rescaling of beta[123] when adaptive_epsilon is used with very small (which is the default) weno_eps | ||
Latest revision as of 04:01, 29 June 2017
As of 13-Mar-2017, several tests fail on the Jenkins build system, but pass on other systems. All tests here were run with code checked out on or after 13-Mar-2017.
Some details are in ticket 2019.
| System | /proc/cpuinfo | /etc/issue | gcc --version |
|---|---|---|---|
| Jenkins build node | Intel Core Processor (Haswell) | Ubuntu 16.04 LTS | gcc (Ubuntu 5.3.1-14ubuntu2) 5.3.1 20160413 |
| login.barrywardell.net | Intel Core Processor (Haswell) | Ubuntu 16.04.2 LTS | gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609 |
| Archimedes | Intel(R) Xeon(R) CPU E5-2630 v4 | Ubuntu 14.04.5 LTS | gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4 |
| Test | Jenkins build node |
login (ubuntu.cfg) | login (build.cfg) | login (ubuntu.cfg without march=native |
login (ubuntu.cfg without march=native, without fast-math |
Archimedes (ubuntu.cfg) |
|---|---|---|---|---|---|---|
| CT_MultiLevel.boostedpuncture/2procs | Fail | Fail | Fail | Fail | Fail | |
| SphericalHarmonicReconGen.SpEC-dat-test/1procs | Fail | |||||
| SphericalHarmonicReconGen.SpEC-h5-test/1procs | Fail | |||||
| SphericalHarmonicRecon.regression_test/2procs | Fail | |||||
| SphericalHarmonicReconGen.SpEC-dat-test/2procs | Fail | |||||
| SphericalHarmonicReconGen.SpEC-h5-test/2procs | Fail | |||||
| GRHydro.GRHydro_test_shock_weno/1procs | Fail | Fail | Pass | Fail | Pass | Pass |
| GRHydro.GRHydro_test_shock_weno/2procs | Fail |
SphericalHarmonicReconGen
- one failure was due to a bug in NaNChecker introduced in 9fc4866
- the other uses HDF5 incorrectly
which however does not seem to be the root cause of the failure
WENO Test
- Roland tried multiple different resolutions (up to 8 times more and 16 times the time resolution, as well as 2 times bigger and offset from 0 grids) none of which prevented NaNs to from on my laptop
next thing to try is to find out which is the exact optimization options that causes the NaN- the option responsible for test failure is
-fassociative-math - the affected file is GRHydro_WENOReconstruct.cc and in particular the rescaling of beta[123] when adaptive_epsilon is used with very small (which is the default) weno_eps
