Improving the new user experience

Summary

We would like to improve the experience of new users of the Einstein Toolkit. Some of us will work on this at the North American Einstein Toolkit workshop in August 2017.

If you have suggestions for improvements, or things which don't work well right now, please add them below.

We probably won't have time during the 2-day workshop to address all that is listed here, but we will make an effort to make progress on one or two items.

Brainstorming

(Roughly in order of how likely we are to work on it at the workshop)

Obstacles faced by new users

• Configuring the ET on a new machine is very difficult (even just compiling, let along interactions with queuing systems etc)
  • It's particularly frustrating to me (Ian) that simfactory cannot figure out even simple things about a new machine (e.g. ppn), and if you don't get this right you aren't allowed to run multi-threaded jobs, etc.
    • We have a patch from Mikael Sahrling which detects the number of cores. Let's commit this during the workshop.
    • SimFactory should be able to figure out everything that it asks for in sim setup (user name, which optionlist to use for a machine, source base dir (https://trac.einsteintoolkit.org/ticket/2056)).
    • Steve is working on automatically detecting the OS to choose the optionlist, and also something to do with which packages need to be installed (https://bitbucket.org/simfactory/simfactory2/branch/os_detect).
• The ET takes a long time to compile
  • Maybe have multiple thornlists
  • Maybe add tags to thorns, so that you can build just one type of thorn (e.g. those for BBH, or for NS, or something like that)
  • Parallel build by default
  • Linking stage very slow - Roland has some ideas
  • Formaline seems to always be a suspect
  • Don't build MPI without warning the user. Maybe abort and give the command or instructions to build it.
• Easy-ish on a new machine, but hard on a machine which already has a lot of packages installed, as they may be conflicting
• The ET has a lot of dependencies, or compiles a lot of libraries (which sometimes don't compile successfully)
  • Is the self-built version of OpenMPI ever actually usable?
  • Erik has talked about using Spack for libraries. Maybe we should push for this. Erik now prefers "nix".
• Many of the examples do not work
  • Test automatically
  • Add comments to indicate which ones will not work with the toolkit
  • List examples on the ET website - those that work.
  • At a previous workshop, this was evaluated, but there was no resolution (Fixing examples)
• There are too many tutorials
  • The following are listed on the wiki:
    • Tutorial for New Users
    • Simplified Tutorial for New Users
    • Getting Started for Cactus Experts
    • Compiling the Einstein Toolkit
  • The names of the tutorials do not allow users to distinguish what they are for. For example, the Tutorial for New Users and Simplified Tutorial for New Users differ in that the former is run on Queen Bee, and the latter is run on a user's own laptop or workstation.
  • Perhaps all the above should be consolidated
• On slower laptops, the build stage regularly hangs and has to be killed and restarted (which almost always solves the problem). Can we figure out which components are responsible, and omit them from quick-start tutorials?
  • RH: if the slower laptop also has less memory (or is a VM) then I would first try and monitor how much memory in particular the linker consumes. Testing this on my workstation it uses 1-2GB of RAM for a full ET build. On a 32bit VM with a cut down thornlist (no Formaline) it uses ~700MB. Similarly some C++ code takes a hug e amount of memory to compile.
• Comments: Some of the documentation (tex) files that come with thorns are blank. If I need more information about how to use a thorn with missing documentation, I usually look in the ccl files for clues. It would be nice to have any additional information from the ccl files in the tex files as well. (Lump all the details about a thorn in the same place.)
• The mailing list and one or two calls I've joined so far were a huge help. But sometimes asking "beginner questions" over public channels makes me feel as if I'm wasting people's time. Up-to-date documentation with lots of details about parameters would be amazing! (Also, of course, time-consuming to write!)
• I don't think this is very important, but a resources page listing some good textbooks or papers for getting started in numerical relativity might be nice for people who come in without prior knowledge.
• Tutorials don't have sophisticated examples
• Important for examples to be able to run on laptops
• Jupyter notebooks as an alternative (replacement?) to tutorials
• Automatically-tested tutorials?