Improved scheduling

Some months ago, we discussed possible ways to make scheduling in Cactus more straightforward for thorn writers. Please see Adding requirements to the Cactus scheduler for the work that was done before. We then had a telecon on 24-Oct-2011 in which we did some brainstorming to introduce new people to the ideas.

Current status:

• There is a branch of the Cactus flesh which supports REQUIRES and PROVIDES directives in schedule.ccl files.

• There is a cut-down WaveToy thorn with very simple schedule requirements.

Possible plan - including long-term vision. Please discuss!

• Change REQUIRES and PROVIDES to READS and WRITES - we seem to have consensus that this makes more sense

• Set up a sequence of functions in CCTK_INITIAL with dependency information between variables. Make Carpet decide the scheduling order within CCTK_INITIAL based on these.

• Make a unigrid simulation work.

• Make Carpet handle regridding/restricting/syncing/prolongating of all variables by calling scheduled functions which say they can write these variables.

• Make boundary conditions work with dependency-based scheduling

• Add READS and WRITES information to all scheduled functions in the Einstein Toolkit

• Deprecate unneeded schedule groups and "before" and "after" and transition entirely to the new dependency-based scheduling (one year from now?)

Issues/thoughts:

• Think about how to deal with MoL. Should we first try to get evolution working without MoL?

• What shall we do with PUGH?

• Do thorns need to declare dependencies on all Cactus variables that they access? e.g. grid scalars, grid arrays, spherical surfaces etc? I think yes. What about flesh variables: cctk_nghostzones etc? I think no.

• Is it too much effort for a thorn writer to declare all the dependencies in the schedule.ccl file?

• We cannot derive the dependencies automatically from the source file. For strict checking, we could make variables which we have not asked for either null pointers or not available at the compiler level - I prefer the latter but the former would be easier to implement first.

• Eventually we can check writes as well as reads by doing checksumming of data after a function is called. This is mostly important for checking that different regions of the grid have been written according to the scheduling.

• Idea: can we eventually eliminate all bins and groups and "before/after" directives, and drive the scheduling entirely based on data dependency? Would need to have "contexts" for variables. e.g. saying that a function writes gxx_initial, and MoL then reads gxx_initial and writes gxx.