Difference between revisions of "PreSync"

From Einstein Toolkit Documentation
Jump to: navigation, search
(Created page with "The purpose of this page is to explain how to properly implement the features provided by the PreSync update to a thorn, as well as how to do this while maintaining backward c...")
 
Line 9: Line 9:
 
== New Thorns ==
 
== New Thorns ==
  
Unsurprisingly, writing a new thorn which will only work with PreSync active is simpler than providing backward compatibility. For most thorns, this is a relatively simple matter. Tackling these item-by-item,
+
Unsurprisingly, writing a new thorn which will only work with PreSync active is simpler than providing backward compatibility. For most thorns, this is a relatively simple matter. There are several places at which new thorns will diverge from old thorns.
# New Macros tet 
+
=== Schedule.ccl ===
etet
+
==== Read/Write Declarations ====
# adsfdsaf
+
The schedule.ccl file will have several changes. First, each scheduled function must declare what variables it reads and writes. These declarations take the general form of
 +
 
 +
schedule func at bin
 +
{
 +
  LANG: C
 +
  READS: thorn::variable/group(region)
 +
  WRITES: thorn::variable/group(region)
 +
}
 +
Every variable (grid functions, scalars, arrays, etc.) which the function func accesses must be included in this list. The variable/group usually gives the variable that is accessed. '''If''' the entire group is accessed, then the group name can be given instead. The region can be either ''interior'', ''boundary'', or ''everywhere''. If a variable is read ''everywhere'', this means that the grid function must have its ghost zones synchronized and boundary conditions applied. If this has not yet happened by the time func runs, then PreSync will automatically schedule these operations. Similarly, writing to the ''interior'' means that the ghost zones are no longer valid and must be synchronized the next time they are needed. These declarations can be simplified in several ways. First, the region is optional. If no region is given, then READs assume ''everywhere'' and WRITEs assume ''interior''. Second, the "thorn::" part can be dropped for variables from the thorn that the schedule.ccl is from. For example, the ML_BSSN thorn could schedule something of the form
 +
schedule func at bin
 +
{
 +
  LANG: C
 +
  READS: TmunuBase::stress_energy_tensor
 +
  WRITES: Xt1
 +
}
 +
where Xt1 is implicitly ML_BSSN::Xt1 and stress_energy_tensor READ access to every variable in the group stress_energy_tensor from the thorn TmunuBase.
 +
 
 +
Many functions have many variables accessed in the same way. In addition to using group names instead of variable names, several other features can simplify these expressions. First, a single line can give several variables/groups with the same access type. Again pretending we are scheduling func in ML_BSSN:
 +
schedule func at bin
 +
{
 +
  LANG: C
 +
  READS: TmunuBase::stress_energy_scalar(everywhere), TmunuBase::stress_energy_vector, TmunuBase::stress_energy_tensor
 +
  WRITES: Xt1(interior), Xt2, Xt3, ML_mom
 +
}
 +
Note that we have mixed group and variable names in the WRITEs declarations. This is perfectly fine. The one case in which group and variable names cannot be used in combination is if some variables in a group are read and others are written. Even if all of them are read/written, the group name cannot currently be used if the other declaration type only has some of the variables. They must be explicitly given as variable names in this case.
 +
 
 +
The second feature is for previous timelevels. In Cactus, the previous timelevels are accessed with an "_p" appended to the end of the variable name. Functions which access previous timelevels of variables must also declare these as separate variables. In the case that one needs access to the previous timelevels for an entire group of variables, the "_p" can instead be appended to the group name in the scheduling:
 +
schedule func at bin
 +
{
 +
  LANG: C
 +
  READS: var1(everywhere), var1_p, var1_p_p
 +
  WRITES: group2(interior), group2_p, group2_p_p
 +
}
 +
 
 +
 
 +
'''Important note:''' previously, there has been no strict enforcement of the public/protected/private status of variables in Cactus. However, the new declarations respect these designations. Therefore, any thorn which needs to access variables from other thorns needs to properly declare ''inherit'' or ''friend'' status of other thorns in the interface.ccl.
 +
 
 +
 
 +
# New Macros <br/> The old flesh provided the macro DECLARE_CCTK_ARGUMENTS for getting access to the variables declared in the interface.ccl of the thorn. PreSync provides
 +
# dfd

Revision as of 23:33, 27 September 2022

The purpose of this page is to explain how to properly implement the features provided by the PreSync update to a thorn, as well as how to do this while maintaining backward compatibility.

Overview

The PreSync update (publication) introduces the concept of data-dependent scheduling to Cactus and Carpet. While not fully data-dependent, this update specifically automates the scheduling of ghost zone synchronization and the application of boundary conditions. The upcoming driver thorn CarpetX is designed from the ground up with PreSync in mind, and thorns will be required to be PreSync-compatible to use CarpetX. Even in Carpet, automating the scheduling of inter-processor communication makes the writing of thorns easier; thorns using PreSync do not need to use the SYNC statements or manually schedule ApplyBCs so long as they are running with PreSync active. Of course, for backward compatibility, thorns will still need to provide scheduling information for these to still properly function with PreSync disabled.

In the following sections the way to implement PreSync is outlined, starting with new thorns without backward compatibility. Following this is a section detailing how one implements these new features in an older thorn while still maintaining backward compatibility (or writing a new thorn which can work with both).

New Thorns

Unsurprisingly, writing a new thorn which will only work with PreSync active is simpler than providing backward compatibility. For most thorns, this is a relatively simple matter. There are several places at which new thorns will diverge from old thorns.

Schedule.ccl

Read/Write Declarations

The schedule.ccl file will have several changes. First, each scheduled function must declare what variables it reads and writes. These declarations take the general form of

schedule func at bin
{
  LANG: C
  READS: thorn::variable/group(region)
  WRITES: thorn::variable/group(region)
}

Every variable (grid functions, scalars, arrays, etc.) which the function func accesses must be included in this list. The variable/group usually gives the variable that is accessed. If the entire group is accessed, then the group name can be given instead. The region can be either interior, boundary, or everywhere. If a variable is read everywhere, this means that the grid function must have its ghost zones synchronized and boundary conditions applied. If this has not yet happened by the time func runs, then PreSync will automatically schedule these operations. Similarly, writing to the interior means that the ghost zones are no longer valid and must be synchronized the next time they are needed. These declarations can be simplified in several ways. First, the region is optional. If no region is given, then READs assume everywhere and WRITEs assume interior. Second, the "thorn::" part can be dropped for variables from the thorn that the schedule.ccl is from. For example, the ML_BSSN thorn could schedule something of the form

schedule func at bin
{
  LANG: C
  READS: TmunuBase::stress_energy_tensor
  WRITES: Xt1
}

where Xt1 is implicitly ML_BSSN::Xt1 and stress_energy_tensor READ access to every variable in the group stress_energy_tensor from the thorn TmunuBase.

Many functions have many variables accessed in the same way. In addition to using group names instead of variable names, several other features can simplify these expressions. First, a single line can give several variables/groups with the same access type. Again pretending we are scheduling func in ML_BSSN:

schedule func at bin
{
  LANG: C
  READS: TmunuBase::stress_energy_scalar(everywhere), TmunuBase::stress_energy_vector, TmunuBase::stress_energy_tensor
  WRITES: Xt1(interior), Xt2, Xt3, ML_mom
}

Note that we have mixed group and variable names in the WRITEs declarations. This is perfectly fine. The one case in which group and variable names cannot be used in combination is if some variables in a group are read and others are written. Even if all of them are read/written, the group name cannot currently be used if the other declaration type only has some of the variables. They must be explicitly given as variable names in this case.

The second feature is for previous timelevels. In Cactus, the previous timelevels are accessed with an "_p" appended to the end of the variable name. Functions which access previous timelevels of variables must also declare these as separate variables. In the case that one needs access to the previous timelevels for an entire group of variables, the "_p" can instead be appended to the group name in the scheduling:

schedule func at bin
{
  LANG: C
  READS: var1(everywhere), var1_p, var1_p_p
  WRITES: group2(interior), group2_p, group2_p_p
}


Important note: previously, there has been no strict enforcement of the public/protected/private status of variables in Cactus. However, the new declarations respect these designations. Therefore, any thorn which needs to access variables from other thorns needs to properly declare inherit or friend status of other thorns in the interface.ccl.


  1. New Macros
    The old flesh provided the macro DECLARE_CCTK_ARGUMENTS for getting access to the variables declared in the interface.ccl of the thorn. PreSync provides
  2. dfd