ET Workshop 2015/new developments - Revision history

Hinder at 03:08, 17 August 2015

2015-08-17T03:08:03Z

Rhaas at 09:38, 13 August 2015

2015-08-13T09:38:45Z

Rhaas at 09:20, 13 August 2015

2015-08-13T09:20:36Z

Rhaas: /* Adaptive mesh refinement and Carpet */

2015-08-13T09:20:08Z

‎Adaptive mesh refinement and Carpet

Rhaas: /* Adaptive mesh refinement and Carpet */

2015-08-13T09:18:22Z

‎Adaptive mesh refinement and Carpet

Rhaas at 08:58, 13 August 2015

2015-08-13T08:58:13Z

Rhaas at 07:12, 13 August 2015

2015-08-13T07:12:14Z

Rhaas at 07:05, 13 August 2015

2015-08-13T07:05:02Z

Hinder at 06:56, 13 August 2015

2015-08-13T06:56:39Z

Hinder at 06:56, 13 August 2015

2015-08-13T06:56:08Z

@@ Line 40: / Line 40: @@
 #* PRACE ony funds big ones, smaller ones through national agencies. (CINECA offers class C allocations for this)
 # Usability
 ## documentation wanted, not just code but also on how to do things
 ## larger set of gallery examples
 ## suggestion to also have a correctness checking framework
-## non-working examples are included in the toolkit. Example parfiles should be commented to make them easier to understand.
+## non-working examples are included in the toolkit. Example parfiles should be commented to make them easier to understand.  (see [https://trac.einsteintoolkit.org/ticket/641 #641])
 == Adaptive mesh refinement and Carpet ==

@@ Line 18: / Line 18: @@
 ## HPC correctness test
 ## Updating private codes to agree with ET developments
-# Community policies
+# Community practises
 ## backwards compatibility. Strict compatibility hurts usefulness.
 ## Cactus may have been to conservative mainaining compatibility
@@ Line 64: / Line 64: @@
 Using a third party AMR infrastructure runs into issues with different data structures used between different infrastructures. Would need to hook up both the AMR and the Cactus specific infrastructure: memory management, scheduler, reduction operations, interpolators. Specific question: replacing CarpetLib by BoxLib. This would be easier since CarpetLib is designed in a similar way than BoxLib. It is not something that can be done in an afternoon though. Erik says that strategically this would be good and was tried a couple times, eg with Enzo. Not clear in advance if CarpetLib is currently performant enough for fine grained AMR. We may expect that it is not, and that the replacement is actually better. So the replacement (BoxLib) would need to be tested with something of similar complexity of the BSSN equation and fourth order stencils. [https://github.com/BoxLib-Codes BoxLib] has examples for the wave equation on its website.

@@ Line 18: / Line 18: @@
 ## HPC correctness test
 ## Updating private codes to agree with ET developments
-# Community practises
+# Community policies
 ## backwards compatibility. Strict compatibility hurts usefulness.
 ## Cactus may have been to conservative mainaining compatibility

←Older revision		Revision as of 09:20, 13 August 2015
Line 63:		Line 63:
	For fine grained AMR one can start with Carpet's ARMToy example. Gradient based indicators can be troublesome, a better method may be self-shadowing which was tried "in the distant" past in Carpet but not yet implemented. The method relies on computing an error estimate between coarse and fine levels are computed just before restriction into the coarse grid happens. Ian Hawke remembers that it was less trivial than expected. If this is used to compute an error estimate on the fine grid then one needs some calls to C++ to transfer the difference from the restricted region into the fine grid.		For fine grained AMR one can start with Carpet's ARMToy example. Gradient based indicators can be troublesome, a better method may be self-shadowing which was tried "in the distant" past in Carpet but not yet implemented. The method relies on computing an error estimate between coarse and fine levels are computed just before restriction into the coarse grid happens. Ian Hawke remembers that it was less trivial than expected. If this is used to compute an error estimate on the fine grid then one needs some calls to C++ to transfer the difference from the restricted region into the fine grid.

−	Using a third party AMR infrastructure runs into issues with different data structures used between different infrastructures. Would need to hook up both the AMR and the Cactus specific infrastructure: memory management, scheduler, reduction operations, interpolators. Specific question: replacing CarpetLib by BoxLib. This would be easier since CarpetLib is designed in a similar way than BoxLib. It is not something that can be done in an afternoon though. Erik says that strategically this would be good and was tried a couple times, eg with Enzo. Not clear in advance if CarpetLib is currently performant enough for fine grained AMR. We may expect that it is not, and that the replacement is actually better. So the replacement (BoxLib) would need to be tested with something of similar complexity of the BSSN equation and fourth order stencils. [~~BoxLib\|~~https://github.com/BoxLib-Codes] has examples for the wave equation on its website.	+	Using a third party AMR infrastructure runs into issues with different data structures used between different infrastructures. Would need to hook up both the AMR and the Cactus specific infrastructure: memory management, scheduler, reduction operations, interpolators. Specific question: replacing CarpetLib by BoxLib. This would be easier since CarpetLib is designed in a similar way than BoxLib. It is not something that can be done in an afternoon though. Erik says that strategically this would be good and was tried a couple times, eg with Enzo. Not clear in advance if CarpetLib is currently performant enough for fine grained AMR. We may expect that it is not, and that the replacement is actually better. So the replacement (BoxLib) would need to be tested with something of similar complexity of the BSSN equation and fourth order stencils. [https://github.com/BoxLib-Codes BoxLib] has examples for the wave equation on its website.

←Older revision		Revision as of 09:18, 13 August 2015
Line 61:		Line 61:
	* this however is limited in how much one can safe		* this however is limited in how much one can safe

−	For fine grained AMR one can start with Carpet's ARMToy example. Gradient based indicators can be troublesome, a better method may be self-shadowing which was tried "in the distant" past in Carpet but not yet implemented. The method relies on computing an error estimate between coarse and fine levels are computed just before restriction into the coarse grid happens. Ian Hawke remembers that it was less trivial than expected.	+	For fine grained AMR one can start with Carpet's ARMToy example. Gradient based indicators can be troublesome, a better method may be self-shadowing which was tried "in the distant" past in Carpet but not yet implemented. The method relies on computing an error estimate between coarse and fine levels are computed just before restriction into the coarse grid happens. Ian Hawke remembers that it was less trivial than expected. If this is used to compute an error estimate on the fine grid then one needs some calls to C++ to transfer the difference from the restricted region into the fine grid.
		+
		+	Using a third party AMR infrastructure runs into issues with different data structures used between different infrastructures. Would need to hook up both the AMR and the Cactus specific infrastructure: memory management, scheduler, reduction operations, interpolators. Specific question: replacing CarpetLib by BoxLib. This would be easier since CarpetLib is designed in a similar way than BoxLib. It is not something that can be done in an afternoon though. Erik says that strategically this would be good and was tried a couple times, eg with Enzo. Not clear in advance if CarpetLib is currently performant enough for fine grained AMR. We may expect that it is not, and that the replacement is actually better. So the replacement (BoxLib) would need to be tested with something of similar complexity of the BSSN equation and fourth order stencils. [BoxLib\|https://github.com/BoxLib-Codes] has examples for the wave equation on its website.

←Older revision		Revision as of 08:58, 13 August 2015
Line 47:		Line 47:
	## non-working examples are included in the toolkit. Example parfiles should be commented to make them easier to understand.		## non-working examples are included in the toolkit. Example parfiles should be commented to make them easier to understand.
	## higher level "Einstein Toolkit" user guide.		## higher level "Einstein Toolkit" user guide.
		+
		+	== Adaptive mesh refinement and Carpet ==
		+	Currently two knowledgeable maintainers: Erik Schnetter and Roland Haas
		+
		+	Concerning the question about scaling and fine grained AMR Carpet already supports this though it may well be less efficient than other codes. Continuous Runge-Kutta method for buffer zone data may work better than Adams-Bashforth. Estimate for Adams-Bashforth is not fully conclusive (pen and paper) making it within 50% of the speed of RK4. Current status is that it is feasible but not tested yet in a big simulation (it is in the code and only requires some changes to enable more time levels).
		+
		+	Ian and Wolfgang state that running without buffer zones is impossible for accuracy. Continuous RK4 seems a cleaner solution. Advantage is that one needs to store fewer time levels. In principle we would try both AB and cont-RK. The AB work stalled since there is now time for a real-world test and a physics problem.
		+
		+	For more fine grained AMR Wolfgang suggests to have a test case with refinement level boundaries inside of the NS. Ian Hawke reports that a refinement level boundary worked fine if the whole box was contained in the star, the worst behaviour was if the buffer zones intersected the star's surface.
		+
		+	Methods to reduce the buffer zone width right now:
		+	* change numerical scheme in buffer zones this requires the user code to know which points are buffer zones
		+	* this however is limited in how much one can safe
		+
		+	For fine grained AMR one can start with Carpet's ARMToy example. Gradient based indicators can be troublesome, a better method may be self-shadowing which was tried "in the distant" past in Carpet but not yet implemented. The method relies on computing an error estimate between coarse and fine levels are computed just before restriction into the coarse grid happens. Ian Hawke remembers that it was less trivial than expected.

@@ Line 9: / Line 9: @@
 # Performance in optimization and usability
 ## AMR, scaling, adaptiveness
-## * reduce focus on home grown solution for GR only
+##* reduce focus on home grown solution for GR only
-## * discuss benefits of Chombo and GRChombo. Ian Hawke mentions bad experience with these frameworks in relativity.
+##* discuss benefits of Chombo and GRChombo. Ian Hawke mentions bad experience with these frameworks in relativity.
 ## Usability
 ##* more examples, better documentation (hypothetical "science with ET", Carpet)

@@ Line 6: / Line 6: @@
 ##* Ian Hawke mentions the possibility of other workflow management systems that exist and have a wide user base.
 ##* desire to include management of simulation data
 # Performance in optimization and usability
 ## AMR, scaling, adaptiveness

@@ Line 3: / Line 3: @@
 ## visualization
 ## simulation management, simfactory
-## * Ian Hinder describes situation of simfactory2 and work on simfactory3 (Hinder, Wardell, Schnetter).
+##* Ian Hinder describes situation of simfactory2 and work on simfactory3 (Hinder, Wardell, Schnetter).
-## *Ian Hawke mentions the possibility of other workflow management systems that exist and have a wide user base.
+##* Ian Hawke mentions the possibility of other workflow management systems that exist and have a wide user base.
-## * desire to include management of simulation data
+##* desire to include management of simulation data
 # Performance in optimization and usability
 ## AMR, scaling, adaptiveness

@@ Line 39: / Line 39: @@
 #* PRACE ony funds big ones, smaller ones through national agencies. (CINECA offers class C allocations for this)
 # Usability
-##* documentation wanted, not just code but also on how to do things
+## documentation wanted, not just code but also on how to do things
-##* larger set of gallery examples
+## larger set of gallery examples
-##* lack of complete documentation. Some part are well documented (Cactus flesh) but newer features are mostly undocumented, for example the tags
+## lack of complete documentation. Some part are well documented (Cactus flesh) but newer features are mostly undocumented, for example the tags
-##* want some high level documentation
+## want some high level documentation
-##* suggestion to also have a correctness checking framework
+## suggestion to also have a correctness checking framework
-##* non-working examples are included in the toolkit. Example parfiles should be commented to make them easier to understand.
+## non-working examples are included in the toolkit. Example parfiles should be commented to make them easier to understand.
-##* higher level "Einstein Toolkit" user guide.
+## higher level "Einstein Toolkit" user guide.