https://docs.einsteintoolkit.org/et-docs/index.php?action=history&feed=atom&title=A_new_I%2FO_file_format 2026-05-13T13:03:04Z Revision history for this page on the wiki MediaWiki 1.31.0 https://docs.einsteintoolkit.org/et-docs/index.php?title=A_new_I/O_file_format&diff=4061&oldid=prev 2015-10-07T02:45:14Z

<p>stress that speed of access must not be limited to single file</p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">←Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 02:45, 7 October 2015</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l16" >Line 16:</td> <td colspan="2" class="diff-lineno">Line 16:</td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* needs to be based on HDF5, or an equivalent portrable, widely-supported file format</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* needs to be based on HDF5, or an equivalent portrable, widely-supported file format</div></td></tr> <tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>* needs to support efficient parallel I/O (including quick discovery of <del class="diffchange diffchange-inline">file </del>contents, quick reading of individual datasets)</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>* needs to support efficient parallel I/O (including quick discovery of <ins class="diffchange diffchange-inline">fileset </ins>contents, quick reading of individual datasets <ins class="diffchange diffchange-inline">from full output of simulation</ins>)</div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* needs to be portable</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* needs to be portable</div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* should be useful for similar projects, e.g. Enzo</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>* should be useful for similar projects, e.g. Enzo</div></td></tr> </table>

Rhaas https://docs.einsteintoolkit.org/et-docs/index.php?title=A_new_I/O_file_format&diff=4060&oldid=prev 2015-10-07T02:38:25Z

<p>add link to mailing list</p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">←Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 02:38, 7 October 2015</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Thoughts on a rather generic AMR I/O file format and library ==</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Thoughts on a rather generic AMR I/O file format and library ==</div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(That would be useful for the Einstein Toolkit and related projects)</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(That would be useful for the Einstein Toolkit and related projects)</div></td></tr> <tr><td class='diff-marker'>−</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div>This is related to the motivation for [https://trac.einsteintoolkit.org/ticket/1033 ticket 1033]</div></td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div>This is related to the motivation for [https://trac.einsteintoolkit.org/ticket/1033 ticket 1033] <ins class="diffchange diffchange-inline">and the discussion on the [http://lists.einsteintoolkit.org/pipermail/users/2015-August/004427.html mailing list].</ins></div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== General Thoughts ==</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== General Thoughts ==</div></td></tr> </table>

Rhaas https://docs.einsteintoolkit.org/et-docs/index.php?title=A_new_I/O_file_format&diff=4059&oldid=prev 2015-10-07T02:31:28Z

<p>add link to ticket 1033</p> <table class="diff diff-contentalign-left" data-mw="interface"> <col class="diff-marker" /> <col class="diff-content" /> <col class="diff-marker" /> <col class="diff-content" /> <tr class="diff-title" lang="en"> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">←Older revision</td> <td colspan="2" style="background-color: #fff; color: #222; text-align: center;">Revision as of 02:31, 7 October 2015</td> </tr><tr><td colspan="2" class="diff-lineno" id="mw-diff-left-l1" >Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Thoughts on a rather generic AMR I/O file format and library ==</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== Thoughts on a rather generic AMR I/O file format and library ==</div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(That would be useful for the Einstein Toolkit and related projects)</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>(That would be useful for the Einstein Toolkit and related projects)</div></td></tr> <tr><td colspan="2">&#160;</td><td class='diff-marker'>+</td><td style="color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #a3d3ff; vertical-align: top; white-space: pre-wrap;"><div><ins style="font-weight: bold; text-decoration: none;">This is related to the motivation for [https://trac.einsteintoolkit.org/ticket/1033 ticket 1033]</ins></div></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== General Thoughts ==</div></td><td class='diff-marker'>&#160;</td><td style="background-color: #f8f9fa; color: #222; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #eaecf0; vertical-align: top; white-space: pre-wrap;"><div>== General Thoughts ==</div></td></tr> </table>

194.94.224.254 https://docs.einsteintoolkit.org/et-docs/index.php?title=A_new_I/O_file_format&diff=4058&oldid=prev 2015-10-06T13:59:09Z

<p>‎<span dir="auto"><span class="autocomment">Ideas</span></span></p> <a href="https://docs.einsteintoolkit.org/et-docs/index.php?title=A_new_I/O_file_format&amp;diff=4058&amp;oldid=4057">Show changes</a>

Eschnett https://docs.einsteintoolkit.org/et-docs/index.php?title=A_new_I/O_file_format&diff=4057&oldid=prev 2015-10-06T13:57:34Z

<p></p> <p><b>New page</b></p><div>== Thoughts on a rather generic AMR I/O file format and library ==<br /> (That would be useful for the Einstein Toolkit and related projects)<br /> <br /> == General Thoughts ==<br /> <br /> === Uses ===<br /> <br /> * generic file I/O, i.e. exporting/importing data<br /> * checkpointing, recovery<br /> * visualization<br /> * post-processing<br /> * long-term archival<br /> <br /> === Necessary Properties ===<br /> <br /> * needs to be based on HDF5, or an equivalent portrable, widely-supported file format<br /> * needs to support efficient parallel I/O (including quick discovery of file contents, quick reading of individual datasets)<br /> * needs to be portable<br /> * should be useful for similar projects, e.g. Enzo<br /> * must support unigrid, AMR, multi-block, DGFE, staggered grids<br /> * must be usable from C, C++, Fortran, Python, Mathematica<br /> * should be implemented in a portable stand-alone library that can easily be used in various projects<br /> * must remain accessible without such a library<br /> <br /> == Ideas ==<br /> <br /> We introduce various concepts that abstract various properties. Data are described in terms of these concepts instead of using ad-hoc descriptions for datasets.<br /> <br /> It seems useful that each concept has abstract properties and several concrete realizations.<br /> <br /> Concepts:<br /> * project (consisting of simulations)<br /> * manifold (consisting of discretizations)<br /> * coordinate systems (various types)<br /> * tangent spaces (what is the connection to a coordinate system?)<br /> * tensors (described via bases) (should bases be related to tangent spaces?)<br /> * generic variable types (such as &quot;floating&quot;), implemented via concrete types (such as float32, float64)<br /> * fields, as defined by the user<br /> <br /> Names should be illustrative only and don&#039;t matter; only the connection between these concepts are relevant.</div>

Eschnett