ET Workshop Summer 2013
- 1 New User's workshop
- 2 Developer's workshop
- 2.1 ET Workshop Summer 2013 (Developers Workshop
- 2.2 In-Workshop Details
- 2.3 Notes
New User's workshop
Summer Einstein Toolkit Workshop 2013
The users workshop is part of the CGWAS and will be held on the last day of the summer school (Friday). Currently the programme lists introduction to numerical relativity, an introduction to the ET and two hands-on sessions. The big hands-on session from last year was too technical and should be reworked if used at all. Instead we could consider running a very short head-on nsns run and analysing the data. The runs takes ~20min at a ridiculously low resolution, but might be sufficient to shows how it's done.
Spacetime Coordinates and Contact Information
The new users workshop is part of CGWAS and is held on the last day of the school (Friday).
Workshop email list: firstname.lastname@example.org, emergencies: Frank Löffler (email@example.com, +1-225-202-1752) or Roland Haas (firstname.lastname@example.org, +1-404-386-3657)
List of Lecturers
Below are the ET volunteers who will be presenting or heading parts of the sessions.
|Roland Haas||Caltech||Introduction to the ET|
|Erik Schnetter||Perimeter Institute||Introduction to numerical relativity|
|Frank Löffler||LSU||Introduction to the ET|
|Joshua Faber||RIT||hands-on session|
|Steve Brandt||LSU||hands-on session|
|Day||Time||Activity||Presenter / Responsible|
|Fri||9:00-10:30||Introduction to Numerical Relativity||Erik Schnetter|
|Fri||11:00-11:40||Introduction to the Einstein Toolkit, Overview||Frank Löffler|
|Fri||11:40-12:30||Introduction to the Einstein Toolkit, Details||Roland Haas|
|Fri||14:00-14:20||Introduction to hands-on environment: simfactory and installing VisIt||Steven Brandt|
|Fri||14:20-14:40||Introduction to hands-on Cactus thorn-writing||Roland Haas|
|Fri||14:40-15:00||Introduction to hands-on visualization within the Einstein Toolkit||Roland Haas|
|Fri||15:30-17:30||Hands-on, center of mass for analysis purposes (simple way)||everyone helping|
|Fri||17:30-18:00||Examples of ET production usage||Christian Reisswig|
- The virtual machine appliance for the workshop is available here and will run in virtualbox.
- A section of zwicky is reserved for the use of workshop participants.
- A public svn repo for any workshop material is available here: https://svn.einsteintoolkit.org/documents/Workshop_Summer_2013
Hands on exercises
|Running and extending (par-file) of two provided examples: perturbed, single BH, and perturbed, single TOV star|
|Hands-on: Using visualization tools to look at meaningful quantities ( TOV profile evolutions ) (slides)|
|Create movie of either BH+GW or TOV oscillations using VisIt (slides, TOV session file, TOV data file, Psi4 session file, Psi4 data file)|
|Hands-on: Implement your own thorn (neutron star tracking), and visualize it, in teams of about 5 people (http://www.tapir.caltech.edu/~pmoesta/handson_analysis.pdf), more information at end of wiki|
ET Workshop Summer 2013 (Developers Workshop
Spacetime Coordinates and Contact Information
|Dates||Sat July 27th - Mon 29th (incl.)|
|Location||Cahill room 370, H323 call ins from AEI, TAT. (call in instructions)|
|Contacts||Workshop email list: email@example.com, emergencies: Frank Löffler (firstname.lastname@example.org, +1-225-202-1752) or Roland Haas (email@example.com, +1-404-386-3657)|
|Hotel||Saga Motor Inn, 1633 E. Colorado Blvd., Pasadena, CA 91106, Preferred reservation line, Phone (626) 795-0431, Fax (626) 792-0559, http://www.thesagamotorhotel.com, $65.00 double bed, one person, $70.00 double bed, two people, Caltech rate (mention when calling)
Vagabond Inn, Pasadena, 1203 East Colorado Blvd., Pasadena, CA 91106, Phone (626) 449-3170, http://www.vagabondinn.com, $79.00 double bed (one or two persons), rate on website differs, best to call and mention Caltech.
|Transportation - Hotel||There are many possible ways of getting from the airport to Pasadena:
At any terminal at LAX, you will find departure platforms for all of the above transportation systems just outside the terminals.
The FlyAway Bus + Metro is usually very convenient. To get on it, just wait by the blue sign that says "FlyAway Busses". Wait until a FlyAway Bus with sign "Union Station" arrives (it departs every ~30min). Jump on it and go until Union Station (its only and final destination). You pay $7 upon arrival (they have change if you don't exactly have $7). At Union Station, go to the platform that says "Metro Gold Line". Pay $1.50 at the ticket machine (one way ticket). Take the Metro Gold Line (direction "Sierra Madre Station") and exit at "Lake Station" (Caltech or Vagabond Inn) or "Allen Station" (best for the Saga Motor Hotel). Check out google maps for further info. Here's a link to Metro Gold Line, including a link to this trip planer (it also knows about FlyAway busses).
|Transportation - Workshop||walking distance, directions.|
|Food options||Chandler's on campus, Ernie's food truck (times), restaurants on Lake, anything on Colorado.|
The room's IP address is 18.104.22.168, it should accept calls. PLEASE DO NOT TEST this right now since the room is often in use for group meetings etc. Instead check the room booking page and send an email to Roland Haas who will operate to room for a test.
We have had best success so far by connecting from another conference room system (Tandberg, Polycom seem to work). Failing that it is possible to use Ekiga for Linux (call h323:22.214.171.124). However we found that the versions of Ekiga in Ubuntu and Debian require specific settings to work. In particular the only audio/video codec combination that seems to work is G722 for audio and H261 for video. Ekiga lets you enable/disable codecs in its options dialog. We do not know of a working client for OSX or Windows at this point (XMeeting claims to be H323 client for OSX).
for the developers' workshop (July 27th to 29th):
|Steven Brandt||Louisiana State University||Vagabond|
|Peter Diener||Louisiana State University||Vagabond|
|Frank Löffler||Louisiana State University||Vagabond|
|Ian Hinder||Albert Einstein Institute||Remote|
|Tanja Bode||Universität Tübingen||Remote|
Want to have a relatively large list of possible topics then select which ones to actually work on.
Ideas for topics:
- AMR grid setup
- AMR scaling
- IO scaling
- Data-driven scheduler
- Cleanup of GRHydro
- Automated performance tests to verify that changes are actually beneficial
- MHD development:
- A-field MHD
- David Rideout offered to give a presentation
- McLachlan: Erik and Ian want to make several changes to the code:
- Allow arguments to be passed via scripts to generate multiple thorns in parallel, rather than sequentially
- Design of more high level APIs to streamline declaration of new objects, e.g. tensors
- Introduce master calculation for BSSN variables, and use filtering to isolate necessary code to be called in a specific run
- Enable making parameters available at either runtime or Kranc time for greater flexibility
- Additional optimizations
|Day||Time||Activity||Presenter / Responsible|
- Wireless access is through the (dedicated) CGWAS network.
Todo for sure:
- DR talk
list of votes on topics:
- (3) MHD, A, s
- (1) GRHydro cleanup
- (3) GRHydro vectorization
- (2) McLachlan
- (3) scaling
- (2) benchmarks
- Ian: Paper recently added to arxiv: http://arxiv.org/abs/1307.6488
- got funded (LSU, GaTech, NCSA, [Beijing])
- Money should benefit the whole Chemora group. Possible get together in Atlanta as Zhihui Du, our Chinese collaborator, will be there for the next year or so. Other venues are possible.
- "PDE" compiler
- Kranc, maybe with Hydro -> CaKernel
- Need more language features in the PDE language
- Use of accelerators (and CPUs)
- OpenCL (for all CUs) - Want this to make sure we can run on any new machine.
- CUDA - Want to be able to use this. CaKernel can do CUDA or OpenCL.
- Machine profile
- PAPI (now also works on Intel MICs) - dynamic feedback for LoopControl or other things.
- Production use
- Not just for exascale
- The concern is that a wide variety of big machines be enabled, too many proposals were targeting exascale.
- Intermediate Black Hole Binary Science Driver - Gatech primarily responsible. However, the Chinese side recently put in a proposal to Titan for this. Getting benefits in GRHydro may, however, be more important.
- Load Balancing - We'll need Peter's load balancer. Using AMPI with Cactus was also promised.
- First steps
- Get the code working again
- Run on one node
- Focus on Supermike 2
- Mailing list (hackathon)
- Quote of the day: Christian Ott: "Have you solved the scaling problem yet?"
- Carpet switched to new internal representation of the box-datastructure, improving scaling of modify-operations of this (initial setup, regridding)
- Since Hydro isn't vectorized, we loose performance on vector machines, unsurprisingly.
David Rideout's Talk
- Directions for Cactus HPC Framework
- Tension between generic nature of framework vs. scientific results
- Tangent discussion: need for elliptic solver to increase userbase, but hard
- More on the tangent: whether to use packages or write from scratch
- Ian: See http://arxiv.org/abs/1305.5576 which will be free software soon. This is a generic multigrid elliptic solver using Carpet.
- Big Data: esp. phone data is important current research topic
- Cosmological simulations can be modeled using similar techniques as networks
- Relevance to Cactus
- Network data as discrete spacetime model
- Discrete quantum gravity
- Unstructured Meshes
- Changes to Cactus to incorporate different grid structures and models may be quite significant, limited experience for current maintainers
Erik Schnetter's Paper
- Performance and Optimization Abstractions for Large Scale Heterogeneous Systems in the Cactus/Chemora Framework: Media:xscale2013.pdf
Staggered A-field Evolution
In the staggered system, B-fields live on faces (gets its own gridfunction) A-field lives on edges (centered versions get a new variable) E-field lives on edges, but may be stored locally (?) reconstruction needs to propagate some quantities, like velocity from centers to edges.
- Changes were made to interfaces and params, and committed to Zelmani (may change as needed)
- Tasks to work on
- Prolongation/restriction/carpetLib operators -- Philipp
- HLLE for the Avec, being careful to consider which variables live where -- Philipp
- Reconstruction for Bvec, velocities (doubly so) at edges -- Frank/Tanja
- Calc Update, Source Terms, averaged B field for fluid fluxes -- Josh
- Documentation of the method -- Josh/???