Services

From Einstein Toolkit Documentation
Revision as of 21:05, 29 February 2024 by Rhaas (talk | contribs) (→‎Mailing lists)
Jump to: navigation, search

The Einstein Toolkit depends on a number of services managed by the ET maintainers. Here we list each of these, and give any relevant information. Ideally, this page should be sufficient for a maintainer to deal with urgent issues relating to these services in case the primary administrator is unavailable.

On 2018-12-10 and 2018-12-17 Steve Brandt and Roland Haas gave a short introduction to the ET infrastructure services. Here are the slides used which may contain information not yet included in this wiki.

Getting Help

  • For LSU sites, you can email CCT IT Support <it-support@cct.lsu.edu>, please mention that you work with Steve Brandt. Ideally, you should have a CCT external user account when sending help mail. If you don't have one, please email the above it-support@cct.lsu.edu address to ask for one.
  • For NCSA, the generic email for help of any kind is help@ncsa.illinois.edu and CC'ing Roland Haas (rhaas@illinois.edu) will help. If you are involved in a service hosted at NCSA we can create an external collaborators NCSA account for you, please contact Roland to create such an account.

Website

  • https://einsteintoolkit.org
  • Location: A VM at CCT
  • Administrator: Steve Brandt
  • Source for website: https://bitbucket.org/einsteintoolkit/www.git
  • Docker source for webserver: https://github.com/stevenrbrandt/et-websites.git
  • Updating the website:
    • Login to the einsteintoolkit.org server as root
    • Go to the et-websites directory
    • Type 'docker-compose build'
    • Type 'docker-compose down' to bring down the server
    • Type 'docker-compose up -d' to restart with the new image
    • the website docker container auto-updates once a day
  • Members list: The members list is updated manually, in response to an email sent by the website's PHP script. The email sends the subscribers email address, whether they'd like to be added to the mailing list, a link to add the user to the mailing list, and a text entry for members.txt (from which the members page is generated). In addition, there is a python script, sortmembers.py, which sorts the members.txt file.
  • Release announcements: Each must be put in the about/releases directory, using the name ET_{year}_{month}_announcement.md. The script mk_release_announcements.py will generate a txt and html file from the md file.
  • the website reports unanswered question candidates on https://www.einsteintoolkit.org/tools/unanswered.php which is driven by the script unanswered.pl in the tools directory of the www repo. Occasionally the script may be unable to contact the wiki page https://docs.einsteintoolkit.org/et-docs/Answered_emails that holds emails flagged as "answered". When this happens only from within the webserver then Roland and Steve observed (at least once) that the ARP cache on the webserver was incorrect and could be reset by contacting the webserver from the wiki server

ThornGuide on einsteintoolkit.org

  • ThornGuide.php is in /www/einstein/www/documentation
  • It draws on the autogenerated http doc files, which are updated once a day by a GitHub action docs.yml
  • the github action trigger the webserver to update itself

Tutorial

git clone  git@bitbucket.org:einsteintoolkit/www
cd www
python3 ./whitelist-code-gen.py  email@example.com
# this adds a md5 hash of the email address to the file tutorials-whitelist.txt
git add tutorials-whitelist.txt 
git commit -m 'Tutorial whitelist: new user added'
git push
# respond to request on mailing list *and* to requestor directly

Wiki

  • https://docs.einsteintoolkit.org/et-docs/
  • Location: A VM and Database at CCT
  • Accounts:
    • Log into docker image as root (docker exec -it wiki bash)
    • To create a user named dvader with password dieoobiwandie, run the command php /var/www/mediawiki/maintenance/createAndPromote.php dvader dieobiwandie
    • To change a password, run the command php /var/www/mediawiki/maintenance/changePassword.php --user=dvader --password=iluvupadme
  • Administrator: Steve Brandt, Roland Haas
  • Source code: https://github.com/stevenrbrandt/et-websites.git

Subversion

git

The organizations should has administrator group and a developer group where administrators are the ET maintainers and developers are ET users who have been granted write access to the repos. We try to keep access control coarse grained, erring on the side of granting write access easily and to all repos rather than micromanaging access.

New code repositories on bitbucket should set up a "repo push" (the default) web-hook to contact https://einsteintoolkit.org/hooks/repo_changed.php?secret=XXX to send commit messages to commits@einsteintoolkit.org where XXX is to be found in https://bitbucket.org/einsteintoolkit/www/src/master/hooks/repo_changed.php#lines-18 (or thereabouts, look for the variable $secret).

New code repositories on github should set up commits@einsteintoolkit.org as the recipient of push event emails in https://github.com/GITHUB_USER_NAME/REPO_NAME/settings/notifications/edit .

github is used by gitter.im to authenticate users but access to https://gitter.im/einsteintoolkit is possible without being in the github einsteintoolkit organization.

Bitbucket ticket repository

Mailing lists

  • http://lists.einsteintoolkit.org/mailman/listinfo
  • Location: A VM at CCT, hostname svn.einsteintoolkit.org
  • Administrator: Steve Brandt. He needs to filter the spam daily. For those who want to know, Russian brides are still a popular subject in spam.
  • archives wrap text emails at the 80 character mark using CSS in /usr/lib/mailman/templates/site/en/article.html
   <STYLE TYPE="text/css">
     pre {
       overflow-x: auto;
       white-space: pre-wrap;
       white-space: -moz-pre-wrap;
       white-space: -pre-wrap;
       white-space: -o-pre-wrap;
       word-wrap: break-word;
       width: 80ch;
     }
   </STYLE>

alias_maps = hash:/etc/aliases, hash:/etc/mailman/aliases

Have to force-allow some SMTP servers (Google.com in particular) for postdfix, currently in a file /etc/postfix/google_acces and /etc/postfix/main.cf 

check_client_access hash:/etc/postfix/google_access
check_client_access hash:/etc/postfix/github_access

and in the access file google_access 

google.com OK
amantadine.ncsa.uiuc.edu OK
outbound.protection.outlook.com OK

and similar or access from MS's github runner hosts (at least 172.160.0.0/11) as they show up in the logs eg on [1].

After modifying which one must run: 

/usr/sbin/postmap google_access
/usr/sbin/postmap github_access
/etc/init.d/postfix restart

Chat channels

Zoom

  • we are using a zoom instance provided by LSU, details described on ET_call_Zoom_Info.
  • Administrator: Steve Brandt

ET call reminders

  • reminders to add to the agenda and for the calls themselves are sent to users@einsteintoolkit.org at Monday 15:18 Central time and Wednesday 17:15 Central time respectively, via cron jobs running on Roland Haas' workstation
  • Administrator: Roland Haas
#!/bin/bash

if [ $(date +%s) -lt  $(date +%s -d  2019-06-21) ] ; then
exit
fi

mail -r users@einsteintoolkit.org -s "Agenda for Thursday's Meeting" users@einsteintoolkit.org <<EOF
Please update the Wiki with agenda items for Thursday's meeting. Thanks!

https://docs.einsteintoolkit.org/et-docs/meeting_agenda

--The Maintainers
EOF

#!/bin/bash

if [ $(date +%s) -lt  $(date +%s -d  2019-06-21) ] ; then
exit
fi

mail -r users@einsteintoolkit.org -s "Einstein Toolkit Meeting Reminder" users@einsteintoolkit.org <<EOF
Hello,

Please consider joining the weekly Einstein Toolkit phone call at
9:00 am US central time on Thursdays. For details on how to connect
and what agenda items are to be discussed, use the link below.

https://docs.einsteintoolkit.org/et-docs/Main_Page#Weekly_Users_Call

--The Maintainers
EOF

CactusCode.org

Website monitoring

Websites are monitored every 15 minutes by https://einsteintoolkit.github.io/uptime using a dummy github einsteintoolkit-uptime with email address maintainers+uptime@einsteintoolkit.org. Currently monitored pages are:

Email updates are sent to <maintainers@einsteintoolkit.org> which has had uptime@einsteintoolkit.org added to its list of allowed posters. An older hand-written monitor of Frank Loeffler's is at https://www.cct.lsu.edu/~knarf/cgi-bin/monitor.cgi .

Access to the monitors is by all ET maintainers via its github repo https://github.com/einsteintoolkit/uptime

Cactus services?

Domain

  • einsteintoolkit.org
  • Owned by LSU
  • Notes: Cannot point subdomains to any machines not controlled directly by LSU

Build and test

Adding a new thorn

The testing system https://github.com/EinsteinToolkit/tests/blob/master/.github/workflows/main.yml does not use GetComponents to check out the code. Instead it uses a git super-repository where all thorns are submodules.

One creates a new submodule in repos</repos> for each git repository, similarly to how GetComponents creates a git checkout for each repository. One then has to manually create symbolic links from <codde>arrangements/&langle;ARRANGEMENT&rangle;/&langle;THORN&rangle; to the checkout. The submodule must be set up to track the main development branch (main, master, etc.).

For example for CarpetX 

git clone https://github.com/einsteintoolkit/tests
cd tests
git submodule update --recursive --init
cd repos
git submodule add -b main https://github.com/eschnett/CarpetX CarpetX

and then update .gitmodules to read 

[submodule "repos/CarpetX"]                                                                                     path = repos/CarpetX
  revision = main
  branch = main
  url = https://github.com/eschnett/CarpetX
  upstreamurl = git@github.com:eschnett/CarpetX
  upstreamtype = git
</code>

And finally create the arrangements etc. use GetComponents

<pre>
cd tests
bin/GetComponents --no-update --root $PWD manifest/einsteintoolkit.th

then use git status to then git add and git commit</commit> the changes.

Git updater

Retrieved from "https://docs.einsteintoolkit.org/et-docs/index.php?title=Services&oldid=8392"