GitSuperRepoUsersGuide

Introduction

In a system such as the Einstein Toolkit which is made up of many independent components, all developed in their own repositories, it can be difficult to keep track of the toolkit as a whole. For example, there is no way to label a specific version of the toolkit without creating branches in all the component central repositories. This means that updating an ET checkout from the current central version is essentially an irreversible operation, as there is no record kept of the version of each component before the update. There is no way to see at a glance what changes you have in your Cactus tree relative to the central repositories.

To solve this problem, we have set up a Git "super-repository" of the Einstein Toolkit. This is a Git repository with no independent content of its own which contains submodules for each of the components of the toolkit. Each of these submodules is a pointer to a specific commit in a component repository. When the super-repository is cloned (with git clone --recursive), the submodule repositories for the components are also cloned, so you get a complete Cactus tree. Each of the submodules is a fully-fledged Git repository, and can be used as such. A commit in the super-repository represents the set of commits in the submodules.

This approach has the following advantages:

• You can see with a single command all the uncommitted changes in all submodules;
• You can easily check out any version of the ET across all submodules - this is not limited to release branches, but includes any version which has a corresponding commit in the super-repository;
• You can see all the changes you are going to get before updating;
• You can roll back an update if you find that the new version doesn't work;
• You can describe the state of the code by a single super-repository commit hash (plus any local changes).

Since some of the components of the ET are maintained in SVN or Mercurial rather than Git, we maintain Git mirrors of these.

This page describes how to check out and work with the Einstein Toolkit super-repository.

Checking out

 git clone --recursive git://git.barrywardell.net/EinsteinToolkit Cactus

This takes about 10 minutes and will create a directory called Cactus containing the entire Einstein Toolkit and all its version history. When it is done, setup the git-module package which provides convenience functionality:

 cd Cactus
 ./bin/git-module setup

This adds a command to add git-module to your PATH in your shell startup file (.bash_profile or .profile). You should run this now to have access to git-module on your PATH:

 source ~/.profile

or

 source ~/.bash_profile

You will need to configure simfactory as usual either by using "sim setup" or by creating a simfactory/etc/defs.local.ini file by hand.

What's new?

To see the commits that are available on the server, use

 git module summary

Commits that you would get by updating are listed in green, and those that you have made locally that you probably want to push are listed in red.

Viewing local modifications

You can see at a glance which of the submodules have local changes using

 git status

You can then go into each submodule and see those changes using git status or git diff.

Viewing history

In addition to the super repository being a git repository, each sub-repository is also a separate fully-fledged Git repository, so you can go into each one and type "git status", "git log", etc. This works for repositories which are SVN or Mercurial upstream, because we use git-svn and hg-git to convert the repositories to Git on the server. You can also use the usual graphical tools (we recommend GitX for Mac OS, and git-gui/gitk for Linux) on each subrepository to visualise the log messages and patches, see which files have local changes, and interactively commit parts of files.

Updating

If you want to pull all changes from the server (those listed in green in "git module summary"), use

 git pull
 git submodule update

Checking out a specific branch

If you want to use a particular Einstein Toolkit release, perhaps because the current development version is unstable, use

 git checkout ET_2011_06
 git module checkout --all

The tree will be very quickly updated to match the release. All changed files will have updated datestamps, so you should be able to trust the Cactus make system to recompile only what is necessary. However, it might be safer to delete any configurations and build them again.

Committing

For sub-repositories which are upstream (as most are), regardless of which revision control system is used upstream you must first initialise the sub-repository for committing:

 git module init-upstream <submodule-path>

from the root Cactus (super-repo) directory. You can get a list of the available submodules with "git module ls". Once you have completed this process for a submodule, you can treat the submodule as a regular git repository and commit as normal. The method of pushing depends on the version control system used upstream.

Git

For thorns whose source repositories are in Git already, you can commit and push from the subrepositories:

 git push upstream

upstream is a special Git remote which represents a repository URL with write-access (e.g. via ssh), as distinct from origin which is likely the read-only mirror that you cloned from.

SVN

For submodules which use SVN upstream, you can commit directly to the source SVN repository using, from the submodule directory,

 git svn dcommit

assuming you have commit rights in the source SVN repository. This will push any local commits to SVN.

Mercurial

This is not working yet due to issues with the Carpet repository. Barry: what would the syntax be if it was working?

For Mercurial upstream repositories, it is more complicated, as there is no git-hg (equivalent to git-svn). In this case, you make commits locally to your submodule as if it were a regular git repository. Then, when you want to push upstream:

 cd <submodule-path>.hg
 hg pull git # Pull from the git submodule
 hg push # Push to the upstream hg repo