GitSuperRepoUsersGuide

Introduction

In a system such as the Einstein Toolkit which is made up of many independent components, all developed in their own repositories, it can be difficult to keep track of the toolkit as a whole. For example, there is no way to label a specific version of the toolkit without creating branches in all the component central repositories. This means that updating an ET checkout from the current central version is essentially an irreversible operation, as there is no record kept of the version of each component before the update. There is no way to see at a glance what changes you have in your Cactus tree relative to the central repositories.

To solve this problem, we have set up a Git "super-repository" of the Einstein Toolkit. This is a Git repository with no independent content of its own which contains submodules for each of the components of the toolkit. Each of these submodules is a pointer to a specific commit in a component repository. When the super-repository is cloned (with git clone --recursive), the submodule repositories for the components are also cloned, so you get a complete Cactus tree. Each of the submodules is a fully-fledged Git repository, and can be used as such. A commit in the super-repository represents the set of commits in the submodules.

This approach has the following advantages:

• You can see with a single command all the uncommitted changes in all submodules;
• You can easily check out any version of the ET across all submodules - this is not limited to release branches, but includes any version which has a corresponding commit in the super-repository;
• You can see all the changes you are going to get before updating;
• You can roll back an update if you find that the new version doesn't work;
• You can describe the state of the code by a single super-repository commit hash (plus any local changes).

Since some of the components of the ET are maintained in SVN or Mercurial rather than Git, we maintain Git mirrors of these.

This page describes how to check out and work with the Einstein Toolkit super-repository.

Checking out

 git clone --recursive git://git.barrywardell.net/EinsteinToolkit Cactus

This takes about 10 minutes and will create a directory called Cactus containing the entire Einstein Toolkit and all its version history. When it is done, you need to set up the git-module package which provides convenience functionality:

 cd Cactus
 ./bin/git-module setup

This edits your Bash startup file (assumed to be ~/.profile), adding git-module to your PATH and enabling bash completion for the git-module package. You should now reload your startup file to have access to git-module on your PATH:

 source ~/.profile

Note that this currently only works if your shell is Bash. If you use another shell, skip the setup and just add '.../Cactus/bin' to your path.

You will need to configure simfactory as usual either by using "sim setup" or by creating a simfactory/etc/defs.local.ini file by hand.

Status

What's new?

Get all current information from the remote repositories (this does not change anything in your working tree):

 git fetch
 git submodule foreach git fetch # I think this is unnecessary; git fetch seems to fetch any submodules which need it

Find out what commits you would get if you updated:

 git log --reverse --oneline -p --submodule master..origin/master $(git submodule|awk '{print $2}')

We should have a git-module command for this, preferably in the format of git-module summary. It should not make assumptions about branch names like I did above.

Viewing local modifications

You can see at a glance which of the submodules have local changes using

 git status

You can then go into each submodule and see those changes using git status or git diff.

Viewing history

In addition to the super repository being a git repository, each sub-repository is also a separate fully-fledged Git repository, so you can go into each one and type "git status", "git log", etc. This works for repositories which are SVN or Mercurial upstream, because we use git-svn and hg-git to convert the repositories to Git on the server. You can also use the usual graphical tools (we recommend GitX for Mac OS, and git-gui/gitk for Linux) on each subrepository to visualise the log messages and patches, see which files have local changes, and interactively commit parts of files.

Changing to a new version

Updating

If you want to pull all changes from the server (those listed in green in "git module summary"), use

 git pull
 git submodule update

from the Cactus directory.

If you want to go back to the version before the pull, for example if the new version doesn't work, you can use

 git reset --hard ORIG_HEAD
 git submodule update

Checking out a specific branch

If you want to use a particular Einstein Toolkit release, perhaps because the current development version is unstable, use

 git checkout ET_2011_06
 git submodule update

The tree will be very quickly updated to match the release. All changed files will have updated datestamps, so you should be able to trust the Cactus make system to recompile only what is necessary. However, it might be safer to delete any configurations and build them again.

You can see a list of available super-repository branches using

 git branch -a

Committing

For sub-repositories which are upstream (as most are), regardless of which revision control system is used upstream you must first initialise the sub-repository for committing:

 git module init-upstream <submodule-path>

from the root Cactus (super-repo) directory. You can get a list of the available submodules with "git module ls". Once you have completed this process for a submodule, you can treat the submodule as a regular git repository and commit as normal. The method of pushing depends on the version control system used upstream.

Git

For thorns whose source repositories are in Git already, you can commit and push from the subrepositories:

 git push upstream

upstream is a special Git remote which represents a repository URL with write-access (e.g. via ssh), as distinct from origin which is likely the read-only mirror that you cloned from.

SVN

For submodules which use SVN upstream, you can commit directly to the source SVN repository using, from the submodule directory,

 git svn dcommit

assuming you have commit rights in the source SVN repository. This will push any local commits to SVN.

Mercurial

This is not working yet due to issues with the Carpet repository.

For Mercurial upstream repositories, make sure git-hg is installed. You make commits locally to your submodule as if it were a regular git repository. Then, when you want to push upstream:

 git hg push