GitSuperRepoUnsorted

This is a list of things that would be useful to include in the documentation somewhere

• A possible way to link the problems and solutions would be with two or three common scenarios. These would highlight a limitation/annoyance and show how our system provides an easy solution.

• A nice summary: The idea is to have a central "group" repository which points to a series of git and git-svn repositories as submodules, checked out using GetComponents. The group repository is what is cloned by users in the group, and will contain both public and private code. Gitolite can handle permissions based on users and branches. The group repository (and its submodules) would be updated from the upstream versions on any commits, and users could pull from the group repository when they wanted to.

• We could also have branches in the repository for the ET releases, and switching between them should be a lot easier than it is now (which is usually a case of "nuke your tree and check out again").

• We should also provide a public example of such a repository, containing e.g. the Einstein Toolkit. This could then be used in our ET demonstration.

Ideas/To-do

• Make it possible for the server to advance certain super-repo branches automatically. There could be some syntax in the .gitmodules file for a branch which said "auto-advance = true" or similar. This would make it possible for users to push their own branches of the super-repo and not have to worry about running the update etc themselves.

Current setup

• A git repository which only contains submodules and a few symlinks.
• Git submodules for each remote repository.
• In the case where the remote repository is an svn server, there is a git-svn clone hosted by us. Users can create and push new branches to this clone (provided they're named numrel/*), but cannot update the branches which come from svn.
• Similarly for remote Mercurial repositories I have a git clone which we host.
• For remote git repositories where we may want 'internal' branches, we host a mirror of the remote repository and allow additional branches to be created. These additional branches are only on our mirror, they are not pushed back upstream. As an example of this, we have a mirror of McLachlan which has a branch with vectorisation enabled.
• All mirrors (git-svn, git-hg and git) are automatically kept up to date with an hourly cron job, as is the 'submodules' branch of the super-repository.
• Commiting back upstream works for git, svn and hg.

git module

• add
• checkout <submodule>|--all
  • Checks out the local version of the named branch (from .gitmodules) for this submodule
  • TODO: if there are repos in .gitmodules in the new branch that don't exist in the old branch, you might have to do git submodule init. Maybe this should be in git-module
• config
• fetch <submodule>|--all
  • Fetches new commits into the given submodules. Does not merge. Does not update anything in super-repo.
• init-upstream
  • Set up the local repository for committing upstream. If the repository is SVN, this command needs to contact the server.
• ls
• mv
• rm
• setup
• update <submodule>|--all
  • Performs a fetch followed by a checkout
  • Following the checkout, this should do a merge --no-ff.
  • It would be convenient to have a verbose option (maybe on by default) which prints all the commits which are being added during the merge.
  • In addition, we could have a --dry-run|-n option which prints all the same things, but does not do the actual merge.
  • With the above, we can do a dry-run to see what we would get, then do a non-dry-run to actually do the update.
  • Optionally, we could also print out at the beginning the commit we started from, in case we want to go back to it

Workflow

The following tasks are things that users should be able to do with our system:

• Download the code
• Update to the "current" version of the code
• Undo an update
• Add components
• Remove components
• Go back to an old version
• See differences between versions
• Make a change to a component
• Commit component changes
• Create patches for components
• Create branches for components
• Publish your local version of the code for others to see

Updating/pulling

It is convenient to update some or all of the submodules in a super-repository at the same time. This means bringing in changes from their upstream repositories. Currently we have "git module update" which 1. fetches, 2. checks out the branch listed in the 'revision' entry of .gitmodules, and 2. pulls the checked out branch with the --ff-only option. This is a very safe setup and can never lead to any conflicts which need to be resolved.