Version control - Revision history

Hinder: /* Arguments against having lots of thorns in a single repository */

2013-08-19T06:47:27Z

‎Arguments against having lots of thorns in a single repository

Knarf: /* Arguments against having lots of thorns in a single repository */

2013-08-17T21:20:15Z

‎Arguments against having lots of thorns in a single repository

Rhaas: add some arguments against large repositories

2013-08-11T19:42:51Z

add some arguments against large repositories

Knarf: /* Miscellaneous repositories */

2013-08-11T18:53:53Z

‎Miscellaneous repositories

131.215.195.89: add argument about inter-thorn consistency

2013-08-11T18:52:39Z

add argument about inter-thorn consistency

Knarf: /* Reasons to Change */

2013-08-11T18:28:22Z

‎Reasons to Change

Knarf at 18:24, 11 August 2013

2013-08-11T18:24:34Z

Knarf at 18:21, 11 August 2013

2013-08-11T18:21:50Z

Knarf: /* Arguments for having lots of thorns in a single repository */

2013-08-11T18:17:12Z

‎Arguments for having lots of thorns in a single repository

Knarf: /* Options */

2013-08-11T18:12:45Z

‎Options

←Older revision		Revision as of 06:47, 19 August 2013
Line 122:		Line 122:

	Finally it seems as if there are ways out there to combine individual sources into a super-repository with git (submodules and subcomponents which can be used even with svn), but it is much harder to split up a repository since on essentially has to keep around several checkouts for the different revisions one wants to access (eg I keep a git-svn copy of EinsteinEvolve/GRHydro for the single reason of preparing the GRHydro updates from Zelmani in it before submitting them to the svn server).		Finally it seems as if there are ways out there to combine individual sources into a super-repository with git (submodules and subcomponents which can be used even with svn), but it is much harder to split up a repository since on essentially has to keep around several checkouts for the different revisions one wants to access (eg I keep a git-svn copy of EinsteinEvolve/GRHydro for the single reason of preparing the GRHydro updates from Zelmani in it before submitting them to the svn server).
		+
		+	==Organise repositories by project==
		+
		+	Another option is to have a single repository for the Einstein thorns, one for Cactus, and one for each of the additional independent projects. The list of repositories would then be:
		+
		+	* Cactus
		+	* Einstein
		+	* Kranc
		+	* EinsteinExact
		+	* McLachlan
		+	* Carpet
		+	* Something about external libraries, to be decided
		+	* SimFactory
		+
		+	One concern which has been raised with this approach is that it makes it harder to mix and match different versions of thorns, e.g. within the Einstein repository. The argument against this is that you should not need to do this, as the current trunk should be fairly stable (aggressive development can happen on branches, if people are making drastic changes). A further advantage is that if people are developing one particular thorn, they are more likely to also be working with the latest versions of all the other thorns.

@@ Line 117: / Line 117: @@
 ==Arguments against having lots of thorns in a single repository==
-Once a certain sized is reached, and assuming that the code actually is developed, then the number of commits to the repository increases and it becomes very difficult for a user/developer to check out a "stable" copy of the repository. At any given revision one thorn will be undergoing rapid changes and with at single repository it is not possible to hold on to an older version of that thorn (which eg might offer all the functionality one needs) while updating another thorn with a crucial bugfix/new feature. This happened to some degree in Zelmani when a new thorn was added for postprocessing production results and thus a new revision was created however this also updated the evolution thorn to their development state which is often unstable. Larger repositories have this problem to an even larger degree (ie the code is constantly changing "under ones feet") which makes it hard to isolate causes of bugs in (presumably fairly isolated) sub-components.
+Once a certain sized is reached, and assuming that the code actually is developed, then the number of commits to the repository increases and it becomes very difficult for a user/developer to check out a "stable" copy of the repository. At any given revision one thorn will be undergoing rapid changes and with at single repository it is not possible to hold on to an older version of that thorn (which eg might offer all the functionality one needs) while updating another thorn with a crucial bugfix/new feature (note: this is not true with Subversion. This problem really stems from the limitation of git that it cannot handle different subdirectories of a checkout living at another revision, nor being able to checkout only part of a repository). This happened to some degree in Zelmani when a new thorn was added for postprocessing production results and thus a new revision was created however this also updated the evolution thorn to their development state which is often unstable. Larger repositories have this problem to an even larger degree (ie the code is constantly changing "under ones feet") which makes it hard to isolate causes of bugs in (presumably fairly isolated) sub-components.
 Philosophically the ET seems to be a loose collection of independently developed modules. Combining unrelated thorns in shared repositories negates this to some degree. Note that in this light, having AEIThorns be a single repo would be ok, since they are all from one institution (and eg Zelmani works this way mostly without problems).
 Finally it seems as if there are ways out there to combine individual sources into a super-repository with git (submodules and subcomponents which can be used even with svn), but it is much harder to split up a repository since on essentially has to keep around several checkouts for the different revisions one wants to access (eg I keep a git-svn copy of EinsteinEvolve/GRHydro for the single reason of preparing the GRHydro updates from Zelmani in it before submitting them to the svn server).

←Older revision		Revision as of 19:42, 11 August 2013
Line 114:		Line 114:

	The main objection raised on the ET call to having one thorn per repository in general was that this leads to a large number of repositories, and this overwhelms the user interface. SourceTree shows submodules (if we go that route) in a hierarchical structure, and lists separately those with uncommitted changes, but SorceTree is Mac (and Windows) only, so also not a solution for everyone. “git status” lists the submodules with uncommitted changes, like "svn status" as well. If people don't want to use submodules or subtrees, we would need to develop tools to manage multiple repositories together. Such tools are needed anyway unless you have a single repository for the whole toolkit, which isn't going to happen.		The main objection raised on the ET call to having one thorn per repository in general was that this leads to a large number of repositories, and this overwhelms the user interface. SourceTree shows submodules (if we go that route) in a hierarchical structure, and lists separately those with uncommitted changes, but SorceTree is Mac (and Windows) only, so also not a solution for everyone. “git status” lists the submodules with uncommitted changes, like "svn status" as well. If people don't want to use submodules or subtrees, we would need to develop tools to manage multiple repositories together. Such tools are needed anyway unless you have a single repository for the whole toolkit, which isn't going to happen.
		+
		+	==Arguments against having lots of thorns in a single repository==
		+
		+	Once a certain sized is reached, and assuming that the code actually is developed, then the number of commits to the repository increases and it becomes very difficult for a user/developer to check out a "stable" copy of the repository. At any given revision one thorn will be undergoing rapid changes and with at single repository it is not possible to hold on to an older version of that thorn (which eg might offer all the functionality one needs) while updating another thorn with a crucial bugfix/new feature. This happened to some degree in Zelmani when a new thorn was added for postprocessing production results and thus a new revision was created however this also updated the evolution thorn to their development state which is often unstable. Larger repositories have this problem to an even larger degree (ie the code is constantly changing "under ones feet") which makes it hard to isolate causes of bugs in (presumably fairly isolated) sub-components.
		+
		+	Philosophically the ET seems to be a loose collection of independently developed modules. Combining unrelated thorns in shared repositories negates this to some degree. Note that in this light, having AEIThorns be a single repo would be ok, since they are all from one institution (and eg Zelmani works this way mostly without problems).
		+
		+	Finally it seems as if there are ways out there to combine individual sources into a super-repository with git (submodules and subcomponents which can be used even with svn), but it is much harder to split up a repository since on essentially has to keep around several checkouts for the different revisions one wants to access (eg I keep a git-svn copy of EinsteinEvolve/GRHydro for the single reason of preparing the GRHydro updates from Zelmani in it before submitting them to the svn server).

@@ Line 102: / Line 102: @@
 * CactusIO
-Cactus flesh, CactusBase, CactusUtils, CactusPUGH etc could conceivably all be in a single Cactus repository, with commit rights being controlled by subdirectory (if using Gitolite <http://gitolite.com/gitolite/vref.html>).  Are these all under the same licence?
+Cactus flesh, CactusBase, CactusUtils, CactusPUGH etc could conceivably all be in a single Cactus repository, with commit rights being controlled by subdirectory, but CactusPUGH doesn't quite fit in and might be separate. All Cactus thorns have the same license, so no problem should stem from this.
 ==Arguments for having lots of thorns in a single repository==

@@ Line 110: / Line 110: @@
 Checkout time is reduced due to requiring a smaller number of connections to the server
 Counter-argument: With parallel checkouts, this might not be an important consideration.
 The main objection raised on the ET call to having one thorn per repository in general was that this leads to a large number of repositories, and this overwhelms the user interface.  SourceTree shows submodules (if we go that route) in a hierarchical structure, and lists separately those with uncommitted changes, but SorceTree is Mac (and Windows) only, so also not a solution for everyone. “git status” lists the submodules with uncommitted changes, like "svn status" as well. If people don't want to use submodules or subtrees, we would need to develop tools to manage multiple repositories together.  Such tools are needed anyway unless you have a single repository for the whole toolkit, which isn't going to happen.

@@ Line 7: / Line 7: @@
 ==Reasons to Change==
-Won’t argue for Git vs SVN here; search the internet for why one or the other is preferred. In the end its about taste and what is it used for (nothing is the best for everything). When the proposal to switch to Git for some components was raised on the ET telecon, nobody objected, and many people voiced support. Further, there was no objection on the mailing list. We conclude that of the active developers who care about the issue, most prefer Git and support a transition.
+Won’t argue for Git vs SVN here; search the internet for why one or the other is preferred. In the end its about taste and what it is used for (nothing is the best for everything). When the proposal to switch to Git for some components was raised on the ET telecon, nobody objected, and many people voiced support. Further, there was no objection on the mailing list. We conclude that of the active developers who care about the issue, most prefer Git and support a transition.
 ==Considerations==

@@ Line 26: / Line 26: @@
 Managing the ET becomes much easier if everything is in one repository system (Git provides mechanisms to treat collections of repositories together; but then Subversion has that as well), so (2) adds some work, both on the management side as well as on the user side. On the other hand, we cannot and should not enforce Git for all repositories, so we will always have a mix.
-Additionally, we can:
+= Repository rearrangements =
 * Require each thorn to be an independent repository
 * Require each arrangement to be an independent repository
@@ Line 40: / Line 41: @@
 There is no (?) established method for treating multiple subcomponents as a single repository if those subcomponents use different VCSs.  We have developed GetComponents for the purpose of checking out such a heterogeneous collection, and support has been added to it for performing some basic operations on the whole repository, but this is a “home-grown” solution, and we would need to develop and support it ourselves.
-=Possible arrangement/thorn repository split=
+==Possible arrangement/thorn repository split==
 The following split is based on logical relationships between the thorns, and ignores the issue of the potential convenience of having a number of thorns in a single repository in the absence of infrastructure for working with sets of repositories.

@@ Line 40: / Line 40: @@
 There is no (?) established method for treating multiple subcomponents as a single repository if those subcomponents use different VCSs.  We have developed GetComponents for the purpose of checking out such a heterogeneous collection, and support has been added to it for performing some basic operations on the whole repository, but this is a “home-grown” solution, and we would need to develop and support it ourselves.
-==Possible arrangement/thorn repository split==
+=Possible arrangement/thorn repository split=
 The following split is based on logical relationships between the thorns, and ignores the issue of the potential convenience of having a number of thorns in a single repository in the absence of infrastructure for working with sets of repositories.
-===Arrangement repositories===
+== Arrangement repositories==
 * CactusBase (nearly all of these thorns are used nearly all of the time)
 * CactusPUGH:
@@ Line 65: / Line 65: @@
-===One repository per thorn===
+==One repository per thorn==
 * CactusNumerical:
@@ Line 91: / Line 91: @@
 ** Groupings of thorns hosted by institutions; no real logical relationship between them, and we probably don’t include all the thorns in these repository groupings in the ET anyway.  The thorns in each arrangement are not related to each other.
-===Miscellaneous repositories===
+==Miscellaneous repositories==
 * Cactus flesh
@@ Line 103: / Line 103: @@
 Cactus flesh, CactusBase, CactusUtils, CactusPUGH etc could conceivably all be in a single Cactus repository, with commit rights being controlled by subdirectory (if using Gitolite <http://gitolite.com/gitolite/vref.html>).  Are these all under the same licence?
-===Arguments for having lots of thorns in a single repository===
+==Arguments for having lots of thorns in a single repository==
 If your tools can only “see” a single repository at a time, it is hard to see all the changes that exist in your current working tree, so you don’t know what might need to be committed or discarded.  Having many thorns per repository means a smaller number of repositories to check manually.

@@ Line 33: / Line 33: @@
 The third option is the most pragmatic.
-It is strongly desirable to be able to manage the entire toolkit (or more generally, an entire Cactus tree) as a single entity.  One can then talk about a specific version of the whole code, can globally revert to a previous version, see history of all thorns together, update the tree, etc.. However, with thorns spread across the globe, controlled by different groups, this is hard to impossible to achieve.
+It is strongly desirable to be able to manage the entire toolkit (or more generally, an entire Cactus tree) as a single entity.  One can then talk about a specific version of the whole code, can globally revert to a previous version, see history of all thorns together, update the tree, etc... Also note that here one of the deficiencies of Git shows: it is not possible to checkout only a sub-tree of a Git repository, while this is not a problem at all for Subversion.
 GetComponents and Heterogeneous Repositories