By clayb on Feb 11, 2010
Why make folks do new things?
I realized that despite the importance for software development, many folks are largely unaware of the features and power their S(ource) C(ode) M(anagement) tool can provide them. In my group, we use Mercurial. It is an awesome SCM and perfect for our team (which is largely Python writing) as it is written in Python.
However, as with any tool under development, there are features which were not mature when we started using Mercurial, or otherwise, we have just never taken advantage of some features despite their obvious use.
For us, we have a release of OpenSolaris every six months roughly and to prevent stopping all non-release development, we will fork our source base into a release and development (non-release targeting) code base. The first time I did this, as the gatekeeper (what we call the team's build maintainer and release engineer) I simply created a second Mercurial repository -- this was for the OpenSolaris 0906 release. This repository mirrored our slim_source Mercurial repository. All was well. This approach illustrated three areas of uncertainty with developer understanding, however:
- Exporting changes from a repo
- Importing changes to a repo
- Viewing and understanding a file's SCM history
Further, this required the team to hold a dead code base around (the now dead slim_0906 repository for the similarly named release). If we want to know what changes made that release we must clone in and go through it trying to compare it to our main-line code base slim_source.
This was awesome to know, for as a gatekeeper one becomes far more intimate with the SCM than most developers need to be. But it also questioned the power lurking underneath the SCM. For Mercurial can address all these issues easily:
- Exporting changes from a repo -- use hg export
- Importing changes to a repo -- use hg import
- Viewing and understanding a file's SCM history -- use hg glog or hg view
- Forking a source code base for a project -- use hg branches
Given these issues and the potential fixes, I figured I should try and help the developer community using our code base learn these new tools. Further, trying to automate as much as I can, using Mercurial named branches was a key thing to keeping all of our source code in one repository for later reference and analysis. Though, it meant going beyond established OpenSolaris territory of Mercurial use as to my knowledge no other team is using named branches on hg.opensolaris.org yet.
Named branches caused some issues!
First, the OpenSolaris developer environment uses community-developed extensions called Cadmium. These are to mimic some tools which Sun's previous TeamWare software provided as the Solaris SCM of old (such as our recommit extension). Other features are simply to establish standard operating procedure for development (like webrev's as can be seen at cr.opensolaris.org). These Cadmium extensions were developed assuming no-one would be working on branched code bases.
Luckily, since we rely on Cadmium's recommit the IPS folks had talked about the issues of using branches before. Rich Lowe in particular already even
had a potential fix for hg recommit ready for codereview! Putting this in place until it is accepted into the OpenSolaris SUNWonbld package simply required sending a heads-up to the slim_source developing community to install the updated Cadmium files. has integrated into build 134 and later (see the changeset log for build 134) a fix for bug
6922488 - cadmium recommit should be more relaxed about in-repository branching. To use the new code do nothing more than pkg image-update to build 134 or newer and your pkg:/developer/build/onbld package will be updated!
However, then as development continued, we found that there was an issue on our gate machine. The gate machine for the OS/Networking consolidation is different than hg.opensolaris.org so that the ON gatekeepers can configure their own Mercurial gate-hooks. On our gate machine hg.opensolaris.org we can not configure our own gate-hooks, and this caused us issue as the gate machine, of course, will accept one making or merging a branch in the source code base.
Mercurial, will of course, let you happily close a branch (e.g. hg commit --close-branch slim_1003). However, we found folks could do interesting things like:
hg clone ssh://firstname.lastname@example.org/hg/caiman/slim_source hg update -C
[do changes] hg branch -f hg commit hg push
Case 2 (simply merge the branches)
hg clone ssh://email@example.com/hg/caiman/slim_source hg update -C
[do changes] hg commit [do more changes] hg commit hg recommit (with an un-patched hg recommit) [hg recommit would say to merge all heads -- each branch has its own head!] hg merge -r hg commit hg push
These two cases, though undesirable to our community, would happily be accepted by hg.opensolaris.org into our slim_source repository and require lots of coordination to revert their affects. So the answer was to write a Mercurial hook. But this hook would be a client-side hook.
As such, with the help of Dave Marker we were able to craft a user-side hook to check if all pushes were continuing on the same branch or if someone is jumping branches. This hook needs to be added to an hgrc file in you path. For a build machine, this can be /etc/hgrc or for a user you can use your ~/.hgrc.
First, download the tar ball with the necessary hook and Python file. Next, modify your hgrc as below and you might have other lines in your [extensions] and [hooks] sections simply append these lines to their correct sections:
[extensions] hook = ~/slim_hooks [hooks] pretxncommit.0 = python:hook.branch_change_chk.branch_change_chk
What about export and import?
Well, unintentionally, developers were exposed to hg export and hg import. As some pushes before the user-hook was available were unfortunately toxic, developers ended up having to learn to export their changes and import them into clean source pulls. Luckily this went very smoothly but a copy of the instructions are reproduced below for interest's sake:
Instructions to move your changes to a non-toxic workspace:
First, determine which changeset you wish to preserve:Use hg log to see what changes you have made to the workspace. For example, here I would want changeset 716:7c6e6a587d85:
 clayb@xsplat:hg log|less changeset: 716:7c6e6a587d85 branch: slim_1003 tag: tip user: Clay Baenziger
date: Fri Feb 05 15:32:44 2010 -0700 description: Test changeset: 715:a836473e08d3 branch: slim_1003 user: Matt Keenan date: Fri Feb 05 14:52:13 2010 +0000 description: 14358 Installer graphic needs changes
Next, export your change:
 clayb@xsplat:hg export 7c6e6a587d85 > /tmp/mychange
Now, pull a clean workspace via:
 clayb@xsplat:hg clone \\ ssh://anon at hg dot opensolaris dot org/hg/caiman/slim_source
Next, select the branch you want to push to:
Confirm it's clean:
 clayb@xsplat:cd slim_source  clayb@xsplat:hg log --template="True\\n" -r 5f70c8ce60b9|| \\ echo "False" abort: unknown revision '5f70c8ce60b9'! False
For slim_1003 (blocker bugs):
 clayb@xsplat:hg update -C slim_1003 6 files updated, 0 files merged, 0 files removed, 0 files unresolved
For default (future development):
 clayb@xsplat:hg update -C default 0 files updated, 0 files merged, 0 files removed, 0 files unresolved
Add your change to the clean repo:
 clayb@xsplat:hg import /tmp/mychange applying /tmp/mychange
Confirm only your change is outgoing (and in the right branch):
 clayb@xsplat:hg outgoing comparing with ssh://anon at hg dot opensolaris dot org/hg/caiman/slim_source searching for changes changeset: 715:b3867e9369ed branch: slim_1003 tag: tip parent: 712:ec8ad8a8fb9c user: Clay Baenziger
date: Fri Feb 05 15:32:44 2010 -0700 description: Test