Sunday Jun 24, 2012

JPRT: A Build & Test System


A while back I did a little blogging on a system called JPRT, the hardware used and a summary on my weblog. This is an update on the JPRT system.

JPRT ("JDK Putback Reliablity Testing", but ignore what the letters stand for, I change what they mean every day, just to annoy people :\^) is a build and test system for the JDK, or any source base that has been configured for JPRT. As I mentioned in the above blog, JPRT is a major modification to a system called PRT that the HotSpot VM development team has been using for many years, very successfully I might add. Keeping the source base always buildable and reliable is the first step in the 12 steps of dealing with your product quality... or was the 12 steps from Alcoholics Anonymous... oh well, anyway, it's the first of many steps. ;\^)

Internally when we make changes to any part of the JDK, there are certain procedures we are required to perform prior to any putback or commit of the changes. The procedures often vary from team to team, depending on many factors, such as whether native code is changed, or if the change could impact other areas of the JDK. But a common requirement is a verification that the source base with the changes (and merged with the very latest source base) will build on many of not all 8 platforms, and a full 'from scratch' build, not an incremental build, which can hide full build problems. The testing needed varies, depending on what has been changed.

Anyone that was worked on a project where multiple engineers or groups are submitting changes to a shared source base knows how disruptive a 'bad commit' can be on everyone. How many times have you heard:
"So And So made a bunch of changes and now I can't build!".
But multiply the number of platforms by 8, and make all the platforms old and antiquated OS versions with bizarre system setup requirements and you have a pretty complicated situation (see

We don't tolerate bad commits, but our enforcement is somewhat lacking, usually it's an 'after the fact' correction. Luckily the Source Code Management system we use (another antique called TeamWare) allows for a tree of repositories and 'bad commits' are usually isolated to a small team. Punishment to date has been pretty drastic, the Queen of Hearts in 'Alice in Wonderland' said 'Off With Their Heads', well trust me, you don't want to be the engineer doing a 'bad commit' to the JDK. With JPRT, hopefully this will become a thing of the past, not that we have had many 'bad commits' to the master source base, in general the teams doing the integrations know how important their jobs are and they rarely make 'bad commits'. So for these JDK integrators, maybe what JPRT does is keep them from chewing their finger nails at night. ;\^)

Over the years each of the teams have accumulated sets of machines they use for building, or they use some of the shared machines available to all of us. But the hunt for build machines is just part of the job, or has been. And although the issues with consistency of the build machines hasn't been a horrible problem, often you never know if the Solaris build machine you are using has all the right patches, or if the Linux machine has the right service pack, or if the Windows machine has it's latest updates. Hopefully the JPRT system can solve this problem. When we ship the binary JDK bits, it is SO very important that the build machines are correct, and we know how difficult it is to get them setup. Sure, if you need to debug a JDK problem that only shows up on Windows XP or Solaris 9, you'll still need to hunt down a machine, but not as a regular everyday occurance.

I'm a big fan of a regular nightly build and test system, constantly verifying that a source base builds and tests out. There are many examples of automated build/tests, some that trigger on any change to the source base, some that just run every night. Some provide a protection gateway to the 'golden' source base which only gets changes that the nightly process has verified are good. The JPRT (and PRT) system is meant to guard the source base before anything is sent to it, guarding all source bases from the evil developer, well maybe 'evil' isn't the right word, I haven't met many 'evil' developers, more like 'error prone' developers. ;\^) Humm, come to think about it, I may be one from time to time. :\^{ But the point is that by spreading the build up over a set of machines, and getting the turnaround down to under an hour, it becomes realistic to completely build on all platforms and test it, on every putback. We have the technology, we can build and rebuild and rebuild, and it will be better than it was before, ha ha... Anybody remember the Six Million Dollar Man? Man, I gotta get out more often.. Anyway, now the nightly build and test can become a 'fetch the latest JPRT build bits' and start extensive testing (the testing not done by JPRT, or the platforms not tested by JPRT).

Is it Open Source? No, not yet. Would you like to be? Let me know. Or is it more important that you have the ability to use such a system for JDK changes?

So enough blabbering on about this JPRT system, tell me what you think.
And let me know if you want to hear more about it or not.

Stay tuned for the next episode, same Bloody Bat time, same Bloody Bat channel. ;\^)


Thursday Mar 27, 2008

OpenJDK, Mercurial, and The Changeset View

Why do I have to create a "Merge" changeset when there was nothing to merge?

For most of us old TeamWare users, and maybe other SCM users, the need for all the Mercurial "Merge" changesets (or as some people politely refer to as 'merge turds') seems confusing and messy. If the changes don't involve the same file, it can be hard to understand why you need a Merge changeset.

What did TeamWare look like?

In TeamWare a 'resolve' was necessary only when there was a conflict, meaning that two people changed the same file. The tool 'filemerge' provided a way to easily deal with each file conflict, but merging changes is and will always be a risky business. Everyone has had an experience with a 'bad merge', they are nasty problems. No Source Code Management (SCM) tool completely removes the need for merging, and our only hope is for the merging tools to help us out here. It is probably true that a Distributed SCM like TeamWare, Mercurial, or git may create the need for more frequent merging, but the end result is often the same as a non-Distributed SCM, so maybe with a DSCM the merge work is also distributed? Anyway, I digress.

With TeamWare, the 'resolve' action resulted in multiple revisions in each SCCS file that had a conflict. The TeamWare tool 'vertool' vertool provided a way to pick an SCCS file and view it's revision history. Again, this was on a per-file basis, and although that created some benefits for developers, like being able to 'putback' just one file change, it also made it a little difficult to record the true state of the entire workspace. Here is a snapshot of vertool in action for anyone that hasn't seen it:

Notice the SCCS revision graph, when conflicts happened, the graph gets a little more complicated, but unless the changes are abandoned, it always connects back up to the main trunk of the graph. With TeamWare, every file was controlled with SCCS, and every file had a graph. The connections between files was never formally managed by TeamWare, but TeamWare provided some tools like 'freezept' to allow you to try and manage it.

And with Mercurial ...

The changes come in changesets or grouped changes to files, which are treated and tracked as changes to the repository. Yes, the changes are made to specific files, but the revision tracking is done for the entire repository. When a merge situation in Mercurial happens, and they will be frequent, a new changeset has to be created to potentially carry any file merge changes, but most importantly to identify the merged or joined results of two changesets. All changesets have at least one parent changeset, but Merge changesets have two parent changesets. Everytime you do an 'hg pull' that adds new changesets in your repository, and your repository has changesets that have not been pushed yet, you have created what is called a 'multiple head' situation and you will need a Merge changeset. A 'head' is a changeset with no descendants, the tip changeset is a head and must be the only head if you want to push your changesets to the OpenJDK repositories (we do not allow any multiple head pushes with the OpenJDK repositories). This unfortunately means that people that do frequent "syncs" with their parent repository may be creating many Merge changesets, that's just the way it is, like Taxes, we will need to learn to live with it.

The 'hg view' command of Mercurial can provide some insight into this Merge business. To use 'hg view' you need to:

  1. Enable the hgk extension in your ~/.hgrc file.
  2. Make sure that the hgk tool in available from your PATH environment variable setting. You may need to download the Mercurial source bundle that matches the version of Mercurial you are using and get the hgk file from the contrib directory.
  3. Make sure the wish tool is available from your PATH environment variable setting. Note that Solaris Express has a /usr/bin/wish that works, and the MacOS 10.5 has a /usr/bin/wish that works, but you may need to do a little searching to find a wish that is acceptable to hgk. Solaris 10 and older machines may have one at /opt/sfw/bin/wishx or /usr/sfw/bin/wisk8.[34]

For example, to see what the most recent changesets pushed to the OpenJDK jdk7/jdk repository look like:

   hg clone yourjdk
   cd yourjdk
   hg view

You should then see something like this:

Looks a little like a Public Transportation System. Notice the groups of changesets created by developers, usually generated one right after the other. If two developers manage to line up (luck), the sequence is simple, but the second one to do a push had to do a pull and create a merge changeset. Layering on top of that is the integrations of the various teams to the master repository, which should appear as major addition to the graph.

Since a changeset is a repository revision this has tremendous benefits. For example, anyone can re-create the state of a repository (all the files) as of any changeset by simply doing:

   hg clone -r 82c85cfd8402 yourjdk trimmedjdk

Creating a separate repository that represents the state of everything as of that specific changeset id (which happens to be a changeset I created, specifically

I hope this has been helpful to at least a few people. Send me comments if I can clarify this more for people.

For more on the hgk extension go to Thanks to Chris Mason for creating this great extension.


Friday Nov 16, 2007

Transitioning from TeamWare to Mercurial

Just thought I would pull together some basic guidelines for anyone transitioning from TeamWare workspaces to Mercurial repositories.

I'm assuming that multiple branches will not be maintained in the Mercurial repositories in this information. Mercurial will allow you to maintain multiple branches of development in a single repository. It's my opinion that multiple branches of development can be done safer and more reliably by just have a separate clone for the separate branches. Anyone experiencing this feature in SCCS files where separate revision trees can be maintained on a file will probably agree.

TeamWare Basics

Skip this section if you are a regular TeamWare user.

A TeamWare workspace consists of a directory of files under SCCS control, each file is managed individually. Throughout the TeamWare workspace are directories called SCCS contain s.filename files which contain the original file as it was first entered into the SCCS directory, plus deltas to convert that original file to the various increasing numeric revisions of the file. A read-only file is kept in the parent directory of the SCCS directory, edits to the file requires you to use the 'sccs edit filename' command. A 'sccs delget filename' command is used to define a new revision of the file. Each revision of a file can contain a comment. SCCS manages source files and revisions to source files. TeamWare manages batches of SCCS files. TeamWare features and SCCS features get blurred sometimes, but SCCS can and is often used independent from TeamWare.

TeamWare allows for any number of workspaces with a child/parent relationship and also has a very nice code merging tool called filemerge. TeamWare allowed you to have partial workspaces. The top of the workspace contains a Codemgr_wsdata directory that holds various TeamWare book-keeping files. It's drawbacks besides not being open source are around performance and the lack of features like changesets and revision markings (tags). Over the years the short-comings have been somewhat corrected with various scripts and tools written by various teams.

Mercurial For TeamWare Users

Mercurial (like TeamWare) allows you to have any number of repositories (assume repository==workspace) and allows you to access repositories via NFS paths or with ssh:// or even http:// paths.

In many ways, at a high functional level, your Mercurial experience will be similar to the experiences you have had with TeamWare, but the details are vastly different, especially if you have become dependent on the specific format of a TeamWare workspace or the contents of the SCCS files.

At the very top of the Mercurial repository is a hidden directory called .hg which holds the Mercurial book-keeping files plus a secure set of all "commited" sources and changes to those sources.

Unlike TeamWare, where the visible source files were read-only until you explicitly used 'sccs edit' to explicitly edit them, the Mercurial "working set" sources are all read-write, and you are free to edit these files at any time. So by default you will have a working set of read-write sources and the more permanent committed files that are saved in your .hg directory.

With Mercurial, all changes to a repository are done with via "changesets", which are originally created with an 'hg commit' somewhere along the line. An ideal changeset would be all the file changes/renames/deletes/adds for one particular bug, but a changeset can be small or very large. New files, deleted files, and renamed files must all be done via a changeset. You use 'hg commit' to commit file changes into a "changeset" in your own repository, it doesn't go anywhere unless someone pulls it from your repository, or you push the changeset somewhere. The 'hg pull' is like the TeamWare bringover command, and 'hg push' is like the TeamWare putback command, well sort of. Both the 'hg push' and 'hg pull' push or pull "changesets" or committed changes to and from the .hg directories of a repositories. So your working set files are NOT automatically updated when the files in the .hg directory changes (where changesets are kept), you must explicitly run 'hg update' to update your working set files. And it's important to note that with Mercurial you do not "pull or push files" but the changesets or changes to the entire repository. This is very different from TeamWare which manages SCCS files, where you could bringover or putback individual files.

The changeset concept is like a repository wide SCCS revision number, one changeset id defines the state of the entire repository. A changeset that has no children changesets is called a "head", and there should only be one head, which is also called the tip. But when you do a pull, you often end up with multiple "head" changesets, and the gola is to perform an 'hg merge' and 'hg commit' a new "merge" changeset that will become the single "head" or "tip". Regardless of any specific file changes that might be conflicting, a merge changeset will always be needed to get back to one "head".

Roughly Equivalent Command Mappings

NOTE: Optimally, the use of 'hg commit' should be done after all the file adds, deletes, renames, and edits are done. An ideal changeset is one that contains all the changes for a particular feature or bug fix.

Action TeamWare Mercurial
Create new workspace/repository workspace create hg init
Create a child workspace/repository bringover -p parent -w child . hg clone parent child
Add a file sccs create filename hg add filename && hg commit
Delete a file workspace filerm filename hg remove filename && hg commit
Rename a file workspace filemv filename1 filename2 hg rename filename1 filename2 && hg commit
Change a file sccs edit filename && vi filename && sccs delget filename vi filename && hg commit
Rename a workspace/repository workspace move oldpath newpath mv oldpath newpath
Delete a workspace/repository workspace delete -f path rm -f -r path
Verify workspace/repository workspace check hg verify
Pull changes from another workspace/repository bringover -p parentpath . && resolve hg pull parentpath && hg update && hg merge && hg commit
Push changes to another workspace/repository putback -p parentpath . hg push parentpath
Get the name of the workspace/repository workspace name hg root
Get the name of the default parent workspace/repository workspace parent hg paths
Resolve merge conflicts resolve hg merge
Freshen the working set files sccs get filename hg update filename
Check for incoming changes bringover -n -p parentpath . hg incoming parentpath
Check on outgoing changes putback -n -p parentpath . hg outgoing parentpath
Details on changes to a single file sccs prs filename || sccs prt filename hg log -v filename && hg annotate filename
Viewing file changes sccs diff filename hg diff filename
Undo file changes sccs unedit filename hg revert filename
Listing uncommited edited files workspace find -c -OR- sccs tell hg status
Listing all managed files workspace updatenames && cat Codemgr_wsdata/nametable | sort | cut -d' ' -f1 hg locate
List a source file annotated with the revisions Various custom scripts hg annotate filename

Using Mercurial Example

Here is a simple example of pulling a repository and making a changeset.

In this example /export2/build_integration/repos/control is a path to a amsll Mercurial repository.

    # # Step 1: Check hg version # Make sure you have access to the right Mercurial. <1> rm -f -r ${HOME}/MercurialExercises/Exercise1 <2> mkdir -p ${HOME}/MercurialExercises/Exercise1/temp <3> hg version Mercurial Distributed SCM (version 0.9.3) Copyright (C) 2005, 2006 Matt Mackall This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # Step 2: Clone a test control repository # Creates your own private repository to play with (may take a few minutes) <9> rm -f -r ${HOME}/MercurialExercises/Exercise1/your_repos <10> mkdir -p ${HOME}/MercurialExercises/Exercise1/your_repos <11> cd ${HOME}/MercurialExercises/Exercise1/your_repos <12> hg clone /export2/build_integration/repos/control control requesting all changes adding changesets adding manifests adding file changes added 1 changesets with 24 changes to 24 files 24 files updated, 0 files merged, 0 files removed, 0 files unresolved # # Step 3: Private clone of the above repository for testing push/pull. # Clone a second private repository. <13> cd ${HOME}/MercurialExercises/Exercise1/your_repos <14> hg clone control control-work 24 files updated, 0 files merged, 0 files removed, 0 files unresolved # # Exercise 1: Step 4: Pretend like something changed and do a 'pull' # Pulling changes now will say 'no changes'. <15> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control <16> hg pull pulling from /export2/build_integration/repos/control searching for changes no changes found <17> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control-work <18> hg pull pulling from ${HOME}/MercurialExercises/Exercise1/your_repos/control searching for changes no changes found # # Step 5: Make a change in your work area and push it # Demonstrates a simple file change, commit and push of a change. <19> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control-work <20> echo '#harmless' >> make/Makefile <21> hg status M make/Makefile <22> hg diff diff -r 48e79d6618ee make/Makefile --- a/make/Makefile Sun Sep 30 17:55:14 2007 -0700 +++ b/make/Makefile Thu Oct 04 19:08:49 2007 -0700 @@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen .PHONY: all build what clobber insane freshen \\ fastdebug_build debug_build product_build setup +#harmless <23> hg commit -m "9999999: Fixed world peace" <24> hg outgoing searching for changes changeset: 1:0d317da1a3c4 tag: tip user: ${USER} date: Thu Oct 04 19:08:53 2007 -0700 summary: 9999999: Fixed world peace <25> hg push pushing to ${HOME}/MercurialExercises/Exercise1/your_repos/control searching for changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files # # Step 6: Inspect the push (watch the working set files) <26> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control <27> tail -1 make/Makefile <28> hg status <29> hg update 1 files updated, 0 files merged, 0 files removed, 0 files unresolved <30> tail -1 make/Makefile #harmless

Using TeamWare and webrev to import a changeset

The tool webrev creates a set of web pages than can be used to browse code changes, but more recent versions create patch files (looks like "diff -r -u") that can be fed into gpatch (GNU patch) or a similar tool to apply the changes.

In this example /export2/build_integration/ws7/control is a path to a TeamWare integration workspace and /export2/build_integration/repos/control is a path to an equivalent Mercurial repository.

    # # Step 1: Bringover a test control workspace
    #   Creates your own private workspace (may take a few minutes)
    <1> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_ws
    <2> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_ws
    <3> cd ${HOME}/MercurialExercises/Exercise2/your_ws
    <4> bringover -q -p /export2/build_integration/ws7/control -w control .
    Parent workspace: /export2/build_integration/ws7/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Examined files: 55
    Bringing over contents changes: 55
    Examined files: 55
    Contents Summary:
          55   create
    # # Step 2: Private child workspace of the above workspace to hold changes.
    <5> cd ${HOME}/MercurialExercises/Exercise2/your_ws
    <6> bringover -q -p control -w control-work .
    Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Examined files: 55
    Bringing over contents changes: 55
    Examined files: 55
    Contents Summary:
          55   create
    # # Step 3: Make changes in the child workspace
    <7> cd ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/make
    <8> sccs edit Makefile
    new delta 1.314
    444 lines
    <9> echo '#harmless' >> Makefile
    <10> sccs delget -y'9999999: Fixed world peace' Makefile
    No id keywords (cm7)
    1 inserted
    0 deleted
    444 unchanged
    No id keywords (cm7)
    445 lines
    <11> cd ..
    <12> putback -n .
    Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Examined files: 55
    Would put back contents changes: 1
    update: make/Makefile
    Examined files: 55
    Contents Summary:
           1   update
          54   no action (unchanged)
    No changes were put back
    <13> webrev -l .
       SCM detected: teamware
     File list from: 'putback -n  .' ...  Done.
          Workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Compare against: ${HOME}/MercurialExercises/Exercise2/your_ws/control
          Output to: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev
       Output Files:
    		 patch cdiffs udiffs sdiffs frames old new
     Generating PDF: Skipped: no output available
         index.html: Done.
    <14> cat webrev/control-work.patch
    --- old/make/Makefile	Thu Oct  4 19:09:19 2007
    +++ new/make/Makefile	Thu Oct  4 19:09:19 2007
    @@ -442,3 +442,4 @@
     .PHONY: all build what clobber insane freshen \\
     	fastdebug_build debug_build product_build setup
    # # Step 4: Clone a test control repository
    #   Creates your own private repository to play with (may take a few minutes)
    <15> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_repos
    <16> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_repos
    <17> cd ${HOME}/MercurialExercises/Exercise2/your_repos
    <18> hg clone /export2/build_integration/repos/control control
    requesting all changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 24 changes to 24 files
    24 files updated, 0 files merged, 0 files removed, 0 files unresolved
    <19> hg clone control control-work
    24 files updated, 0 files merged, 0 files removed, 0 files unresolved
    # # Step 5: Import the patch into the repository
    <20> cd ${HOME}/MercurialExercises/Exercise2/your_repos/control-work
    <21> gpatch -u -p1 < ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev/control-work.patch
    patching file make/Makefile
    <22> hg status
    M make/Makefile
    <23> hg diff
    diff -r 48e79d6618ee make/Makefile
    --- a/make/Makefile	Sun Sep 30 17:55:14 2007 -0700
    +++ b/make/Makefile	Thu Oct 04 19:09:32 2007 -0700
    @@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen
     .PHONY: all build what clobber insane freshen \\
     	fastdebug_build debug_build product_build setup
    <24> hg commit -m "9999999: Fixed world peace"
    <25> hg outgoing
    searching for changes
    changeset:   1:23e0962ced6d
    tag:         tip
    user:        ${USER}
    date:        Thu Oct 04 19:09:36 2007 -0700
    summary:     9999999: Fixed world peace
    <26> hg push
    pushing to ${HOME}/MercurialExercises/Exercise2/your_repos/control
    searching for changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 1 changes to 1 files

Beginner Gotchas for TeamWare Users

Not setting up your ~/.hgrc file. The name you define in ~/.hgrc with "[ui]" and "username=" is the name that will be permanently recorded in the changesets you create with 'hg commit'. I don't recommend adding your email address in username, but that's up to you, just keep in mind it will be public information when your changesets reach a public repository. TeamWare/SCCS used your system username, but very few TeamWare workspaces were ever made public.
Forgot the 'hg update' After an 'hg pull' (aka bringover), don't forget the 'hg update', or use 'hg pull -u'. The default pull and push just updates the changesets and doesn't update your read-write working set of files. You need to be careful about updating the working set files on shared repositories they could get updated while others are viewing them.
Forgot to merge After an 'hg pull', you need to 'hg update', and if you have changesets that you have not committed you will also need to 'hg merge' and 'hg commit'. If you forget you will end up with multiple heads and a more difficult time merging later.
Forgot to commit after a merge (multiple heads) After 'hg merge' you need to 'hg commit'. The merge just prepares you for the 'hg commit' of a merge changeset. If you forget you will end up with multiple heads and a more difficult time merging later.
Making accidental edits Mercurial working set files are always read-write and ready to edit, no 'sccs edit' action is necessary. Use 'hg status' to monitor what files you have changed.
Using the wrong relative path File paths supplied to 'hg' commands are relative to the current directory, the TeamWare bringover and putback commands want paths relative to the root of the workspace, regardless of the current directory.
Not defining the file .hgignore for 'hg status' The 'hg status' command tells you what outstanding changes you have in your working set, by default it looks in '.' or the entire directory, but if there are files created during a build, you want 'hg status' to ignore those files. Make sure you define the .hgignore file so that 'hg status' will only find files in the directories you want managed by the repository. TeamWare never really helped with the problem of forgetting to 'sccs create' your files, 'hg status' solves this common problem.
Using NFS/UFS for team integration areas TeamWare for the most part was designed around sharing data via NFS or UFS file systems. Mercurial can work the same way, but when using it for team integration areas we recommend the use of the ssh:// parent path mechanisms described in the Mercurial Book . Unless everyone in the team or group is in the same Unix group, have the same default group, and all use 'umask 2', using NFS/UFS will be problematic. Mercurial obeys the strict Unix rules of file creation and permissions, and over time TeamWare has adjusted itself (perhaps improperly) to avoid the file permission issues you can see with Mercurial.
Too quick on the 'hg commit' Once a changeset is created (the 'hg commit'), and pushed, it's pretty permanent. Make sure that before the 'hg commit' happens that the changes are correct, reviewed, the right ones, and complete, otherwise you'll be creating yet another one to correct your mistakes.
Doing a push with outstanding working set changes The 'hg push' will not detect any outstanding changes to your working set, it just pushes the existing changesets. ALWAYS use 'hg status' before an 'hg push' to make sure you have created all your changesets with 'hg commit', unless of course you have changes you don't want to push.
Committing a sensitive file Accidental additions of sensitive source files can be a big problem. Completely removing a sensitive file that has been accidently added to a repository can be a real problem. be very careful what files you add to a repository! Adding non-open source files to an 'open source' repository will inflict major pain on many people.
Doing anything to the .hg files Don't mess with the .hg data files, if you do you are INSANE, leave that to the Mercurial professionals. If you suspect they have been corrupted, use 'hg verify' to check. Backups are always important, so make sure you keep a relatively recent backup repository. If you can't 'hg rollback', save the repository somewhere, clone a fresh copy from your parent, remove the working set files completely from the clone, and copy in the working set from your corrupted repository (but not the .hg files). Now you can use the standard 'hg status' and 'hg diff' to see what file changes you may have lost and adjust.
Using SCCS keywords Mercurial by default does not support anything like SCCS keywords in files. You should remove these or find another solution.
Looking for putback comments or history files Changeset comments represent BOTH the SCCS comments and the effective TeamWare putback comment.
Using problematic filenames Watch out for directory and filenames that only differ in case (e.g. test and Test), at least on the Mac and Windows these can be troublesome. Long pathnames (>255 characters) can also be a problem.

Converting a TeamWare Workspace to a Mercurial Repository

Converting a TeamWare workspace to a Mercurial repository (without history) is pretty trivial:

       bringover -p your_workspace -w /tmp/repo .
       cd /tmp/repo
       workspace parent -u
       rm -f -r Codemgr_wsdata
       rm -f -r deleted_files
       foreach i ( `find . -name SCCS` )
          ( cd $i/.. && sccs edit SCCS )
          rm -f -r $i
       hg init
       hg add

A simple source tree can be turned into a Mercurial repository with just hg init; hg add. Turning a TeamWare workspace into a plain source tree is relatively simple too, I just create a separate workspace, purge a few files, make sure all the sources are in 'edit' mode, and remove the SCCS directories.

Performance Comparisons and Data

Nothing but good news in this area, for both time and space.

Many of the past tricks used to speed up TeamWare bringovers and putbacks, especially over slow connections should not be necessary with Mercurial, it is very fast. The initial 'hg clone' of a repository should be considerably faster, but the most important actions of 'hg pull' or 'hg push' will be so much faster you may question if the action actually happened. Unlike TeamWare, only the changesets are transported, and many fewer files are accessed and in a more efficient manner.

The size of the repositories should also be smaller (at least 50% smaller) than the equivalent TeamWare workspace, this isn't surprising due to the lack of compression and age of SCCS file.


Various blogs on JDK development procedures, including building, build infrastructure, testing, and source maintenance.


« July 2016