Transitioning from TeamWare to Mercurial

Just thought I would pull together some basic guidelines for anyone transitioning from TeamWare workspaces to Mercurial repositories.

I'm assuming that multiple branches will not be maintained in the Mercurial repositories in this information. Mercurial will allow you to maintain multiple branches of development in a single repository. It's my opinion that multiple branches of development can be done safer and more reliably by just have a separate clone for the separate branches. Anyone experiencing this feature in SCCS files where separate revision trees can be maintained on a file will probably agree.

TeamWare Basics

Skip this section if you are a regular TeamWare user.

A TeamWare workspace consists of a directory of files under SCCS control, each file is managed individually. Throughout the TeamWare workspace are directories called SCCS contain s.filename files which contain the original file as it was first entered into the SCCS directory, plus deltas to convert that original file to the various increasing numeric revisions of the file. A read-only file is kept in the parent directory of the SCCS directory, edits to the file requires you to use the 'sccs edit filename' command. A 'sccs delget filename' command is used to define a new revision of the file. Each revision of a file can contain a comment. SCCS manages source files and revisions to source files. TeamWare manages batches of SCCS files. TeamWare features and SCCS features get blurred sometimes, but SCCS can and is often used independent from TeamWare.

TeamWare allows for any number of workspaces with a child/parent relationship and also has a very nice code merging tool called filemerge. TeamWare allowed you to have partial workspaces. The top of the workspace contains a Codemgr_wsdata directory that holds various TeamWare book-keeping files. It's drawbacks besides not being open source are around performance and the lack of features like changesets and revision markings (tags). Over the years the short-comings have been somewhat corrected with various scripts and tools written by various teams.

Mercurial For TeamWare Users

Mercurial (like TeamWare) allows you to have any number of repositories (assume repository==workspace) and allows you to access repositories via NFS paths or with ssh:// or even http:// paths.

In many ways, at a high functional level, your Mercurial experience will be similar to the experiences you have had with TeamWare, but the details are vastly different, especially if you have become dependent on the specific format of a TeamWare workspace or the contents of the SCCS files.

At the very top of the Mercurial repository is a hidden directory called .hg which holds the Mercurial book-keeping files plus a secure set of all "commited" sources and changes to those sources.

Unlike TeamWare, where the visible source files were read-only until you explicitly used 'sccs edit' to explicitly edit them, the Mercurial "working set" sources are all read-write, and you are free to edit these files at any time. So by default you will have a working set of read-write sources and the more permanent committed files that are saved in your .hg directory.

With Mercurial, all changes to a repository are done with via "changesets", which are originally created with an 'hg commit' somewhere along the line. An ideal changeset would be all the file changes/renames/deletes/adds for one particular bug, but a changeset can be small or very large. New files, deleted files, and renamed files must all be done via a changeset. You use 'hg commit' to commit file changes into a "changeset" in your own repository, it doesn't go anywhere unless someone pulls it from your repository, or you push the changeset somewhere. The 'hg pull' is like the TeamWare bringover command, and 'hg push' is like the TeamWare putback command, well sort of. Both the 'hg push' and 'hg pull' push or pull "changesets" or committed changes to and from the .hg directories of a repositories. So your working set files are NOT automatically updated when the files in the .hg directory changes (where changesets are kept), you must explicitly run 'hg update' to update your working set files. And it's important to note that with Mercurial you do not "pull or push files" but the changesets or changes to the entire repository. This is very different from TeamWare which manages SCCS files, where you could bringover or putback individual files.

The changeset concept is like a repository wide SCCS revision number, one changeset id defines the state of the entire repository. A changeset that has no children changesets is called a "head", and there should only be one head, which is also called the tip. But when you do a pull, you often end up with multiple "head" changesets, and the gola is to perform an 'hg merge' and 'hg commit' a new "merge" changeset that will become the single "head" or "tip". Regardless of any specific file changes that might be conflicting, a merge changeset will always be needed to get back to one "head".

Roughly Equivalent Command Mappings

NOTE: Optimally, the use of 'hg commit' should be done after all the file adds, deletes, renames, and edits are done. An ideal changeset is one that contains all the changes for a particular feature or bug fix.

Action TeamWare Mercurial
Create new workspace/repository workspace create hg init
Create a child workspace/repository bringover -p parent -w child . hg clone parent child
Add a file sccs create filename hg add filename && hg commit
Delete a file workspace filerm filename hg remove filename && hg commit
Rename a file workspace filemv filename1 filename2 hg rename filename1 filename2 && hg commit
Change a file sccs edit filename && vi filename && sccs delget filename vi filename && hg commit
Rename a workspace/repository workspace move oldpath newpath mv oldpath newpath
Delete a workspace/repository workspace delete -f path rm -f -r path
Verify workspace/repository workspace check hg verify
Pull changes from another workspace/repository bringover -p parentpath . && resolve hg pull parentpath && hg update && hg merge && hg commit
Push changes to another workspace/repository putback -p parentpath . hg push parentpath
Get the name of the workspace/repository workspace name hg root
Get the name of the default parent workspace/repository workspace parent hg paths
Resolve merge conflicts resolve hg merge
Freshen the working set files sccs get filename hg update filename
Check for incoming changes bringover -n -p parentpath . hg incoming parentpath
Check on outgoing changes putback -n -p parentpath . hg outgoing parentpath
Details on changes to a single file sccs prs filename || sccs prt filename hg log -v filename && hg annotate filename
Viewing file changes sccs diff filename hg diff filename
Undo file changes sccs unedit filename hg revert filename
Listing uncommited edited files workspace find -c -OR- sccs tell hg status
Listing all managed files workspace updatenames && cat Codemgr_wsdata/nametable | sort | cut -d' ' -f1 hg locate
List a source file annotated with the revisions Various custom scripts hg annotate filename

Using Mercurial Example

Here is a simple example of pulling a repository and making a changeset.

In this example /export2/build_integration/repos/control is a path to a amsll Mercurial repository.

    # # Step 1: Check hg version # Make sure you have access to the right Mercurial. <1> rm -f -r ${HOME}/MercurialExercises/Exercise1 <2> mkdir -p ${HOME}/MercurialExercises/Exercise1/temp <3> hg version Mercurial Distributed SCM (version 0.9.3) Copyright (C) 2005, 2006 Matt Mackall This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # Step 2: Clone a test control repository # Creates your own private repository to play with (may take a few minutes) <9> rm -f -r ${HOME}/MercurialExercises/Exercise1/your_repos <10> mkdir -p ${HOME}/MercurialExercises/Exercise1/your_repos <11> cd ${HOME}/MercurialExercises/Exercise1/your_repos <12> hg clone /export2/build_integration/repos/control control requesting all changes adding changesets adding manifests adding file changes added 1 changesets with 24 changes to 24 files 24 files updated, 0 files merged, 0 files removed, 0 files unresolved # # Step 3: Private clone of the above repository for testing push/pull. # Clone a second private repository. <13> cd ${HOME}/MercurialExercises/Exercise1/your_repos <14> hg clone control control-work 24 files updated, 0 files merged, 0 files removed, 0 files unresolved # # Exercise 1: Step 4: Pretend like something changed and do a 'pull' # Pulling changes now will say 'no changes'. <15> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control <16> hg pull pulling from /export2/build_integration/repos/control searching for changes no changes found <17> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control-work <18> hg pull pulling from ${HOME}/MercurialExercises/Exercise1/your_repos/control searching for changes no changes found # # Step 5: Make a change in your work area and push it # Demonstrates a simple file change, commit and push of a change. <19> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control-work <20> echo '#harmless' >> make/Makefile <21> hg status M make/Makefile <22> hg diff diff -r 48e79d6618ee make/Makefile --- a/make/Makefile Sun Sep 30 17:55:14 2007 -0700 +++ b/make/Makefile Thu Oct 04 19:08:49 2007 -0700 @@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen .PHONY: all build what clobber insane freshen \\ fastdebug_build debug_build product_build setup +#harmless <23> hg commit -m "9999999: Fixed world peace" <24> hg outgoing searching for changes changeset: 1:0d317da1a3c4 tag: tip user: ${USER} date: Thu Oct 04 19:08:53 2007 -0700 summary: 9999999: Fixed world peace <25> hg push pushing to ${HOME}/MercurialExercises/Exercise1/your_repos/control searching for changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files # # Step 6: Inspect the push (watch the working set files) <26> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control <27> tail -1 make/Makefile <28> hg status <29> hg update 1 files updated, 0 files merged, 0 files removed, 0 files unresolved <30> tail -1 make/Makefile #harmless

Using TeamWare and webrev to import a changeset

The tool webrev creates a set of web pages than can be used to browse code changes, but more recent versions create patch files (looks like "diff -r -u") that can be fed into gpatch (GNU patch) or a similar tool to apply the changes.

In this example /export2/build_integration/ws7/control is a path to a TeamWare integration workspace and /export2/build_integration/repos/control is a path to an equivalent Mercurial repository.

    # # Step 1: Bringover a test control workspace
    #   Creates your own private workspace (may take a few minutes)
     
    <1> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_ws
    <2> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_ws
    <3> cd ${HOME}/MercurialExercises/Exercise2/your_ws
    <4> bringover -q -p /export2/build_integration/ws7/control -w control .
    Parent workspace: /export2/build_integration/ws7/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control
    
    Examined files: 55
    
    Bringing over contents changes: 55
    
    Examined files: 55
    
    Contents Summary:
          55   create
     
    # # Step 2: Private child workspace of the above workspace to hold changes.
     
    <5> cd ${HOME}/MercurialExercises/Exercise2/your_ws
    <6> bringover -q -p control -w control-work .
    Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    
    Examined files: 55
    
    Bringing over contents changes: 55
    
    Examined files: 55
    
    Contents Summary:
          55   create
     
    # # Step 3: Make changes in the child workspace
     
    <7> cd ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/make
    <8> sccs edit Makefile
    1.313
    new delta 1.314
    444 lines
    <9> echo '#harmless' >> Makefile
    <10> sccs delget -y'9999999: Fixed world peace' Makefile
    No id keywords (cm7)
    1.314
    1 inserted
    0 deleted
    444 unchanged
    1.314
    No id keywords (cm7)
    445 lines
    <11> cd ..
    <12> putback -n .
    Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    
    Examined files: 55
    
    Would put back contents changes: 1
    
    update: make/Makefile
    
    Examined files: 55
    
    Contents Summary:
           1   update
          54   no action (unchanged)
    
    No changes were put back
    <13> webrev -l .
       SCM detected: teamware
     File list from: 'putback -n  .' ...  Done.
          Workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Compare against: ${HOME}/MercurialExercises/Exercise2/your_ws/control
          Output to: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev
       Output Files:
    	make/Makefile
    		 patch cdiffs udiffs sdiffs frames old new
     Generating PDF: Skipped: no output available
         index.html: Done.
    <14> cat webrev/control-work.patch
    --- old/make/Makefile	Thu Oct  4 19:09:19 2007
    +++ new/make/Makefile	Thu Oct  4 19:09:19 2007
    @@ -442,3 +442,4 @@
     .PHONY: all build what clobber insane freshen \\
     	fastdebug_build debug_build product_build setup
     
    +#harmless
     
    # # Step 4: Clone a test control repository
    #   Creates your own private repository to play with (may take a few minutes)
     
    <15> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_repos
    <16> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_repos
    <17> cd ${HOME}/MercurialExercises/Exercise2/your_repos
    <18> hg clone /export2/build_integration/repos/control control
    requesting all changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 24 changes to 24 files
    24 files updated, 0 files merged, 0 files removed, 0 files unresolved
    <19> hg clone control control-work
    24 files updated, 0 files merged, 0 files removed, 0 files unresolved
     
    # # Step 5: Import the patch into the repository
     
    <20> cd ${HOME}/MercurialExercises/Exercise2/your_repos/control-work
    <21> gpatch -u -p1 < ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev/control-work.patch
    patching file make/Makefile
    <22> hg status
    M make/Makefile
    <23> hg diff
    diff -r 48e79d6618ee make/Makefile
    --- a/make/Makefile	Sun Sep 30 17:55:14 2007 -0700
    +++ b/make/Makefile	Thu Oct 04 19:09:32 2007 -0700
    @@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen
     .PHONY: all build what clobber insane freshen \\
     	fastdebug_build debug_build product_build setup
     
    +#harmless
    <24> hg commit -m "9999999: Fixed world peace"
    <25> hg outgoing
    searching for changes
    changeset:   1:23e0962ced6d
    tag:         tip
    user:        ${USER}
    date:        Thu Oct 04 19:09:36 2007 -0700
    summary:     9999999: Fixed world peace
    
    <26> hg push
    pushing to ${HOME}/MercurialExercises/Exercise2/your_repos/control
    searching for changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 1 changes to 1 files
    
              

Beginner Gotchas for TeamWare Users

Gotcha
Why?
Not setting up your ~/.hgrc file. The name you define in ~/.hgrc with "[ui]" and "username=" is the name that will be permanently recorded in the changesets you create with 'hg commit'. I don't recommend adding your email address in username, but that's up to you, just keep in mind it will be public information when your changesets reach a public repository. TeamWare/SCCS used your system username, but very few TeamWare workspaces were ever made public.
Forgot the 'hg update' After an 'hg pull' (aka bringover), don't forget the 'hg update', or use 'hg pull -u'. The default pull and push just updates the changesets and doesn't update your read-write working set of files. You need to be careful about updating the working set files on shared repositories they could get updated while others are viewing them.
Forgot to merge After an 'hg pull', you need to 'hg update', and if you have changesets that you have not committed you will also need to 'hg merge' and 'hg commit'. If you forget you will end up with multiple heads and a more difficult time merging later.
Forgot to commit after a merge (multiple heads) After 'hg merge' you need to 'hg commit'. The merge just prepares you for the 'hg commit' of a merge changeset. If you forget you will end up with multiple heads and a more difficult time merging later.
Making accidental edits Mercurial working set files are always read-write and ready to edit, no 'sccs edit' action is necessary. Use 'hg status' to monitor what files you have changed.
Using the wrong relative path File paths supplied to 'hg' commands are relative to the current directory, the TeamWare bringover and putback commands want paths relative to the root of the workspace, regardless of the current directory.
Not defining the file .hgignore for 'hg status' The 'hg status' command tells you what outstanding changes you have in your working set, by default it looks in '.' or the entire directory, but if there are files created during a build, you want 'hg status' to ignore those files. Make sure you define the .hgignore file so that 'hg status' will only find files in the directories you want managed by the repository. TeamWare never really helped with the problem of forgetting to 'sccs create' your files, 'hg status' solves this common problem.
Using NFS/UFS for team integration areas TeamWare for the most part was designed around sharing data via NFS or UFS file systems. Mercurial can work the same way, but when using it for team integration areas we recommend the use of the ssh:// parent path mechanisms described in the Mercurial Book . Unless everyone in the team or group is in the same Unix group, have the same default group, and all use 'umask 2', using NFS/UFS will be problematic. Mercurial obeys the strict Unix rules of file creation and permissions, and over time TeamWare has adjusted itself (perhaps improperly) to avoid the file permission issues you can see with Mercurial.
Too quick on the 'hg commit' Once a changeset is created (the 'hg commit'), and pushed, it's pretty permanent. Make sure that before the 'hg commit' happens that the changes are correct, reviewed, the right ones, and complete, otherwise you'll be creating yet another one to correct your mistakes.
Doing a push with outstanding working set changes The 'hg push' will not detect any outstanding changes to your working set, it just pushes the existing changesets. ALWAYS use 'hg status' before an 'hg push' to make sure you have created all your changesets with 'hg commit', unless of course you have changes you don't want to push.
Committing a sensitive file Accidental additions of sensitive source files can be a big problem. Completely removing a sensitive file that has been accidently added to a repository can be a real problem. be very careful what files you add to a repository! Adding non-open source files to an 'open source' repository will inflict major pain on many people.
Doing anything to the .hg files Don't mess with the .hg data files, if you do you are INSANE, leave that to the Mercurial professionals. If you suspect they have been corrupted, use 'hg verify' to check. Backups are always important, so make sure you keep a relatively recent backup repository. If you can't 'hg rollback', save the repository somewhere, clone a fresh copy from your parent, remove the working set files completely from the clone, and copy in the working set from your corrupted repository (but not the .hg files). Now you can use the standard 'hg status' and 'hg diff' to see what file changes you may have lost and adjust.
Using SCCS keywords Mercurial by default does not support anything like SCCS keywords in files. You should remove these or find another solution.
Looking for putback comments or history files Changeset comments represent BOTH the SCCS comments and the effective TeamWare putback comment.
Using problematic filenames Watch out for directory and filenames that only differ in case (e.g. test and Test), at least on the Mac and Windows these can be troublesome. Long pathnames (>255 characters) can also be a problem.

Converting a TeamWare Workspace to a Mercurial Repository

Converting a TeamWare workspace to a Mercurial repository (without history) is pretty trivial:

    
       bringover -p your_workspace -w /tmp/repo .
       cd /tmp/repo
       workspace parent -u
       rm -f -r Codemgr_wsdata
       rm -f -r deleted_files
       foreach i ( `find . -name SCCS` )
          ( cd $i/.. && sccs edit SCCS )
          rm -f -r $i
       done
       hg init
       hg add
    

A simple source tree can be turned into a Mercurial repository with just hg init; hg add. Turning a TeamWare workspace into a plain source tree is relatively simple too, I just create a separate workspace, purge a few files, make sure all the sources are in 'edit' mode, and remove the SCCS directories.

Performance Comparisons and Data

Nothing but good news in this area, for both time and space.

Many of the past tricks used to speed up TeamWare bringovers and putbacks, especially over slow connections should not be necessary with Mercurial, it is very fast. The initial 'hg clone' of a repository should be considerably faster, but the most important actions of 'hg pull' or 'hg push' will be so much faster you may question if the action actually happened. Unlike TeamWare, only the changesets are transported, and many fewer files are accessed and in a more efficient manner.

The size of the repositories should also be smaller (at least 50% smaller) than the equivalent TeamWare workspace, this isn't surprising due to the lack of compression and age of SCCS file.

Comments:

Post a Comment:
Comments are closed for this entry.
About

Various blogs on JDK development procedures, including building, build infrastructure, testing, and source maintenance.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today