Friday Apr 04, 2008

Clop Clop Clop

I wake up at 4AM wondering why I've woken up. I lay there with my eyes open and I hear a strange sound "clop clop clop". What is that sound? It sounds like dress shoes, no, more like high heels on the concrete walk outside the bedroom windows.

The windows and blinds are closed, then there is the sound again "clop clop clop". It has GOT to be someone walking around my backyard. Why would someone be walking around my backyard? I lay there wondering what to do. I could grab my king size MagLite flashlight and step outside in my boxers and see who is there, would that be a wise move? I should probably buy a shotgun for these kind of events, wonder if the wife would approve of that kind of purchase, I could keep it under the bed. She of course is sleeping soundly. Probably not the time to talk to her about buying a shotgun.

There it is again "clop clop clop"... then something different, "clup clup clup", a deeper sound, so I have a woman in high heels and a man in dress shoes walking around my back yard. The flashlight in boxers probably would not be a good idea, besides I'm very comfy in my warm bed right now. Should I call 911? I haven't really seen them, but they could be evil assassins, like that lady assassin on "CSI: NY" TV show. Oh geez, I should just get up and see who is in my yard, what is the big deal?

Then I hear it again, "clop clop clop" and a "clup clup clup". Ok, don't be silly, if someone is walking around my yard at 4AM, there must be a perfectly innocent reason. Maybe they are real estate agents, humm, at 4AM, nah. A neighbor looking for something? In high heels? At 4AM? I might do something like that (not in high heels) but none of my neighbors are that strange. Was I expecting a relative to show up? And the door was locked so they are walking around in high heels outside our bedroom window? Nah.

All of a sudden the sound changes... it moves, it moves so that it sounds like it is coming from inside the room. What the heck? I sit up in bed and look down at the floor below the windows, and there is the dog, licking itself, "clop clop clop ... clup clup clup".

I wonder if I could still talk the wife into the shotgun?

Sunday Mar 30, 2008

My Author List

I like to read novels, fiction mostly, favoring detective or mystery novels. Just thought I would share some opinions on a few. Keep in mind that I am no expert on writing, I just know what I like.

  • Dean Koontz is one of my very favorites. He manages to take a fairly unbelievable situation or concept and hook you line and sinker so that by the end of the book, you are scratching your head wondering how he came up with this story. His more recent Odd Thomas and Frankenstein books are fantastic. He seems to just get better and better.
  • Thomas Perry is a great read. His Jane books are about people finding new identities and I loved every one of them, and wished there were more. I've read everything he has written and jump at the chance to read more.
  • John Sanford is a good read, maybe a little creepy, but his Kidd series of novels are some of my favorite books. I do find it hard to remember which ones of the Prey novels I have read, but have probably read 80% of them.
  • Robert Ludlum, what can you say, his Bourne novels are on the big screen. Fairly complex and somewhat longer novels, but always well done. I'm sure I haven't read all of his novels, but probably half of them.
  • Clive Cussler is an adventure action novelist and I love his books, and have read them all, including the new ones he is starting to co-author with others (including his son). Some people might find this books a little too much James Bondish but he always manages to tie in some history (which I like) with some current event or concern about the environment or some kind of risk to the planet.
  • Robert B. Parker books are pure detective books, in the spirit of Raymond Chandler. Always a good read.
  • Walter Mosley is a different author. I first started reading his books when Clinton was in the White House and mentioned him as one of his favorite authors. His books are unique, and I like them, but many people may find then crude or overly explicit, so they are not for everybody. It certainly gives you a different perspective of the L.A.
  • Sue Grafton made my life easy by having her books be titled via the alphabet, e.g. "A" Is for Alibi, "B" Is for Burglar, "C" Is for Corpse, ... I was instructed by the person that introduced me to these books to try and start with "A", which did make it easier to keep track. Although centered around the same female detective, the stories are all unique and different. I enjoy all of these books.
  • Robert Crais wrote a series on a L.A. detective named Elvis which I found very entertaining. Very easy read, a bit on the macho side. One of his later non-Elvis books "Hostage" was turned into a movie, it was good book, and a good movie.
  • Erle Stanley Gardner is most known for his Perry Mason novels. I'm a huge Perry Mason fan, both the T.V. series and the books, although I've read only a handful of the books. I love the details in the books and shows about cars and the California towns (not just L.A. where Perry Mason has his business). Erle was a master at keeping you guessing about 'who dun it', but everyone always laughs when the person that really did it breaks down on the stand and confesses.
  • Tony Hillerman books are all centered around the Four Corners area of New Mexico and Arizona. His books are unique, and if you don't appreciate Navajo history and culture you may not appreciate these mysteries. I like the different setting these books provide from your standard detective novels.
  • Jack Higgins has written many books, but I've only read the Sean Dillon books. His books are a little predictable, with a bit of an Englishman slant.
  • Julia Navarro is a new author and I just finished her book "The Brotherhood of the Holy Shroud". I liked it, and it even got me fascinated with the Shroud of Turin, to the point that I'm now reading a book about the Shroud and it's history. I'm looking forward to more books from Julia.
  • J.A. Jance is a great author, and if I have missed reading any of her books I didn't mean to. Always a good read, and I look forward to all her books.
  • Lisa Scottoline are lawyer mysteries, and are pretty good books. I try and limit my books about lawyers, I prefer detectives.
  • Kyle Mills hasn't written many books, but I will read anything he writes. His books Rising Phoenix, Storming Heaven, Sphere of Influence, and Free Fall were all excellent. Unique plots and good reads.
  • Greg Iles hasn't written that many books, but the ones I have read are good. Hard to find his books in the used book stores I go to.
  • Janet Evanovich is a hilarious read. The humor and situations created in her books are always entertaining. The "One for the ...", "Two for the ...", series was very entertaining.
  • Stephen Hunter was an author I started reading a long time ago. His books center around snipers usually, and in some way gaining justice. I've read all his books, and wished he would write more.
  • Michael Connelly is an excellent writer. His books are entertaining and always hold your attention. I've read all his books and look forward to more.
  • Anne Perry is an interesting author. Her stories are unique historical mysteries. Interesting enough, she is a convicted murder. I haven't read any of her books recently, suffered from keeping track of which ones I'd read and which ones I hadn't.
  • Patricia Cornwell usually writes about coroner stories. I haven't read any of these in a long time. Had trouble keeping them sorted out, but drifted back to detective stories.
  • John Saul gives me the creeps. The books are good, very good, but maybe too good. Dean Koontz I can take, but for some reason Saul got on my "If I want to be creeped out" list. (I don't read Stephen King either, I guess I'm not a horror person).
  • Len Deighton is new to me. I just finished City of Gold, and found it different and interesting. I'll be checking out more books from him in the future.
  • William Diehl is also new to me. I just finished Reign in Hell, which was a pretty good book. I'll be looking for more.
  • Ridley Pearson is fairly new to me. Just finished reading my first book by him, looking forward to reading more.
  • Leonard Goldberg writes medical mysteries. I just finished Brainwaves, and I liked it. A good read. Not sure I want to read a bunch of medical thrillers, but he will stay on my list.
  • John Le Carre is not for me. I find his books entirely too complex and hard to read. I just can't easily get into his books, not sure why.
  • Robin Cook is a medical thriller author, probably the, primary one. I recently finished Seizure, which is what got me interested in the Shroud of Turin and led me to Julia Navarro. I've read quite a few of his books, all excellent.
  • Michael Crichton, what can you say about this guy. His books are usually scientific thrillers, and written very well. You can never go wrong with his books, unless you have seen the movie, in which case I tend to get frustrated when they don't match up. So I try and read the books first, and when the movie comes out I can be a movie critic when it doesn't follow the book.
  • John Lescroart is relatively new to me but I've liked the few books I have read.
  • John Case is actually a husband and wife team, and I have only read a few books. Good books, I will read more from John Case.
  • Jonathan Kellerman is a psycho thriller author. I read a few of these but they tend to be a bit more disturbing than the typical detective novel, good books, but I try and limit my intake of psycho thrillers.
  • Gore Vidal is much more than a historical fiction writer, but that's what I have read from him. Books like Burr, 1876, Empire, and Lincoln. I haven't read any recently, but whenever I get a need for some American History (one of my hobbies), I'll look for a good Gore Vidal historical fiction.

Well, I'm sure I left out some, but maybe this helps someone find a good book to read.


Friday Nov 16, 2007

Transitioning from TeamWare to Mercurial

Just thought I would pull together some basic guidelines for anyone transitioning from TeamWare workspaces to Mercurial repositories.

I'm assuming that multiple branches will not be maintained in the Mercurial repositories in this information. Mercurial will allow you to maintain multiple branches of development in a single repository. It's my opinion that multiple branches of development can be done safer and more reliably by just have a separate clone for the separate branches. Anyone experiencing this feature in SCCS files where separate revision trees can be maintained on a file will probably agree.

TeamWare Basics

Skip this section if you are a regular TeamWare user.

A TeamWare workspace consists of a directory of files under SCCS control, each file is managed individually. Throughout the TeamWare workspace are directories called SCCS contain s.filename files which contain the original file as it was first entered into the SCCS directory, plus deltas to convert that original file to the various increasing numeric revisions of the file. A read-only file is kept in the parent directory of the SCCS directory, edits to the file requires you to use the 'sccs edit filename' command. A 'sccs delget filename' command is used to define a new revision of the file. Each revision of a file can contain a comment. SCCS manages source files and revisions to source files. TeamWare manages batches of SCCS files. TeamWare features and SCCS features get blurred sometimes, but SCCS can and is often used independent from TeamWare.

TeamWare allows for any number of workspaces with a child/parent relationship and also has a very nice code merging tool called filemerge. TeamWare allowed you to have partial workspaces. The top of the workspace contains a Codemgr_wsdata directory that holds various TeamWare book-keeping files. It's drawbacks besides not being open source are around performance and the lack of features like changesets and revision markings (tags). Over the years the short-comings have been somewhat corrected with various scripts and tools written by various teams.

Mercurial For TeamWare Users

Mercurial (like TeamWare) allows you to have any number of repositories (assume repository==workspace) and allows you to access repositories via NFS paths or with ssh:// or even http:// paths.

In many ways, at a high functional level, your Mercurial experience will be similar to the experiences you have had with TeamWare, but the details are vastly different, especially if you have become dependent on the specific format of a TeamWare workspace or the contents of the SCCS files.

At the very top of the Mercurial repository is a hidden directory called .hg which holds the Mercurial book-keeping files plus a secure set of all "commited" sources and changes to those sources.

Unlike TeamWare, where the visible source files were read-only until you explicitly used 'sccs edit' to explicitly edit them, the Mercurial "working set" sources are all read-write, and you are free to edit these files at any time. So by default you will have a working set of read-write sources and the more permanent committed files that are saved in your .hg directory.

With Mercurial, all changes to a repository are done with via "changesets", which are originally created with an 'hg commit' somewhere along the line. An ideal changeset would be all the file changes/renames/deletes/adds for one particular bug, but a changeset can be small or very large. New files, deleted files, and renamed files must all be done via a changeset. You use 'hg commit' to commit file changes into a "changeset" in your own repository, it doesn't go anywhere unless someone pulls it from your repository, or you push the changeset somewhere. The 'hg pull' is like the TeamWare bringover command, and 'hg push' is like the TeamWare putback command, well sort of. Both the 'hg push' and 'hg pull' push or pull "changesets" or committed changes to and from the .hg directories of a repositories. So your working set files are NOT automatically updated when the files in the .hg directory changes (where changesets are kept), you must explicitly run 'hg update' to update your working set files. And it's important to note that with Mercurial you do not "pull or push files" but the changesets or changes to the entire repository. This is very different from TeamWare which manages SCCS files, where you could bringover or putback individual files.

The changeset concept is like a repository wide SCCS revision number, one changeset id defines the state of the entire repository. A changeset that has no children changesets is called a "head", and there should only be one head, which is also called the tip. But when you do a pull, you often end up with multiple "head" changesets, and the gola is to perform an 'hg merge' and 'hg commit' a new "merge" changeset that will become the single "head" or "tip". Regardless of any specific file changes that might be conflicting, a merge changeset will always be needed to get back to one "head".

Roughly Equivalent Command Mappings

NOTE: Optimally, the use of 'hg commit' should be done after all the file adds, deletes, renames, and edits are done. An ideal changeset is one that contains all the changes for a particular feature or bug fix.

Action TeamWare Mercurial
Create new workspace/repository workspace create hg init
Create a child workspace/repository bringover -p parent -w child . hg clone parent child
Add a file sccs create filename hg add filename && hg commit
Delete a file workspace filerm filename hg remove filename && hg commit
Rename a file workspace filemv filename1 filename2 hg rename filename1 filename2 && hg commit
Change a file sccs edit filename && vi filename && sccs delget filename vi filename && hg commit
Rename a workspace/repository workspace move oldpath newpath mv oldpath newpath
Delete a workspace/repository workspace delete -f path rm -f -r path
Verify workspace/repository workspace check hg verify
Pull changes from another workspace/repository bringover -p parentpath . && resolve hg pull parentpath && hg update && hg merge && hg commit
Push changes to another workspace/repository putback -p parentpath . hg push parentpath
Get the name of the workspace/repository workspace name hg root
Get the name of the default parent workspace/repository workspace parent hg paths
Resolve merge conflicts resolve hg merge
Freshen the working set files sccs get filename hg update filename
Check for incoming changes bringover -n -p parentpath . hg incoming parentpath
Check on outgoing changes putback -n -p parentpath . hg outgoing parentpath
Details on changes to a single file sccs prs filename || sccs prt filename hg log -v filename && hg annotate filename
Viewing file changes sccs diff filename hg diff filename
Undo file changes sccs unedit filename hg revert filename
Listing uncommited edited files workspace find -c -OR- sccs tell hg status
Listing all managed files workspace updatenames && cat Codemgr_wsdata/nametable | sort | cut -d' ' -f1 hg locate
List a source file annotated with the revisions Various custom scripts hg annotate filename

Using Mercurial Example

Here is a simple example of pulling a repository and making a changeset.

In this example /export2/build_integration/repos/control is a path to a amsll Mercurial repository.

    # # Step 1: Check hg version # Make sure you have access to the right Mercurial. <1> rm -f -r ${HOME}/MercurialExercises/Exercise1 <2> mkdir -p ${HOME}/MercurialExercises/Exercise1/temp <3> hg version Mercurial Distributed SCM (version 0.9.3) Copyright (C) 2005, 2006 Matt Mackall This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. # # Step 2: Clone a test control repository # Creates your own private repository to play with (may take a few minutes) <9> rm -f -r ${HOME}/MercurialExercises/Exercise1/your_repos <10> mkdir -p ${HOME}/MercurialExercises/Exercise1/your_repos <11> cd ${HOME}/MercurialExercises/Exercise1/your_repos <12> hg clone /export2/build_integration/repos/control control requesting all changes adding changesets adding manifests adding file changes added 1 changesets with 24 changes to 24 files 24 files updated, 0 files merged, 0 files removed, 0 files unresolved # # Step 3: Private clone of the above repository for testing push/pull. # Clone a second private repository. <13> cd ${HOME}/MercurialExercises/Exercise1/your_repos <14> hg clone control control-work 24 files updated, 0 files merged, 0 files removed, 0 files unresolved # # Exercise 1: Step 4: Pretend like something changed and do a 'pull' # Pulling changes now will say 'no changes'. <15> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control <16> hg pull pulling from /export2/build_integration/repos/control searching for changes no changes found <17> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control-work <18> hg pull pulling from ${HOME}/MercurialExercises/Exercise1/your_repos/control searching for changes no changes found # # Step 5: Make a change in your work area and push it # Demonstrates a simple file change, commit and push of a change. <19> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control-work <20> echo '#harmless' >> make/Makefile <21> hg status M make/Makefile <22> hg diff diff -r 48e79d6618ee make/Makefile --- a/make/Makefile Sun Sep 30 17:55:14 2007 -0700 +++ b/make/Makefile Thu Oct 04 19:08:49 2007 -0700 @@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen .PHONY: all build what clobber insane freshen \\ fastdebug_build debug_build product_build setup +#harmless <23> hg commit -m "9999999: Fixed world peace" <24> hg outgoing searching for changes changeset: 1:0d317da1a3c4 tag: tip user: ${USER} date: Thu Oct 04 19:08:53 2007 -0700 summary: 9999999: Fixed world peace <25> hg push pushing to ${HOME}/MercurialExercises/Exercise1/your_repos/control searching for changes adding changesets adding manifests adding file changes added 1 changesets with 1 changes to 1 files # # Step 6: Inspect the push (watch the working set files) <26> cd ${HOME}/MercurialExercises/Exercise1/your_repos/control <27> tail -1 make/Makefile <28> hg status <29> hg update 1 files updated, 0 files merged, 0 files removed, 0 files unresolved <30> tail -1 make/Makefile #harmless

Using TeamWare and webrev to import a changeset

The tool webrev creates a set of web pages than can be used to browse code changes, but more recent versions create patch files (looks like "diff -r -u") that can be fed into gpatch (GNU patch) or a similar tool to apply the changes.

In this example /export2/build_integration/ws7/control is a path to a TeamWare integration workspace and /export2/build_integration/repos/control is a path to an equivalent Mercurial repository.

    # # Step 1: Bringover a test control workspace
    #   Creates your own private workspace (may take a few minutes)
    <1> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_ws
    <2> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_ws
    <3> cd ${HOME}/MercurialExercises/Exercise2/your_ws
    <4> bringover -q -p /export2/build_integration/ws7/control -w control .
    Parent workspace: /export2/build_integration/ws7/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Examined files: 55
    Bringing over contents changes: 55
    Examined files: 55
    Contents Summary:
          55   create
    # # Step 2: Private child workspace of the above workspace to hold changes.
    <5> cd ${HOME}/MercurialExercises/Exercise2/your_ws
    <6> bringover -q -p control -w control-work .
    Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Examined files: 55
    Bringing over contents changes: 55
    Examined files: 55
    Contents Summary:
          55   create
    # # Step 3: Make changes in the child workspace
    <7> cd ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/make
    <8> sccs edit Makefile
    new delta 1.314
    444 lines
    <9> echo '#harmless' >> Makefile
    <10> sccs delget -y'9999999: Fixed world peace' Makefile
    No id keywords (cm7)
    1 inserted
    0 deleted
    444 unchanged
    No id keywords (cm7)
    445 lines
    <11> cd ..
    <12> putback -n .
    Parent workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control
    Child workspace:  ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Examined files: 55
    Would put back contents changes: 1
    update: make/Makefile
    Examined files: 55
    Contents Summary:
           1   update
          54   no action (unchanged)
    No changes were put back
    <13> webrev -l .
       SCM detected: teamware
     File list from: 'putback -n  .' ...  Done.
          Workspace: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work
    Compare against: ${HOME}/MercurialExercises/Exercise2/your_ws/control
          Output to: ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev
       Output Files:
    		 patch cdiffs udiffs sdiffs frames old new
     Generating PDF: Skipped: no output available
         index.html: Done.
    <14> cat webrev/control-work.patch
    --- old/make/Makefile	Thu Oct  4 19:09:19 2007
    +++ new/make/Makefile	Thu Oct  4 19:09:19 2007
    @@ -442,3 +442,4 @@
     .PHONY: all build what clobber insane freshen \\
     	fastdebug_build debug_build product_build setup
    # # Step 4: Clone a test control repository
    #   Creates your own private repository to play with (may take a few minutes)
    <15> rm -f -r ${HOME}/MercurialExercises/Exercise2/your_repos
    <16> mkdir -p ${HOME}/MercurialExercises/Exercise2/your_repos
    <17> cd ${HOME}/MercurialExercises/Exercise2/your_repos
    <18> hg clone /export2/build_integration/repos/control control
    requesting all changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 24 changes to 24 files
    24 files updated, 0 files merged, 0 files removed, 0 files unresolved
    <19> hg clone control control-work
    24 files updated, 0 files merged, 0 files removed, 0 files unresolved
    # # Step 5: Import the patch into the repository
    <20> cd ${HOME}/MercurialExercises/Exercise2/your_repos/control-work
    <21> gpatch -u -p1 < ${HOME}/MercurialExercises/Exercise2/your_ws/control-work/webrev/control-work.patch
    patching file make/Makefile
    <22> hg status
    M make/Makefile
    <23> hg diff
    diff -r 48e79d6618ee make/Makefile
    --- a/make/Makefile	Sun Sep 30 17:55:14 2007 -0700
    +++ b/make/Makefile	Thu Oct 04 19:09:32 2007 -0700
    @@ -442,3 +442,4 @@ sponsors-bringover: sponsors-freshen
     .PHONY: all build what clobber insane freshen \\
     	fastdebug_build debug_build product_build setup
    <24> hg commit -m "9999999: Fixed world peace"
    <25> hg outgoing
    searching for changes
    changeset:   1:23e0962ced6d
    tag:         tip
    user:        ${USER}
    date:        Thu Oct 04 19:09:36 2007 -0700
    summary:     9999999: Fixed world peace
    <26> hg push
    pushing to ${HOME}/MercurialExercises/Exercise2/your_repos/control
    searching for changes
    adding changesets
    adding manifests
    adding file changes
    added 1 changesets with 1 changes to 1 files

Beginner Gotchas for TeamWare Users

Not setting up your ~/.hgrc file. The name you define in ~/.hgrc with "[ui]" and "username=" is the name that will be permanently recorded in the changesets you create with 'hg commit'. I don't recommend adding your email address in username, but that's up to you, just keep in mind it will be public information when your changesets reach a public repository. TeamWare/SCCS used your system username, but very few TeamWare workspaces were ever made public.
Forgot the 'hg update' After an 'hg pull' (aka bringover), don't forget the 'hg update', or use 'hg pull -u'. The default pull and push just updates the changesets and doesn't update your read-write working set of files. You need to be careful about updating the working set files on shared repositories they could get updated while others are viewing them.
Forgot to merge After an 'hg pull', you need to 'hg update', and if you have changesets that you have not committed you will also need to 'hg merge' and 'hg commit'. If you forget you will end up with multiple heads and a more difficult time merging later.
Forgot to commit after a merge (multiple heads) After 'hg merge' you need to 'hg commit'. The merge just prepares you for the 'hg commit' of a merge changeset. If you forget you will end up with multiple heads and a more difficult time merging later.
Making accidental edits Mercurial working set files are always read-write and ready to edit, no 'sccs edit' action is necessary. Use 'hg status' to monitor what files you have changed.
Using the wrong relative path File paths supplied to 'hg' commands are relative to the current directory, the TeamWare bringover and putback commands want paths relative to the root of the workspace, regardless of the current directory.
Not defining the file .hgignore for 'hg status' The 'hg status' command tells you what outstanding changes you have in your working set, by default it looks in '.' or the entire directory, but if there are files created during a build, you want 'hg status' to ignore those files. Make sure you define the .hgignore file so that 'hg status' will only find files in the directories you want managed by the repository. TeamWare never really helped with the problem of forgetting to 'sccs create' your files, 'hg status' solves this common problem.
Using NFS/UFS for team integration areas TeamWare for the most part was designed around sharing data via NFS or UFS file systems. Mercurial can work the same way, but when using it for team integration areas we recommend the use of the ssh:// parent path mechanisms described in the Mercurial Book . Unless everyone in the team or group is in the same Unix group, have the same default group, and all use 'umask 2', using NFS/UFS will be problematic. Mercurial obeys the strict Unix rules of file creation and permissions, and over time TeamWare has adjusted itself (perhaps improperly) to avoid the file permission issues you can see with Mercurial.
Too quick on the 'hg commit' Once a changeset is created (the 'hg commit'), and pushed, it's pretty permanent. Make sure that before the 'hg commit' happens that the changes are correct, reviewed, the right ones, and complete, otherwise you'll be creating yet another one to correct your mistakes.
Doing a push with outstanding working set changes The 'hg push' will not detect any outstanding changes to your working set, it just pushes the existing changesets. ALWAYS use 'hg status' before an 'hg push' to make sure you have created all your changesets with 'hg commit', unless of course you have changes you don't want to push.
Committing a sensitive file Accidental additions of sensitive source files can be a big problem. Completely removing a sensitive file that has been accidently added to a repository can be a real problem. be very careful what files you add to a repository! Adding non-open source files to an 'open source' repository will inflict major pain on many people.
Doing anything to the .hg files Don't mess with the .hg data files, if you do you are INSANE, leave that to the Mercurial professionals. If you suspect they have been corrupted, use 'hg verify' to check. Backups are always important, so make sure you keep a relatively recent backup repository. If you can't 'hg rollback', save the repository somewhere, clone a fresh copy from your parent, remove the working set files completely from the clone, and copy in the working set from your corrupted repository (but not the .hg files). Now you can use the standard 'hg status' and 'hg diff' to see what file changes you may have lost and adjust.
Using SCCS keywords Mercurial by default does not support anything like SCCS keywords in files. You should remove these or find another solution.
Looking for putback comments or history files Changeset comments represent BOTH the SCCS comments and the effective TeamWare putback comment.
Using problematic filenames Watch out for directory and filenames that only differ in case (e.g. test and Test), at least on the Mac and Windows these can be troublesome. Long pathnames (>255 characters) can also be a problem.

Converting a TeamWare Workspace to a Mercurial Repository

Converting a TeamWare workspace to a Mercurial repository (without history) is pretty trivial:

       bringover -p your_workspace -w /tmp/repo .
       cd /tmp/repo
       workspace parent -u
       rm -f -r Codemgr_wsdata
       rm -f -r deleted_files
       foreach i ( `find . -name SCCS` )
          ( cd $i/.. && sccs edit SCCS )
          rm -f -r $i
       hg init
       hg add

A simple source tree can be turned into a Mercurial repository with just hg init; hg add. Turning a TeamWare workspace into a plain source tree is relatively simple too, I just create a separate workspace, purge a few files, make sure all the sources are in 'edit' mode, and remove the SCCS directories.

Performance Comparisons and Data

Nothing but good news in this area, for both time and space.

Many of the past tricks used to speed up TeamWare bringovers and putbacks, especially over slow connections should not be necessary with Mercurial, it is very fast. The initial 'hg clone' of a repository should be considerably faster, but the most important actions of 'hg pull' or 'hg push' will be so much faster you may question if the action actually happened. Unlike TeamWare, only the changesets are transported, and many fewer files are accessed and in a more efficient manner.

The size of the repositories should also be smaller (at least 50% smaller) than the equivalent TeamWare workspace, this isn't surprising due to the lack of compression and age of SCCS file.

Wednesday May 23, 2007

NeoOffice Patch 4 is much faster

NeoOffice 2.1 Patch 4

The latest NeoOffice (OpenOffice for the Mac) version 2.1 Patch 4 is really fast. Anyone using NeoOffice on the Mac should get this version from and install the latest Patch 4.


Wednesday May 02, 2007

Mercurial Best Practices

Mercurial Best Practices

Sunset Bay, Oregon.

Just a draft of a some 'best practices' we should consider when we start using Mercurial. This is not gospel, not official, not perfect, just a basis for discussion.

Also, some of these best practices might be enforceable via Mercurial hooks, and that seems like a good idea to me, but I haven't tried hooks like this yet.

Proposed Mercurial Best Practices (or Best Intentions?):

  • Integration repositories should never have an 'hg pull' operation done to them. An integration repository is any repository that is the target of an 'hg push', and changes to integration repositories should only come in via a 'hg push'. By default, 'hg push' will not let you push multiple heads. The best way to update an integration repository is to clone a separate 'merge' repository instance, pull the changes from the integration parent into this 'merge' clone, perform the appropriate testing to verify the merge is acceptable, and then push the resulting merged changesets into the integration repository. This protects all other team members by providing a clean repository to pull from and to push to.
  • Integration repositories should NEVER have multiple heads. Separate integration repositories should be used for separate development efforts or source branches.
  • The 'hg status' command should never list files that should be ignored, e.g. the file .hgignore must be defined to ignore all files that should not be managed in the repository.
  • The creation of a changeset ('hg commit') should be done when the change is in a final state and ready to be 'committed' to a repository. Changesets should not be created for preliminary changes or experimental changes. Consider using patches for changes that have not been reviewed, built, and at least had preliminary testing done. Multiple changesets that have never left the nest (never pushed or pulled), could be merged together via a fresh clone, copy of the working set files, and a fresh changeset created in the clone. I'd recommend delaying the 'hg commit' until you are relatively sure that you are done rather than hassle with merging changesets.
  • The first line of a changeset comment should be a standalone description of the changeset and be less than 65 characters, e.g. "Changes for: 8140072 and 1466620", "Sync changes, w/details", or for a simple one bug fix, just "Fixed NNNNNNN: Synopsis".
  • The non-merge changeset comment must include at least one bugid, and the comment should include at least one line per bug of the form:
      Fixed NNNNNNN: Synopsis
  • Each changeset should fix one or more bugs completely. Partial fixes should be rare, and should use the line:
      Partial NNNNNNN: Synopsis
  • In the rare case where an additional changeset is needed for a fix or feature the changeset comments should repeat the line:
      Fixed NNNNNNN: Synopsis
    as needed.
  • The comments should not include: login names, user names, copyright notices, email addresses, ads, spam, jokes, or foul language. These comments will be publicly visible and should comment the changeset in a professional manner.
  • The comments should be in English, using only the simple ASCII character set.
  • The comments should not include code snippets and should be kept as brief as possible, or be of a size appropriate to the changes it represents. Major features may require a more detailed changeset comment, but the changeset comments should not be used as documentation on a feature, rather it should describe why the changes were needed to support that feature.

Ideally, Mercurial changesets should ultimately provide a complete bundle for a fix or feature, I know there may be exceptions to the rule, but if we set down the rules early we can start creating clean and self-documenting changesets.

Let me know what you think.


Tuesday Apr 17, 2007

The Mercurial Mindset - Food for Thought

The Mercurial Mindset

So I'm sitting in my office at home and I hear this noise in our Ornamental Pear tree outside. It's not a Mercurial user, just a squirrel, eating fruit from a fruitless tree. Why a fruitless tree is bearing fruit is beyond me, but the squirrel likes it. Hopefully the new Mercurial users will be as happy as Rocky here. ;\^)

I've been deep into the technical and developer concerns around Mercurial conversion issues for a few months now. I see the light at the end of the tunnel, most of the time anyway. ;\^) So consider this a perspective from deep in the trenches.

One brave documentation team inside Sun has actually stepped up and done the conversion from Teamware to Mercurial, and although these are not software developers, and Mercurial isn't managing Java or C source files, the experience is interesting. Some of the problems they ran into:

  • Using NFS, file/directory ownership and permissions for the groups was a problem.

    The standard Unix rules on user/group permissions seem to be a problem for teams that have members that belong to different Unix groups. Now you might think this is obvious, but it's often not. Teams using Mercurial with NFS or local file systems need to make sure that they have their umask and groups setup right for standard Unix file sharing.

    As far as I can tell Mercurial is obeying the umask/group rules and doing the right thing, but Teamware appeared to have the avoided this issue by making sure all directories created with a 'putback' (e.g. 'hg push') had wide-open permissions (read-write for everyone), and then I assume it re-creates files as needed in those directories. In any case, Teamware did not seem to care that multiple team members didn't share the same groups, or it did this on purpose.

  • Multiple Heads (by accident?)

    Mercurial allows for multiple branches of development, but in general I don't think this is a good idea for anyone starting out. In many cases repositories would end up with multiple heads, which is normal after an 'hg pull' that has a conflict. But the state was very confusing for new users. Doing an 'hg push' will fail telling you to 'hg update', and it would tell you to 'hg merge' and it would tell you to 'hg commit'. Well, in fact, you do need to 'hg update', 'hg merge', and 'hg commit', but the error messages didn't seem to help explain what was happening.

    One confusing item in particular was the fact that even if the merge was empty, you still need to follow the 'hg merge' with an 'hg commit'. Quiting the editor when the file says "no changes", will often terminate the commit and leave you in the same state. Once the pattern is understood, they could conform, but it was considered 'odd' to be committing what appeared to be nothing.

    Repeated 'hg pull' actions after this state just created more and more heads or unmerged changesets, which made it even more confusing. Once the pattern of 'hg pull && hg update && hg merge && hg commit' was established, things went much smoother.

  • The basic steps.

    The basic steps of doing a change and pushing the change were not obvious to some people.

    With Teamware you would:

    bringover -p /nfs-path/parent-workspace -w /path/workspace .
    cd /path/workspace
    (cd some_dir && sccs edit files && vi/emacs files && sccs delget filename)
    bringover -p /nfs-path/parent-workspace .
    putback .   -OR   putback some_directory/filename

    The Teamware bringover and putback is a per-file operation, with the default operation being the entire workspace, but the "I'm putting back files" attitude seems to be rather burned into the soul or something. When a resolve happens, it's on a per-file basis, and you each file is resolved separately.

    With Mercurial you would:

    hg clone /nfs-http-ssh-path/parent-repository /path/repository
    cd /path/repository
    vi/emacs any files you want
    hg commit
    hg pull && hg update
    hg merge && hg commit
    hg push

    The merge could be unnecessary, but if 'hg heads' lists more than one head, you need to 'hg merge && hg commit' to lop it off. (The 'hg commit' creates the changeset).

    With Mercurial you push and pull changesets, not files. Although you can create a changeset for one file change, you still don't push files. And if changeset A was created after changeset B, you can't skip B, you'd need to push both, even if these two changesets change completely unrelated files.

  • No partial workspaces.

    I've come to the conclusion that this is a good thing. Over the years I have been amazed at the times two supposedly unrelated file changes can cause a build failure or a runtime failure. Allowing partial workspaces or partial pushes is a risk to anyone that pulls these changes into their own repository. The integrity of the repository should be a top priority, and you can only guarantee that when you operate on the entire repository.

    Maybe after we get more experience with the merge issues I'll change my mind.

  • Merge issues.

    I've gotten myself into a merge nightmare a few times, tried kdiff3 on the Mac but it didn't seem to work for me, or I need to RTFM. Simple merges were fine, but the nasty ones where the same lines were changed were challenging for me, and for others too. The Teamware filemerge tool ('f' command during a resolve) was quite nice, and I'm beginning to miss it. Hooking filemerge into Mercurial is possible but on a file by file basis, and when you get those 100 files to merge, that may be awkward. But it's only a matter of time before I find a good merge tool and configuration.

Hope this helps someone. I'll try and talk about the poor CVS or Subversion mindset problems with Mercurial next time, although I'm not an experienced CVS or Subversion expert, I can address the benefits of a DSCM or Distributed Source Code Management system like Mercurial.


Various blogs on JDK development procedures, including building, build infrastructure, testing, and source maintenance.


« July 2016