Sunday Mar 02, 2008

NetBeans, Mercurial, Ant, Mac OS X, and getting the right PATH set

It took be some time, but I figured out how to change the environment variables that the launched Mac applications will get. Why? Because I wanted NetBeans to be running with a PATH environment variable setting that matched what Ant got when I used the build.xml file from the command line. When running the "<exec>" Ant task to use an executable, full paths or modifying the PATH is very platform specific, hard to maintain, and a huge pain. Having the PATH set properly in the environment is the best way.

Now I could launch "netbeans" from a command line and it would work, but here is the answer for setting the environment for applications launched:

<> cat ~/.MacOSX/environment.plist 
PATH = "/Users/ohair/ant/bin:/Users/ohair/findbugs/bin:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/sbin:/sbin";

The directory ~/.MacOSX will need to be created. Apparently this file is read in at login time, so if you change it you will need to logout and log back in again for any change to make a difference. In my case I added the path to my Ant, my Findbugs, and /usr/local/bin which contains the Mercurial (hg) I want to have available in the PATH. I'm not sure NetBeans will actually use the version of Ant in the PATH, but that hasn't been an issue for me.

Why Does hg need to be in the PATH?

It was the need for /usr/local/bin (hg) in the path (/usr/local/bin) that got me started on all this because in my Ant file I wanted the build to automatically pull the version information out of the repository and make it available to the built product as a property setting. Effectively I need to run:

    # Get the last changeset
    hg tip --template '{node|short}\\n'
    # Get the latest tag with a Version string in it
    hg log -l 1 --template '{desc|firstline}\\n' -k "Version:"
    # Get the date of the last changeset
    hg tip --template '{date|shortdate}\\n'

I used Ant rules something like this:

    <target name="hgpropfile" description="Create property file">
        <exec executable="hg" outputproperty="">
            <arg value="tip"/>
            <arg value="--template"/>
            <arg value="{node|short}\\n"/>
        <exec executable="hg" outputproperty="hg.last.tag.summary">
            <arg value="log"/>
            <arg value="-l"/>
            <arg value="1"/>
            <arg value="-k"/>
            <arg value="Version:"/>
            <arg value="--template"/>
            <arg value="{desc|firstline}\\n"/>
        <exec executable="hg" outputproperty="">
            <arg value="tip"/>
            <arg value="--template"/>
            <arg value="{date|shortdate}\\n"/>
<!-- Indentation is critical here -->
<echo file="${dist.dir}/" append="false">
product.version=${} ${hg.last.tag.summary} [${}]

This property and it's value would need to be read in or made available at runtime, the actual Java code would just need to getProperty("product.version") to get the version string.

I used a tag to track the version code name, with the most recent tag containing "Version:" provides the product code name. The date and changeset id come from the tip, or most recent changeset.

Automating the creation of a new version code name tag can be done with a special Ant target used when needed. Effectively it needs to run:

    hg tag -f -m "Version: Name" TAG-YYYY-MM-DD
I just manually created a file with a few hundred code names and pick one based on the day of the year. This "AllVersions" file could look as simple as:
    Humor Risk (1921), previewed once and never released; thought to be lost
    The Cocoanuts (1929), released by Paramount Pictures
    Animal Crackers (1930), released by Paramount
    The House That Shadows Built (1931), released by Paramount (short subject)
    Monkey Business (1931), released by Paramount
    Horse Feathers (1932), released by Paramount
    Duck Soup (1933), released by Paramount
    A Night at the Opera (1935), released by MGM
    A Day at the Races (1937), released by MGM
    Room Service (1938), released by RKO Radio Pictures
    At the Circus (1939), released by MGM
    Go West (1940), released by MGM
    The Big Store (1941), released by MGM
    A Night in Casablanca (1946), released by United Artists
    Love Happy (1949), released by United Artists
    The Story of Mankind (1957), released by Warner Brothers
But ideally you would want enough code names in the list to avoid the name getting re-used too many times. The "hg tip --template '{node|short}\\n'" is your real version, these code names are just a way to help people quickly identify a version.

The Ant target looks something like:

    <target name="new_version" description="Create new version tag">
            <format property="" pattern="D"/>
            <format property="date.ymd" pattern="yyyy-MM-dd"/>
        <property name="versions.file" value="AllVersions"/> 
        <exec executable="wc" outputproperty="version.count.temp" input="${versions.file}">
            <arg value="-l"/>
        <exec executable="sed" outputproperty="version.count" inputstring="${version.count.temp}">
           <arg value="-e"/>
           <arg value="s@\^[\\ ]\*\\([1-9][0-9]\*\\)$@\\1@"/>
        <exec executable="expr" outputproperty="version.selection">
            <arg value="${}"/>
            <arg value="%"/>
            <arg value="${version.count}"/>
        <exec executable="sed" outputproperty="" input="${versions.file}">
            <arg value="-e"/>
            <arg value="${version.selection},${version.selection}p"/>
            <arg value="-n"/>
        <exec executable="hg">
            <arg value="tag"/>
            <arg value="-f"/>
            <arg value="-m"/>
            <arg value="Version: ${}"/>
            <arg value="TAG${date.ymd}"/>

Or in a Makefile you could do a similar thing:

DATE_YEAR_DAY:=$(shell date +%j)
DATE_YMD:=$(shell date +%Y-%m-%d)
VERSION_COUNT:=$(shell cat $(VERSIONS_FILE) | wc -l | sed -e 's@\^[\\ ]\*\\([1-9][0-9]\*\\)$@\\1@')
        hg tag -f -m "Version: $(VERSION_NAME)" TAG$(DATE_YMD)

Now creating tags like this may not be advised or necessary with all repositories, but the basic principle can work in many situations. For example, with the OpenJDK project, the Release Engineering people will create the major milestone tags, and using those you could effectively identify a JDK version with a name (e.g. JDK 7 Build 23), and an exact changeset id. The trick is to get the version information from the repository, into the build tool (make or ant), into the product installation or baked into the product executable, and then available at runtime by the product plus easily seen when looking at an installation of the product. One issue I see is dealing with the situation where you are building a plain source tree without the Mercurial data or the Mercurial tools, somehow when the plain source tree is created the version data would need to be left in the source bundle.

Hope this is of some use to people. I'm sure there might be a better way, so if anyone has any ideas please add your comments. Ultimately I'd like a product to be able to provide enough details to a user so that the original source tree could be made quickly available. Given the changeset id, the exact and complete source could be re-created with hg clone --rev, of course that gets more complicated with a forest, but still pretty simple.


Friday Feb 29, 2008

Why is NetBeans so slow? Here is a tip.

I use NetBeans 6 all the time on my Mac laptop and have learned how to configure it to run more effectively for me. NetBeans, like any Java application needs room to work, and the default memory settings for Java and NetBeans are usually not what I consider ideal. Of course how would they know what is ideal, you need to tell it. Granted my solution may not be ideal for others, but it's worth playing with. I'm using jdk5 update 13 on my Mac, but these same setting should apply to jdk6 also. And it's not just for The Mac or even NetBeans 6, this should work on any platform.

KellyMacBook<3> cat ~/.netbeans/6.0/etc/netbeans.conf
netbeans_default_options="-J-Xms256m -J-Xmx512m -J-XX:PermSize=32m -J-XX:MaxPermSize=160m
-J-Xverify:none -J-Dapple.laf.useScreenMenuBar=true -J-XX:+UseConcMarkSweepGC
-J-XX:+CMSClassUnloadingEnabled -J-XX:+CMSPermGenSweepingEnabled"

Any change to this file will require a reboot (there should be one line in the file above, I had to break it up to get the blog to display it in a readable way).

Also, turn on the View->Toolbars->Memory to keep track of the NetBeans memory usage, clicking on it will force a GC, and you can get a feeling for how much memory your NetBeans session needs, making adjustments to the maximums above. And my recommendation is that the maximum (-J-Xmx) setting should never be more than the RAM on the machine minus 512m, or system memory thrashing may occur.

Additional information can be found at the NetBeans FAQ on GC pauses and the NetBeans FAQ on configuration.

The above settings make a world of difference in the way NetBeans performs for me.



Various blogs on JDK development procedures, including building, build infrastructure, testing, and source maintenance.


« April 2014