Thursday Jul 30, 2009

Integrating JSLint more tightly into NetBeans

Updated on August 3rd, 2009, to reflect jslint4java release 1.3.1

In my previous blog entry, Netbeans, JavaScript, Ant, Hudson, and JSLint: Oh my!, I described an approach for integrating JSLint into my NetBeans project.  While this approach worked, I still had to navigate to each JavaScript issue manually.  This is a huge pain, something that I haven't done since my earliest days as a developer, before I learned about Emacs' next error functionality.  NetBeans also has a really nice "Next Error" function that automatically navigates the user to the next error detected by the compiler.  Unfortunately, NetBeans didn't recognize the native output of JSLint.   This got me thinking about building a custom wrapper around JSLint so I can transform the output to whatever NetBeans would be happy with.

At roughly the same time, Dominic Mitchell posted a comment in response to my blog entry, pointing me towards jslint4java, which not only wraps JSLint and transforms its output, but it also provides a nice JSLint task interface to Ant.  This looked very promising, so I gave the just recently released jslint4java version 1.3 a whirl, but unfortunately, NetBeans did not recognize the output as errors.

Netbeans not recognizing output as an error

After some experimenting, I learned that NetBeans expects compiler errors to have exactly the same format as javac error messages.  As an example, version 1.3 of jslint4java outputs:

utilities.js:145:22:Use '===' to compare with 'undefined'

Whereas NetBeans expects the following output:

/Users/ashamash/projects/omalley/src/trunk/CQS/web/js/utilities.js:145:22: Use '===' to compare with 'undefined'.

There are two subtle differences:

  • All filenames need to have the full path (e.g. /Users/ashamash/projects/omalley/src/trunk/CQS/web/js/utilities.js vs. just utilities.js)
  • There needs to be a space between the colon and the error description (e.g. 145:22: Use vs. 145:22:Use)

With these two changes, NetBeans is a lot happier:

NetBeans recognizing output as an error

I sent Dominic the code changes needed to implement this, and after a couple of rounds of collaboration, the current development version of jslint4java has these modifications in it, as well as release jslint4java release 1.3.1 (released July 31st, 2009).

To integrate this version into NetBeans builds, follow these steps:

  1. Download jslint4java release 1.3.1 from http://jslint4java.googlecode.com/files/jslint4java-1.3.1-dist.zip, unpack it somewhere, note the path to the jslint4java-1.3.1.jar file.
  2. Define the properties needed in nbproject/project.properties:
    #
    # cqs.build.jslint: true/false setting to determine whether to run the
    # jslint javascript analysis utility on the individual javascript
    # files.  Default is true for production purposes, but developers can
    # set it to false in the file private/private.properties.  It is
    # highly recommended that this be left to true.
    #
    # cqs.build.jslint.failonerror: this true/false setting determines
    # whether the build should fail or not if jslint detects any errors.
    # Currently set to false until we fix all the javascript code so it
    # passes jslint.
    #
    cqs.build.jslint=true
    cqs.build.jslint.failonerror=false
    #
    cqs.js.jslint.files=\\
           js/utilities.js,\\
           js/file2.js,\\
           js/file3.js
    
  3. Now, you are ready to integrate jslint4java into your project's build.xml file, this is the Ant task that we use.  In our project, we keep an explicit list of JavaScript files we want to lint, so we can separate them from 3rd party code.  Another approach, described in the jslint4java docs, is to analyze all the JavaScript files in a given directory.  You can do what works best for you. 
        <target name="doJSLint4Java">
            <echo level="info" message="doJSLintWithJSLint4Java: running on files ${cqs.js.jslint.files}...." />
            
            <property name="jslint4java.dir" value="../tools/jslint4java-1.3.1" />
            <property name="jslint4java.jar.file" value="${jslint4java.dir}/jslint4java-1.3.1.jar"/> 
            <available file="${jslint4java.jar.file}" property="jslint4java.jar.file.available" />
            <fail unless="jslint4java.jar.file.available" message="jslint4java Jar file not found - expected at ${jslint4java.jar.file}" />
    
            <taskdef name="jslint4java"
                     classname="com.googlecode.jslint4java.ant.JSLintTask"
                     classpath="${jslint4java.jar.file}" />
    
            <jslint4java haltOnFailure="${cqs.build.jslint.failonerror}">⁞
              <formatter type="plain" />
              <filelist id="jslint.filelist" dir="${basedir}/web" files="${cqs.js.jslint.files}"/>
            </jslint4java>
    
            <echo level="info" message="doJSLintWithJSLint4Java: finished running...." />
        </target>
    
    
  4. For our project, we use "The Good Parts" JSLint setting, and embed the appropriate settings as a comment at the top of our JavaScript files.  You can use the tool at http://jslint.com/ to help you figure out which settings are best for your project.

And that's it.  You should read about best practices with JSLint, etc.  Thanks to Dominic Mitchell for making this tool available!

If you want to build it from scratch, you can follow these steps:

  1. Download the current snapshot of jslint4java, use git:
    $ git clone git://github.com/happygiraffe/jslint4java.git
    Initialized empty Git repository in /Users/ashamash/projects/jslint4java/src/jslint4java/.git/
    remote: Counting objects: 2548, done.
    remote: Compressing objects: 100% (862/862), done.
    remote: Total 2548 (delta 1028), reused 2346 (delta 940)
    Receiving objects: 100% (2548/2548), 1021.89 KiB | 232 KiB/s, done.
    Resolving deltas: 100% (1028/1028), done.
    $ 
    
  2. To build it, use Maven:
    $ mvn -Pdist clean package
    [INFO] Scanning for projects...
    [INFO] Reactor build order: 
    [INFO]   jslint4java parent
    [INFO]   jslint4java
    [INFO]   jslint4java ant task
    [INFO]   jslint4java docs
    [INFO]   jslint4java distribution
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java parent
    [INFO]    task-segment: [clean, package]
    [INFO] ------------------------------------------------------------------------
    [INFO] [clean:clean]
    [INFO] Deleting file set: /Users/ashamash/projects/jslint4java/src/jslint4java/target (included: [\*\*], excluded: [])
    [INFO] [site:attach-descriptor]
    [INFO] [javadoc:jar {execution: attach-javadocs}]
    [INFO] Not executing Javadoc as the project is not a Java classpath-capable package
    [INFO] Preparing javadoc:aggregate
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java parent
    [INFO] ------------------------------------------------------------------------
    [WARNING] Removing: aggregate from forked lifecycle, to prevent recursive invocation.
    [INFO] No goals needed for project - skipping
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java
    [INFO] ------------------------------------------------------------------------
    [INFO] No goals needed for project - skipping
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java ant task
    [INFO] ------------------------------------------------------------------------
    [INFO] No goals needed for project - skipping
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java docs
    [INFO] ------------------------------------------------------------------------
    [INFO] No goals needed for project - skipping
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java distribution
    [INFO] ------------------------------------------------------------------------
    [INFO] No goals needed for project - skipping
    [WARNING] The dependency: com.googlecode.jslint4java:jslint4java-ant:jar:1.4-SNAPSHOT can't be resolved but has been found in the reactor.
    This dependency has been excluded from the plugin execution. You should rerun this mojo after executing mvn install.
    [INFO] [javadoc:aggregate {execution: aggregate-javadoc}]
    [WARNING] The dependency: [com.googlecode.jslint4java:jslint4java-ant:jar:1.4-SNAPSHOT] can't be resolved but has been found in the reactor (probably snapshots).
    This dependency has been excluded from the Javadoc classpath. You should rerun javadoc after executing mvn install.
    [WARNING] IGNORED to add some artifacts in the classpath. See above.
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java
    [INFO]    task-segment: [clean, package]
    [INFO] ------------------------------------------------------------------------
    [INFO] [clean:clean]
    [INFO] Deleting file set: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/target (included: [\*\*], excluded: [])
    [INFO] [resources:resources]
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] Copying 1 resource
    [INFO] [compiler:compile]
    [INFO] Compiling 6 source files to /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/target/classes
    [INFO] [resources:testResources]
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] skip non existing resourceDirectory /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/src/test/resources
    [INFO] [compiler:testCompile]
    [INFO] Compiling 5 source files to /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/target/test-classes
    [INFO] [surefire:test]
    [INFO] Surefire report directory: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/target/surefire-reports
    
    -------------------------------------------------------
     T E S T S
    -------------------------------------------------------
    Running com.googlecode.jslint4java.OptionTest
    Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.049 sec
    Running com.googlecode.jslint4java.IssueTest
    Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.122 sec
    Running com.googlecode.jslint4java.JSLintTest
    Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.119 sec
    Running com.googlecode.jslint4java.OptionParserTest
    Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.005 sec
    Running com.googlecode.jslint4java.UtilTest
    Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec
    
    Results :
    
    Tests run: 33, Failures: 0, Errors: 0, Skipped: 0
    
    [INFO] [jar:jar]
    [INFO] Building jar: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/target/jslint4java-1.4-SNAPSHOT.jar
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java ant task
    [INFO]    task-segment: [clean, package]
    [INFO] ------------------------------------------------------------------------
    [INFO] [clean:clean]
    [INFO] Deleting file set: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target (included: [\*\*], excluded: [])
    [INFO] [resources:resources]
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] Copying 1 resource
    [INFO] [compiler:compile]
    [INFO] Compiling 5 source files to /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/classes
    [INFO] [resources:testResources]
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] Copying 6 resources
    [INFO] [compiler:testCompile]
    [INFO] Compiling 1 source file to /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/test-classes
    [INFO] [surefire:test]
    [INFO] Surefire report directory: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/surefire-reports
    
    -------------------------------------------------------
     T E S T S
    -------------------------------------------------------
    Running com.googlecode.jslint4java.ant.PlainResultFormatterTest
    Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.09 sec
    
    Results :
    
    Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
    
    [INFO] [antrun:run {execution: default}]
    [INFO] Executing tasks
    
    allTests:
    [au:antunit] Build File: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/test-classes/antunit/tests.xml
    [au:antunit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 3.573 sec
    [au:antunit] Target: testArbitraryResource took 0.836 sec
    [au:antunit] Target: testFailsOnError took 0.862 sec
    [au:antunit] Target: testNamesInOutputHavePath took 0.433 sec
    [au:antunit] Target: testNoResources took 0.331 sec
    [au:antunit] Target: testOptionParams took 0.378 sec
    [au:antunit] Target: testSuccess took 0.277 sec
    [au:antunit] Target: testIssue6 took 0.393 sec
    [INFO] Executed tasks
    [INFO] [jar:jar]
    [INFO] Building jar: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/jslint4java-ant-1.4-SNAPSHOT.jar
    [INFO] [shade:shade {execution: make-shaded-jar}]
    [INFO] Including com.googlecode.jslint4java:jslint4java:jar:1.4-SNAPSHOT in the shaded jar.
    [INFO] Including rhino:js:jar:1.7R1 in the shaded jar.
    [INFO] Attaching shaded artifact.
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java docs
    [INFO]    task-segment: [clean, package]
    [INFO] ------------------------------------------------------------------------
    [INFO] [clean:clean]
    [INFO] Deleting file set: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-docs/target (included: [\*\*], excluded: [])
    [INFO] [resources:resources {execution: filter-docs}]
    [INFO] Using 'UTF-8' encoding to copy filtered resources.
    [INFO] Copying 6 resources
    [INFO] [site:attach-descriptor]
    [INFO] [assembly:single {execution: zip}]
    [INFO] Reading assembly descriptor: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-docs/src/main/assembly/zip.xml
    [INFO] Building zip: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-docs/target/jslint4java-docs-1.4-SNAPSHOT.zip
    [INFO] ------------------------------------------------------------------------
    [INFO] Building jslint4java distribution
    [INFO]    task-segment: [clean, package]
    [INFO] ------------------------------------------------------------------------
    [INFO] [clean:clean]
    [INFO] Deleting file set: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-dist/target (included: [\*\*], excluded: [])
    [INFO] [site:attach-descriptor]
    [INFO] [assembly:single {execution: dist}]
    [INFO] Reading assembly descriptor: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-dist/src/main/assembly/dist.xml
    [INFO] Building zip: /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-dist/target/jslint4java-1.4-SNAPSHOT-dist.zip
    [INFO] 
    [INFO] 
    [INFO] ------------------------------------------------------------------------
    [INFO] Reactor Summary:
    [INFO] ------------------------------------------------------------------------
    [INFO] jslint4java parent .................................... SUCCESS [10.958s]
    [INFO] jslint4java ........................................... SUCCESS [9.327s]
    [INFO] jslint4java ant task .................................. SUCCESS [6.435s]
    [INFO] jslint4java docs ...................................... SUCCESS [3.104s]
    [INFO] jslint4java distribution .............................. SUCCESS [1.791s]
    [INFO] ------------------------------------------------------------------------
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESSFUL
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time: 32 seconds
    [INFO] Finished at: Mon Aug 03 12:10:02 EDT 2009
    [INFO] Final Memory: 48M/81M
    [INFO] ------------------------------------------------------------------------
    
  3. Note the path to the jslint4java-ant/target/jslint4java-1.4-SNAPSHOT-shaded.jar file, this is the jar file you'll need to reference in the build.xml file.
    $ find `pwd` -name '\*.jar' -print
    /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java/target/jslint4java-1.4-SNAPSHOT.jar
    /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/jslint4java-ant-1.4-SNAPSHOT-shaded.jar
    /Users/ashamash/projects/jslint4java/src/jslint4java/jslint4java-ant/target/jslint4java-ant-1.4-SNAPSHOT.jar
    

During this exercise, I was exposed to two new (for me) technologies, git and Maven (I'm personally an svn and ant type of person).  I especially liked the Maven integration in NetBeans.  Both experiences were quite good, but that's a story for another day.

Tuesday Jul 07, 2009

Using svnsync to migrate an SVN repository to another site

I recently had to migrate our SVN repository from one SVN service to another.  Normally, this is done via svnadmin dump and svnadmin load, but I did not have administrative command line access to one of the environments, so I had to use the svnsync tool to accomplish this.  Took me a while to figure out how to do it, and naturally and I didn't write it down, so when I had to do this the second time around, I forced myself to blog it, so when I had to do it the third time, I woul dhave some reference material.  This blog entry really has little to do with sharing with you (the reader), but rather sharing with "future me" ;-)

Anyway, the tool to use is svnsync, but the docs are a bit sparce.  Mirroring the repository is a two step process.

  1. First, the destination repository has to be set up via the svnsync init command, which confusingly takes the DESTINATION repository URL argument before the SOURCE repository URL (this one stumped for a while):

    svnsync init --sync-username username --sync-password password DEST_REPOSITORY_URL SRC_REPOSITORY_URL

  2. Once the destination repository is set up, you can use the svnsync sync command to initiate the copy.  In case the sync fails (e.g. due to network failures), this command can be restarted as frequently as necessary to copy the entire repository.  The primary use of svnsync is to create mirrors, so some folks even run this out of cron to periodically sync the repositories:

    svnsync sync --sync-username username --sync-password password DEST_REPOSITORY_URL

Friday Jun 19, 2009

Netbeans, JavaScript, Ant, Hudson, and JSLint: Oh my!

I agree with Douglas Crockford on two points he makes in his book:

  • ... JavaScript became the language of the Web by default, making its popularity almost completely independent of its qualities as a programming language ...
  • Most programming languages contain goo⁞d and bad parts, but JavaScript has more than its share of the bad, having been developed and released in a hurry before it could be refined.

The first point is becoming more debatable, as other RIA technologies (including Sun's JavaFX) become more prominent, but let's leave that aside for now.

As for the second point, there is a lot developers can do to make the experience with JavaScript a whole lot better.  Don't get me wrong, JavaScript can be amazing, but it can also be a nightmare.  Part of the nightmare is that the development environments and tools for JavaScript are not yet on par with other environments.  My current project involves writing a whole lot of JavaScript.  We've gone through the usual JavaScript cycles - we've been burned by both programming errors as well as deployment errors.

This blog entry describes how we've integrated JSLint into our Continuous Build/Integration environment, so we can better control the code that we write.  There are other techniques that we've implemented to improve our experience with JavaScript that I'll describe in future blogs:

  • Build time automated concatenation of all the JavaScript files into a single file that has a unique file name, so that it can be cached.  This of course implies automated munging of the .html/.jsp pages that source the JavaScript files...
  • Selenium - for automated testing of our Web UI - eases the fear of writing and refactoring our JavaScript code.

First, download the components that you'll need, and install them in a tools directory somewhere:

  1. Rhino - JavaScript for Java - Rhino is an open-source implementation of JavaScript written entirely in Java. It is typically embedded into Java applications to provide scripting to end users.
  2. JSLint itself - get this version which is prebundled to work with Rhino.

This blog assumes you already have a Netbeans project created.  Add the following properties to your nbproject/project.properties file.  The CQS prefix that I use is the name of our project, you should change it to match yours.  As for whether you want your build to fail based on whether JSLint passes or not, that's your call, but I highly recommend it, otherwise developers (myself included) will tend to ignore the output of JSLint.

#
# cqs.build.jslint: true/false setting to determine whether to run the
# jslint javascript analysis utility on the individual javascript
# files.  Default is true for production purposes, but developers can
# set it to false in the file private/private.properties.  It is
# highly recommended that this be left to true.
#
# cqs.build.jslint.failonerror: this true/false setting determines
# whether the build should fail or not if jslint detects any errors.
#
cqs.build.jslint=true
cqs.build.jslint.failonerror=true 

Next, define your list of JavaScript files you want to analyze. 

cqs.js.jslint.files=\\
       js/utilities.js,\\
       js/3rdparty.js,\\
       js/tooltipsprop.js,\\
       js/publish2.js,\\
       js/manageusers.js,\\
       js/login.js,\\
       js/detailhtmlformatter.js,\\
       js/staticsearch.js,\\
       js/searchonready.js,\\
       js/mylibrary.js

Another technique others use is to analyze all the .js files in a given directory, this approach is described in various blogs such as this one.  However, we maintain a list of JavaScript files that are required, since this is used elsewhere in our build scripts (e.g. the automated concatenation aspects of our build script use it).

Next, you'll be making a series of changes to your build.xml file.  First, some stuff that we need to define, since we use it later.  The targets use the propertyregex task defined in ant-contrib, so you'll need to define this task in order to use these targets, this needs to come before the target definitions.  I don't know how people used Ant before ant-contrib was made available:

<property name="tools.dir" value="${basedir}/../tools" />
<property name="ant.contrib.jar.file" value="${tools.dir}/ant/lib/ant-contrib-1.0b3.jar" />
<available file="${ant.contrib.jar.file}" property="ant.contrib.jar.file.available" />
<fail unless="ant.contrib.jar.file.available" message="Ant contrib library not installed" />
<taskdef name="propertyregex" classname="net.sf.antcontrib.property.RegexTask">
    <classpath>
        <fileset dir="${tools.dir}/ant/lib" />
    </classpath>
</taskdef> 

Modify the appropriate Netbeans build target to include a "jslint" target:

<target name="-pre-dist" depends="jslint" >
</target>

These are the actual Ant targets for implementing jslint:

<target name="jslint">
    <if>
        <istrue value="${cqs.build.jslint}"/>
        <then>
            <antcall target="doJSLint" inheritAll="true" inheritRefs="true" />
        </then>
        <else>
            <echo message="Not running JSLint - JSLint is disabled." />
        </else>
    </if>
</target> <!-- jslint -->

<target name="doJSLint">
    <echo level="info" message="JSLint: running...." />
    <property name="rhino.jar.file" value="${tools.dir}/rhino1_7R2/js.jar"/>
    <property name="jslint.js.file" value="${tools.dir}/jslint/jslint.js"/>

    <available file="${rhino.jar.file}" property="rhino.jar.file.available" />
    <fail unless="rhino.jar.file.available" message="Rhino Jar file not found - expected at ${rhino.jar.file}" />

    <available file="${jslint.js.file}" property="jslint.js.file.available" />
    <fail unless="jslint.js.file.available" message="jslint.js file not found - expected at - ${jslint.js.file}" />

    <propertyregex property="cqs.js.jslint.files.spacedelim" input="${cqs.js.jslint.files}" regexp="," replace=" "/>

    <echo level="info" message="JSLint: running on ${cqs.js.jslint.files.spacedelim}...." />
    <exec dir="${basedir}/web" executable="java" failonerror="${cqs.build.jslint.failonerror}">
        <arg line="-jar ${rhino.jar.file} ${jslint.js.file} ${cqs.js.jslint.files.spacedelim}"/>
    </exec>

    <echo level="info" message="JSLint: finished running...." />
</target> <!-- doJSLint -->
And that's it!  JSLint will now be run as part of the regular build.  Within Netbeans, you can just "deploy" the project, that will run it.  On the command line, "ant dist" will do the trick.

We use Hudson for continous build/continuous integration, so every time a source file is checked in, a full build, including a JSLint pass, is done.

Friday May 22, 2009

Debugging raised to a whole new level

Usually, I can figure out the answers to my own problems.  After all, that's my job as an engineer - I solve problems.  But today, I'm totally stumped.

I discovered a bug this morning.  This is not new.  I find lots of bugs all the time.  Usually, I introduce a lot of them.  A good day is one where I fix more bugs than I introduce.  Those rarely happen.  ;-)

Here is the bug that I saw today in the morning, when I started up my day.  The first thing I do every day is look at the Sun Software Library (to see what new stuff has been posted there).  The Sun Software Library is what I work on.

a friend and a companion

At first, I thought it was a fancy computer virus, but then I realized, wait a second, I'm using a Mac.

So I tried my debugging tool of choice: netbeans.

Alas, this is one bug that not even Netbeans can tackle.

So what to do?  An ant crawled into my monitor, and is now walking around sandwiched between the LCD panel and the plastic/glass/whatever the clear panel is on top of the LCD display.  Well, I'll make the best of it, I need the extra brain power to help me with writing code anyway.  I hear that ants can do amazing things, maybe I can put this one to work for me?

This ant knows PHP better than I do!  It led me to exactly what I was looking for.


And JavaScript?  Well, that can't possibly have been meant for human beings, the ant is way better at it than I am ;-)

Monday Apr 06, 2009

Deploying Liferay into an existing Glassfish Container

In a previous blog entry, I described how to get started with LifeRay.  I wanted to use the latest version of GlassFish, so I went through the effort to figure out how to deploy LifeRay into an existing instance of GlassFish.  This blog describes how to do this.

Follow these steps to get LifeRay deployed on an existing (or new) instance of GlassFish:

  1. Download GlassFish v2.1 from https://glassfish.dev.java.net/downloads/v2.1-b60e.html and follow the installation instructions on this page.
  2. Start the domain, test it, make sure it is working.  These examples assume you used the "setup.xml" file that is distributed with GlassFish.  If not, please adjust accordingly.

    $ ./bin/asadmin start-domain domain1

  3. Download the Liferay Portal 5.2.2 WAR file as described here: http://www.liferay.com/web/guest/downloads/additional, and deploy it.

    $ ./bin/asadmin deploy ~/downloads/liferay-portal-5.2.2.war
    Command deploy executed successfully.


  4. Take a look at the log file, you will see lots of java.lang.ClassNotFoundException  exceptions:

    $ less domains/domain1/logs/server.log

  5. In preparation for fixing the errors, shut down GlassFish:

    $ ./bin/asadmin stop-domain domain1
    Domain domain1 stopped.


  6. Download the Liferay Portal 5.2.2 Dependencies as described here: http://www.liferay.com/web/guest/downloads/additional, and unpack the ZIP file somewhere.
  7. Copy the JAR files in the ZIP file into the domain/domain1/lib directory:

    $ cp ~/downloads/liferay-portal-dependencies-5.2.2/\*.jar domains/domain1/lib

  8. Copy the three extra JAR files needed from the LifeRay deployment into the domains/domain1/lib directory as well:

    $ cp domains/domain1/applications/j2ee-modules/liferay-portal-5.2.2/WEB-INF/lib/xalan.jar domains/domain1/lib
    $ cp domains/domain1/applications/j2ee-modules/liferay-portal-5.2.2/WEB-INF/lib/xercesImpl.jar domains/domain1/lib
    $ cp domains/domain1/applications/j2ee-modules/liferay-portal-5.2.2/WEB-INF/lib/serializer.jar domains/domain1/lib


  9. Start up the app server again:

    $ ./bin/asadmin start-domain domain1

  10. You are now good to go, with LifeRay deployed
  11. A suggestion: you may start seeing "java.lang.OutOfMemoryError: PermGen space" errors in the server log file.  I started seeing these errors as soon as I deployed my app in addition to LifeRay, using JDK 1.6 on MacOS 10.5.6.  The default 192MB setting for GlassFish does not seem to be sufficient.  I increased the setting to 384MB by editing the file domains/domain1/config/domain.xml:

    $ grep -i perm domains/domain1/config/domain.xml
            <jvm-options>-XX:MaxPermSize=384m</jvm-options>

Wednesday Apr 01, 2009

Getting started with Glassfish, Liferay, Netbeans, PortalPack, and HelloWorld Portlet

Recently, I had a task to build a HelloWorld portlet deployed to a liferay portal server.  I looked around, couldn't find any up to date blog entries that really covered this topic.  After I went through the startup pain, I figured I'd share my findings.

The examples below work on Mac OS X v10.5.6 using Java 1.6.

I followed these steps to get it working.

First major step is to get Glassfish working with Liferay on your Mac:

  1. Download the Liferay portal bundled with Glassfish from http://downloads.sourceforge.net/lportal/liferay-portal-glassfish-darwin-5.2.2.jar.
  2. Follow the Quick Installation Instructions - Installing GlassFish from a JAR file from Liferay, with the following additional steps:
    1. To finish the Liferay installation, you need to start up the Glassfish app server.  The following command will work, assuming you did not change the domain name in the setup.xml file:

      $ ./bin/asadmin start-domain domain1
    2. Look in the log files:

      $ tail -f domains/domain1/logs/server.log

      Look for these kind of entries:

      [AutoDeploy] Selecting file /Users/ashamash/projects/omalley/liferay-portlet/glassfish/domains/domain1/autodeploy/liferay-portal.war for autodeployment.
    3. Ultimately, you will see an error message:

      java.lang.NoSuchMethodError: org.apache.xml.utils.DefaultErrorHandler.<init>(Z)V
              at org.apache.xalan.processor.TransformerFactoryImpl.<init>(TransformerFactoryImpl.java:1002)

    4. Stop the application server:

      $ ./bin/asadmin stop-domain domain1
    5. Copy the following libraries into the ./lib directory:

      $ cd domains/domain1/applications/j2ee-modules/liferay-portal/WEB-INF/lib/
      $ cp xercesImpl.jar serializer.jar xalan.jar ../../../../../../../lib

      $ cd ../../../../../../..
    6. Start the application server again.  It should start up OK this time.

      $ ./bin/asadmin start-domain domain1
    7. Be patient, it takes 30-60 seconds to start up Liferay.
    8. Now, you can navigate to your portal at http://localhost:8080/.
  3. Now that you have Liferay working for you, you can log in via the credentials provided on the Quick Start Guide:
    • Email address: bruno@7cogs.com
    • Password: bruno
You are now ready to build your first HelloWorld portlet.  I did this with Netbeans and the Portal Pack.  Follow these steps:
  1. Start with Netbeans 6.5
  2. Add the Portal Pack.  I did this by downloading portal-pack-plugin-3_0_all.zip from the Portal Pack downloads page.
  3. Go through the Getting Started with Portal Pack 3.0 tutorial, it's really good, it took me a grand total of 10 minutes to build the HelloWorld portlet.

The advantage of using a Netbeans Portal Pack is that Netbeans takes care of all the packaging and deployment stuff for you.  I was able to build, deploy, and run my portlet simply by running the project.  Pretty neat stuff.

Netbeans

Tuesday Dec 02, 2008

Using Brio Intelligence to query MySQL databases on Solaris

I'm part of the engineering group responsible for building the Sun Software Library.  The deployment architecture for this site is the classic 3-tier based model, with glassfish and MySQL acting as the application server and database server, respectively.

Quite understandably, our (internal to Sun) users wanted all sorts of reports to understand the data.  As engineers, we wanted to avoid building custom reports, since we felt like we'd end up in an endless loop of "can you get me this data" and "can you get me that data".  So we looked for tools that enable our end users to query the data directly and produce the reports themselves.

There are two tools that are popular within Sun for these kind of purposes: OpenOffice Base and Brio (originally from Hyperion, which was acquired by Oracle).  We are using the following:

  • MySQL 5.0.51 Source distribution
  • Brio Intelligence v6.6.4 on a Solaris 10 based SPARC system

About Brio

This blog entry describes how to get Brio to query a MySQL database, which unfortunately was not trivial. The assumption is that Brio and MySQL are already installed, and the database contains schema and data.

Special Kudos to May Mu, Donna Ling, and Wendy Klinke at Sun for helping figure this out.

Brio Intelligence (or at least the version that we have) does not support integration with MySQL directly (the way OpenOffice Base does).  In order to get Brio to query against MySQL, you need to install the MySQL ODBC driver on your Brio system, and configure the ODBC driver to query MySQL.  This is not hard to do, just follow these steps:

  1. Download the MySQL ODBC driver.  You specifically need version 3.51.06, which comes in a file named "MyODBC-3.51.06-sun-solaris2.9-sparc.tar.gz".  I tried it with the latest version (3.51.27), and it doesn't work, as this later version seems to be trying to link against libodbcinst.so.1.  This issue doesn't exist with the older version of the driver.

  2. Unpack the file by typing in "gunzip < MyODBC-3.51.06-sun-solaris2.9-sparc.tar.gz | tar xfvp -", as follows:
    $ gunzip < MyODBC-3.51.06-sun-solaris2.9-sparc.tar.gz | tar xfvp -
    x MyODBC-3.51.06-sun-solaris2.9-sparc, 0 bytes, 0 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/ChangeLog, 12595 bytes, 25 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/COPYING, 19106 bytes, 38 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/README, 1949 bytes, 4 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/INSTALL-BINARY, 5253 bytes, 11 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/RELEASE-NOTES, 5478 bytes, 11 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/odbc.ini, 851 bytes, 2 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3-3.51.06.so, 354904 bytes, 694 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3.a, 187388 bytes, 366 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3.la, 839 bytes, 2 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3.lai, 840 bytes, 2 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3_r-3.51.06.so, 359276 bytes, 702 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3_r.a, 187388 bytes, 366 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3_r.la, 855 bytes, 2 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3_r.lai, 856 bytes, 2 tape blocks
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3.so symbolic link to libmyodbc3-3.51.06.so
    x MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3_r.so symbolic link to libmyodbc3_r-3.51.06.so      
  3. You now need to edit the odbc.ini file that came with the driver to include information about your database.  You can use your favorite text editor for this, I personally use Emacs.  The odbc.ini file looks as follows (the same file demonstrates how to set up multiple databases, I only include one for simplicity):
    ;
    ;  odbc.ini configuration for MyODBC and MyODBC 3.51 Drivers
    ;
    
    [ODBC Data Sources]
    database_name      = database_description
    
    [database_name]
    Driver       = /home/as32693/downloads/MyODBC-3.51.06-sun-solaris2.9-sparc/libmyodbc3.so
    Description  = Connector/ODBC 3.51 Driver DSN
    SERVER       = bco132.central.sun.com
    PORT         = 3306
    USER         = reports
    Password     = 
    Database     = catalog_reporting
    OPTION       = 3
    SOCKET       = 
    
    You should replace the database_name, database_description, and SERVER, USER, and Database options with the appropriate values.  database_name and database_description can be anything you want (they show up inside of Brio, when you are prompted to choose the database to query).  You need to change the Driver to the actual directory you are using.  SERVER, USER, and Database need to be valid, your MySQL DBA can provide you this information.

  4. You now need to set two environment variables that the MyODBC library needs.  These can be put into your appropriate files (e.g. .cshrc or .bashrc).  I personally put them into a file named "run-brio" as follows:
    #!/bin/bash
    
    export ODBCSYSINI=/home/as32693/downloads/MyODBC-3.51.06-sun-solaris2.9-sparc
    export ODBCINI=$ODBCSYSINI/odbc.ini
    
    /usr/dist/exe/brio
  5. And now you can run Brio by typing "run-brio", and configure it to use an ODBC connection.  The following sequence of images demonstrate this.



  6. Select ODBC as the "Connection Software" and "Type of Database".



  7. When prompted, select your database host (this is the same name that was in the odbc.ini file), and type in your username and password.



  8. If you get the following error message: 'MySQL][ODBC 3.51 Driver]Client does not support authentication protocol requested by server: consider upgrading MySQL client", which looks like the following:



    Then you need to ask your MySQL administrator to change the authentication method on the user provided, as described in this post http://forums.mysql.com/read.php?38,2511,2828#msg-2828 and http://dev.mysql.com/doc/refman/5.0/en/old-client.html.  It's also quite possible that the later ODBC drivers work with Brio, and support the latest authentication schemes.  If you get this to work, let me know.

  9. If your username and password are accepted, press "finish", you'll be prompted to save the OCE file.  If you plan on reporting against this database frequently, you should save the OCE file, so you can connect to this database through it every time.



  10. And you are now all set!  Here is how Brio looks like connecting to a MySQL database.  The attributes table is unique to our schema.


I hope this is useful to you.


About

Ari Shamash is the software engineering manager of the Sun Software Library engineering team at Sun Microsystems.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today