Tuesday Nov 03, 2009

Hudson: A Continous Integration Tool

Continuous Integration Tools

     A continuous integration tool is an automated build system that checks out the most current code from the source code repository, builds it, and makes the resulting artifacts available for download. Such tools come handy when your application has multiple modules and there are multiple engineers working on them. These tools could be used to integrate these various modules, build the system and also maybe run some tests to ensure that everything is fine.

Hudson: What is it?

     Hudson is an open-source continous integration tool, which has become very popular for some time now. It provides various features like options for checking out the source code from various version controlling systems like CVS, SVN, etc., setting the version of Java to be used, the machine(s) on which to run the integrations/builds, notifying an user or a group of users about a build failure, scheduling the job execution, etc. More information can be obtained from the Hudson project site at http://hudson-ci.org/.

Master-Slave Configuration

          One interesting feature of Hudson is its provision to run a job in a master-slave configuration, i.e., there would be a Hudson master machine which would take up all the requests like defining a job, configuring it, triggering it, etc., while there will be a set of slave machines on which the executions would actually happen.

        Lets say you want to define a job which has to be run on more than one platform. You define it in the Hudson master, the interface for all your configurations, and then tie it to all the machines that you would want to run the job on. The underlying mechanism of Hudson takes care of all the communication between the Hudson master and each of the slaves which are registered with it.

        Now, we will see how to setup Hudson - the master and slave(s).

Setting up the Hudson Master

    The master could be setup on any OS, for convenience lets assume we are setting it up on Linux.

  • If you are using Ubuntu, be sure to upgrade to 8.04 to avoid problems with RSA keys (keys generated by keygenerator in 7.10 are blacklisted!)
  • Login as someone with root role (e.g. uadmin)
  • sudo useradd -d /space/hudson -m hudson - this creates user hudson with home in /space/hudson (Folder hudson does not need to exist)
  • sudo passwd hudson changes the password for user hudson
  • Login as hudson user
  • Create folder $HOME/jdks and install there jdk1.6.0
  • ssh-keygen -t rsa generate public / private RSA key, public key is used for ssh login to slaves without passwords
  • =touch /space/hudsonserver/master = - creates foo file master as a workaround for hudson issue #936- parent project occuppies executor on slave
  • Download hudson.war bits the Hudson site at http://hudson-ci.org/
  • Install webserver (e.g. tomcat) and deploy Hudson
  • If you do not want to run Hudson in Tomcat, use built-in server Winstone:
    • run java -jar hudson.war to start Hudson master on port 8080
    • NOTE: best idea is to use something like this script (we use in this script non-default port 18080 because this is second instance of Hudson on the same machine):
               # kill running hudson
               kill `ps aux|awk '$13 == "./hudson.war" {print $2}'` 2> /dev/null
               # nohup new hudson
               nohup /space/jdks/jdk1.6.0_05/bin/java -jar ./hudson.war --httpPort=18080 --ajp13Port=18009 &

  • Once you have the war file deployed, you could launch the application.
  • Various settings like different JDKs, MAVEN_HOME could be made at the "Manage Hudson" page by clicking on the "Manage Hudson" link in the side pane.

Setting up the Hudson Slave

The following steps could be used to create a slave on a linux/solaris machine:

  • Create an user "hudson" with home in /space/hudson.
    • sudo useradd -d /space/hudson -m hudson
  • Set a password for this user
    • sudo passwd hudson
  • Login as user "hudson".
  • Create folders "ant" and "jdks" in /space/hudson.
  • Download and unzip ant into folder "ant".
  • Download and unzip various versions of jdk into jdks.
  • Copy the id_rsa.pub of the Hudson master to file /space/hudson/.ssh/authorized_keys. This enables the Hudson master to establish a remote connection to the slave without having to enter the login credentials. To do this:
    • Setup a FTP connection to the Hudson master.
    • Get the id_rsa.pub file from the hidden folder ".ssh".
    • Close the FTP connection.
    • Copy this id_rsa.pub file to /space/hudson/.ssh/authorized_keys. Create the directory .ssh if it doesn't exist already.
    • To verify that the Hudson master's key is successfully added to this slave's "ssh" keys, try setting up a ssh connection from the Hudson master to this slave node, as "ssh hudson@your_hudson_slave". This should setup a connection without prompting for a password.
  • Copy the slave agent "slave.jar" to /space/hudson. The jar file can be obtained from the archive "hudson.war" which has been downloaded to setup the Hudson master. The hudson.war file may be downloaded from the site http://hudson.dev.java.net.
  • Write a small shell script which sets the various paths and starts the slave agent. Name it runSlave.sh. A typical script file would be like:
    • cat /space/hudson/runSlave.sh
    • echo "Starting the slave agent on the node XYZ..."
    • export JAVA_HOME=/space/hudson/jdks/jdk1.6.0_06
    • export ANT_HOME=/space/hudson/ant/apache-ant-1.7.1
    • export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH
    • java -jar slave.jar
    • echo "Slave agent started..."
      • This shell script would be called by the Hudson master. Running the script directly would not start the slave agent, this gets started only when run from the Hudson master.

  • Register this slave machine with the Hudson master in the "Manage Hudson" page.

You have your Hudson Master/Slave configuration ready. Now, you could just go ahead and define, configure and schedule your jobs to run according to your choice.

 In case you have any queries you could consider sending a mail to the Hudson user's mailing list - users@hudson.dev.java.net  which is a pretty active mailing list.


Naresh worked at Sun Microsystems for two years. During these two years he had worked on the Project Metro and Project Jersey.


« July 2016