Hudson: A Continous Integration Tool

Continuous Integration Tools

     A continuous integration tool is an automated build system that checks out the most current code from the source code repository, builds it, and makes the resulting artifacts available for download. Such tools come handy when your application has multiple modules and there are multiple engineers working on them. These tools could be used to integrate these various modules, build the system and also maybe run some tests to ensure that everything is fine.

Hudson: What is it?

     Hudson is an open-source continous integration tool, which has become very popular for some time now. It provides various features like options for checking out the source code from various version controlling systems like CVS, SVN, etc., setting the version of Java to be used, the machine(s) on which to run the integrations/builds, notifying an user or a group of users about a build failure, scheduling the job execution, etc. More information can be obtained from the Hudson project site at http://hudson-ci.org/.

Master-Slave Configuration

          One interesting feature of Hudson is its provision to run a job in a master-slave configuration, i.e., there would be a Hudson master machine which would take up all the requests like defining a job, configuring it, triggering it, etc., while there will be a set of slave machines on which the executions would actually happen.

        Lets say you want to define a job which has to be run on more than one platform. You define it in the Hudson master, the interface for all your configurations, and then tie it to all the machines that you would want to run the job on. The underlying mechanism of Hudson takes care of all the communication between the Hudson master and each of the slaves which are registered with it.

        Now, we will see how to setup Hudson - the master and slave(s).

Setting up the Hudson Master

    The master could be setup on any OS, for convenience lets assume we are setting it up on Linux.

  • If you are using Ubuntu, be sure to upgrade to 8.04 to avoid problems with RSA keys (keys generated by keygenerator in 7.10 are blacklisted!)
  • Login as someone with root role (e.g. uadmin)
  • sudo useradd -d /space/hudson -m hudson - this creates user hudson with home in /space/hudson (Folder hudson does not need to exist)
  • sudo passwd hudson changes the password for user hudson
  • Login as hudson user
  • Create folder $HOME/jdks and install there jdk1.6.0
  • ssh-keygen -t rsa generate public / private RSA key, public key is used for ssh login to slaves without passwords
  • =touch /space/hudsonserver/master = - creates foo file master as a workaround for hudson issue #936- parent project occuppies executor on slave
  • Download hudson.war bits the Hudson site at http://hudson-ci.org/
  • Install webserver (e.g. tomcat) and deploy Hudson
  • If you do not want to run Hudson in Tomcat, use built-in server Winstone:
    • run java -jar hudson.war to start Hudson master on port 8080
    • NOTE: best idea is to use something like this script (we use in this script non-default port 18080 because this is second instance of Hudson on the same machine):
               #!/bin/bash               
               # kill running hudson
               kill `ps aux|awk '$13 == "./hudson.war" {print $2}'` 2> /dev/null
               # nohup new hudson
               nohup /space/jdks/jdk1.6.0_05/bin/java -jar ./hudson.war --httpPort=18080 --ajp13Port=18009 &

  • Once you have the war file deployed, you could launch the application.
  • Various settings like different JDKs, MAVEN_HOME could be made at the "Manage Hudson" page by clicking on the "Manage Hudson" link in the side pane.

Setting up the Hudson Slave

The following steps could be used to create a slave on a linux/solaris machine:

  • Create an user "hudson" with home in /space/hudson.
    • sudo useradd -d /space/hudson -m hudson
  • Set a password for this user
    • sudo passwd hudson
  • Login as user "hudson".
  • Create folders "ant" and "jdks" in /space/hudson.
  • Download and unzip ant into folder "ant".
  • Download and unzip various versions of jdk into jdks.
  • Copy the id_rsa.pub of the Hudson master to file /space/hudson/.ssh/authorized_keys. This enables the Hudson master to establish a remote connection to the slave without having to enter the login credentials. To do this:
    • Setup a FTP connection to the Hudson master.
    • Get the id_rsa.pub file from the hidden folder ".ssh".
    • Close the FTP connection.
    • Copy this id_rsa.pub file to /space/hudson/.ssh/authorized_keys. Create the directory .ssh if it doesn't exist already.
    • To verify that the Hudson master's key is successfully added to this slave's "ssh" keys, try setting up a ssh connection from the Hudson master to this slave node, as "ssh hudson@your_hudson_slave". This should setup a connection without prompting for a password.
  • Copy the slave agent "slave.jar" to /space/hudson. The jar file can be obtained from the archive "hudson.war" which has been downloaded to setup the Hudson master. The hudson.war file may be downloaded from the site http://hudson.dev.java.net.
  • Write a small shell script which sets the various paths and starts the slave agent. Name it runSlave.sh. A typical script file would be like:
    • cat /space/hudson/runSlave.sh
    • echo "Starting the slave agent on the node XYZ..."
    • export JAVA_HOME=/space/hudson/jdks/jdk1.6.0_06
    • export ANT_HOME=/space/hudson/ant/apache-ant-1.7.1
    • export PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH
    • java -jar slave.jar
    • echo "Slave agent started..."
      • This shell script would be called by the Hudson master. Running the script directly would not start the slave agent, this gets started only when run from the Hudson master.

  • Register this slave machine with the Hudson master in the "Manage Hudson" page.

You have your Hudson Master/Slave configuration ready. Now, you could just go ahead and define, configure and schedule your jobs to run according to your choice.

 In case you have any queries you could consider sending a mail to the Hudson user's mailing list - users@hudson.dev.java.net  which is a pretty active mailing list.


Comments:

Note that Sun provides support for Hudson through their Sun GlassFish Portfolio offering. See http://wiki.hudson-ci.org/display/HUDSON/Commercial+Support

Posted by Eduardo Pelegri-Llopart on November 04, 2009 at 08:02 AM PST #

11

Posted by guest on April 11, 2010 at 10:24 PM PDT #

Hi

I had recently configured connected master (Windows ) and slave ( Mac)

Now I want to create a job which build on slave. can you please mention steps to follow while creating jobs that can execute on slave.

I had created a script file , which can build on mac system

Posted by Configure jobs on master for slave on January 11, 2011 at 10:23 PM PST #

Hi,

its simple to achieve what you want.
Simply create a job, and in the jobs configuration tie it to the slave that you had installed on the Mac.
This can be done by checking the checkbox "Restrict where this project can be run", and in the Label field enter the slave's label or name.

Posted by Naresh on January 12, 2011 at 04:42 PM PST #

Thank you Shri.Naresh,

Basically , I have one more question. I had created a script file by name "BuildScript.h" and enabled execute shell command under build section,
I had provide the buildscript.sh, but hudson Master(Windows) is not recogonizing the command.

any idea ?

Again thank you for the response.

regards
Dwarak

Posted by Dwarak on January 12, 2011 at 04:54 PM PST #

Dwarak,

are you trying to run the sh file on your Hudson master or is it running on the Hudson master rather than the Mac slave?

Are you sure you have tied the job to the Mac slave?

Posted by Naresh on January 12, 2011 at 05:14 PM PST #

shri.Naresh,

I'm providing command sh file on master (windows), but the file is located on mac pc (Slave).

Yes I connected the slave using JNLP.

I also got this error

/var/folders/gk/gkBq445TE-8JiQ+NSlDd0U+++TI/-Tmp-/hudson5699702322382949051.sh: line 2: /Volumes/Data/TCG_Repo/Code: No such file or directory
Finished: FAILURE

This is SVN local path on slave "/Volumes/Data/TCG_Repo/", but it not recognizing.
Regards
Dwarak

Posted by Dwarak on January 12, 2011 at 06:47 PM PST #

Nice tutorial.....

Well done........

Posted by jayaraj on January 21, 2011 at 03:57 PM PST #

Just found this command "ssh-copy-id" which can be used to copy your id_rsa.pub file to the authorized keys of a remote machine -

ssh-copy-id [-i [identity_file]] [user@]machine

So, no more worries of overwriting existing entries :)

Posted by Naresh on January 23, 2011 at 06:20 PM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Naresh worked at Sun Microsystems for two years. During these two years he had worked on the Project Metro and Project Jersey.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today