Thursday Feb 02, 2012

Oracle XE and Hudson Co-Existence

A small roadblock that hit me this morning that is worth documenting if only for my own reference.  I run my main Hudson install on my Linux server somewhere in the wilds(?) of Texas, however, I wanted to check something out with Hudson as a Windows service, something which I've not done for a few months.  

Starting Hudson with java -jar hudson-2.2.0.war however failed with a port bind error. It seems that port 8080 was in use.  I was a bit confused because I didn't have an Apache, Tomcat or OC4J running at that address, something else, something new, had grabbed the port.  Fortunately, as ever, I had TCPview installed and that quickly showed me that the owning process was the TNS Listener process from my newly installed Oracle XE 11g edition. 

Now I could have simply booted Hudson on a different port using the --httpPort argument, however, the chances are that I will be wanted to start other stuff on 8080 as well so XE had to be kicked off of that port for it's HTTP listener. This is done quickly, and without any re-starting, from SQLPlus:

Enter user-name: system
Enter password:
Connected to:
Oracle Database 11g Express Edition Release - Production
SQL> begin
  2  dbms_xdb.sethttpport('8282');
  3  end;
  4  /
PL/SQL procedure successfully completed.
SQL> exit
Disconnected from Oracle Database 11g Express Edition Release - Production

Short and sweet, Oracle now sits on 8282 and 8080 is free for Hudson. 

Thursday Dec 08, 2011

Automating Remote Deployment to Oracle WebLogic from Hudson

As part of my effort to streamline the continuous build processes around my sample and prototype portfolio, I use Hudson to trigger new builds of each sample as it is updated. Thus far I had been getting as far as testing, building (using OJDeploy), precompiling and archiving, but I'm not gotten around to deploying as part of those jobs. 

The main reason for this was due to the fact that the Hudson deployer plugin does not support WebLogic as a target (this is due in turn to a limitation of the underlying Cargo CodeHaus project) . Anyway I digress.

So rather than using the deploy step in a Hudson FreeStyle project, we need to add a normal build step at the end of the chain to handle the WebLogic deployment. Now as it happens, this is pretty easy to do, and given that you'll generally be building ADF projects with OJDeploy in any case, you have everything to hand, and no real additional setup is required. If you're not building and deploying an ADF project then the additional step you will need here is to ensure that there is a WebLogic server installation available on the Hudson machine that is executing the job.

The actual command line to trigger the deploy is pretty simple, however, at it's most basic level it takes a plain-text username and password for the WebLogic administrator.  This is probably not the kind of information that you want to leave sprinkled throughout your job configurations and console logs, so preventing that is the first step.

(Note: Even so I'm running in a pretty simple dev/test environment where the WebLogic is within the same network and I'm not using SSL, if you read the documentation references below you'll see how to beef this up for a more secure environment.)

Step 1. Conceal your Identity

The utility that we'll be using to do the deploy is weblogic.Deployer (more later, but see here for the 10.3.5 doc on this).  As I mentioned, that takes username and password parameters which we don't want, however, it also provides a way to externalize the credentials and that's what I'll use here.  This is a step that you need to carry out on the Hudson executor machine and it takes place outside of Hudson all together. We'll use the weblogic.WLST utility to generate an encrypted credential store to hold the username and password for the remote server I'm about to deploy to. You will need WebLogic or JDeveloper installed on the machine to get access to the  weblogic.Deploy and weblogic.WLST commands.  I'm signifying that location with ${WLS_HOME} here. For the sake of illustration I'm logged in as the user that runs the Hudson process and I'll store the new credential files in a sub-directory of that users home directory called wlskeys. I'm using the bash shell here:

source ${WLS_HOME}/wlserver_10.3/server/bin/
java weblogic.WLST

Initializing WebLogic Scripting Tool (WLST) ...
Welcome to WebLogic Server Administration Scripting Shell
Type help() for help on available commands

wls:/offline> connect('weblogic','secret_admin_password',

Connecting to t3://duncan_wls:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'adf_domain'.
Warning: An insecure protocol was used to connect to the server. To ensure on-the-wire security, the SSL port or Admin port should be used instead.

wls:/adf_domain/serverConfig> storeUserConfig('~/wlskeys/',

Creating the key file can reduce the security of your system if it is not kept in a secured location after it is created.
Do you want to create the key file? y or n
The username and password that were used for this WebLogic Server connection are stored in /home/hudson/wlskeys/ and /home/hudson/wlskeys/duncan_wls_userConfig.key.

wls:/adf_domain/serverConfig> exit()
Exiting WebLogic Scripting Tool.

So there you have it. If you now peek in the specified directory you will be able to see the generated files.  It goes without saying that the files in question are ones to protect! You can go ahead and generate as many of these file pairs as you have servers for Hudson to deploy to.

Step 2. The Deploy command from Hudson 

The actual deploy step from Husdon is defined as a Execute Shell step in your job. To cut down on typing and allow for different directory structures across the Hudson slaves I'm using,  I've predefined some variables in the general Hudson config and overridden them appropriately on the slave node configuration (Node Properties -> Environment Variables):

  • ${WLS_HOME} WebLogic Home directory
  • ${WLS_KEYSTORE} The directory on which I save the properties and key files generated in Step 1 
  • ${TESTING_SERVER} The URL for the Admin Server (e.g. t3://duncan_wls:7001)

 Then the Hudson build step looks looks this:

source ${WLS_HOME}/wlserver_10.3/server/bin/
java weblogic.Deployer -adminurl ${TESTING_SERVER}
  -userconfigfile ${WLS_KEYSTORE}/
  -userkeyfile ${
  -deploy ${WORKSPACE}/deploy/myapp.ear -upload

The -upload parameter is the important one here, that will take the EAR file that Hudson has build and transfer it to the remote machine before trying the install.

Variations on the Theme

This approach is, of course, just one way of achieving this. Other options would include using the FTP-Publisher plugin to push the file to the remote server, and if this was set up to the right place and WebLogic was auto-deploying in development mode you could bypass the need for the WebLogic install on the Hudson executors. However, given that for an ADF application we really need to use OJDeploy to build the app we'll have this stuff to hand anyway. 

Other variations would include using WLST or the WebLogic ANT tasks rather than weblogic.Deploy. The end result and approach are identical in these cases.

I'm sure that the WebLogic plugin for Maven is probably an even better route to go for this entire process, however, I've not Maven-ised my main codebase yet so that will have to wait for another day.

Thursday Aug 11, 2011

Arcane zip commands for your ADF App and Hudson

I'm in the process of automating the builds of some of my sample applications that I create for proof of concepts and the like, and based on the fact that I'm heavily involved in the Hudson project now it made a lot of sense to use Hudson to help me. Now this entry is not about Hudson per-say, it is, however, related in that it's the type of syntax that you would use in a shell command executed from Hudson.

In this case I have a standard ADF project (Model, ViewController and the like)  which has been checked out from Subversion by Hudson and built with OJDeploy.  What I wanted to do in this case was also create a clean source archive from the checked out sources, without having to do a separate SVN export to get a clean copy.  So this involved working out the correct syntax of the unix zip command to exclude the artifacts (such as classes, WAR files and .svn directories) that I didn't want in the source zip file.

Anyway here's the command that creates the source zip in my top level deploy directory - hopefully it will save you a bunch of time if you ever need to do the same thing, it's one of those fiddly to get right things that always takes up way too much time.

The following is all one command, I've just split it up onto two lines for readability:

zip -r ./deploy/${JOB_NAME}_${BUILD_NUMBER} * 
    -x 'deploy/*' -x */deploy/* -x *classes/* -x *.data* -x *.svn*

This command is issued as a Hudson build step from the ${WORKSPACE} which is also the ADF Application workspace root directory


Hawaii, Yes! Duncan has been around Oracle technology way too long but occasionally has interesting things to say. He works in the Development Tools Division at Oracle, but you guessed that right? In his spare time he contributes to the Hudson CI Server Project at Eclipse
Follow DuncanMills on Twitter

Note that comments on this blog are moderated so (1) There may be a delay before it gets published (2) I reserve the right to ignore silly questions and comment spam is not tolerated - it gets deleted so don't even bother, we all have better things to do with our lives.
However, don't be put off, I want to hear what you have to say!


« July 2016