Úterý VII 22, 2008

Hudson as a tool for testing? Yes!

Lots of folks in NetBeans started to use Hudson as a tool for continuous integration, or just for building NetBeans from various clones, or to build test, whatever you can imagine. Just take a look at Deadlock machine and you will see a plenty of NetBeans related jobs. We (NetBeans QE) recently got an idea to utilize Hudson as our testing infrastructure. Yes, NetBeans have a lot of automated tests that are being ran daily and test results are evaluated for potential problems. So you have tests, you have machines, but you still need an infrastructure to connect this together. We chose Hudson, because it's a great tool, and what's really cool - is extensible by various plugins (available on Hudson's web, or you can write your own if you wish). The basic idea was to use Hudson as "master-slave" architecture, where you simply set up on master what do you want to run, and master distributes the work to the slaves, that execute the jobs. More detailed approach to this - configure on master which tests do you want to run on which platforms, and they will get executed on appropriate slaves. And getting one level lower - to achieve this, you have to have something we called generic Ant script, which will provide all the stuff needed to execute test:
  1. Download IDE - the IDE build which will be tested
  2. Download NetBeans Test Distribution - Test Distribution contains binaries of all tests from NetBeans repo.
  3. Run selected tests - Specify which tests from Test Distribution will run in tested IDE
So i wrote and Ant script with plenty of useful targets to provide this functionality (Yeah, by this I learned a lot how to write Ant scrips:-) ). This Ant script is generic, and all the stuff like where to take IDE build, where to take Test Distribution, which tests run, what to do with results etc. are controlled by passing corresponding properties. Once we have this working, the only thing we have to do, is to distribute the Ant script to machine we want to run tests on, and provide some better UI than command line for the main properties, which control the test execution. Therefore me and Max Sauer extended Hudson by Test Run Plugin (sources are not public right now), that enables Hudson to copy our generic Ant script on target slave machines and extends job configuration with our panel for setting properties for testrun. With all this stuff working, people can simply log in on master, select one or more slave machines on which will tests run, configure which build will be tested by which test (through nice GUI :-) ), run the job and take a nap while watching the progress of tests execution on target machines. When test run is finished, you get a nice overview of test results (yes, built-in feature of Hudson).
The main benefits we take from Hudson (aka features we really like):
  • Master-slave architecture - you can have various machine with various architectures/OS added into testing farm.
  • JUnit results processing - nice HTML, results trends, charts,...
  • Scheduler - triggering of testrun by another job (which can listen e.g. if new production build is available), or "cron-like" settings.
  • Email notification - send email if tests do not pass.
  • SCP plugin - we can send test results to remote machine that can parse them and store into DB.
  • Authentication - simply configure and point Hudson to your LDAP server, really out of box functionality.
  • Multiple JDK installations - specify location where are JDKs located and voila - you can run whole matrix of tests - one axis is JDKs, second is platform.
  • ...many many others:-)
And conclusion? Hudson is not only contentious integration tool, it can be almost whatever you like.

Pátek III 14, 2008

Project with sharable libraries (AKA How to avoid broken references)

Till now:

If you tried to share with someone your NetBeans project, where you depend on some libraries that you've defined in IDE, you probably received complains: "Your project does not work for me, I've got there broken references, everything's red underlined,.....and...I don't have time to do some tweaking, resolving...ummmm....maybe I'm not so interested in your project..:-)" Well, that's because your friend's IDE was missing definitions of libraries, that your project needs to work.

Since now:

If you share your NetBeans project (e.g. using VCS), your friend just simply checks out your project and libraries with their definitions. Project is then ready to use out of the box.

Example of typical workflow:

  1. Create you project under some versioned folder. When creating project, don't forget to check Project Shared With Other Users checkbox. Do not change location for sharable libraries, it's good to have them on the same level as project folder(s).
  2. Add dependencies on libraries that your project needs. Notice that when you invoke Add library on your project, you are operating with shared libraries (i.e. libraries stored in location you defined), not global libraries. Good practice is that when you are adding new library into shared location, choose that you want to copy library JAR(s) into shared location. It ensures that definitions and JARs are physically at the same location.
  3. Once you do some coding, it's time to check in your sources into VCS. Notice that also newly added libraries are offered for commit.
  4. That's it! If someone checks out your project (as well as libraries folder as location for shared libraries), project is ready to use out of the box.

TIP: If you have some projects created by NetBeans 6.0 and you want to make them sharable, there's a wizard which can make project to use sharable libraries. Just go to project properties, and on Libraries panel you'll see Make sharable... button. This button triggers a wizard, which switches your project into sharable concept and imports necessary libraries to your shared location.

For those who are interested here is some documentation of this great new feature. Feel free to try it out in NetBeans 6.1 Beta.


Tomas Musil


« červenec 2016