This was originally posted on my dev2dev blog November 21st, 2007. ALSB has been rebranded as Oracle Service Bus.
Scheduling tasks for Java environments seem to come up fairly often. Often times cron or Windows's Scheduled Tasks might be used, but that's not necessarily ideal if your application servers are spread across multiple machines because your scheduling mechanism could now have a single point of failure on that particular OS instance. In this post, I'll review a customer situation that is well-suited for a scheduled polling solution and discuss the various options that I researched in WebLogic to implement it.
The Use Case - A Database Event Generator
One of my customers recently purchased AquaLogic Service Bus (ALSB) and AquaLogic Data Services Platform (ALDSP). One of their main use cases is to be able to monitor database tables for new records. This particular customer has an aversion to database triggers, which is probably the most common solution to this type of problem. Therefore, we need to rely on something external to the database to detect these events. Unfortunately, this is not an out-of-the-box feature of either ALSB or ALDSP today, although it is a feature of WebLogic Integration (WLI). One of the options for WLI's event generator is to use a query-based polling approach, which executes a select statement that returns new records, publishes the events, and updates each record to indicate it has been processed. It does this on a schedule. I thought that a similar approach could be implemented fairly easily in a regular J2EE application. For example, a session bean method called on a regular basis that we could deploy to the ALSB server since it has WLS underneath the covers.
Timer Options in WebLogic Server
So the question now becomes, what mechanism can be used to schedule the invocation of the session bean on a regular basis? A key requirement is that the solution should work well in a clustered environment such that it will fail-over to another server in a cluster, yet doesn't redundantly execute on each server in the cluster. I searched around and came up with several options for scheduling with timers in WebLogic.
- EJB 2.1 Timer Service
- Workshop Timer Control
- CommonJ Timer Manager
EJB 2.1 Timer Service
The EJB 2.1 Timer Service is supported in WLS 9.2 and WLS 10 and initially seemed like a good solution since it is a J2EE standard. However, upon a little more investigation I ruled it out mainly because the timer object cannot be migrated from server to server. Of course, it can take advantage of Whole Server Migration, but that is not currently implemented at this customer, and it has additional infrastructure requirements, such as a SAN, that add complexity. Additionally, the Timer Service is not supported with WebLogic Clustering. There are 2 possible compromises mentioned in the documentation, but neither one of those is ideal.
Workshop Timer Control
This mechanism is covered very well with a nice tutorial in the documentation and is very easy to develop. However, this approach is also not designed to operate in a cluster with support for fail-over to another managed server. Furthermore, using web-services may not be optimal with respect to reliability for once and only once messaging.
CommonJ Timer Manager
The CommonJ Timer and Work Manager specification was jointly developed between BEA and IBM to address the limitations of Threads and Timers when used in a managed container. See the main page for the specification on dev2dev or review the documentation for additional detail and examples. The WebLogic Job Scheduler functionality is specifically designed to work in a clustered environment and therefore, provides a solution that meets the main requirements.
A Simple Prototype
For this post, I'll discuss a simple prototype that I created that works well in a single-server environment. Some of the details of how the Job Scheduler works will prevent this approach from being used without modification in a clustered environment, but I'll save those details for a subsequent posting.
Event Generator Components
StartupServlet and web.xml
The init() method registers the TimerListener with the TimerManager that is configured web.xml, which also tells this servlet to start up automatically when the application starts. The web.xml also has some other configuration components such as the sql to execute, how often to check for new rows, the max to process at one time etc. By putting these details in web.xml, I can easily use a Deployment Plan to change these values later without changing my code. The Here's how easy it is to create the TimerManager in web.xml:
<resource-ref>
<res-ref-name>timer/eventGeneratorTimer</res-ref-name>
<res-type>commonj.timers.TimerManager</res-type>
<res-auth>Container</res-auth>
<res-sharing-scope>Shareable</res-sharing-scope>
</resource-ref>The TimerManager javadoc discusses this is detail.
TimerListener
A normal TimerManager is configured on the environment. The TimerListener’s responsibility is to invoke the SessionBean. In this simple prototype, I had the StartupServlet implement the TimerListener interface itself. This aspect of the prototype will need adjustment when moving to the JobScheduler, but this illustrates the concept nicely for a prototype.
SessionBean
The session bean responsibility is to query the database table for events by checking a processed column on the table (or using an association table). For each new row that is found, mark it as processed and send the row id to a jms queue, which is presumably picked up later by an ALSB proxy or some other component. By making this transactional we can ensure once and only once behavior if we use an XA database driver and XA enabled jms connection factory.
JMS Queue
The queues only responsibility is to hold messages containing row ids of db events. Admittedly, this is not as sophisticated as the WLI solution, which can contain more data about the row, but we could always add more data later if necessary.
Database Table
For this scenario, I created a simple SHIPMENT table in the AQUA schema of my Oracle XE database. The PROCESSED column is what will tell the Event Generator whether this row needs to be processed or not. Since we'll be querying on that, we'd better have that indexed as well. The assumption here is that some other process inserts these rows, and now we want to detect shipments and do something like send out an Advance Shipment Notice via ALSB.
CREATE TABLE AQUA.SHIPMENT (
SHIPMENT_ID NUMBER(38,0) NOT NULL,
SHIPMENT_DATE TIMESTAMP(6) DEFAULT SYSDATE,
CUST_ID NUMBER(38,0) DEFAULT 0,
PROCESSED NUMBER(1,0) DEFAULT 0,
PRIMARY KEY(SHIPMENT_ID)
)CREATE INDEX AQUA.PROCESSED_IDX ON AQUA.SHIPMENT(PROCESSED)
try it out
My project code is posted here. I used ALSB 2.6, which runs on WebLogic Server 9.2 MP1 as my server runtime. I used Workshop 10.1 as my IDE as it has many improvements over the Workshop 9.2.1 that ships with ALSB 2.6. You can use my code with Workshop 9.2.1, but the Eclipse projects will not be able to be imported without Eclipse version issues, so you'll just have to use the source files if you go that route and recreate the projects manually.
You'll have to setup a datasource, jms connection factory, and jms queue in the WLS console. Since I used an ALSB, I already had a connection factory, and just setup a queue and a datasource. Once you have these setup, you'll need to update the annotations in the DBEventGeneratorSessionBean.java so the resource references match the names you used in the console.
Once you deploy the application, you see see log statements like this, with new entries every 10 seconds:
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>
To see if the event generator works, execute some SQL like this:
DELETE FROM AQUA.SHIPMENT
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(1)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(2)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(3)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(4)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(5)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(6)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(7)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(8)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(9)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(10)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(11)
Now the log should show something like this:
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() found 10 rows>
<queryForEvents() updates complete>
<queryForEvents() jms messages sent>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() found 1 rows>
<queryForEvents() updates complete>
<queryForEvents() jms messages sent>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>
This illustrates that we're checking for up to 10 new records every 10 seconds. When you go look at your JMS Queue, it should now have 11 messages , each with a SHIPMENT_ID of one of the inserts you issued. This is the part where we'd connect an ALSB proxy to the JMS queue, which could send an email, an ASN, etc. That's an entry for another time.
