X
  • November 21, 2007

WebLogic Scheduling - A polling approach to implement a DB event generator for ALSB

Guest Author

This was originally posted on my dev2dev blog November 21st, 2007.  ALSB has been rebranded as Oracle Service Bus.

Scheduling tasks for Java environments seem to come up fairly often.  Often times cron or Windows's Scheduled Tasks might be used, but that's not necessarily ideal if your application servers are spread across multiple machines because your scheduling mechanism could now have a single point of failure on that particular OS instance.  In this post, I'll review a customer situation that is well-suited for a scheduled polling solution and discuss the various options that I researched in WebLogic to implement it.

The Use Case - A Database Event Generator

One of my customers recently purchased AquaLogic Service Bus (ALSB) and AquaLogic Data Services Platform (ALDSP).  One of their main use cases is to be able to monitor database tables for new records.  This particular customer has an aversion to database triggers, which is probably the most common solution to this type of problem.  Therefore, we need to rely on something external to the database to detect these events.  Unfortunately, this is not an out-of-the-box feature of either ALSB or ALDSP today, although it is a feature of WebLogic Integration (WLI).  One of the options for WLI's event generator is to use a query-based polling approach, which executes a select statement that returns new records, publishes the events, and updates each record to indicate it has been processed.  It does this on a schedule.  I thought that a similar approach could be implemented fairly easily in a regular J2EE application.  For example, a session bean method called on a regular basis that we could deploy to the ALSB server since it has WLS underneath the covers. 

Timer Options in WebLogic Server

So the question now becomes, what mechanism can be used to schedule the invocation of the session bean on a regular basis?  A key requirement is that the solution should work well in a clustered environment such that it will fail-over to another server in a cluster, yet doesn't redundantly execute on each server in the cluster.  I searched around and came up with several options for scheduling with timers in WebLogic.

  1. EJB 2.1 Timer Service
  2. Workshop Timer Control
  3. CommonJ Timer Manager

EJB 2.1 Timer Service

The EJB 2.1 Timer Service is supported in WLS 9.2 and WLS 10 and initially seemed like a good solution since it is a J2EE standard.  However, upon a little more investigation I ruled it out mainly because the timer object cannot be migrated from server to server.  Of course, it can take advantage of Whole Server Migration, but that is not currently implemented at this customer, and it has additional infrastructure requirements, such as a SAN, that add complexity.  Additionally, the Timer Service is not supported with WebLogic Clustering.  There are 2 possible compromises mentioned in the documentation, but neither one of those is ideal.

Workshop Timer Control

This mechanism is covered very well with a nice tutorial in the documentation and is very easy to develop.  However, this approach is also not designed to operate in a cluster with support for fail-over to another managed server.  Furthermore, using web-services may not be optimal with respect to reliability for once and only once messaging.

CommonJ Timer Manager

The CommonJ Timer and Work Manager specification was jointly developed between BEA and IBM to address the limitations of Threads and Timers when used in a managed container.  See the main page for the specification on dev2dev or review the documentation for additional detail and examples.  The WebLogic Job Scheduler functionality is specifically designed to work in a clustered environment and therefore, provides a solution that meets the main requirements.

A Simple Prototype

For this post, I'll discuss a simple prototype that I created that works well in a single-server environment.  Some of the details of how the Job Scheduler works will prevent this approach from being used without modification in a clustered environment, but I'll save those details for a subsequent posting.

Event Generator Components

StartupServlet and web.xml

The init() method registers the TimerListener with the TimerManager that is configured web.xml, which also tells this servlet to start up automatically when the application starts.  The web.xml also has some other configuration components such as the sql to execute, how often to check for new rows, the max to process at one time etc.  By putting these details in web.xml, I can easily use a Deployment Plan to change these values later without changing my code.   The Here's how easy it is to create the TimerManager in web.xml:

   <resource-ref>
            <res-ref-name>timer/eventGeneratorTimer</res-ref-name>
            <res-type>commonj.timers.TimerManager</res-type>
            <res-auth>Container</res-auth>
            <res-sharing-scope>Shareable</res-sharing-scope>
    </resource-ref>

The TimerManager javadoc discusses this is detail.

TimerListener

A normal TimerManager is configured on the environment.  The TimerListener’s responsibility is to invoke the SessionBean.  In this simple prototype, I had the StartupServlet implement the TimerListener interface itself.  This aspect of the prototype will need adjustment when moving to the JobScheduler, but this illustrates the concept nicely for a prototype.

SessionBean

The session bean responsibility is to query the database table for events by checking a processed column on the table (or using an association table).  For each new row that is found, mark it as processed and send the row id to a jms queue, which is presumably picked up later by an ALSB proxy or some other component.  By making this transactional we can ensure once and only once behavior if we use an XA database driver and XA enabled jms connection factory.

JMS Queue

The queues only responsibility is to hold messages containing row ids of db events.  Admittedly, this is not as sophisticated as the WLI solution, which can contain more data about the row, but we could always add more data later if necessary.

Database Table

For this scenario, I created a simple SHIPMENT table in the AQUA schema of my Oracle XE database.  The PROCESSED column is what will tell the Event Generator whether this row needs to be processed or not.  Since we'll be querying on that, we'd better have that indexed as well.  The assumption here is that some other process inserts these rows, and now we want to detect shipments and do something like send out an Advance Shipment Notice via ALSB.

CREATE TABLE AQUA.SHIPMENT (
    SHIPMENT_ID           NUMBER(38,0) NOT NULL,
    SHIPMENT_DATE         TIMESTAMP(6) DEFAULT SYSDATE,
    CUST_ID            NUMBER(38,0) DEFAULT 0,
    PROCESSED        NUMBER(1,0) DEFAULT 0,
    PRIMARY KEY(SHIPMENT_ID)
)

CREATE INDEX AQUA.PROCESSED_IDX ON AQUA.SHIPMENT(PROCESSED)

try it out

My project code is posted here.  I used ALSB 2.6, which runs on WebLogic Server 9.2 MP1 as my server runtime.  I used Workshop 10.1 as my IDE as it has many improvements over the Workshop 9.2.1 that ships with ALSB 2.6.  You can use my code with Workshop 9.2.1, but the Eclipse projects will not be able to be imported without Eclipse version issues, so you'll just have to use the source files if you go that route and recreate the projects manually.

You'll have to setup a datasource, jms connection factory, and jms queue in the WLS console.  Since I used an ALSB, I already had a connection factory, and just setup a queue and a datasource.  Once you have these setup, you'll need to update the annotations in the DBEventGeneratorSessionBean.java so the resource references match the names you used in the console.

Once you deploy the application, you see see log statements like this, with new entries every 10 seconds:

<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>

To see if the event generator works, execute some SQL like this:

DELETE FROM AQUA.SHIPMENT
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(1)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(2)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(3)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(4)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(5)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(6)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(7)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(8)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(9)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(10)
INSERT INTO AQUA.SHIPMENT(SHIPMENT_ID) VALUES(11)

Now the log should show something like this:

<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() found 10 rows>
<queryForEvents() updates complete>
<queryForEvents() jms messages sent>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() found 1 rows>
<queryForEvents() updates complete>
<queryForEvents() jms messages sent>
<timerExpired() with period 10000>
<queryForEvents()>
<queryForEvents() no events to process>

This illustrates that we're checking for up to 10 new records every 10 seconds.  When you go look at your JMS Queue, it should now have 11 messages , each with a SHIPMENT_ID of one of the inserts you issued.  This is the part where we'd connect an ALSB proxy to the JMS queue, which could send an email, an ASN, etc.  That's an entry for another time.

Join the discussion

Comments ( 5 )
  • Mischa Thursday, November 26, 2009
    Hi James
    I just miss a tiny bit of information:
    Where ist the timer instance configured? I see, you lookup a timer/eventGeneratorTimer object, but who puts it on the JNDI tree? I can't find a tab or something on the WLS console (10.3) where I could configure timers. And I don't think the timer instance is magically configured when the timer is looked up on JNDI...
    Thanks
    Mischa
  • james.bayer Saturday, November 28, 2009
    Mischa,
    The timer manager is configured via the resource reference.
    http://download.oracle.com/docs/cd/E12839_01/apirefs.1111/e13941/commonj/timers/TimerManager.html
    "Creating and Configuring a Timer Manager
    TimerManagers are configured during deployment via deployment descriptors. The TimerManager definition may also contain additional implementation-specific configuration information.
    Once a TimerManager has been defined in a deployment descriptor, instances of it can be accessed using a JNDI lookup in the local Java environment Each invocation of the JNDI lookup() for a TimerManager returns a new logical instance of a TimerManager.
    The TimerManager interface is thread-safe.
    "
    If you're using the Job Scheduler (which works in a cluster), then you need to use this style and make sure the prereqs are set up such as leasing, etc:
    http://download.oracle.com/docs/cd/E12839_01/web.1111/e13733/toc.htm#sthref10
    Thanks,
    James
  • james.bayer Monday, November 30, 2009
    James
    Thanks, it works now. I just misunderstood the to be a reference to a resource configured elsewhere on the app-server. But in fact, it this element is more like a factory than like a reference. Strange nameing...
    Cheers
    Mischa
  • Pierluigi Vernetto Friday, December 17, 2010
    Excellent article!
    yet this needs updating: in EJB 3.0 the Timer services are supported in a clustered environment:
    http://shaoxiongyang.blogspot.com/2010/10/how-to-use-ejb-3-timer-in-weblogic-10.html
    and this *could* make this approach a winner over the other 2 mentioned in this article....
  • james.bayer Friday, December 17, 2010
    Thanks for the link to the clustered EJB3 blog. That is a very nice write-up.
    James
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.