Tuesday Aug 06, 2013

Oracle BAM Adapter Batching Deep Dive

In the real business scenario of integrating SOA/AIA applications with Oracle Business Activity Monitoring (BAM), one common requirement is that such integration should not impact the operations of business systems running on SOA/AIA in terms of high availability and performance. To achieve this goal, one option is to use BAM Adapter or BAM Sensor Actions with Batching enabled. This article provides deep dive of how batching and its parameters work in Oracle BAM Adapter.

Why You Need Batching

When talking about batching, the first impression you may have is that you put a set of data into a package or a batch, and send it across for processing as a whole. In the BAM Adapter context, batching not only allows a group of XML payloads to be sent to the BAM Server side, but also changes the behavior of a synchronous invocation to BAM Server API to Async.

In mission critical applications, the preferred approach to integrate Oracle BAM with live data feed is using the Fire and Forget Message Exchange Pattern (MEP). Fire and Forget MEP has two major benefits compared to synchronous integration pattern:

  • High Performance - Threads invoking BAM Adapter APIs are not blocked by down stream operations. Thus the slow execution of down stream code should not impact the performance of the core business system.
  • High Availability - The failures including application failure such as errors due to invalid payload,  and system failures such as system outage due to maintenance, will not impact the normal operations of the main system. For example, in the Order to Cash (O2C) flow of AIA Communication PIPs, data integration between AIA and BAM should be in an asynchronous manner, thus, failures and outage of BAM will not impact AIA operations.

When Oracle BAM Adapter or BAM Sensor Actions is used, you should always enable batching to leverage the Fire and Forget MEP. In the next section, you will see how BAM Adapter works with batching enabled.

How Batching Works

Oracle BAM Adapter is a JCA compliant adapter that primarily converts Oracle BAM operations (Insert, Upsert, Update, and Delete) into the standard Web Services Interface described by WSDL.

By default, BAM Adapter operates in a synchronous manner as shown in the following diagram.

The caller thread T1 initiated in Oracle BPEL Process Manager invokes BAM Adapter operations in a synchronous way, and the same thread T1 continues executing BAM Server (Active Data Cache) API.

The behavior changes if batching is enabled shown in the following diagram.

BAM Batching Diagram

Oracle BAM Adapter works in the following way when batching is enabled:

  1. The caller thread T1 sends the request in the XML format to Oracle BAM Adatper.
  2. Oracle BAM Adapter processes the request and delegates the operation to BatchProcessor, which is a module in BAM to handle batch processing.
  3. BatchProcessor inserts the request message into a batch and release the caller thread T1.
  4. BatchProcessor determines when to send the batch to an internal queue based on the following rules: The number of messages in the batch reaches the threshold specified by the Batch_Lower_Limit attribute or the timer specified by the Batch_Timeout attribute expires (5 secs by default).
  5. The batch in the queue is scheduled to be flushed to BAM by an external thread T2. The queue size is specified by the Number_Batches atribute.

So you can see that the caller thread T1 is not block by BAM server operations.

Summary

In summary, batching should be always enabled if integrating BAM with mission critical applications (for example, O2C PIP) using Oracle BAM Adapter.



Thursday Nov 25, 2010

BAM Report URL SSO Simulation

[Read More]

Tuesday Nov 23, 2010

BAM 11g Ant Deployment

[Read More]

Monday Jul 05, 2010

Configuring XA For AQ JMS Adapter With Oracle RAC

This note describes the best practice of configurating XA for AQ JMS Adapter on SOA 10.1.3.4 with Oracle RAC Database that hosts AQ based JMS topics or queues,  For more details about Oracle SOA XA and RAC Database Configuration, please refer to the Oracle White Paper - Oracle SOA Suite 10g XA and RAC Database Configuration Guide, which is available on http://www.oracle.com/technology/tech/soa/ha/index.html.

 

Software and Patch Requirements

  • SOA suite must be patched to 10.1.3.4 MLR8 or above.
  • Oracle Database 10.2.0.3+ and 11.1.0.6+ are XA certified.
  • JTA/XA Patch 8442423 and Patch 8328051.
  • Patch 9213052 (Fixes JDBC connection issue when one RAC instance hosting the AQJMS is down)
  • Patch 9442233

Configuring AQ JMS Adapter

  1. Creating AQ Users and Granting Privileges
    Create a AQ user if such user does not exist in the Oracle Database Instance. Then grant the user necessary privileges to enqueue and dequeue messages. To achieve this, run the following sql scripts.
    
    connect / as sysdba
    drop user jmsuser cascade;
    create user jmsuser identified by welcome1;
    grant connect, resource, aq_administrator_role to jmsuser;
    grant execute on sys.dbms_aqadm to jmsuser;
    grant execute on sys.dbms_aq to jmsuser;
    grant execute on sys.dbms_aqin to jmsuser;
    grant execute on sys.dbms_aqjms to jmsuser;
    exec dbms_aqadm.grant_system_privilege('ENQUEUE_ANY','jmsuser');
    exec dbms_aqadm.grant_system_privilege('DEQUEUE_ANY','jmsuser');
    
  2. Creating JMS Queues or Topics
    Connect to DB instance as the AQ user. Run the following script to create and start JMS queue.
    
    connect jmsuser/welcome1
    exec dbms_aqadm.create_queue_table(queue_table=>'JMSDEMO_QUEUE_TABLE',
                       queue_payload_type=>'sys.aq$_jms_text_message',
                       multiple_consumers=>false);
    exec dbms_aqadm.create_queue(queue_name=>'JMSDEMO_QUEUE',
                       queue_table=>'JMSDEMO_QUEUE_TABLE');
    exec dbms_aqadm.start_queue(queue_name=>'JMSDEMO_QUEUE');
    commit;
    
  3. Granting permissions to XA recovery user
    Create a new user "xauser" if such user does not exist in your Database instance, then grant permissions to this user. The SQL script is,
    
    create user xauser identified by welcome1;
    grant select on dba_pending_transactions to xauser;
    grant execute on sys.dbms_system to xauser;
    
  4. Configuring OC4J Resource Provider
    Resource Provider is an abstraction that enables to plug in third-party message providers for JMS connections. To configure AQ JMS Resource Provider, you need to specify the following properties in $ORACLE_HOME/j2ee/OC4J_SOA/config/application.xml,
    • Resource Provider name - unique identifier of the resource provider
    • Resource Provider class - "oracle.jms.OjmsContext"
    • JNDI name of the Data source that is used to connect to database hosting AQ queues or topics.

    For example,
    
    <RESOURCE-PROVIDER name="aqjmsRAC" class="oracle.jms.OjmsContext">
      <description></description>
      <property name="datasource" value="jdbc/aqjmsRACDS"></property>
    </RESOURCE-PROVIDER>
  5. Configuring Data Source Used by AQ JMS Resource Provider
    To configure data source used by AQ JMS, Open $ORACLE_HOME/j2ee/OC4J_SOA/config/data-sources.xml in a text editor, then add connection pool and data source by specifying
    • XA Connection factory class which is oracle.jdbc.xa.client.OracleXADataSource
    • JDBC Connection URL - Provides username, password and JDBC connection string that are used to connect to DB. The syntax to connect to RAC DB is as follows,
      
      url="jdbc:oracle:thin:@(DESCRIPTION=
                 (ADDRESS_LIST=(LOAD_BALANCE=on)
                 (ADDRESS=(PROTOCOL=tcp)(HOST=host1-vip)(PORT=1521))
                 (ADDRESS=(PROTOCOL=tcp)(HOST=host2-vip)(PORT=1521)))
                 (CONNECT_DATA=(SERVICE_NAME=ORCLSVC)))
      
    • Configure XA recovery required by OC4j Transaction Manager to conduct recovery process for database resources. Add the following after connection factory setting.
      
       <xa-recovery-config>
                  <password-credential>
                     <username>xauser</username>
                     <password>welcome1</password>
                  </password-credential>
       </xa-recovery-config>
      
    • Config connection pool properties. The following properties are recommended.
      • min-connections - Specify the minimal connections in the pool
      • max-connections - Specify the maximal connecitons in the pool
      • validate-connection and validate-connection-statement - these are recommended settings that enables validating JDBC connection when getting a connection from pool
    • Configure a data source that uses the connection pool. The data source JNDI name should match the one specified in AQ JMS Resource Provider configuration.
    Here is the example of data source configuration.
    
    <connection-pool name="aqjms_rac_pool" max-connections="50" min-connections="20" 
        validate-connection="true" validate-connection-statement="select 1 from dual">
          <connection-factory factory-class="oracle.jdbc.xa.client.OracleXADataSource" 
          user="jmsuser" password="welcome1" 
          url="jdbc:oracle:thin:@(DESCRIPTION=
               (ADDRESS_LIST=(LOAD_BALANCE=on)
               (ADDRESS=(PROTOCOL=tcp)(HOST=host1-vip)(PORT=1521))
               (ADDRESS=(PROTOCOL=tcp)(HOST=host2-vip)(PORT=1521)))
               (CONNECT_DATA=(SERVICE_NAME=ORCLSVC)))" 
          commit-record-table-name="">
             <xa-recovery-config>
                <password-credential>
                   <username>xauser</username>
                   <password>welcome1</password>
                </password-credential>
             </xa-recovery-config>
             <fatal-error-codes>
                <error-code code="17008"/>
             </fatal-error-codes>
          </connection-factory>
    </connection-pool>
    <managed-data-source connection-pool-name="aqjms_rac_pool" 
        jndi-name="jdbc/aqjmsRACDS" name="aqjmsRACDS" 
        manage-local-transactions="false"/>
    
  6. Configuring JMS Adapter
    To configure JMS Adapter, Open $ORACLE_HOME/j2ee/OC4J_SOA/application-deployments/default/JmsAdapter/oc4j-ra.xml in a text editor, then specify
    • location - The JNDI name of JMS Adapter Connection Factory.
    • connectionFactoryLocation - The AQ JMS Connection Factory Location. Set this property to java:comp/resource/$RESOURCE_PROVIDER/$CONNECTION_FACTORIES/$NAME, where
      • $RESOURCE_PROVIDER is the Resource Provider name for AQ JMS.
      • $CONNECTION_FACTORIES can be either QueueConnectionFactories or TopicConnectionFactories. For XA configuration, it must use either XAQueueConnectionFactories or XATopicConnectionFactories.
      • $NAME - Can be any name for the connection factory
    • isTopic - Set to false if this is a queue
    • isTransacted - Set to false. JMS adapter will use local transaction instead of global transaction if this property is set to true. Always set this property to false when using XA
    • username - AQ JMS username
    • password - AQ JMS password
    For example, the configuration looks like
    
    <CONNECTOR-FACTORY location="eis/AqjmsRAC/DemoQueue" connector-name="Jms Adapter">
       <CONFIG-PROPERTY name="connectionFactoryLocation" 
          value="java:comp/resource/aqjmsRAC/XAQueueConnectionFactories/myQCF" />
       <CONFIG-PROPERTY name="factoryProperties" value="" />
       <CONFIG-PROPERTY name="acknowledgeMode" value="AUTO_ACKNOWLEDGE"/>
       <CONFIG-PROPERTY name="isTopic" value="false" />
       <CONFIG-PROPERTY name="isTransacted" value="false" />
       <CONFIG-PROPERTY name="username" value="jmsuser" />
       <CONFIG-PROPERTY name="password" value="welcome1" />
       <CONNECTION-POOLING use="none">
       </CONNECTION-POOLING>
       <SECURITY-CONFIG use="none">
           </SECURITY-CONFIG>
       </CONNECTOR-FACTORY>
    
Designing BPEL Process To Produce/Consume Messsages From AQ JMS Queues/Topics
This note will not describe how to use JMS Adapter in BPEL/ESB to produce or consume messages. However, I would like to share some tips and tricks of using JMS Adapters here.
  1. Always check the wsdl file generated by JMS Adapter Wizard. Verify if location value for jca:address element matches the JNDI name specified in JMS Adapter Connection Factory in oc4j-ra.xml
  2. UseMessageListener attribure for jca:operation element in the wsdl file must be set to false. JMS Adapter only supports polling approach for inbound operations on SOA 10.1.3.4 or above version.
  3. Just a clarification that cacheConnections property is set to true by default for JMS Adapter partnerlink in BPEL, You don't need to change this to false.
[Read More]

Monday Dec 14, 2009

How To Configure BAM 11g With LDAP Servers

[Read More]
About

bocadmin_ww

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today