Monday Jun 17, 2013

Fuzzy Searching sypport in OUAF V4.2.0.0.0

One of the most common requirements in products using Oracle Utilities Application Framework is searching for records using various criteria. In Oracle Utilities Application Framework we provide a set of Query Zones that allow SQL to be used to retrieve information using whatever criteria is configured. In most cases we support precise and/or wildcarding (this is configurable). In Oracle Utilities Application Framework V4.2.0.0.0 we added support for inprecise or fuzzy searching using the facilities provided in Oracle Text.

Oracle Text is a component of all editions of the Oracle Database that allows for special indexes and additional clauses to provide a wide range of text searching and processing options. One of the major features of Oracle Text in Oracle Database 11gR2 is Name searching. This facility allows implementations to specify addtional SQL clauses to implement fuzzy or inprecise searching names and addresses in 20+ languages natively in the database.

For example, say you were looking for a customer with the name Bill. Fuzzy searching would allow Bill, William, Willy, Billy etc to be returned as well as they are valid variations of the name. This applies to surnames and addresses as well. For example, if you were looking at a surname of Smith, you could get results back of valid variations such as Smythe, or Smithe etc...

This Oracle Text facility is now interfaced using a new @fuzzy helper function (that is what I call it) to get the SQL query to implement the new SQL clauses easily in Query zones.

The process to implement this is summarized as follows:

  • Enable Oracle Text on the database. By default, it is not enabled. This is a one off DBA DCL statement.
  • Grant access to Oracle Text to the product users.This is a one off DBA DCL statement.
  • Create Oracle Text indexes with the necessary parameters on the columns you want to apply the searching on. This will build special entries and variations for the SQL to use for you. The index creation allows you to customize the attributes and dictionaries used by the search to return the variations.
  • Use the @fuzzy helper function in your Query zone SQL against the columns indexed and with the necessary configuration.

The @fuzzy helper function provides a number of parameters:

@fuzzy(term, score, numresults, weighting) where:

 term Value to search on
 score The degress of fuzziness. Values between 1 - 80. The higher the number the more precise the search. Default is 60.
 numresults Number of variations to consider for the term. Number between 1 and 5000. Default is 100.
 weighting Whether the results are returned in order of score/weight. Specify WEIGHT to enable this. 

In the query zone you can default the values or provide them as parameters. For example:

To pass user data as term parameter and use default values of the remaining fuzzy operator parameters the zone SQL will need to be defined as following:

SELECT USER_ID, LAST_NAME from SC_USER WHERE CONTAINS(LAST_NAME, @fuzzy(:F1)) > 0;

To pass user data as term parameter in fuzzy operator, and to set score to 70, number results to 6, and to specify weight the zone SQL will need to be defined as following.

SELECT USER_ID, LAST_NAME from SC_USER WHERE CONTAINS(LAST_NAME, @fuzzy(:F1, 70, 6, ‘weight’)) > 0;

To pass user data as term parameter and a score in fuzzy operator, to set number results to 6, and to specify weight the zone SQL will need to be defined as following:

SELECT USER_ID, LAST_NAME from SC_USER WHERE CONTAINS(LAST_NAME, @fuzzy(:F1, :F2, 6, ‘weight’)) > 0;

Skip unnecessary parameters using the appropriate number of commas. For example:

SELECT USER_ID, LAST_NAME from SC_USER WHERE CONTAINS(LAST_NAME, @fuzzy(:F1,,,‘weight’)) > 0;

For more details of this facility and some general advice about Oracle Text refer to the Using Oracle Text for Fuzzy Searching Oracle Utilities Application Framework (Doc Id: 1561930.1) whitepaper available from My Oracle Support.

The key to the right results is the configuration of the Oracle Text Index. Refer to the Oracle Text documentation

Software Configuration Management Series Updated

A series of whitepapers has been available for Oracle Utilities Application Framework for the last few releases that deals with Software Configuration Management. This is a broad set of topics that cover aspects of managing code, data and environments in an implementation. These documents have been updated to reflect the latest advice and new facilities available within the Oracke Utilities Application Framework to handle this information.

The documents cover the following topics:

  • Concepts - General concepts and introduction.
  • Environment Management - Principles and techniques for creating and managing environments.
  • Version Management - Integration of Version control and version management of configuration items.  Updated to include mode information about revision control.
  • Release Management - Packaging configuration items into a release. Updated to include advice around Configuration Migration Assistant.
  • Distribution - Distribution and installation of releases across environments. Updated to include advice around Configuration Migration Assistant. 
  • Change Management - Generic change management processes for product implementations.
  • Status Accounting - Status reporting techniques using product facilities. Updated to include advice around Configuration Migration Assistant.
  • Defect Management - Generic defect management processes for product implementations.
  • Implementing Single Fixes - Discussion on the single fix architecture and how to use it in an implementation.
  • Implementing Service Packs - Discussion on the service packs and how to use them in an implementation.
  • Implementing Upgrades - Discussion on the the upgrade process and common techniques for minimizing the impact of upgrades.

The documents are available from My Oracle Support at Doc Id: 560401.1.

Thursday Jun 13, 2013

New version of What's New in FW4 whitepaper

A new updated version of the What's New in FW4 has been released which includes the latest information about the features of Oracle Utilities Application Framework up to an including version 4.2.0.1.0.

The whitepaper also features updated information for existing functionality including references to new whitepapers and more content describing key new features. This whitepaper is design to help customers who are considering upgrading from products using V2.x of the Oracle Utilities Application Framework to products using Oracle Utilities Application Framework V4.x.

The whitepaper is available in My Oracle Support at What's New In Oracle Utilities Application Framework V4 (Doc Id: 1177265.1).

Monday Jun 03, 2013

New Oracle Service Bus Integration Whitepaper

As part of Oracle Utilities Application Framework V4.2.0.1.0 a new set of Oracle Service Bus protocol adapters are available to allow Oracle Service Bus to process outbound communications initiated from the product. This complements the inbuilt and Oracle Service Bus integration for inbound communications that already has been released over the last few Oracle Utilities Application Framework releases.

The Oracle Service Bus Integration allows the following:

  • Definition of XAI Inbound Services as Business Services within Oracle Service Bus with support for Oracle Web Services Manager functionality for flexible security of inbound calls.
  • Optionally, using a Proxy Service on the Business Service to allow you to load balance Web Services calls across clusters or managed services.
  • A set of protocol adapters than can be easily installed within Oracle Service Bus to implement outbound communications for Outbound Messages (by Outbound Message Type) and for Oracle Utilities Customer Care And Billing and Oracle Enterprise Taxation And Policy Management, Notification and Workflow (by NDS Type),

A whitepaper has been released outlining the integration with step by step instructions on configuration and includes advice for customers migrating from the Multipurpose Listener (MPL). Customers migrating from the MPL should read this whitepaper and attend appropriate training to fully exploit the Oracle Service Bus integration.

The whitepaper is available from My Oracle Support - Oracle Service Bus Integration Oracle Utilities Application Framework (Doc Id: 1558279.1).

Friday Apr 26, 2013

Updated Best Practices and Production Configuration Whitepapers

The whitepapers available for the Oracle Utilties Application Framework are updated on a regular basis to include new and updated advice for past and the latest versions of the Oracle Utilties Application Framework. As part of that update cycle the following whitepapers have been updated:

  • Technical Best Practices for Oracle Utilities Application Framework Based Products (Doc Id: 560367.1) - Updated and new advice around security, database connectivity and monitoring.
  • Batch Best Practices for Oracle Utilities Application Framework based products (Doc Id: 836362.1) - Updated advice for Coherence tuning and monitoring.
  • Production Environment Configuration Guidelines (Doc Id: 1068958.1) - Latest advice from customer experiences and benchmarks for configuring production. Additonal advice has been added to optimize Coherence for increased batch stability.

The whitepapers are available from My Oracle Support.

Tuesday Apr 16, 2013

Installing OUAF in Oracle WebLogic native mode

As in a previous post it is possible to install an Oracle Utilities Application Framework based product in native mode as an alternative to the embedded mode offered by the default installation routine.

The basic process for this is as follows:

  • Install Oracle WebLogic as per the Oracle WebLogic documentation. If you are running Oracle ExaLogic then this may be already done as part of the Oracle ExaLogic deployment.
  • Using the Configuration Wizard shipped with Oracle WebLogic (or via Oracle Enterprise Manager) create a domain, create an administration server with the console deployed to it and create servers/clusters you want to deploy the product within.
  • You may need to alter the server startup definitions within the server definition to add additional java options and java memory settings for the product. The defaults shipped with Oracle WebLogic reflect the defaults of tha java version used and typically are set too small for the product.
  • Install the Oracle Utilities Application Framework based product as per the Installation Guide with a few additional settings specifically when specifying the host and port numbers (they should match the values you specified in the Configuration Wizard). You can optionally setup the product to use data sources, if you want to use them. If you do this then you need to deploy the data sources to the server allocated to the product BEFORE you deploy the product. 
  • Set the base location of the software in the startup for the servers you want to deploy the product. This tells the runtime to refer to the additional runtime components and product configuration files at runtime. This is simply making sure the environment variable (SPLEBASE) is set appropriately prior to server startup.
  • Create an XML Registry setting in the domain so that the correct parsers for the platform are used for the product.
  • Define your security realm settings for your domain including creating the product authentication group, JNDI user for JNDI lookups and initial product users.
  • Configure the domain to use advanced settings for each component of the product. This is simply setting a configuration flag on the domain realm.
  • Deploy the product Business Application Server (SPLService) into the servers using the deployment facilities of Oracle WebLogic or Oracle Enterprise Manager.
  • Deploy the product Web Application Server (SPLWeb) into the servers using the deployment facilities of Oracle WebLogic or Oracle Enterprise Manager. You will need to set the deployment order so that this component starts AFTER the startup of the Business Application Server.

The diagram below illustrates the process in detail:

Native Install Process

The product is now installed ready to be configured, managed and operated from the facilities in Oracle WebLogic Administration console or via Oracle Enterprise Manager.

This may look like a bit of effort but most of the work is wizard driven asking questions and simply settings the correct values. For a full detailed description of the process refer to Native Installation Oracle Utilities Application Framework (Doc Id: 1544969.1) whitepaper available from My Oracle Support.

Monday Apr 15, 2013

Supporting CLIENT-CERT

One of the additional configuration options for the authentication of the product is to implement a Single Sign On solution or implement client certificates. Whilst most of the configuration for these features is performed in the Single Sign On product and/or J2EE Application Server, the Oracle Utilities Application Framework has to be configured to use that facility.

In most cases, to use these facilities the login configuration for the product has to be changed from FORM or BASIC to CLIENT-CERT. This informs the product that the credentials will be passed directly from the J2EE Application Server (via the Single Sign On solution, security providers or via client certificates).

To make this change the following process must be performed:

  • Logon to the machine that houses the environment to change as the product administrator.
  • Take a copy of  the web.xml.template to cm.web.xml.template in the same directory the original is located (in Oracle Utilities Application Framework V2.x it is located in the etc directory of the environment; in Oracle Utilities Application Framework V4.x it is located in the templates directory). This will inform the Oracle Utilities Application Framework to use this new template instead of the base template.
  • Edit the cm.web.xml.template file and replace the login-config section with a section configuring the CLIENT-CERT configuration. For example:

Replace:

    <login-config>
      <auth-method>@WEB_WLAUTHMETHOD@</auth-method>
      <form-login-config>
         <form-login-page>@WEB_FORM_LOGIN_PAGE@</form-login-page>
         <form-error-page>@WEB_FORM_LOGIN_ERROR_PAGE@</form-error-page>
      </form-login-config>
   </login-config>

With:

    <login-config>
      <auth-method>CLIENT-CERT</auth-method>
   </login-config> 

Note: For Oracle Utilities Application Framework V4.x customers this may need to be repeated for the templates for AppViewer (web.xml.appViewer.template) and online help (web.xml.help.template) if you wish to include those components in the same solution.

  • Ensure the environment is shutdown prior to implementing any changes.
  • Execute the initialSetup[.sh] utility to implement the changes and rebuild the EAR files.

Note: As the web.xml file has been changed and EAR file rebuilt, customers using native mode will have to redeploy the SPLWeb application to reflect the change.

  • Optionally, changes can be verified by viewing the web.xml files generated under the etc\conf subdirectory.
  • Restart the product.

The product now is configured to use the CLIENT-CERT option.

Friday Apr 12, 2013

Using Native Installations of Oracle WebLogic

By default, when using Oracle WebLogic, the Oracle Utilities Application Framework based product is installed in embedded mode. This means that the configuration files and files within the product are used directly by Oracle WebLogic, including configuration files necessary for Oracle WebLogic itself (e.g. config.xml). The installation process generates the necessary configuration files and then using generated versions of Oracle WebLogic utilities points the Oracle WebLogic runtime to the files in the product environment. The term embedded is used to describe the fact that Oracle WebLogic uses files embedded in the product rather than its own files; it simply provides the runtime for the environment.

This default approach has advantages and disadvantages:

Advantages Disadvantages
Simple and easy to implement configuration, ideal for development and other non-production environments Changes to domain configuration within Oracle WebLogic console must be reflected in configuration files using user exits or custom templates to retain changes across patches/upgrades.
One Oracle WebLogic installation can be shared across many environments on the same host Does not support clustering without complex manual changes to configuration files
Default security setup Administration Server deployed with product
Common configuration change scenarios are handled by configuration settings Oracle Enterprise Manager does not recognize Oracle WebLogic targets without manual configuration of discovery
  Utilities provided must be used to manage product
  Limited changes to some features (such as domain)

This setup is ideal for development and other non-production environments where you need multiple copies of the product on a single host but may not be appropriate for production environments where advanced security setup and clustering are typically required.

The alternative is to install the product in what is termed, native mode. Typically Oracle WebLogic J2EE Web Applications are deployed directly to Oracle WebLogic and managed using capabilities within Oracle WebLogic. This has the advantage of gaining full access to the Oracle WebLogic facilities like advanced configuration and more importantly the ability cluster the product across multiple nodes. Oracle Utilities Application Framework V2.2 and above, can be installed using this mode with minor changes to the installation process. It is also possible to convert an embedded installation into a native installation with minor changes, if migration to this mode is appropriate.

The native mode allows the product to have access to support using the features of Oracle WebLogic with fewer configuration steps than embedded mode. The advantages and disadvantages of this mode are outlined in the table below:

Advantages Disadvantages
Native Support for Clustering/Managed Servers Support for multiple environments per domain is limited at present stage. Multiple WebLogic installs may be required if multiple environments are on the same host
Changes to domain do not require manual changes to templates Requires some manual effort in setting up domain, servers and security for environment
Administration Server can be separated from product servers   
Oracle Enterprise Manager natively recognizes Oracle WebLogic targets  
Native facilities in Oracle WebLogic and/or Oracle Enterprise Manager can be used to configure, operate and monitor the product  
Configuration and operational facilities of Oracle WebLogic can be used (including documented variations)  

The two modes have different attributes and different approaches applicable to different situations. The following recommendations should be consider when deciding which mode to use:

  • It is not impractical use different modes for different environments. One mode will not usually satisfy all the needs of all environments.
  • It is recommended to use native mode for production implementations as it offers flexibility, cluster support, separation of the Administration function and the ability to use the advanced configuration elements of Oracle WebLogic as well as Oracle Enterprise Manager (if applicable).
  • It is recommended to use native mode if each environment is housed in a separate virtual machine, which is common in virtualized implementations. This will allow configuration at the virtual machine level to be used and reduces maintenance efforts.
  • It is recommended to use embedded mode if more than one copy of the product exists on the same virtual or non-virtual host. The ability to share a common copy of Oracle WebLogic is reduces the maintenance efforts for multiple environments.
  • It is recommended to use embedded mode for development environments where java based development is taking place. This setup supports the use of the expanded mode features of Oracle WebLogic used by the Oracle Utilities SDK, which requires access to expanded directories for multi-user development.

For more information about native mode installation refer to the new whitepaper Native Installation for Oracle Utilities Application Framework (Doc Id: 1544969.1). For customers considering clustering and/or Oracle ExaLogic also refer to the Implementing Oracle ExaLogic and/or Oracle WebLogic Clustering (Doc Id: 1334558.1) whitepaper available from My Oracle Support.

Thursday Apr 11, 2013

Updated Oracle ExaLogic and Clustering Whitepaper

The Implementing Oracle ExaLogic and/or Oracle WebLogic Clustering  (Doc Id: 1334558.1) whitepaper available from My Oracle Support has been updated with additional advice.

The updated advice is around setting Java options within the console (instead of within a command utility) as well as adjusting timeouts that may occur during the deployment of the product EAR files.

Wednesday Apr 03, 2013

Using Oracle WebLogic datasources with OUAF based products

By default, the online (and XAI) components of the Oracle Utilities Application Framework use Universal Connection Pool (or c3p0 for Oracle Utilities Application Framework V2.x) for connection pooling. If you wish to use Oracle WebLogic JDBC based connection pooling for integration to other products or to use the GridLink features of Oracle WebLogic, then Oracle Utilities Application Framework needs to be configured to use Oracle WebLogic datasources.

The following process can be used for this:

  • Define the JDBC datasource to the product database, using the Oracle WebLogic console, with the following attributes:
    • The JNDI Name for the datasource should contain a directory name such as jdbc. For example, jdbc/demodb. The name of the JNDI should reflect your site requirements.
    • Ensure that Global Transaction Support is disabled on the JDBC connection. This is not appropriate for this integration.
    • The XA JDBC driver may be used but the product does not take advantage of the XA feature set.
    • The Instance or Service driver may be used but if there is no site preference use the Service driver.
    • The Database User and Password to use should be any database user with read/write access to the product such as CISUSER or SPLUSER. The value can correspond to the value of the DBUSER configuration variable in the ENVIRON.INI, if no site preference exists.

     

  • If your site is using Oracle WebLogic in embedded mode the above change will change the Oracle WebLogic config.xml to add the JDBC resource. To retain changes over upgrade and patches do the following:
    • For products using Oracle Utilities Application Framework V2.x, you should copy the config.xml.template (or config.xml.win.template for Windows) in the etc directory and create a cm version (add cm. as the name prefix) and add the following lines: 
<jdbc-system-resource>
    <name>demodb</name>
    <target>myserver</target>
    <descriptor-file-name>jdbc/demodb-jdbc.xml</descriptor-file-name>
</jdbc-system-resource>
<internal-apps-deploy-on-demand-enabled>false</internal-apps-deploy-on-demand-enabled>

Note: The values for name, target and descriptor-file-name tags should be altered to suit your JDBC connection. The internal-apps-deploy-on-demand-enabled is not required but is included for sequence reference purposes.

    •  For products using Oracle Utilities Application Framework V4.x then the above XML code should be placed in the cm_config.xml.exit_3.include file (or CM_config.xml.win.exit_3.include for Windows) in the templates directory. This is using the new user exit method introduced in Oracle Utilities Application Framework V4.x which avoids the need for a cm template altogether.
  •   To use the new JDBC datasource for online the following must be performed:
    • Copy the hibernate.properties.web.tempate to cm.hibernate.properties.web.tempate within the relevant directory. For Oracle Utilities Application Framework V2.x the directory is the etc subdirectory, and Oracle Utilities Application Framework V4.x the directory is the template subdirectory. This will tell the utilities to override the base template with the custom template.
    • Add the following lines to the top of the cm template file:
hibernate.connection.datasource = <jdbc_jndi>
hibernate.connection.username = <jndi_user>
hibernate.connection.password = <jndi_user_password>

where :

<jdbc_jndi> - Full JNDI name for the JDBC connection
<jndi_user> - A user to access the JNDI (Substitution variable @WEB_WLSYSUSER@ can be used in the template).
<jndi_user_password> - The password for the user to access the JNDI (Substitution variable @WEB_WLSYSPASS@ can be used in the template).

    • Remove the following lines from the cm template file depending on the version of the Oracle Utilities Application Framework:  
       Framework Version  Lines to remove
       2.2.x, 4.0.x All hibernate.connection.url entries including any ifdef statements
      All hibernate.c3p0.* entries
       4.1.x, 4.2.x All hibernate.connection.url entries including any ifdef statements
      All hibernate.ucp.* entries
    • Change the hibernate.transaction.factory_class setting to org.hibernate.transaction.JDBCTransactionFactory
    • Change the hibernate.connection.provider_class to org.hibernate.connection.DatasourceConnectionProvider
    • Save the file. For Oracle Utilities Application Framework V4.2 and above the sample cm template is shown below:
hibernate.connection.driver_class = @DBDRIVER@
hibernate.connection.datasource = jdbc/demodb
hibernate.connection.username = @WEB_WLSYSUSER@
hibernate.connection.password = @WEB_WLSYSPASS@
hibernate.dialect = @DIALECT@
hibernate.show_sql = false
hibernate.max_fetch_depth = 2
hibernate.transaction.factory_class = org.hibernate.transaction.JDBCTransactionFactory
hibernate.jdbc.fetch_size = 100
hibernate.jdbc.batch_size = 30
hibernate.query.factory_class = org.hibernate.hql.internal.classic.ClassicQueryTranslatorFactory
hibernate.cache.use_second_level_cache = false
hibernate.query.substitutions = true 'Y', false 'N'
hibernate.connection.provider_class=org.hibernate.connection.DatasourceConnectionProvider
hibernate.connection.release_mode=on_close
#ouaf_user_exit hibernate.properties.exit.include
#ouaf_user_exit hibernate.properties.web.exit.include
  • Execute the initialSetup[.sh] command to apply the changes and new templates.

Note: For products using the Oracle Utilities Application Framework V4.x you can execute the initialSetup[.sh] –t command to apply the changes. This avoids an EAR rebuild.

  • Restart the server to verify the connection is active 

Now the data source is used from Oracle WebLogic for online as well as XAI. This technique does not apply to batch as Oracle WebLogic is not used for batch, this will continue to use UCP or c3p0 (depending on the version of Oracle Utilities Application Framework is used).

Tuesday Mar 26, 2013

Loading XAI Inbound Services in Oracle Service Bus

It is possible to use Oracle Service Bus to process interfaces to the Oracle Utilities Application Framework to support the advanced facilities available in Oracle Service Bus or simply use the proxy capbilities of Oracle Service Bus. This advice applies to Oracle Utilities Application Framework V2.2 and above.

The first step in this process is to import XAI Inbound Services into Oracle Service Bus:

  • When creating an XAI Inbound Service a WSDL is generated for the service and is listed on the XAI Inbound Service Maintenance screen. This should be noted as it will be used in Oracle Service Bus.
  • Logon to the Oracle Service Bus console (a.k.a. Change Center) and start an Edit Session. This will enable the ability to maintain configuration data within Oracle Service Bus.
  • Create or Navigate to a Folder within an Project to want to store the resources within Oracle Service Bus.
  • Create a Resource of type Resources from URL and specify the WSDL URL for the service as well as a Resource Name to be used to identify the resource within Oracle Service Bus and the Resource Type of WSDL. Use the Import feature to import the WSDL definition into Oracle Service Bus.
  • Remember to Activate your Edit session to apply the configuration.

Now that the WSDL is imported within Oracle Service Bus it can be used for various configuratoin activities within Oracle Service Bus including creating a Business Service. To create a Business Service you must do the following:

  • Logon to the Oracle Service Bus console (a.k.a. Change Center) and start an Edit Session. This will enable the ability to maintain configuration data within Oracle Service Bus.
  • Create or Navigate to a Folder within an Project to want to store the Business Service within Oracle Service Bus.
  • Create a Resource of type Business Service and specify the name and documentation for your service. In Service Type, select WSDL Web Service and select the WSDL you imported as the desired WSDL to use.
  • Select the Binding to Submit the configuration to specify the binding in the WSDL to use.
  • In the Transport Configuration screen specify HTTP protocol and the URI's you wish to use for the Business Service. By default the URI in the WSDL can be used but you alter this for your site as well as place a number of URI's in the configuration to support clustering the Business Service. If you use this facility remember to configure the Load Balancing Algorithm to your requirements on the same screen.
  • In the HTTP Transport configuaration screen, you need to specify the Authenticaton to use. You can specify the Basic configuration, by default, but it is recommended to set use the Policies facility in Oracle Service Bus as it seperates the security from the definition implemention. In this case, select None for Authentication. In this case, this tells Oracle Service Bus that security will be supplied externally. 
  • Fill in the rest of the configuration according to your requirements and save the Business Service.
  • Remember to Activate your Edit session to apply the configuration.

The Business Service is almost ready to use. We need to resolve the security settings. The preferred way is to create a credential in Oracle WebLogic using the named credentials as specified in XAI Best Practices (Doc Id: 942074.1) available from My Oracle Support. Once those are created they can be attached to the Business Service using this technique:

  • Logon to the Oracle Service Bus console (a.k.a. Change Center) and start an Edit Session. This will enable the ability to maintain configuration data within Oracle Service Bus.
  • Open the Business Service and navigate to the Policies Tab for the configuration.
  • Select OWSM Policies to indicate you want to use Oracle Web Services Manager policies.
  • Add the oracle/wss_username_token_client_policy to apply to the Business Service. Customers using SSL can use oracle/wss_username_token_over_ssl_client_policy instead.
  • Update the Business Service.
  • At this point you may see a save Conflict. This occurs if the default credential (named basic.credential) does not exist in Oracle WebLogic. You can resolve this by creating the basic.credential or changing the credential used to the one you created earlier. The latter can be resolved by editing the Business Service and navigating to the Security tab and specifying the name of the created credential in the Override Value. The conflict is resolved.
  • Remember to Activate your Edit session to apply the configuration.

The Business Service is now ready to be used within Oracle Service Bus. To verify you can use the Test Console to verify the transaction.

Note: When using the Test Console, it will default transactionType used in XAI Inbound Services to a value that may not be appropriate for your test. Ensure the value is correct before executing your test.

In the next few blog entries I will outline some additional features in Oracle Service Bus you can use with Oracle Utilities Application Framework.

Friday Feb 08, 2013

Configuration Migration Assistant Part 8 - Differences from ConfigLab

For customers on Oracle Utilities Customer Care And Billing as well as Oracle Enterprise Taxation and Policy Management who use ConfigLab will ask what the Configuration Migration Assistant offers over ConfigLab. Here is the summary:

  • Simpler Terminology - One of the issues with users of ConfigLab was the confusing terminology used. You had supporting and supported environments and then Compare Source and Synchronize processes. In Configuration Migration Assistant we removed all that terminology and use the easier understandable Source and Target. There is no equivalent of Compare Source and Synchronize as it just export and import.
  • Minimal Setup Required - One of the other issues with ConfigLab is that you explicitly define the data migration relationships between environments and that required technical setup, called Environment registration. This process created db links between environments and needed to be maintained by the DBA across upgrades or patches. For the Configuration Migration Assistant there is no such technical setup. The only setup is to define the export and import directories on the Master Configuration record in each environment. This is retained across patches and upgrades automatically.
  • Reusable Objects - In ConfigLab, the relationships and criteria where stored in an object called DB Process and each relationship and criteria was in a DB Instruction. The issue is that structure is not reusable as you had to duplicate it to make changes. In Configuration Migration Assistant different objects are used to store relationships (Migration Plans) and criteria (Migration Requests). This allows for reuse and minimizes the need for duplications.
  • Supports data types used in product - Since the introduction of ConfigLab, the Oracle Utilities Application Framework introduced support for CLOB and XML data types but ConfigLab had issues detecting changes within these data types. In Configuration Migration Assistant you can not only detect changes in nodes within CLOB and XML data types you can now specify relationships using XPATH to model relationships within these data types.
  • Export files can be checked into version management software - One of the requirements from our partners was the ability to take the export files generated by Configuration Migration Assistant and check them into a code repository with any java code and/or Bundles. ConfigLab did not feature this facility.
  • Both Changed and Unchanged data is displayed - One of the key feedback from customers using ConfigLab was that just as important as telling them what has changed, it was important to tell them what has NOT been changed as a result of the import and compare process. This is a feature in Configuration Migration Assistant. This feature was only implied in ConfigLab.
  • Export files can be reused across environments - In line with other migration tools, it was important to be able to export once but use many times. For example, the export can represent a point in time configuration that can be migrated across environments regardless of the target state. ConfigLab could not share migrations as it was always considered live migrations.
  • Export files can be used to restore an environment - One of the features of the Configuration Migration Assistant is the ability to import data that was previously exported from the same environment. This acts as a restore like facility. ConfigLab did not feature this facility (well at least not after multiple steps and complex configuration).
  • Objects with system generated keys not supported - ConfigLab was available to migrate any data from environment to environment including master and transaction data. Feedback from our customers suggested it was not optimal for master/transaction data even with lower volumes. This is partly issues around system generated keys in the target environment. Configuration Migration Assistant is restricted to objects without system generated keys to avoid this issue. This means it cannot be used for master or transaction data. Oracle uses another product called Test Data Management Pack which is part of the Oracle Enterprise Management family of products for this purpose. We will be featuring a discussion of that product in a future posting and whitepaper.
  • Delete transactions not supported -  One of the issues we would get feedback on from ConfigLab customers was delete transactions. If the record did not exist in the Supported environment but in the Supporting environment then a delete statement would be generated. The issue is that you cannot use data that was used for any transaction due to data integrity rules. This would cause ConfigLab to generate errors when attempting to apply changes causing concerns from customers. It is in fact, expected behavior as ConfigLab was protecting data integrity and the errors indicated that. This still concerned customers so to avoid this issue, we do not support delete transactions in Configuration Migration Assistant. These should be manually managed.
  • Same monitor process used for import/export and apply changes - One of the issues with ConfigLab was the sheer number of batch jobs that were available. We simplified this by providing a single configurable monitor process to manage the objects over the lifecycle of the objects. This reduces maintenance overhead and maximizes flexibility. 

In the long run, the Configuration Migration Assistant will replace ConfigLab.

For more information about this aspect of the Configuration Migration Assistant and other aspects refer to the Configuration Migration Assistant Overview (Doc Id: 1506830.1) whitepaper available from My Oracle Support.

Thursday Feb 07, 2013

Configuration Migration Assistant Part 7 - Migration Import

After data is exported it can be imported into target environments. The original source environment can also be a target environment if you wish to use export to restore the data (Note: Deletes are not supported in Configuration Migration Assistant).

Note: Before importing data ensure that the Master Configuration Record for the Migration Assistant Configuration has been setup.

Migration Assistant Configuration

To execute an import, access the target environment and perform the following tasks:

  • Register the Migration Data Set Imports to import exported data into the environment.
  • Execute F1-MGDPR on the target environment to import the data to create a Migration Data Set outlining the changes and the data that is unchanged.
  • Optionally, Approve or Reject individual changes
  • After the approval process has been completed for the import, the Migration Data Set must be marked as Ready To Apply.
  • Execute F1-MGDPR on the target environment to apply the approved changes.

The process is illustrated as follows:

Migration Import process

The first step in the import process is to login to the target environment and create a Migration Data Set Import record using the Administration Menu --> M --> Migration Data Set Import menu option. Fill in the following information:

  • Import Directory - The import directory taken from the Master Configuration Record for the Migration Assistant Configuration.
  • File Name - Name of the export file in the Import Directory (do not include file suffix in name).
  • File Suffix - The File Suffix taken from the Master Configuration Record for the Migration Assistant Configuration.
  • Default Status For Add - The status value for any detected change that results in an Add. The valid values are:
    • Approved - Add is pre-approved.
    • Rejected - Add is pre-rejected
    • Needs Approval -  Add requires a manual approval or rejected
  • Default Status for Change - The status value for any detected change that results in an Update. The valid values are:
    • Approved - Change is pre-approved.
    • Rejected - Change is pre-rejected
    • Needs Approval -  Change requires a manual approval or rejected
For example: 

Migration Data Set Import

Once the Migration Data Set Import has been saved, as with the Migration Data Set Export it registers the intent to import, it does not perform the import at this time. To import the data, execute the F1-MGDPR batch process to execute the import and compare process. Note: Remember the L2 cache must be switched off for the threadpool used for F1-MGDPR. This will generate a Migration Data Set which will contain all the changes and also the data that has not been changed. To view the status of the Migration Data Set, open the Migration Data Set Import search dialog. For example:

Migration Data Set Import Search

Clicking on the Migration Data Set will list all the changes and unchanged data with the relevant defaults used on the Migration Data Set Import record. For example:

Migration Data Set

This will list the Migration Objects in the Migration Data Set. Note: If you want to filter out the Unchanged data use the filters on the zone and use Save Preferences to save your preferences. When selecting a change the SQL for the change is displayed. For example:

Example Change

At this point the change can be Approved or Rejected. Note: The VERSION field is included in each change to ensure that if the record changes between the approval process and the apply changes process then this change will be rejected and the import must be re-performed. This protects integrity of the objects compared.

Once all the changes have been approved the Migration Data Set Import should be marked Ready to Apply. For example:

Ready To Apply

To apply the changes execute F1-MGDPR batch process which will apply Approved changes for any Migration Data Set's that are Ready To Apply. Note: Remember the L2 cache must be switched off for the threadpool used for F1-MGDPR. To check the status of the Migration Data Set Import check the status on the Migration Data Set Import Query screen. If ALL the changes have been applied then the status will be Applied and if any change failed due to a business rule will be Applied With Errors. For example:

Migration Data Set Import Search

The state transition for an Migration Data Set Import is as follows:

For the Migration Data Set Import:

  • Initially Migration Data Set Import is set to Pending state. Before the F1-MGDPR process is executed the import may be manually Cancelled.
  • At the start of the process the Migration Data Set is created and the Migration Data Set Import is marked Ready to Compare.
  • During the compare process the Migration Data Set Import is marked as Comparing.
  • If the compare process encounters an error the Migration Data Set Import is marked in Error and import stopped.
  • If there are NO additions or changes in the export in the target environment, then the Migration Data Set Import is marked as Unchanged, otherwise is marked as Awaiting Approval.
  • After the manual approval process the Migration Data Set Import is marked as Ready to Apply.  Before the F1-MGDPR process is executed the import may be manually set to Cancel Apply.
  • During the Apply Changes process, using the F1-MGDPR batch process, the Migration Data Set Import is marked as Applying.
  • After the Apply Changes process, if all changes are applied successfully then the Migration Data Set Import set to Applied. If any errors occurred it is marked to Applied With Errors.

Migration Data Set Import state transition

For more information about this aspect of the Configuration Migration Assistant and other aspects refer to the Configuration Migration Assistant Overview (Doc Id: 1506830.1) whitepaper available from My Oracle Support.

Wednesday Feb 06, 2013

Configuration Migration Assistant Part 6 - Migration Export

Now that the configuration steps have been completed the next step is to use the Configuration Migration Assistant to migrate data across environments. The first step is to export the data from the source environment.

The steps to perform to export data using the Configuration Migration Assistant are as follows:

  • Create a Migration Data Set Export record to register the intent to export.
  • Execute the Migration Data Set Monitor batch process to perform the export.
  • Optionally, check the exported file into a code repository for cross reference with any customized code.

 For example:

Export Process Overview

The first step in the process is to create a Migration Data Set Export record. To do this use the Administration --> M --> Migration Data Set Export menu item and provide the following information:

  • Migration Request - Name of the Migration Request to Migrate
  • Export Directory - The directory taken from the Master Configuration Record for this environment. You cannot override the directory.
  • File Name - Name of the export file. The name of the file must conform to operating system constraints. We advise you do not embed blanks in the file name. Also do not include the filename suffix.
  • Export Description - Short description to attach within the export file.
  • Source Environment Reference - Defaults to the URL of the source environment but can be changed to reference the source of the information.

For example:

Migration Data Set Export

You may of noticed that Migration Request is a free format field. The easiest way to populate that field automatically is initiating the Export from the Migration Request maintenance screen. This allows you to verify the criteria prior to registering the export request. For example:

Export function from Migration Request

After saving this Migration Data Set Export, the request is registered within the product. Remember this does not perform the actual export it just registers the intent. The F1-MGDPR batch process exports the file. As the Migration Data Set Export has been registered then the request can be cancelled anytime prior to the execution of the F1-MGDPR batch process using the Migration Data Set Export maintenance dialog. For example:

Migration Data Set Export display

Now, you can register any number of Migration Data Set Exports as necessary for migrations.

To export the data the Migration Data Set Monitor batch process (F1-MGDPR) must be executed. This can be executed using any method (online submission, command line, batch scheduler etc) available to the environment by an authorized user.

The generic F1-MGDPR job is used to execute exports, process imports as well as apply changes and requires only one parameter DIST-THD-POOL. This is the threadpool to use for the export. As the product is processing data that is potentially cached, the job must run with the L2 cache turned off. This can be done using the following command:

Linux/UNIX

threadpoolworker.sh -l2 OFF -p <threadpoolname>=<number of threads> 

Windows

threadpoolworker  -l2 OFF -p <threadpoolname>=<number of threads> 

where

<threadpoolname> - Threadpool name to use in DIST-THD-POOL.

<number of threads> - Maximum limit for number of batch threads

Note: starttpw.sh can be also used on Linux/UNIX.

Note: Running this process using a threadpool where L2 cache is enabled may cause unexpected results.

During the execution of the export batch process the Migration Data Set Export will transition to a number of states:

  • The Migration Data Set Export starts in a Pending Status. This indicates that the export should be initiated the next time the F1-MGDPR process is executed.
  • At anytime before the F1-MGDPR is executed the Migration Data Set Export can be Cancelled manually using the Migration Data Set Export maintenance function.
  • During the execution the data to be migrated is identified and Migration Data Set and Migration Objects are created. The Migration Data Set groups all the records for a particular export request. The Migration Data Set Export is transitioned to Set Up Data status to indicate this activity is being performed.
  • The Migration Objects in the Migration Data Set are reviewed for duplications and relationships to be combined into transactions. This allows them to be exported in the correct sequence. The Migration Data Set Export is transitioned to Combine Transactions to indicate this activity is being performed.
  • Once the objects are checked and combined they are ready to export, with the state transitioned to Ready To Export to indicate this activity is being performed.
  • If there is some sort of processing error during any of the above transitions the Migration Data Set Export is transitioned to Error state and the batch process terminated for that Migration Data Set Export.
  • Once all the objects in the Migration Data Set Export are written to the file indicated the Migration Data Set Export is set to an Exported state. The date and time are recorded for auditing purposes.
For example the state transition model for Migration Data Set Export objects: 

Export State Transition

The state transitions can be tracked on the Log tab for the Migration Data Set Export maintenance function. For example:

Log entries for state transition

After executing the F1-MGDPR process the Migration Data Set Export will be updated with the file name and the date/time exported. For example:

Updated Migration Data Set Export

Now the data has been exported to a file. The file is in an internal format and SHOULD NOT be altered manually except through this tool. Any attempt to edit the file may result in unexpected results and even data corruption. The file can be optionally checked into your site code repository if you wish to keep it with any additional customizations such as java code etc.

The file is now available for import which we will cover in the next blog entry.

For more information about this aspect of the Configuration Migration Assistant and other aspects refer to the Configuration Migration Assistant Overview (Doc Id: 1506830.1) whitepaper available from My Oracle Support.

Monday Feb 04, 2013

Configuration Migration Assistant Part 5 - Migration Requests

Once you have Migration Plans created the next step is assembling them into Migration Requests. The Migration Request is a collection of unrelated Migration Plans and associated filter criteria to decide the scope of the migration.

To create a Migration Request navigate to the Administration Menu and select the M --> Migration Request menu option and fill in the following:

  • Migration Request - Name the Migration Request (it should be prefixed with CM to seperate it from the Migration Requests delivered with the product).
  • Description - A short description to describe the Migration Request
  • Detailed Descrption - A detailed description of the Migration Request
  • Migration Plans - A list of Migration Plans to include in this Migration Request and the Selection Criteria to use to subset the requests. This means the following:
    • Migration Plan - The Migration Plan to include in the Migration Request. The Migration Plans in the Request are NOT related. Any relationships are documented in the Migration Plans.
    • Selection Type - The criteria to select the subset of records in the Primary object in the Migration Plan. The Configuration Migration Assistant supports SQL based, XPath based or Algorithm based criteria.
    • Key Selection - The selection criteria to use in the format as specified in the Selection Type. This is SQL WHERE clause, XPATH statement or algorithm to use for selection. You can yse the help icon to find examples.

   For example:

Example Migration Request

The above example uses the (1=1) SQL clause to indicate that ALL records are migrated.

Note: The product supplies the majority of the Migration Requests you would use in the implementation. This step is only necessary if you wanted to copy a base Migration Request and alter it or add custom Business Objects to Migration Plans/Requests.

This concludes the configuration of the Configuration Migration Assistant. The next blog entries on this subject will discuss the execution components of the feature.

For more information about this aspect of the Configuration Migration Assistant and other aspects refer to the Configuration Migration Assistant Overview (Doc Id: 1506830.1) whitepaper available from My Oracle Support.

About

Anthony Shorten
Hi, I am Anthony Shorten, I am the Principal Product Manager for the Oracle Utilities Application Framework. I have been working for over 20+ years in the IT Business and am the author of many a technical whitepaper, manual and training material. I am one of the product managers working on strategy and designs for the next generation of the technology used for the Utilities and Tax markets. This blog is provided to announce new features, document tips and techniques and also outline features of the Oracle Utilities Application Framework based products. These products include Oracle Utilities Customer Care and Billing, Oracle Utilities Meter Data Management, Oracle Utilities Mobile Workforce Management and Oracle Enterprise Taxation and Policy Management. I am the product manager for the Management Pack for these products.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
9
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today