Friday Dec 11, 2015

Introducing WLS JMS Multi-tenancy

Introduction

Multi-tenancy (MT) is the main theme of the WebLogic Server 12.2.1 release. It enhances the Oracle Platform for Software-as-a-Service (SaaS) and Platform-as-a-Service (PaaS) use cases. The main benefits of WebLogic multi-tenancy are increased density, tenant isolation, and simplified cloud configuration and management.

This article introduces multi-tenancy support for WebLogic JMS, the messaging component in WebLogic Server. 

Key MT Concepts

Some of you may have already learned from other blogs (for example Tim’s blog about Domain Partitions for Multi Tenancy) about some of the key concepts in WebLogic MT. But for the benefit of a broader audience, here is a quick review of those concepts before we get into JMS specifics.

WebLogic Multi-tenancy introduces the concepts of domain partition (also known as partition), resource group (RG), and resource group template (RGT).  

A Partition is conceptually a slice of a WebLogic domain, where resources and applications for different tenants can be configured and deployed in isolation on the same WebLogic server or in the same cluster. This improves overall density. Partitions define the isolation boundaries for JNDI, security, runtime MBeans, application persistent data, work managers and logging.  Furthermore, Partitions running on the same server instance have their own lifecycle, for example, a partition can be shut down at any time without impacting other partitions.

A Resource Group is simply a collection of functionally related resources and applications. A RG can be targeted and managed independently of other resource groups in the same partition. Resource groups can be defined not only inside a partition, but also at the domain level. As with partitions, RGs in the same partition (or at the domain level) that are running on the same server instance have their own lifecycle.

A Resource Group Template provides a templating mechanism to reduce the administrative overhead of configuring WebLogic resources and applications for SaaS use cases where the same resources and applications need to run in multiple partitions. It offers a configure-once-and-use-everywhere capability, where a common set of configuration artifacts can be specified in a RGT, and can then be referenced from RGs in different partitions. A RGT is not targetable, and resources in a RGT will not deploy unless the RGT is referenced by a deployed RG.

Note that the resources and applications configured or deployed in a partition (directly inside RGs or via RGs referencing a RGT) are scoped to that partition.

Understanding JMS Resources in MT

In a similar way to other WebLogic configuration artifacts, JMS resources such as JMS servers, SAF agents, path services, persistent stores, messaging bridges, JMS system modules, app-deployment JMS modules, Java EE 7 resource definition modules, and JMS applications can all now be configured and deployed in a RG, either directly or via RGTs, as well as in the ‘classic’ way, which is always directly at the domain level. Note that it is perfectly acceptable to combine both partition and ‘classic’ configuration together in the same domain.   

Resources and applications in different partitions are isolated from one another. For example, you can configure a JMS destination with the same JNDI name in multiple partitions running in the same cluster, and these destinations will be managed via independent runtime MBean instances, and can be independently secured via partition-specific security realms. In addition to non-persistent state, the persistent data (for example, persistent messages and durable subscriptions) in such JMS destinations are also isolated from one another.

Configuring JMS Resources in MT

The following configuration snippets show how JMS resources configured in a multi-tenant environment differs from traditional non-MT JMS configuration.

As you can see, partition-scoped JMS resources are embedded in a resource group in a partition (alternatively, they can be embedded in a Resource Group Template, which is in turn referenced by a Resource Group).

In addition, resources in a resource group are never individually targeted. Instead, the whole resource group is targeted via a virtual target, which is itself targeted in the normal way.  If a RG targets to a virtual target that is in turn targeted to a WL cluster, all JMS resources and applications in the RG will also be targeted to that cluster.

As we will see later, a virtual target not only provides the targeting information of a RG, it also defines the accessing point of a partition. For more information about resource group targeting and virtual targets, check out Joe's blog about Partition Targeting and Virtual Targets.

You might have noticed that I did not discuss configuring individual JMS resources for each server in a WL cluster, nor did I mention configuring “migratable targets” to add high availability. I have good news for you! Neither is needed or even supported in MT. They have been replaced with greatly enhanced WebLogic JMS cluster-targeting and HA support; my colleague Kathiravan blogs about it in 12.2.1 WebLogic JMS: Dynamic Scalability and High Availability Made Simple.

Although system level JMS resources (such as JMS servers, SAF agents, persistent stores, messaging bridges, path services, and JMS modules) are scoped differently in a MT configuration, their respective attributes are specified in exactly the same way as in a non-MT configuration.

Various validation and targeting rules are enforced to ensure that WebLogic MT JMS configuration is isolated, self contained, and easy to manage. One basic and high-level rule in configuring JMS in MT is that a JMS configuration artifact may only reference other configuration artifacts that are in the same scope. For example, a resource group scoped JMS server can only reference a persistent store that is also defined in the same resource group. These rules are enforced by configuration validation checks and by errors and warnings that are logged at runtime.

Accessing JMS Resources in MT

A JMS application designed for multi-tenancy accesses JMS resources in the same way as ‘classic’ JMS applications, by looking up JMS resources in a JNDI name space. The difference is that in a MT environment, a WebLogic JNDI InitialContext is associated with a particular scope (i.e. the domain or a partition), when it is created. 

A MT application can have multiple JNDI contexts that refer to the same WebLogic cluster but are scoped to different partitions. An initial context, once created, sticks to its scope until it is closed. This means that all JNDI operations using a partition-scoped JNDI context instance are performed using the partition-specific area of the JNDI space.

The scope of a JNDI context is determined by the “provider URL” supplied when the initial context is created.

Once an application successfully establishes a partition-scoped JNDI initial context, it can use this context to look up JMS connection factories and destinations in the same way as in a non-MT environment; except that now the application can only access partition-scoped JMS resources.

Let us look at some specific use cases and see how an application can establish an initial context to a particular partition in each of the use cases.

Use Case 1 - Local Intra-partition Access

When a Java EE application needs to access a JMS destination in its local partition in the same cluster (or on the same non-clustered managed server), the application can just create an initial context without supplying a provider URL.

Example 1: Null Provider URL

  Context ctx = new InitialContext();

  Object cf = ctx.lookup("jms/mycf1");

  Object dest = ctx.lookup("jms/myqueue1");

This initial context will be scoped to the partition in which the application is deployed.

Use Case 2 - Local Inter-partition Access

If a Java EE application needs to access a JMS destination (or other resource) in a different partition than the partition to which it is deployed, and the partition is in the same cluster (or on the same managed server) then it can use either a partition-scoped JNDI name or a provider URL with the "local:" protocol.

Using Partition Scoped JNDI Names

A JNDI name can be decorated with a namespace prefix to indicate its scope.

Example 2.1: given the partition configuration in the above examples, the following code can be used to access a JMS destination that is configured in "partition1".

Context ctx = new InitialContext();

Object cf = ctx.lookup("partition:partition1/jms/mycf1");

Object dest = ctx.lookup("partition:partition1/jms/myqueue1");

Similarly a Java EE application in a partition can access a domain level JNDI resource in the same cluster using a partition scoped initial context with the "domain:" namespace prefix, for example "domain:jms/mycf2".

Using a provider URL with the "local:" Protocol

Alternatively, one can specify a "local:" provider URL when creating an initial context to a specific partition.

Example 2.2: given the partition configuration in the above examples, the following code can be used to access a JMS destination that is configured in "partition1".

Hashtable<String, String> env = new Hashtable<>();

env.put(Context.PROVIDER_URL, "local://?partitionName=partition1");

env.put(Context.SECURITY_PRINCIPAL, "weblogic");

env.put(Context.SECURITY_CREDENTIALS, "welcome1");

Context ctx = new InitialContext(env);

Object cf = ctx.lookup("jms/mycf1");

Object dest = ctx.lookup("jms/myqueue1");

The initial context will be associated with "partition1" for its lifetime.

Similarly, a Java EE application in a partition can access a domain level JNDI resource in the same cluster using “local://?partitionName=DOMAIN” as the provider URL. 

Use Case 3 - General Partition Access

A third way for a Java EE application or client to access a JMS destination in a partition is to use a "partition URL". A partition URL is intended to be used when the JMS destination is in a remote cluster (or on a remote non-clustered managed server).  A typical "partition URL" is t3://hostname:port, or t3://host:port/URI-prefix.

Partition URLs may only be used by Java EE applications or clients using WLS 12.2.1 or later (older versions should use dedicated partition ports: see below).

Example 3: given the partition configuration in the examples above, the following code can be used to access a JMS destination that is configured in "partition1".

Note that "/partition1" in the provider URL below is the uri-prefix configured in the VirtualTarget for partition1.

Hashtable<String, String> env = new Hashtable<>();

env.put(Context.PROVIDER_URL, "t3://abcdef00:7001/partition1");

env.put(Context.SECURITY_PRINCIPAL, "weblogic");

env.put(Context.SECURITY_CREDENTIALS, "welcome1");

Context ctx = new InitialContext(env);

Object cf = ctx.lookup("jms/mycf1");

Object dest = ctx.lookup("jms/myqueue1");

Although it is not a best practice, a “partition URL” can also be used to access another partition in the same JVM/cluster.

Use Case 4 – Dedicated Partition Ports

One last option is to setup dedicated ports for each partition, and configuring these is described in Joe's blog about Partition Targeting and Virtual Targets.

Configuring dedicated partition ports enables applications that use ‘classic’ URLs to access a partition, and is mainly intended to enable clients and applications that are running on releases older than 12.2.1 to access partitions in a 12.2.1 or later domain.

Such older clients and applications do not support the use of host name and URI-prefix to access a partition. An attempt to use them from an older client will simply fail or may silently access the domain level JNDI name space.

What’s next?

I hope this article helps you to understand the basics of JMS MT! It is time to start exploring this new and exciting capability. You can find more information for messaging in MT in the Configuring Messaging chapter of Oracle® Fusion Middleware Using WebLogic Server Multitenant.

Thursday Nov 19, 2015

WLS Replay Statistics

Starting in the 12.1.0.2 Oracle thin driver, the replay driver has statistics related to replay. This is useful to understand how many connections are being replayed. It should be completely transparent to the application so you won’t know if connection replays are occurring unless you check.

The statistics are available on a per connection basis or on a datasource basis. However, connections on a WLS datasource don’t share a driver-level datasource object so the latter isn’t useful in WLS. WLS 12.2.1 provides another mechanism to get the statistics at the datasource level.

The following code sample shows how to print out the available statistics for an individual connection using the oracle.jdbc.replay.ReplayableConnection interface, which exposes the method to get a oracle.jdbc.replay.ReplayStatistics object. See https://docs.oracle.com/database/121/JAJDB/oracle/jdbc/replay/ReplayStatistics.html for a description of the statistics values.

if (conn instanceof ReplayableConnection) {
  ReplayableConnection rc = ((ReplayableConnection)conn);
  ReplayStatistics rs = rc.getReplayStatistics(
    ReplayableConnection.StatisticsReportType.FOR_CURRENT_CONNECTION);
  System.out.println("Individual Statistics");
  System.out.println("TotalCalls="+rs.getTotalCalls());
  System.out.println("TotalCompletedRequests="+rs.getTotalCompletedRequests());
  System.out.println("FailedReplayCount="+rs.getFailedReplayCount());
  System.out.println("TotalRequests="+rs.getTotalRequests());
  System.out.println("TotalCallsTriggeringReplay="+rs.getTotalCallsTriggeringReplay());
  System.out.println("TotalReplayAttempts="+rs.getTotalReplayAttempts());
  System.out.println("TotalProtectedCalls="+rs.getTotalProtectedCalls());
  System.out.println("SuccessfulReplayCount="+rs.getSuccessfulReplayCount());
  System.out.println("TotalCallsAffectedByOutages="+rs.getTotalCallsAffectedByOutages()); 
  System.out.println("TotalCallsAffectedByOutagesDuringReplay="+  
      rs.getTotalCallsAffectedByOutagesDuringReplay());  
  System.out.println("ReplayDisablingCount="+rs.getReplayDisablingCount());
}

Besides a getReplayStatistics() method, there is also a clearReplayStatistics() method.

To provide for a consolidated view of all of the connections associated with a WLS datasource, the information is available via a new operation on the associated runtime MBean. You need to look-up the WLS MBean server, get the JDBC service, then search for the datasource name in the list of JDBC datasource runtime MBeans, and get the JDBCReplayStatisticsRuntimeMBean. This value will be null if the datasource is not using a replay driver, if the driver is earlier than 12.1.0.2, or if it’s not a Generic or AGL datasource. To use the replay information, you need to first call the refreshStatistics() operation that sets the MBean values, aggregating the values for all connections on the datasource. Then you can call the operations on the MBean to get the statistics values, as in the following sample code. Note that there is also a clearStatistics() operation to clear the statistics on all connections on the datasource. The following code shows an example of how to print the aggregated statistics from the data source.

public void printReplayStats(String dsName) throws Exception {
  MBeanServer server = getMBeanServer();
  ObjectName[] dsRTs = getJdbcDataSourceRuntimeMBeans(server);
  for (ObjectName dsRT : dsRTs) {
    String name = (String)server.getAttribute(dsRT, "Name");
    if (name.equals(dsName)) {
      ObjectName mb =(ObjectName)server.getAttribute(dsRT,  
        "JDBCReplayStatisticsRuntimeMBean");
      server.invoke(mb,"refreshStatistics", null, null);
      MBeanAttributeInfo[] attributes = server.getMBeanInfo(mb).getAttributes();
      System.out.println("Roll-up");
      for (int i = 0; i <attributes.length; i++) {
        if(attributes[i].getType().equals("java.lang.Long")) {
          System.out.println(attributes[i].getName()+"="+
            (Long)server.getAttribute(mb, attributes[i].getName()));
        }
      }
    }
  }
}

MBeanServer getMBeanServer() throws Exception {
  InitialContext ctx = new InitialContext();
  MBeanServer server = (MBeanServer)ctx.lookup("java:comp/env/jmx/runtime");
  return server;
}
ObjectName[] getJdbcDataSourceRuntimeMBeans(MBeanServer server) 
  throws Exception {
  ObjectName service = new ObjectName( "com.bea:Name=RuntimeService,Type=\
weblogic.management.mbeanservers.runtime.RuntimeServiceMBean");
  ObjectName serverRT = (ObjectName)server.getAttribute(service,  "ServerRuntime");
  ObjectName jdbcRT = (ObjectName)server.getAttribute(serverRT,  "JDBCServiceRuntime");
  ObjectName[] dsRTs = (ObjectName[])server.getAttribute(jdbcRT,
    "JDBCDataSourceRuntimeMBeans");
  return dsRTs;
}

Now run an application that gets a connection, does some work, kills the session, replays, then gets a second connection and does the same thing. Each connection successfully replays once. That means that the individual statistics show a single replay and the aggregated statistics will show two replays. Here is what the output might look like.

Individual Statistics
TotalCalls=35
TotalCompletedRequests=0
FailedReplayCount=0
TotalRequests=1
TotalCallsTriggeringReplay=1
TotalReplayAttempts=1
TotalProtectedCalls=19
SuccessfulReplayCount=1
TotalCallsAffectedByOutages=1
TotalCallsAffectedByOutagesDuringReplay=0
ReplayDisablingCount=0

Roll-up
TotalCalls=83
TotalCompletedRequests=2
FailedReplayCount=0
TotalRequests=4
TotalCallsTriggeringReplay=2
TotalReplayAttempts=2
TotalProtectedCalls=45
SuccessfulReplayCount=2
TotalCallsAffectedByOutages=2
TotalCallsAffectedByOutagesDuringReplay=0
ReplayDisablingCount=0

Looking carefully at the numbers, you can see that the individual count was done before the connections were closed (TotalCompletedRequests=0) and the roll-up was done after both connections were closed. 

You can also use WLST to get the statistics values for the datasource. The statistics are not visible in the administration console or FMWC in WLS 12.2.1.

Thursday Nov 12, 2015

WLS UCP Datasource

WebLogic Server (WLS) 12.2.1 introduces a new datasource type that uses the Oracle Universal Connection Pool (UCP) as an alternative connection pool.  The UCP datasource allows for configuration, deployment, and monitoring of the UCP connection pool as part of the WLS domain.  It is certified with the Oracle Thin driver (simple, XA, and replay drivers). 

The product documentation is at http://docs.oracle.com/middleware/1221/wls/JDBCA/ucp_datasources.htm#JDBCA746 .  The goal of this article  is not to reproduce that information but to summarize the feature and provide some additional information and screen shots for configuring the datasource.

A UCP data source is defined using a jdbc-data-source descriptor as a system resource.  With respect to multi-tenancy, these system resources can be defined at the domain, partition, resource group template, or resource group level. 

The configuration  for a UCP data source is pretty simple with the standard datasource parameters.  You can  name it, give it a URL, user, password and JNDI name.  Most of the detailed configuration and tuning comes in the form of UCP connection properties.  The administrator can configure values for any setters supported by oracle.ucp.jdbc.PoolDataSourceImpl except LogWriter  (see oracle.ucp.PoolDaaSourceImpl) by just removing the "set" from the attribute name (the names are case insensitive).  For example,

ConnectionHarvestMaxCount=3


Table 8-2 in the documentation lists all of the UCP attributes that are currently supported, based on the 12.1.0.2 UCP jar that ships with WLS 12.2.1.

There is some built-in validation of the (common sense) combinations of driver and connection factory:

Driver

Factory (ConnectionFactoryClassName)

oracle.ucp.jdbc.PoolDataSourceImpl (default)

oracle.jdbc.pool.OracleDataSource

oracle.ucp.jdbc.PoolXADataSourceImpl

oracle.jdbc.xa.client.OracleXADataSource

oracle.ucp.jdbc.PoolDataSourceImpl

oracle.jdbc.replay.OracleDataSourceImpl

To simplify the configuration, if the "driver-name" is not specified, it will default to oracle.ucp.jdbc.PoolDataSourceImpl  and the ConnectionFactoryClassName connection property defaults to the corresponding entry from the above table.

Example 8.1 in the product documentation gives a complete example of creating a UCP data source using WLST.   WLST usage is very common for application configuration these days.

Monitoring is available via the weblogic.management.runtime.JDBCUCPDataSourceRuntimeMBean.  This MBean extends JDBCDataSourceRuntimeMBean so that it can be returned with the list of other JDBC MBeans from the JDBC service for tools like the administration console or your WLST script.  For a UCP data source, the state and the following attributes are set: CurrCapacity, ActiveConnectionsCurrentCount, NumAvailable, ReserveRequestCount, ActiveConnectionsAverageCount, CurrCapacityHighCount, ConnectionsTotalCount, NumUnavailable, and WaitingForConnectionSuccessTotal.

The administration console and FMWC make it easy to create, update, and monitor UCP datasources.

The following images are from the administration console. For the creation path, there is a drop-down that lists the data source types; UCP is one of the choices.  The resulting data source descriptor datasource-type set to "UCP". 

http://aseng-wiki.us.oracle.com/asengwiki/download/attachments/643530931/ucp1.jpg?version=3&modificationDate=1443495917000&api=v2

The first step is to specify the JDBC Data Source Properties that determine the identity of the data source. They include the datasource names, the scope (Global or Multi Tenant Partition, Resource Group, or Resource Group Template) and the JNDI names. 

http://aseng-wiki.us.oracle.com/asengwiki/download/attachments/643530931/ucp2.jpg?version=3&modificationDate=1443495999000&api=v2

The next page handles the user name and password, URL, and additional connection properties. Additional connection properties are used to configure the UCP connection pool. There are two ways to provide the connection properties for a UCP data source in the console. On the Connection Properties page, all of the available connection properties for the UCP driver are displayed so that you only need to enter the property value.  On the next page for Test Database Connection, you can enter a propertyName=value directly into the Properties text box.  Any values entered on the previous Connection Properties page will already appear in the text box.  This page can be used to test the specified values including the connection properties.

http://aseng-wiki.us.oracle.com/asengwiki/download/attachments/643530931/ucp5.jpg?version=1&modificationDate=1443496172000&api=v2

The Test Database Connection page allows you to enter free-form values for properties and test a database connection before the data source configuration is finalized. If necessary, you can provide additional configuration information using the Properties, System Properties, and Encrypted Properties attributes.

http://aseng-wiki.us.oracle.com/asengwiki/download/attachments/643530931/ucp3.jpg?version=1&modificationDate=1404361954000&api=v2

The final step is to target the data source. You can select one or more targets to which to deploy your new UCP data source. If you don't select a target, the data source will be created but not deployed. You will need to deploy the data source at a later time before you can get a connection in the application.

For editing the data source, minimal tabs and attributes are exposed to configure, target, and monitor this data source type. 

http://aseng-wiki.us.oracle.com/asengwiki/download/attachments/643530931/ucp4.jpg?version=1&modificationDate=1404361954000&api=v2

The capabilities in FMWC are similar to the administrative console but with a different look and feel.

If you select JDBC Data Sources from the WebLogic Domain drop-down, you will see a list of existing data sources with their associated data source type, scope, and if applicable RG, RGT and Partition.