Thursday Apr 28, 2016

Testing WLS and ONS Configuration

Introduction

Oracle Notification Service (ONS) is installed and configured as part of the Oracle Clusterware installation. All nodes participating in the cluster are automatically registered with the ONS during Oracle Clusterware installation. The configuration file is located on each node in $ORACLE_HOME/opmn/conf/ons.config. See the Oracle documentation for further information. This article focuses on the client side.

Oracle RAC Fast Application Notification (FAN) events are available starting in database 11.2. This is the minimum database release required for WLS Active GridLink. FAN events are notifications sent by a cluster running Oracle RAC to inform the subscribers about the configuration changes within the cluster. The supported FAN events are service up, service down, node down, and load balancing advisories (LBA).

fanWatcher Program

You can optionally test your ONS configuration independent of running WLS. This tests the connection from the ONS client to the ONS server but not configuration of your RAC services. See https://blogs.oracle.com/WebLogicServer/entry/fanwatcher_sample_program for details to get, compile, and run the fanWatcher program. I’m assuming that you have WLS 10.3.6 or later installed and you have your CLASSPATH set appropriately. You would run the test program using something like

java fanWatcher "nodes=rac1:6200,rac2:6200" database/event/service

If you are using the database 12.1.0.2 client jar files, you can handle more complex configurations with multiple clusters, for example DataGuard, with something like

java fanWatcher "nodes.1=site1.rac1:6200,site1.rac2:6200
nodes.2=site2.rac1:6200,site2.rac2:6200" database/event/service

Note that a newline is used to separate multiple node lists. You can also test with a wallet file and password, if the ONS server is configured to use SSL communications.

Once this program is running, you should minimally see occasional LBA notifications. If you start or stop a service, you should see an associated event.

Auto ONS

It’s possible to run without specifying the ONS information using a feature call auto-ONS. The auto-ONS feature cannot be used if you are running with

- an 11g driver or 11g database. Auto-ONS depends on protocol flowing between the driver and the database server and this feature was added in 12c.

- pre-WLS 12.1.3. Auto-ONS is supported starting in WLS 12.1.3.

- an Oracle wallet with SSL communications. Configuration of the wallet requires also configuring the ONS information.

- complicated ONS topology. In general, auto-ONS can figure out what you need but there are cases where you need to specify it precisely. In WLS 12.2.1, an extension of the ONS configuration allows for specifying the exact topology using a property node list. See http://docs.oracle.com/middleware/1221/wls/JDBCA/gridlink_datasources.htm#BABGJBIC for more information.

If you have some configurations that use an 11g driver or database and some that run with 12c driver/database, you may want to just specify the ONS information all of the time instead of using the auto-ONS simplification. The fanWatcher link above indicates how to test fanWatcher using auto-ONS.

WLS ONS Configuration and Testing

The next step is to ensure that you have end-to-end configuration running. That includes the database service for which events will be generated to the AGL datasource that processes the events for the corresponding service.

On the server side, the database service must be configured RCLB enabled. RCLB is enabled for a service if the service GOAL (NOT CLB_GOAL) is set to either SERVICE_TIME or THROUGHPUT. See the Oracle documentation for further information on using srvctl to set this when creating the service.

On the WLS side, the key pieces are the URL and the ONS configuration.

The URL is configured using a long format with this service name specified. The URL can use an Oracle Single Client Access Name (SCAN) address, for example,

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=scanname)(PORT=scanport))(CONNECT_DATA=(SERVICE_NAME=myservice)))

or multiple non-SCAN addresses with LOAD_BALANCE=on, for example,

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=ON)(ADDRESS=(PROTOCOL=TCP)(HOST=host1)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=myservice)))

Defining the URL is a complex topic - see the Oracle documentation for more information.

As described above, the ONS configuration can be implicit using auto-ONS or explicit. The trade-offs and restrictions are also described above. The format of the explicit ONS information is described at http://docs.oracle.com/middleware/1221/wls/JDBCA/gridlink_datasources.htm#BABGJBIC.

If you create the datasource using the administration console with explicit ONS configuration, there is a button to click on to test the ONS configuration. This tests doing a simple handshake with the ONS server.

Of course, the first real test of your ONS configuration with WLS is deploying the datasource, either when starting the server or when targeting the datasource on a running server.

In the administration console, you can look at the AGL runtime monitoring page for ONS, especially if using auto-ONS, to see the ONS configuration.

You can look at the page for instances and check the affinity flag and instance weight attributes that are updated on LBA events. If you stop a service using something like

srvctl stop service -db beadev -i beadev2 -s otrade

that should also show up on this page with the weight and capacity going to 0.


If you look at the server log (for example servers/myserver/logs/myserver.log) you should see a message tracking the outage like the following.

…. <Info> <JDBC> … <Data source JDBC Data Source-0 for service otrade received a service down event for instance [beadev2].>

If you want to see more information like the LBA events, you can enable the JDBCRAC debugging using –Dweblogic.debug.DebugJDBCRAC=true. For example,

... <JDBCRAC> ... lbaEventOccurred() event=service=otrade, database=beadev, event=VERSION=1.0 database=beadev service=otrade { {instance=beadev1 percent=50 flag=GOOD aff=FALSE}{instance=beadev2 percent=50 flag=UNKNOWN aff=FALSE} }

There will be a lot of debug output with this setting so it is not recommended for production.

Wednesday Apr 27, 2016

Migrating from Generic Data Source to Active GridLink

Earlier, I wrote an article about how to migrate from a Multi Data source (MDS) for RAC connectivity to Active GridLink (AGL). This is needed to move from the older datasource technology to the newer technology, both supporting Oracle RAC. The information is now in the public documentation set at http://docs.oracle.com/middleware/1221/wls/JDBCA/gridlink_datasources.htm#JDBCA690.

There are also many customers that are growing up from a standalone database to an Oracle RAC cluster. In this case, it’s a migration from a GENERIC datasource to an AGL datasource. This migration is pretty simple.

No changes should be required to your applications.  A standard application looks up the datasource in JNDI and uses it to get connections.  The JNDI name won’t change.

The only changes necessary should be to your configuration and the necessary information is generally provided by your database administrator.   The information needed is the new URL and optionally the configuration of Oracle Notification Service (ONS) on the RAC cluster. The latter is only needed if you are running with

- an 11g driver or 11g database. Auto-ONS depends on protocol flowing between the driver and the database server and this feature was added in 12c.

- pre-WLS 12.1.3. Auto-ONS is supported starting in WLS 12.1.3.

- an Oracle wallet with SSL communications. Configuration of the wallet requires also configuring the ONS information.

- complicated ONS topology. In general, auto-ONS can figure out what you need but there are cases where you need to specify it precisely. In WLS 12.2.1, an extension of the ONS configuration allows for specifying the exact topology using a property node list. See http://docs.oracle.com/middleware/1221/wls/JDBCA/gridlink_datasources.htm#BABGJBIC for more information.

The URL and ONS attributes are configurable but not dynamic. That means that the datasource will need to be shutdown and restarted after the change. The simplest way to do this is to untarget the datasource, make the changes, and then re-target the datasource.

The recommended approach to migrate from a GENERIC to AGL datasource is to use WLST. The URL must be changed in the JDBCDriverParams object. The new JDBCOracleParams object (it generally doesn’t exist for a GENERIC datasource) needs to have FAN enabled set to true and optionally set the ONS information.

The following is a sample WLST script with the new values hard-coded. You could parameterize it and make it more flexible in handling multiple datasources. If you are using an Oracle wallet for ONS, that needs to be added to the JDBCOracleParams object as well.

# java weblogic.WLST file.py
import sys, socket, os
hostname = socket.gethostname()
datasource="JDBC Data Source-0"
connect("weblogic","welcome1","t3://"+hostname+":7001")
edit()
startEdit()
cd("/JDBCSystemResources/" + datasource )
targets=get("Targets")
set("Targets",jarray.array([], ObjectName))
save()
activate()
startEdit()
cd("/JDBCSystemResources/" + datasource + "/JDBCResource/" +
datasource + "/JDBCDriverParams/" + datasource )
set("Url","jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=(" +
"ADDRESS=(PROTOCOL=TCP)(HOST=dbhost)(PORT=1521)))" +
"(CONNECT_DATA=(SERVICE_NAME=otrade)))")
cd("/JDBCSystemResources/" + datasource + "/JDBCResource/" +
datasource + "/JDBCOracleParams/" + datasource )
set("FanEnabled","true")
set("OnsNodeList","dbhost:6200")
# The following is for WLS 12.1.2 and 12.1.3 if not setting FanEnabled true, which is not recommended
#set("ActiveGridlink","true")
# The following is for WLS 12.2.1 and later

#cd("/JDBCSystemResources/" + datasource + "/JDBCResource/" +
# datasource )
#set("DatasourceType", "AGL")
save()
activate()
startEdit()
cd("/JDBCSystemResources/" + datasource )
set("Targets", targets)
save()
activate()

In WLS 12.1.2 and 12.1.3, there is an explicit ActiveGridlink flag that can be used to identify an AGL datasource, even if FanEnabled is not set to true (which is not recommended) and auto-ONS is used (12.1.2 is the first release in which auto-ONS is supported). In the script above, uncomment the line to set it only if FanEnabled and OnsNodeList are not set.

Starting in WLS 12.2.1, there is an explicit datasource type at the JDBCResource level. If that is set to GENERIC, you must re-set it using set("DatasourceType", "AGL"). In the script above, uncomment the lines to set it.  In this case, the ActiveGridlink flag is not necessary.

In the administrative console, the database type is read-only and there is no mechanism to change the database type. You can try to get around this by setting the URL, FAN Enabled box, and ONS information. However, in 12.2.1 there is no way to re-set the Datasource Type in the console and that value overrides all others.

Wednesday Apr 30, 2014

NEC Touts Oracle WebLogic Active GridLink for RAC Benefits

Japanese multinational, NEC corporation, plans on offering the unique Active GridLink for RAC, providing intelligent integration between Oracle WebLogic Server and Oracle Real Application Clusters (RAC), to customers for its higher performance, availability, and simplified operations. Check out Naoto Kashiwagi, Assistant Manager, NEC discuss Active GridLink benefits in great detail.

Thursday Oct 17, 2013

So what is Active GridLink for RAC?

I had referred to Active GridLink for RAC in my blog yesterday and since then got several questions on this topic. So I decided to re-visit Active GridLink. With the release of version 11g, Oracle WebLogic Server started to provide strong support for the Real Application Clusters (RAC) features in Oracle Database 11g, minimizing database access time while allowing transparent access to rich pooling management functions that maximizes both connection performance and availability. WebLogic is the only application server in the marketplace which has been fully integrated and certified with Oracle Database RAC 11g without losing any rich functionality. Active GridLink provides Fast Connection Failover (FCF), Runtime Connection Load-Balancing (RCLB), and RAC instance graceful shutdown. With the key foundation for providing deeper integration with Oracle RAC, this single data source implementation in Oracle WebLogic Server supports the full and unrestricted use of database services as the connection target for a data source.


For more details and to understand how our customer NEC leverages this capability, read the whitepapers on this topic.

Get in depth ‘how-to’ details from this youtube video from our resident expert, Frances Zhao.

Friday Aug 09, 2013

Oracle and NEC Joint White Paper: WebLogic Server Active GridLink for Oracle RAC Testing Scenarios and Results

Oracle and NEC have been jointly invested on verifying and testing the Active GridLink for RAC solution implemented within Oracle WebLogic Server which gives the rich functionality for  Oracle Database Real Application Clusters integration.
The combination of Oracle WebLogic Server Data Source and Connection Pooling solutions and Oracle RAC provides a high-end mission-critical environment offering performance, high scalability and availability features. Load-balancing and Affinity capabilities offersignificant performance improvement for online transaction processing scenarios, as well as improving throughput and total response time. Failover solution gives end-to-end rapid failure detection supporting graceful shutdown for planned and unplanned Oracle RAC node outages.
In this paper, we start with a brief introduction to Oracle RAC and an overview of the Oracle RAC features supported in Oracle WebLogic Server 11g and 12c. We then focus on details of the effort that have been jointly done with Oracle and NEC with all the detailed testing scenarios and testing results, along with the analysis. The background and overview of NEC’s test of Active GridLink for RAC will be covered in details. The technical details about Runtime Connection Load Balancing, Web Session Affinity, Fast Connection Failover, and how to remove and add the additional RAC node with zero-downtime will be discussed with different use cases.


Download the white paper! And please share your comments
http://www.oracle.com/technetwork/middleware/weblogic/overview/activegridlinkwhitepaperoraclenec-1987937.pdf


Enjoy!

Wednesday Jul 17, 2013

WebLogic Active GridLink - An Oracle and NEC Joint White Paper

Oracle and NEC have been jointly invested on verifying and testing the Active GridLink for RAC solution implemented within Oracle WebLogic Server which gives the rich functionality for Oracle Database Real Application Clusters integration.

To learn about additional enhancements for Oracle WebLogic - Oracle Database integration, please join the Online Launch Event on July 31st.

In this published white paper, we started with a brief introduction to Oracle RAC and an overview of the Oracle RAC features supported in Oracle WebLogic Server 12c. We then focused on details of the effort that have been jointly done with Oracle and NEC with all the detailed testing scenarios and testing results, along with the analysis. The background and overview of NEC’s test of Active GridLink for RAC will be covered in details. The technical details about Runtime Connection Load Balancing, Web Session Affinity, Fast Connection Failover, and how to remove and add the additional RAC node with zero-downtime will be discussed with different use cases.

Check out this Japanese version paper:

http://www.nec.co.jp/middle/oracle/files/gc_wp-gridlink-gridcenter-nec.pdf

English version is coming soon, please check back later!

Wednesday Apr 24, 2013

Getting Connections with Active GridLink

As a follow-up to an earlier blog, someone asked "Can you please tell us more about gravitationShrinkFrequencySeconds and DRCP ?"   DRCP support isn’t shipping yet (wait until the next version of the Database and of WLS) so I can’t talk about it.

I think that people understand how connections are allocated in generic data sources and multi data sources (MDS).   Let me talk generally about how connections are allocated in Active GridLink (AGL). With the former (generic and MDS), connection allocation is pretty much dependent on WLS.  With AGL, it is also very dependent on the database configuration and runtime statistics.  The runtime information is provided to WLS via Oracle Notification Service (ONS) in the form of up and down events and Runtime Load Balancing events.

Connections are added to the pool initially based on the configured initial capacity. Connect time load balancing based on the listener(s) is used to spread the connections across the instances in the RAC cluster. For that to work correctly, you must either specify a Single Client Access Name (SCAN) address or use LOAD_BALANCE=ON for multiple non-SCAN addresses.  If you have multiple addresses and you forget to use LOAD_BALANCE=on, then all of the connections will end up on the first RAC instance – not what you want.  Connection load balancing is intended to give out connections based on load on the available RAC instances.  It’s not perfect.  If you have two instances and they are evenly loaded, there should be approximately 50% of the connections on each instance (but don’t enter a bug if they split up 49/51).  There a general problem with load balancing that the statistics can lag reality.  If you have big swings in demand on the two instances, then you can end up creating a connection on the more heavily loaded instance.

If you go to get a connection from the pool, runtime load balancing is used to pick the instance.  That’s again based on the load on the various instances and the information is updated by default every 30 seconds.  If a connection is available on the desired instance, then you have a hit on the pool and you get the available connection.

If you go to get a connection from the pool and one doesn’t exist on the desired instance, then you have a miss.  In this case, a connection is added to the pool on demand based on connection load balancing.  That is, if you go to reserve a connection and there isn’t one available in the pool, then a new one is created and it is done using connection load balancing as described above.  There’s an assumption here that the connect time and runtime load balancing match up.

If you are running within an XA transaction, then that takes priority in determining the connection that you get.  There are restrictions on XA support within a RAC cluster such that all processing for an XA transaction branch must take place on one instance in the cluster (the restrictions are complicated so the one-instance limitation is the only safe approach).  Additionally, performance is significantly better using a single instance so you want that to happen anyway.  The first time that you get a connection, it’s via the mechanisms described above (runtime load balancing or connection load balancing).  After that, any additional requests for connections within the same XA transaction will get a connection from the same instance.  If you can’t get a connection with “XA affinity,” the application will get an exception.

Similar to XA affinity, if you are running within a Web session, you will get affinity for multiple connections created by the same session.  Unlike XA affinity, “Session Affinity” is not mandatory.  If you can’t get a connection to the same instance, it will go to another instance (but there will be a performance penalty).

When you take down a RAC instance, it generates a “planned down event”.  Any unused connections for that instance are released immediately and connections in use are released when returned to the pool.  If a RAC instance fails, it generates an “unplanned down event.”  In that case, all connections for that instance are destroyed immediately.

When a RAC instance becomes available, either a new instance or an instance is restarted, it generates an “up event.”  When that occurs, connections are proactively created on the new instance.

The pool can get filled up with connections to the “wrong” instance (heavily loaded) if load changes over time.  To help that situation, we release unused connections based on runtime load balancing information.  When gravitation shrinking occurs, one unused connection is destroyed on a heavily loaded instance.  The default period for gravitation shrinking is 30 seconds but you can control it by setting the system property “weblogic.jdbc.gravitationShrinkFrequencySeconds” to an integer value.  We don’t actively create a connection at this point.  We wait until there is demand for a new connection and then the connection load balancing will kick in.

Finally, normal shrinking happens if not disabled.  When this occurs, half of the unused connections down to minimum capacity are destroyed.  The algorithm currently just takes the first set of connections on the list without regard to load balancing (it’s random with respect to instances).  The default period is 900 seconds and you can configure this using ShrinkFrequencySeconds.

There are possible improvements that could be made with respect to pool misses, and gravitational and normal shrinking.  And the database team is working on improving the load balancing done on their end. Still, it works pretty well with instances dynamically coming and going and the load changing over time.

Friday Apr 19, 2013

Migrating from Multi Data Source to Active GridLink

Multi data source (MDS) for RAC connectivity has been supported in WebLogic Server since 2005.  As the popularity of Oracle RAC has grown, so has the use of MDS.  Now with the introduction of Active GridLink (AGL) in early 2011, users of MDS want to migrate to AGL.  There is not an automated mechanism to do so but it’s not the difficult.

First, no changes should be required to your applications.  A standard application looks up the MDS in JNDI and uses it to get connections.  By giving the AGL the same JNDI name as the MDS, the process is exactly the same in the application to use a data source name from JNDI.

The only changes necessary should be to your configuration.   AGL is composed of information from the MDS and the member generic data sources, combined into a single AGL descriptor.  The only additional information that is needed is the configuration of Oracle Notification Service (ONS) on the RAC cluster.  In many cases, the ONS information will consist of the same host names as used in the MDS and the only additional information is the port number, and that can be simplified by the use of a SCAN address.

The MDS descriptor doesn’t have much information in it.  It has a list of the member generic datasources, which will help you figure out where to get the remaining information that you need.  It has a JNDI name, which must become the name of your new AGL to keep things transparent to the application.  If you want to run the MDS in parallel with the AGL, then you will need to give the AGL a new name but the application must also be changed to use a new JNDI name.  You don’t need to worry about the algorithm type.

Each of the member generic datasources will have its own URL.  As described in Appendix B Using Multi Data Sources with Oracle RAC , it will look like the following.

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=host1-vip)(PORT=1521)) (CONNECT_DATA=(SERVICE_NAME=dbservice)(INSTANCE_NAME=inst1)))

Each member should have its own host and port pair.  They will likely have the same service and often have the same port on different hosts.  The URL for the AGL is a combination of the host and port pairs.

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=host1-vip)(PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=host2-vip)(PORT=1521)))
(CONNECT_DATA=(SERVICE_NAME=dbservice))

It is preferable to use an Oracle Single Client Access Name (SCAN) address instead of multiple host or Virtual IP (VIP) addresses.  Then the URL would look something like this.

jdbc:oracle:thin:@(DESCRIPTION=(ADDRESS_LIST=
(ADDRESS=(PROTOCOL=TCP)(HOST=scanaddress)(PORT=1521)))
(CONNECT_DATA=(SERVICE_NAME=dbservice))

It’s a lot simpler and makes changes to the nodes in the cluster transparent.  This section isn’t intended to be a complete guide to writing Oracle URL’s – see the Oracle RAC Administration Guide.

Assuming that you will replace the MDS with the AGL, you will need to delete the MDS and the generic data sources from the configuration using the administration console and add a single AGL data source.  The process is described earlier in this chapter.  Give it the same JNDI name as your MDS had.  Select whether your generic data sources used an XA or non-XA driver. You can enter the complete URL as described above.  The user and password should be the same as what you had on one (hopefully all) of the MDS members.  When you get to the “Test GridLink Datasource Connection” page, click on the “Test All Listeners” button to see that you specified the new URL correctly.  The next page is where you need the new information for the ONS connections.  Specify one or more host:port pairs. For example, “host1-vip:6200” or “scanaddress:6200”.  If possible, use a single SCAN address and port.  Make sure that FAN enabled is checked.  On the next page, test the ONS connections.  Finally, you are ready to deploy the data source.

There are many data source parameters that you can’t configure on the creation flow.  You need to go back and edit the AGL data source configuration.  The parameters that you set should generally be based on the parameters that you were using in the MDS member data sources.  Hopefully they were all the same; if not, you need to decide what the right values are.

Tuesday Feb 12, 2013

5 Reasons to Upgrade to WebLogic Webcast: Your Questions Answered

Do you need to optimize your middleware performance and manageability? Are you looking to consolidate your application infrastructure to a modern shared-services or cloud infrastructure? To answer these questions and more we hosted a live webcast with Mike Lehmann, Senior Director of Product Management for Oracle WebLogic Server and Oracle iAS recently.


The live webcast was a major success thanks to all of you who attended and participated. It generated great enthusiasm and many questions most of which we answered during the live session. However, by popular demand, we have enclosed a link below to the all of the questions and answers from the webcast.

Q&A from 5 Reasons to Upgrade to Oracle WebLogic Server

It’s not too late

If you missed the live webcast, don’t worry. You can view it on demand at:

5 Reasons to Upgrade to Oracle WebLogic Server on demand

Listen to Mike discuss iAS to WebLogic upgrade paths, tools, and best practices for various data center configurations and highlight the benefits of upgrading to WebLogic and describes some of its rich capabilities including:

  • Lightweight, Modern Development Experience
  • High Density, High Performance Virtualization
  • Complete Visibility, Diagnosibility and Management of WebLogic and iAS


For more information, follow WebLogic on:
Twitter: @OracleWebLogic
LinkedIn: http://www.linkd.in/oracle_weblogic
Facebook: https://www.facebook.com/oracleweblogic
YouTube: www.youtube.com/oracleweblogic


Friday Feb 08, 2013

Deltek Highlights Smooth Upgrade to WebLogic, Plus Cool New Features

Listen to Dmitri Tyles, Senior Director of Development at Deltek -- a global provider of enterprise software and information solutions for project-oriented businesses -- discuss their reasons for upgrading to WebLogic 12c and their experience with the upgrade process. Deltek has about 14,000-plus customers and 1.8 million users Dmitri Tyles, Senior Director of Development at Deltek -- a global provider of enterprise software and information solutions for project-oriented businesses -- discuss their reasons for upgrading to WebLogic 12c and their experience with the upgrade process. Deltek has about 14,000-plus customers and 1.8 million users around the world in 80 countries. Needless to say the stakes are high. They were able to upgrade "Costpoint" with 1,500 applications to WebLogic 12c in about six weeks. They did run into a couple defects along the way which were reported to Oracle and fixes received pretty quickly. The rest of the upgrade was pretty smooth, Dmitri says. He goes on to talk about a particular feature which was really key for Deltek -- the support Oracle RAC. WebLogic always supported Oracle RAC, Dmitri claims, but the previous solution required a fair amount of manual maintenance from a WebLogic administrator. So for example, if a RAC administrator on a database side would add another node to the cluster, a WebLogic administrator would have to go in and manually update the same settings on WebLogic Server and add this node and register it. With the Active GridLink solution that Oracle has included in WebLogic, this is no longer necessary. So WebLogic is completely aware of all the changes happening to the RAC and, as such, all this ongoing maintenance is completely eliminated. 
About

The official blog for Oracle WebLogic Server fans and followers!

Stay Connected

Search

Archives
« May 2016
SunMonTueWedThuFriSat
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    
       
Today