Friday Sep 28, 2012

OSB, Service Callouts and OQL

Oracle Fusion Middleware customers use Oracle Service Bus (OSB) for virtualizing Service endpoints and implementing stateless service orchestrations. Behind the performance and speed of OSB, there are a couple of key design implementations that can affect application performance and behavior under heavy load. One of the heavily used feature in OSB is the Service Callout pipeline action for message enrichment and invoking multiple services as part of one single orchestration. Overuse of this feature, without understanding its internal implementation, can lead to serious problems.

This series will delve into OSB internals, the problem associated with usage of Service Callout under high loads, diagnosing it via thread dump and heap dump analysis using tools like ThreadLogic and OQL (Object Query Language) and resolving it.

The first section in the series will mainly cover the threading model used internally by OSB for implementing Route Vs. Service Callouts.

The second section of the "OSB, Service Callouts and OQL" blog posting will delve into thread dump analysis of OSB server and detecting threading issues relating to Service Callout and using Heap Dump and OQL to identify the related Proxies and Business services involved.

The final section of the series will focus on the corrective action to avoid Service Callout related OSB serer hangs. Before we dive into the solution, we need to briefly discus about Work Managers in WLS.

Please refer to the blog posting for more details.

Monday Mar 05, 2012

ThreadLogic version 0.95 released

ThreadLogic version 0.95 is now available for public download.

Key addition is support for externalizing the Advisories and group definitions. Users can use the pre-defined AdvisoryMap.xml to come up with custom advisories with their own definition of what constitutes an advisory - name, pattern/keyword, health, description, advice and let ThreadLogic tag the matching threads with your custom advisory when it finds a match. Similarly the group definitions can be modified or enhanced to include new groupings. This is in addition to other smaller bug fixes and code cleanup.

For more details on how add customized advisories and groups and more features of version 0.95, please check my blog posting.

Feedback and suggestions welcome.

Friday Feb 17, 2012

Introducing ThreadLogic

Eric Gross and Sabha Parameswaran, from Oracle FMW Architects Team (The A-team) are happy to introduce ThreadLogic, an open source Thread Dump Analyzer, to Oracle Fusion Middleware community and customers. 

Motivation behind ThreadLogic

The current set of TDA tools (Samurai/TDA) do not mine the thread dumps. Neither do they provide a more detailed view of what each thread is doing. Most of the existing tools tend to limit themselves to reporting the state (locked/waiting/running) or the lock information.  They don't go into details of the type of activity within a thread, should it be treated as normal or deserving a closer look? Can a pattern or anti-pattern be applied against them? Any possible optimizations? Are there any hot spots? Any classification of threads based on their execution cycles? 

We decided to create ThreadLogic to address these deficiencies. It is based on a fork of the TDA open source tool with new capabilities to analyze and provide advice based on a set of extensible advisories and thread grouping while supporting all JVM thread dumps. Also, a thorough and in-depth analysis of WebLogic Server thread dumps is provided. Both the thread grouping and advisories are extensible where user can add new patterns to match and tag or group threads. Please check the attached document for more details of the tool.

The latest stable bits (version 0.9) and documentation can be downloaded from here.  For more details on the tool's capabilities, please check the documentation page.

Our hope is that this tool would go a long way in improving and helping both internal teams (Support/Dev/Field) and external customers in terms of better analysis and faster issue resolutions.

Feedback welcome.

Thursday Feb 09, 2012

Analyzing Thread Dumps In Middleware

How to analyse Thread dumps, for improving Middleware Performance (at App Server or Application level) as well as for general troubleshooting? This series goes into when and how to capture thread dumps, analysis methodologies, review of TDA tools and an enhanced version of existing TDA tool that can analyze more in depth with special focus on WebLogic Server related dumps as well as some real world samples of thread dump analysis before concluding the series. 

Analyzing Thread Dumps - Series:

Part 4: TDA A-Team and real world samples

Part 3: TDA tools

Part 2: How to capture and analyze Thread Dumps, navigate lock chains

Part 1: Basics of Thread states and thread locking

[Read More]

The primary contributors to this blog are comprised of the Exalogic and Cloud Application Foundation contingent of Oracle's Fusion Middleware Architecture Team, fondly known as the A-Team. As part of the Oracle development organization, The A-Team supports some of Oracle's largest and most strategic customers worldwide. Our mission is to provide deep technical expertise to support various Oracle field organizations and customers deploying Oracle Fusion Middleware related products. And to collect real world feedback to continuously improve the products we support. In this blog, our experts and guest experts will focus on Exalogic, WebLogic, Coherence, Tuxedo/mainframe migration, Enterprise Manager and JDK/JRockIT performance tuning. It is our way to share some of our experiences with Oracle community. We hope our followers took away something of value from our experiences. Thank you for visiting and please come back soon.


« July 2016