How to Use RDA to Generate WLS Thread Dumps At Specified Intervals?

Introduction

There are many ways to generate a thread dump of a WebLogic Managed Server. For example, take a look at:

Taking Thread Dumps - [an excellent blog post on the Middleware Magic site]

or 

Different ways to take thread dumps in WebLogic Server (Document 1098691.1)

There is another method - use Remote Diagnostic Agent! The solution described below is not documented, but it is relatively straightforward to execute. One advantage of using RDA to collect the thread dumps is RDA will also collect configuration, log files, network, system, performance information at the same time.

Instructions

1. Not familiar with Remote Diagnostic Agent? Take a look at my previous blog "Resolve SRs Faster Using RDA - Find the Right Profile"

2. Choose a profile, which includes the WebLogic Server data collection modules (for example the profile "WebLogicServer"). At RDA setup time you should see the prompt below:

-------------------------------------------------------------------------------
S301WLS: Collects Oracle WebLogic Server Information
-------------------------------------------------------------------------------
Enter the location of the directory where the domains to analyze are located
(For example in UNIX, <BEA Home>/user_projects/domains or <Middleware
Home>/user_projects/domains)
Hit 'Return' to accept the default
(/oracle/11AS/Middleware/user_projects/domains)
> 

For a successful WLS connection, ensure that the domain Admin Server is up and
running.

Data Collection Type:
  1  Collect for a single server (offline mode)
  2  Collect for a single server (using WLS connection)
  3  Collect for multiple servers (using WLS connection)

Enter the item number
Hit 'Return' to accept the default (1)
> 2

Choose option 2 or 3.

Note: Collect for a single server or multiple servers using WLS connection means that RDA will attempt to connect to execute online WLST commands against the targeted server(s). The thread dumps are collected using the WLST function - "threadDumps()". If WLST cannot connect to the managed server, RDA will proceed to collect other data and ignore the request to collect thread dumps. If in the final output you see no Thread Dump menu item, then it's likely that the managed server is in a state which prevents new connections to it. If faced with this scenario, you would have to employ alternative methods for collecting thread dumps.

3. The RDA setup will create a setup.cfg file in the RDA_HOME directory. Open this file in an editor. You will find the following parameters which govern the number of thread dumps and thread dump interval.

#N.Number of thread dumps to capture
WREQ_THREAD_DUMP=10
#N.Thread dump interval
WREQ_THREAD_DUMP_INTERVAL=5000

The example lines above show the default settings. In other words, RDA will collect 10 thread dumps at 5000 millisecond (5 second) intervals. You may want to change this to something like:

#N.Number of thread dumps to capture
WREQ_THREAD_DUMP=10
#N.Thread dump interval
WREQ_THREAD_DUMP_INTERVAL=30000

However, bear in mind, that such change will increase the total amount of time it takes for RDA to complete its run.

4. Once you are happy with the setup.cfg, run RDA. RDA will collect, render, generate and package all files in the output directory.

5. For ease of viewing, open up the RDA Start html file - "xxxx__start.htm". The thread dumps can be found under the WLST Collections for the target managed server(s). See screenshots below

Screenshot 1:RDA Start Page - Main Index

Screenshot of RDA Start Page

Screenshot 2: Managed Server Sub Index

Screenshot of Managed Server Sub Index

Screenshot 3: WLST Collections

Screenshot of WLST Collections - pointing out Thread Dump menu option

Screenshot 4: Thread Dump Page - List of dump file links

Screenshot of Thread Dump Page - list of dump file links

Screenshot 5: Thread Dump Dat File Link

Screenshot of what you see when you click a dump file link

Additional Comment:

A) You can view the thread dump files within the RDA Start Page framework, but most likely you will want to download the dat files for in-depth analysis via thread dump analysis tools such as:

If you are new to thread dump analysis - take a look at this recorded Support Advisor Webcast 

Oracle WebLogic Server: Diagnosing Performance Issues through Java Thread Dumps
[Slidedeck from webcast in PDF format]

B) I have logged a couple of enhancement requests for the RDA Development Team to consider:

  1. Add timestamp to dump file links, dat filename and at the top of the body of the dat file
  2. Package the individual thread dumps in a zip so all dump files can be conveniently downloaded in one go.
Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

This is the blog of the Oracle Fusion Middleware Proactive Support Delivery Team. Here we will provide information about our activities, publications, product related information and more. Feedback welcome.

Follow OracleMWSupport on Twitter

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today