Wednesday Nov 27, 2013

How to get Work Manager Information in a WLS or OSB Thread Dump

To analyze issues related to performance or work load, it would be nice to know which work manager is running a given thread. In Oracle Service Bus, a dispatch policy defined for a proxy or a business service relates to a work manager definition in WLS. While analyzing a thread dump, you would like to know which work manager is running a given thread.

Starting with WebLogic Server version 10.3.6, work manager information is shown in a thread dump if work manager / self tuning debugging is enabled. This debug flag can be enabled in the WLS console:

Go to http://<admin-server>:<admin-port>/console - Environment - <your server> - Debug tab:

ENABLE DEBUG FLAG FOR WORK MANAGER SELF TUNING 

Enable the DebugSelfTuning debug flag in the work manager section.

After that, work manager information is automatically added to a thread dump. This is a sample stack trace which has work manager information included:

"[ACTIVE] ExecuteThread: '14' for queue: 'weblogic.kernel.Default (self-tuning)' for workmanager: XBus Kernel@null@MyCustomWorkManager" id=123 idx=0x1d4 tid=21936 prio=5 alive, waiting, native_blocked, daemon
-- Waiting for notification on: java/lang/Object@0xe574a240[fat lock]
at jrockit/vm/Threads.waitForNotifySignal(JLjava/lang/Object;)Z(Native Method)
at java/lang/Object.wait(J)V(Native Method)
at java/lang/Object.wait(Object.java:485)
at com/bea/wli/sb/pipeline/PipelineContextImpl$SynchronousListener.waitForResponse(PipelineContextImpl.java:1620)
^-- Lock released while waiting: java/lang/Object@0xe574a240[fat lock]
at com/bea/wli/sb/pipeline/PipelineContextImpl.dispatchSync(PipelineContextImpl.java:562)
at stages/transform/runtime/WsCalloutRuntimeStep$WsCalloutDispatcher.dispatch(WsCalloutRuntimeStep.java:1391)
at stages/transform/runtime/WsCalloutRuntimeStep.processMessage(WsCalloutRuntimeStep.java:236)
at com/bea/wli/sb/pipeline/debug/DebuggerRuntimeStep.processMessage(DebuggerRuntimeStep.java:74)
at com/bea/wli/sb/stages/StageMetadataImpl$WrapperRuntimeStep.processMessage(StageMetadataImpl.java:346)
at com/bea/wli/sb/stages/impl/SequenceRuntimeStep.processMessage(SequenceRuntimeStep.java:33)
at com/bea/wli/sb/pipeline/PipelineStage.processMessage(PipelineStage.java:84)

 

Monday Nov 11, 2013

Scripted SOA Diagnostic Dumps for PS6 (11.1.1.7)

When you upgrade to SOA Suite PS6 (11.1.1.7) you acquire a new set of Diagnostic Dumps in addition to what was available in PS5. With more than a dozen to choose from and not wanting to run them one at a time, this blog post provides a sample script to collect them all quickly and hopefully easily. There are several ways that this collection could be scripted and this is just one example.




What is Included:
  • wlst.properties: Ant Properties
  • build.xml
  • soa_diagnostic_script.py: Python Script


What is Collected:
  • 5 contextual thread dumps at 5 second intervals
  • Diagnostic log entries from the server
  • WLS Image which includes the domain configuration and WLS runtime data
  • Most of the SOA Diagnostic Dumps including those for BPEL runtime, Adapters and composite information from MDS


Instructions:
  1. Download the package and extract it to a location of your choosing
  2. Update the properties file 'wlst.properties' to match your environment
  3. Run 'ant' (must be on the path)
  4. Collect the zip package containing the files (by default it will be in the script.output location)


Properties Reference:
  • oracle_common.common.bin: Location of oracle_common/common/bin
  • script.home: Location where you extracted the script and supporting files
  • script.output: Location where you want the collections written
  • username: User name for server connection
  • pwd: Password to connect to the server
  • url: T3 URL for server connection, '<host>:<port>'
  • dump_interval: Interval in seconds between thread dumps
  • log_interval: Duration in minutes that you want to go back for diagnostic log information
Script Package

Dynamic Monitoring Service (DMS) Configuration Dumping and CPU Utilization


There was recently a report of CPU spikes on a system that were occuring at precise 3 hour intervals. Research revealed that the spikes were the result of the Dynamic Monitoring Service generating a metrics dump and writing it under the server 'logs' folder for every WLS server in the domain. This blog provides some information on what this is for and how to control it.


The Dynamic Monitoring Service is a facility in FMw (JRF to be more precise) that collects runtime data on the components deployed to WebLogic. Each component is responsible for how much or how little they use the service and SOA collects a fair amount of information. To view what is collected on any running server you can use the following URL, http://host:port/dms/Spy and login with admin credentials.


DMS is essentially always running and collecting this information in the runtime and to protect against loss of this data it also runs automatic backups, by default at the 3 hour interval mentioned above. Most of the management options for DMS are exposed through WLST but these settings are not so we must open the dms_config.xml file which can be found in DOMAIN_HOME/config/fmwconfig/servers/<server_name>/dms_config.xml.


The contents are fairly short and at the bottom you will find the following entry:

<dumpConfiguration>
    <dump intervalSeconds="10800" maxSizeMBytes="75" enabled="true"/>
</dumpConfiguration>

The interval of 10800 seconds corresponds to the 3 hours and the maximum size is 75MB. The file is written as an archive to DOMAIN_HOME/servers/<server_name>/logs/metrics. This archive contains the dump in XML format.


You can disable the dumps all together by simply setting the 'enabled' value to 'false' or of course you could modify the other parameters to suit your needs. Disabling the dumps will NOT impact DMS collections or display at runtime. It will only eliminate these periodic backups.


About

This is the official blog of the SOA Proactive Support Team. Here we will provide information on our activities, publications, product related information and more. Additionally we look forward to your feedback to improve what we do.

Search

Categories
Archives
« November 2013 »
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
7
8
9
10
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
28
29
30
       
Today