Tuesday Jun 25, 2013

Example WLST Script to Obtain JDBC and JTA MBean Values

Introduction

Following on from the blog entry "Get an Offline or Online WebLogic Domain Summary Using WLST!", I have had a request to create a smaller example which only collects a selection of JDBC (System Resource) and JTA configuration and runtime MBeans values. So, here it is.

Download Sample Script

You can grab the sample script by clicking here.

Instructions to Run:

1. After download, extract the zip to the machine hosting the WebLogic environment. You should have three directories along with a readme.txt

  • output
  • Sample_Output
  • scripts

2. In the scripts directory, find the start wrapper script startWLSTJDBCSummarizer.sh (Unix) or startWLSTJDBCSummarizer.cmd (MS Windows). Open the appropriate file in an editor and change the environment variable settings to suit your system.

Example - startWLSTDomainSummarizer.cmd

set WL_HOME=D:\product\FMW11g\wlserver_10.3
set DOMAIN_HOME=D:\product\FMW11g\user_projects\domains\MyDomain
set WLST_OUTPUT_PATH=D:\WLSTDomainSummarizer\output\
set WLST_OUTPUT_FILE=WLST_JDBC_Summary_Via_MBeans.html
call "%WL_HOME%\common\bin\wlst.cmd" WLS_JDBC_Summary_Online.py

Note: The WLST_OUTPUT_PATH directory value must have a trailing slash. If there is no trailing slash, the script will error and not continue. 

3. Run the shell / command line wrapper script. It should launch WLST and kick off "WLS_JDBC_Summary_Online.py". This will hit you with some prompts e.g.

Is your domain Admin Server up and running and do you have the connection details? (Y /N ):
Y
Enter connection URL to Admin Server e.g t3://mymachine.acme.com:7001 :
t3://localhost:7001
Enter weblogic username:
weblogic
Enter weblogic username password (function prompt 1):
welcome1

(Note: the value typed in for password will not be echoed back to the console).

4. If the scripts run successfully, you should get a HTML summary in the specified output directory. See example screenshots below:

Screenshot 1 - JDBC System Resource Tab Page

 Screenshot 2 - JTA Tab Page

5. For the HTML to render correctly, ensure the .js and .css files provided (review the output directory created by the zip file extraction) are accessible. For example, to view the HTML locally (without using a web server), place the HTML output, jquery-ui.js, spry.js and wlstsummarizer.css in the same directory.

Disclaimer

This is a sample script. I have tested it against WebLogic Server 10.3.6 domains on MS Windows and Unix.  I cannot guarantee that the script will run error free or produce the expected output on your system. If you have any feedback add a comment to the blog. I will endeavour to fix any problems with my WLST code.

Credits

JQuery: http://jquery.com/
Spry (Adobe) : https://github.com/adobe/Spry
http://www.red-team-design.com/cool-headings-with-pseudo-elements

Friday May 31, 2013

How to Query Fusion Middleware Metadata Schemas Via JDBC

Introduction

You are dealing with a Fusion Middleware product, such as SOA, which uses Metadata Schemas, typically created in an Oracle Database via the Repository Creation Utility (RCU). As a Middleware administrator you would like to run a script which queries the schemas. However, the database resides on a different host from the Middleware Home, and there is no SQL*Plus or similar tool available. 

The solution? Although Fusion Middleware products do not ship SQL*Plus, Weblogic Server does include JDBC drivers. This presents an opportunity to try out / brush up your java program coding skills.

Example Solution

Having recently dabbled in WLST to query Weblogic Domain MBeans - Get an Offline or Online WebLogic Domain Summary Using WLST! - I have written an example java program to tackle the problem outlined above. The program utilizes the standard Oracle JDBC driver to:

  • establish a connection to an Oracle Database (which can be remote from the machine hosting Weblogic)
  • attempts three queries listed below
  • renders the output in HTML. (I have reused the html / javascript techniques I employed for the WLST Domain Summarizer to present the results in a nice readable format.)

Query ALL_USERS for owners / "prefixes" of Fusion Middleware schemas.

SELECT USERNAME, CREATED
FROM ALL_USERS
WHERE USERNAME LIKE  '%_MDS'
OR USERNAME LIKE '%_IAU'
OR USERNAME LIKE '%_IAU_%'
OR USERNAME LIKE '%_WLS'
OR USERNAME LIKE '%_STB'
OR USERNAME LIKE '%_UMS'
OR USERNAME LIKE '%_UCSCC'
OR USERNAME LIKE '%_OPSS' 
OR USERNAME LIKE '%_SOAINFRA'
ORDER BY 2

Schema Version Registry Summary

SELECT COMP_ID, COMP_NAME, VERSION, STATUS, UPGRADED
FROM
SCHEMA_VERSION_REGISTRY
ORDER BY 1,2

SOAINFRA - Cube Instance Summary

SELECT COMPOSITE_NAME, COMPONENT_NAME, COMPONENTTYPE,
(CASE WHEN STATE=1 THEN 'OPEN AND RUNNING'
WHEN STATE=2 THEN 'OPEN AND SUSPENDED'
WHEN STATE=3 THEN 'OPEN AND FAULTED' 
WHEN STATE=4 THEN 'CLOSED AND PENDING'
WHEN STATE=5 THEN 'CLOSED AND COMPLETED'
WHEN STATE=6 THEN 'CLOSED AND FAULTED'
WHEN STATE=7 THEN 'CLOSED AND CANCELLED'
WHEN STATE=8 THEN 'CLOSED AND ABORTED'
WHEN STATE=9 THEN 'CLOSED AND STALE'
WHEN STATE=10 THEN 'NON-RECOVERABLE' ELSE STATE || ''  END) AS STATE,
COUNT(*) AS NUM_OF_CUBE_INST
FROM " + v_dbuser_soainfra + ".CUBE_INSTANCE
GROUP BY COMPOSITE_NAME, COMPONENT_NAME, COMPONENTTYPE,STATE
ORDER BY COUNT(*) DESC

Download

You can grab the zip package containing  the class, java source and example output by clicking here.

Instructions

1. After download, extract the zip.

2. cd to the directory labelled "class"

3. Edit the LaunchMetadataCollector.sh (Unix) or LaunchMetadataCollector.cmd (Windows) and change the environment variable setting - WL_HOME - to point at your Weblogic Server home.

4. Then run the LaunchMetadataCollector wrapper script. This will prompt for database connection details e.g.

>Enter the database hostname or ip address: mydbhost.acme.com
>Enter the database port number: 1521
>Enter the database sid: orcl
>Enter the database username: system
>Enter the password: 
>Enter the password again: 
[Optional] Enter the database user which owns the SOAINFRA schema: DEV

5. If the program runs successfully you should see the result of the queries in an output file labelled MetadataCollectorOutput.html

If you want to amend / adapt / enhance the program feel free to take a look at the java source file which is located in the "source" directory.

Monday May 13, 2013

My Take on Getting Started With WebLogic Scripting Tool (WLST)

Introduction

WebLogic Scripting Tool (WLST) has been a long established favourite for experienced WebLogic Server administrators. Over the past two / three weeks I have spent time getting to know WLST - and by getting to know I mean developing a script to produce a solution rather than randomly running three or four commands :-). For the final result of my efforts, see my previous blog entry

Get an Offline or Online WebLogic Domain Summary Using WLST! 

This is a short summary of the resources I used. 

Initial reading and understanding

If you are never encountered WLST, spend a few minutes reading:

Oracle Fusion Middleware Administrator's Guide 11g Release 1 (11.1.1)
3.5.1 Getting Started Using the Oracle WebLogic Scripting Tool (WLST)

Oracle Fusion Middleware WebLogic Scripting Tool Command Reference 11g
Overview of WLST Command Categories

Unsurprisingly, you will also find yourself using the Command Reference as a look up for available commands, functionality and syntax.

WLST Code Snippets and Examples

(There are many more WLST postings (questions, examples, discussions) to be found on the web.)

Jython

WLST is based on the Java scripting interpreter, Jython and as such supports standard Jython features such as local variables, conditional variables, and flow-control statements. Therefore, to achieve something more interesting and exciting you must prepared to learn (by doing) a little Jython. Fortunately, the Jython documentation is comprehensive and provides some good examples:

Use an editor which recognizes Jython / Python

Jython (like it's "parent" language Python) is very sensitive to syntax errors. In particular, the language requires the use of indents when coding conditions (e.g IF statements) and loops. Attempt to write Jython using a plain text editor, and before long stress levels will rise. You can make life easier by using an editor which recognizes Jython. Editors like Oracle Eclipse ** and SCITE assist the developer by for example:

  • auto-formatting lines of code
  • color coding of syntax
  • providing option to show line numbers

** Eclipse requires the Python and Jython Interpreters to be configured. Instructions can be found here.

My Oracle Support Documents

And finally, if you have access to My Oracle Support, here are a few knowledge articles which provide some nice WLST examples:

  • How To Debug WLST Jython Scripts? [ID 1360744.1]
  • WLST sample script To Obtain Managed Server Status [ID 1373532.1]
  • WLST Script for monitoring and getting statistics of WebLogic JMS Destinations. [ID 1327086.1]
  • How to get JDBC and Connection Pool data from a WLST or shell script [ID 1453818.1]
  • How to Startup or Shutdown Multiple WebLogic Managed Servers With a WLST Script [ID 1383150.1]
  • How To Take Thread Dumps With WLST [ID 1274713.1]

Tuesday May 07, 2013

Get an Offline or Online WebLogic Domain Summary Using WLST!

Introduction

Further to blog entry "WLST Offline Script to Create Summary of WebLogic Domain", I have created a solution which extends the capability to collecting an online summary if a connection to the domain's Admin Server can be established.

Search the web, and you will find many WLST examples which summarize managed server status and performance, JMS, JDBC and more. In this respect, my efforts are nothing new. However, I thought I would share as my scripts have the following "bells and whistles".

1. If you are unable to connect to the domain's Admin Server, an offline summary of the domain will automatically be obtained

2. The script renders the collected data in HTML, using some javascript to give the final result some interactive formatting for ease of reading. See screenshot snippet below:

Screenshot : Snippet of WLST Summarizer HTML Output

Screenshot showing WLST Summarizer HTML output

3. If a managed server has a "FAILED" health state or there is 1 or more thread hogs in its JVM thread pool, three thread dumps will be captured (15 seconds apart).

Download Sample Script

You can grab the sample script by clicking here.

Instructions to Run:

1. After download, extract the zip to the machine hosting the WebLogic environment. You should have three directories along with a readme.txt

  • output
  • Sample_Output
  • scripts

2. In the scripts directory, find the start wrapper script startWLSTDomainSummarizer.sh (Unix) or startWLSTDomainSummarizer.cmd (MS Windows). Open the appropriate file in an editor and change the environment variable settings to suit your system.

Example - startWLSTDomainSummarizer.cmd

set WL_HOME=D:\product\FMW11g\wlserver_10.3
set DOMAIN_HOME=D:\product\FMW11g\user_projects\domains\MyDomain
set WLST_OUTPUT_PATH=D:\WLSTDomainSummarizer\output\
set WLST_OUTPUT_FILE=WLST_Domain_Summary_Via_MBeans.html

call "%WL_HOME%\common\bin\wlst.cmd" StartDomainSummarizer.py

Note: The WLST_OUTPUT_PATH directory value must have a trailing slash. If there is no trailing slash, the script will error and not continue. 

3. Run the shell / command line wrapper script. It should launch WLST and kick off "StartDomainSummarizer.py". This initial python script will hit you with some prompts e.g.

Is your domain Admin Server up and running and do you have the connection details? (Y /N ):
Y
Enter connection URL to Admin Server e.g t3://mymachine.acme.com:7001 :
t3://localhost:7001
Enter weblogic username:
weblogic
Enter weblogic username password (function prompt 1):
welcome1

(Note: the value typed in for password will not be echoed back to the console).

Depending on your responses it will call "WLSDomainInfoOffline.py" or "WLSDomainInfoOnline.py". 

4. If the scripts run successfully, you should get a HTML summary in the specified output directory. 

5. For the HTML to render correctly, ensure the .js and .css files provided (review the output directory created by the zip file extraction) are accessible. For example, to view the HTML locally (without using a web server), place the HTML output, jquery-ui.js, spry.js and wlstsummarizer.css in the same directory.

Disclaimer

This is a sample script. I have tested it against WebLogic Server 10.3.6 domains on MS Windows and Unix.  I cannot guarantee that the script will run error free or produce the expected output on your system. If you have any feedback add a comment to the blog. I will endeavour to fix any problems with my WLST code.

Credits

JQuery: http://jquery.com/
Spry (Adobe) : https://github.com/adobe/Spry
http://www.red-team-design.com/cool-headings-with-pseudo-elements

Friday Apr 19, 2013

WLST Offline Script to Create Summary of WebLogic Domain

Introduction

I have always believed it is important to find out the bigger picture early on in a problem investigation. For example, if a managed server is exhibiting problematic behaviour, as well as I looking at the logs, I like to know the context - for example:

  • Is the server part of a cluster?
  • How many servers / clusters are in the Weblogic Domain?
  • Is the domain spread across multiple machines?
  • Are System Resources such JDBC and JMS in play?

Weblogic Server scripting ( WLST ) has been around for many years. Consequently there are a large number of examples to be found in blogs, websites and forums. The vast majority assume you can run the script in online mode. What if, however, a WLST online connection cannot be established? Ok, sure we can manually peruse the DOMAIN_HOME/config/config.xml. It is not a friendly a solution partly because some of the configuration is held in other xml files referenced by the config.xml. There has to be another way, I thought, after all WebLogic Server holds domain configuration in MBeans. This got me thinking about WLST.


WLST's readDomain Function

We can use WLST's readDomain function, in offline mode, to load the domain MBean configuration hierarchy / tree. It is as simple this:

Example: Unix Machine:

WL_HOME=/oracle/middleware/wlserver_10.3
DOMAIN_HOME=/oracle/middleware/user_projects/domains/MyDomain
$WL_HOME/common/bin/wlst.sh
wls:/offline/readDomain('/oracle/middleware/user_projects/domains/MyDomain')

Now you are ready to navigate the "tree" using "cd" and "ls" style syntax

wls:/offline/MyDomain/ls()

drw-   AnyMachine
drw-   AppDeployment
drw-   Cluster
drw-   EmbeddedLDAP
drw-   FileStore
drw-   JDBCSystemResource
drw-   JMSServer
drw-   JMSSystemResource
drw-   Library
drw-   MigratableTarget
drw-   Security
drw-   SecurityConfiguration
drw-   Server
drw-   ShutdownClass
drw-   StartupClass
drw-   WLDFSystemResource
-rw-   Active                                        false
-rw-   AdminServerName                               AdminServer
-rw-   AdministrationMBeanAuditingEnabled            false
-rw-   AdministrationPort                            9002
.... etc
wls:/offline/MyDomain/cd ('Server')
wls:/offline/MyDomain/Server/ls()
drw-   AdminServer
drw-   MyManagedServer
drw-   MyManagedServer01
wls:/offline/MyDomain/Server/cd ('MyManagedServer')
wls:/offline/MyDomain/Server/MyManagedServer/ls()
drw-   DataSource
drw-   NetworkAccessPoint
drw-   SSL
drw-   ServerDiagnosticConfig

-rw-   AcceptBacklog                                 300
-rw-   AdminReconnectIntervalSeconds                 10
-rw-   AdministrationPort                            0
-rw-   AdministrationPortEnabled                     false
-rw-   AdministrationProtocol                        null
...
-rw-   JavaCompilerPostClassPath                     null
-rw-   JavaCompilerPreClassPath                      null
-rw-   JavaStandardTrustKeyStorePassPhraseEncrypted  null
-rw-   JdbcLoginTimeoutSeconds                       0
-rw-   KeyStores                                     null
-rw-   ListenAddress                                 127.0.0.1
-rw-   ListenDelaySecs                               0
-rw-   ListenPort                                    7021
-rw-   ListenPortEnabled                             true

... etc
wls:/offline/MyDomain/Server/MyManagedServer/foo = get('ListenPort')
wls:/offline/MyDomain/Server/MyManagedServer/print foo

7021

 

Sample Script and Output

To write a script, which automates the capture and print out of attributes useful to a summary, is challenging because the functions to traverse and search the MBean tree are limited compared to online mode. You are pretty much stuck with the cd(), ls() and get(). However, with a bit of coding creativity with IF and FOR loop syntax you can programmatically explore the tree and return output in a nice format - in my sample I have chosen HTML tables.

Sample Output

To view sample output, go here

Screenshot below shows snipper of sample output.


Download Sample Script

You can grab the sample script by clicking here.

Instructions to Run:

1. Edit the environment variables in the start wrapper script startWLSofflineCollection.sh (Unix) or startWLSofflineCollection.cmd (MS Windows) to suit your system

Example - startWLSofflineCollection.sh

WL_HOME=/oracle/middleware/wlserver_10.3
DOMAIN_HOME=/oracle/middleware/user_projects/domains/MyDomain; export DOMAIN_HOME
WLST_OUTPUT_PATH=/temp/wlst/; export WLST_OUTPUT_PATH
WLST_OUTPUT_FILE=WLST_MBean_Config_Summary.html; export WLST_OUTPUT_FILE
${WL_HOME}/common/bin/wlst.sh CollectWLSConfigOffline.py

Note: The WLST_OUTPUT_PATH directory value must have a trailing slash. If there is no trailing slash script will error and not continue.

2. Then run the wrapper script. It should launch WLST and run the script. The script will collect MBean values and render them in a HTML file.
(Once the script has invoked the WLST shell, the execution of the python script should not take longer than 5 to 10 seconds.)

Disclaimer

This is a sample script. I have tested it against WebLogic Server 10.3.6 domains on MS Windows and Unix.  I cannot guarantee that the script will run error free or produce the expected output on your system. That said, a number of colleagues have also run the script and with their feedback I have ironed out a few problems. If you have any feedback add a comment to the blog. I will endeavour to fix any problems with my WLST code. 

Credits

I found the following blogs particularly useful.

Java / Oracle SOA blog
[http://biemond.blogspot.co.uk/2010/02/invoking-fmw-application-mbeans-in.html]

WLST By Examples
[http://wlstbyexamples.blogspot.co.uk/]


Some Pitfalls (which required coding around)

Checking whether Configuration MBean Hierarchy Exists

If a service or functionality, such as Clusters, JDBC, JMS, Filestore are not configured, the related MBean hierarchy will not be visible when listing at domain level. Example: If Cluster, JDBC, JMS and Filestore have not been configured, the ls() will show:

wls:/offline/MyDomain/ls()

drw-   AnyMachine
drw-   AppDeployment
drw-   EmbeddedLDAP
drw-   Library
drw-   MigratableTarget
drw-   Security
drw-   SecurityConfiguration
drw-   Server
drw-   ShutdownClass
drw-   StartupClass
drw-   WLDFSystemResource

A programmatic attempt to cd to a directory not listed will return an error e.g.

Error: cd() failed. Do dumpStack() to see details.

To work-around, the script contains a custom findMBean function. The function places the contents of the ls() in an array using the returnmap parameter. We can then loop through the array and check whether a given string - the MBean directory name exists e.g.

def findMBean(v_pattern):
        # get a listing of everything in the current directory
    mydirs = ls(returnMap='true');

        v_compile_pattern = java.util.regex.Pattern.compile(v_pattern);

    found = 'Nope not here';

    for mydir in mydirs:
        x = java.lang.String(mydir);
        v_matched = v_compile_pattern.matcher(x);
        if v_matched.find():
            found = 'true';

        return found;

AnyMachine v Machines

In WebLogic Server 10.3, if a domain has one or more machines configured (i.e nodemanager related machines) the ls() at domain level will show

wls:/offline/MyDomain/ls()

drw-   AnyMachine

A cd ('AnyMachine'), however, will fail. The correct cd syntax is

wls:/offline/MyDomain/cd ('Machines')

Identifying Target Information

Features such as JDBC System Resources, Clusters may or may not have targets. An attempt to cd to an MBean directory or get an MBean attribute which does not exist will throw an error and prevent the script from continuing. To work-around we can use TRY / EXCEPT syntax. Example - snippet from sample script which deals with the capture of JDBC System Resource targets.

try:
            v_any_targets = '';
            v_jdbc_target = get('Target');

            # Even if the get fails, the variable is assigned a value of none, set the flag variable accordingly
            if v_jdbc_target == 'None':
                v_any_targets = 'None';
                v_no_of_targets = 1;
            else:   
                # If the get has succeeded then set flag accordingly and obtain length of array returned by the get
                # The array length will be used to determine the HTML rowspan value

                v_any_targets ='Use v_jdbc_target';
                v_no_of_targets = len(v_jdbc_target);

        except:

            # Setting flag and rowspan variable here as well .. belt and braces
            v_any_targets = 'None';
            v_no_of_targets = 1;

            # The exception will still display to standard out, which may cause alarm
            # So adding this message telling the user the exception is expected and can be ignored
            print "IGNORE this exception";   

Thursday Dec 13, 2012

How to Run Apache Commands From Oracle HTTP Server 11g Home

Every now and then you come across a problem when there is nothing in the "troubleshooting manual" which can help you. Instead you need to think outside the box. This happened to me two or three years back. Oracle HTTP Server (OHS) 11g did not start. The error reported back by OPMN was generic and gave no clue, and worse the HTTP Server error log was empty, and remained so even after I had increased the OPMN and HTTP Server log levels. After checking configuration files, operating system resources, etc I was still no nearer the solution.

And then the light bulb moment!

OHS is based on Apache - what happens if I attempt to start HTTP Server using the native apache command. Trouble was the OHS 11g solution has its binaries and configuration files in separate "home" directories

  • ORACLE_HOME contains the binaries
  • ORACLE_INSTANCE contains the configuration files

How to set the environment so that native apache commands run without error? Eventually, with help from a colleague, the knowledge article

How to Start Oracle HTTP Server 11g Without Using opmnctl [ID 946532.1]

was born!

To be honest, I cannot remember the exact cause and solution to that OHS problem two or three years ago. But, I do remember that an attempt to start HTTP Server using the native apache command threw back an error to the console which led me to discover the culprit was some unusual filesystem fault.

The other day, I was asked to review and publish a new knowledge article which described how to use the apache command to dump a list of static and shared loaded modules. This got me thinking that it was time
[ID 946532.1] was given an update. The result

How To Run Native Apache Commands in an Oracle HTTP Server 11g Environment [
ID 946532.1]

Highlights:

  • Title change
  • Improved environment setting scripts
    • Interactive, should be no need to manually edit the scripts (although readers are welcome to do so)
    • Automatically dump out some diagnostic information
  • Inclusion of some links to other troubleshooting collateral

To view the knowledge article you need a My Oracle Support login. For convenience, you can obtain the scripts via the links below.

MS Windows:

Wrapper cmd script - calls main cmd script [After download, remove the ".txt" file extension]
Main cmd script - sets OHS 11g environment to run Apache commands [After download, remove the ".txt" file extension]

Unix:

Shell script - sets OHS 11g environment to run Apache commands on Unix

Please note: I cannot guarantee that the scripts held in the blog repository will be maintained. Any enhancements or faults will applied to the scripts attached to the knowledge article.

Lastly, to find out more about native apache commands, refer to the Apache Documentation

apachectl - Apache HTTP Server Control Interface
[http://httpd.apache.org/docs/2.2/programs/apachectl.html]

httpd - Apache Hypertext Transfer Protocol Server
[http://httpd.apache.org/docs/2.2/programs/httpd.html]

About

This is the blog of the Oracle Fusion Middleware Proactive Support Delivery Team. Here we will provide information about our activities, publications, product related information and more. Feedback welcome.

Follow OracleMWSupport on Twitter

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today