Wednesday Dec 10, 2008

Using the new wshang.ksh version 3.1

I just downloaded the new  wshang.ksh version 3.1 and tried it. Go to  the download location.

(Note: above url is not valid any more. Please download it from My Oracle Support (MOS) site from note: Doc ID 1325651.1 - Download Location For Wshang Script - Gather Debug Data Script For Troubleshooting Oracle iPlanet Web Server or Oracle iPlanet Web Proxy Server Hangs.)

It is very easy to use.

After I downloaded and gunzipped it, I just ran it,

 APPLE:/export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3> ./wshang.ksh
./wshang.ksh[65]: date+%y%m%d%HH%MM%SS:  not found

 This script collects 3 snapshots of the following information
 at a 15 seconds interval against a web server process:
        prstat -L -a (will try 'top' on pre-Solaris 8 systems)
        pmap -x
        netstat -an
 And collections 30 seconds of:
        truss -feald -vall -wall -rall

Data will be collected to the location /tmp/wshang by default.
Data location should have enough disk space for large core files.
Would you like to specify a different directory for data?

Specify a directory:

The following is a list of web server processes:

Solaris 8 or greater
Web Server 7.0/6.1 processes
    364   webservd -r /export/home/iws6.1sp2 -d /export/home/iws6.1sp2/https-ap
    11478 webservd -r /export/home0/case/11330103-admin-gu/61sp10 -d /export/ho
    11640 webservd -d /export/home0/case/11330103-admin-gu/70u3/https-APPLE.low
    12091 webservd -d /export/home/iws7.0u3/admin-server/config -r /export/home
    12742 webservd -r /export/home0/case/11315437-web-session/FVB/run -d /expor
    12814 webservd -r /export/home0/case/11315437-web-session/FVB/run -d /expor
    12860 webservd -r /export/home/iws6.1sp10 -d /export/home/iws6.1sp10/https-
    12919 webservd -r /export/home/iws6.1sp10 -d /export/home/iws6.1sp10/https-
    13879 webservd -d /export/home/iws7.0u3/https-java-mon/config -r /export/ho
    13573 webservd -d /export/home/iws7.0u3/ -r
    15402 webservd -r /export/home/iws6.1sp10 -d /export/home/iws6.1sp10/https-
    15497 webservd -d /export/home0/case/11330103-admin-gu/70u3-2/admin-server/
Web Server 6.0 processes
Proxy Server 4.0 processes
    14065 proxyd -r /export/home/ips4.0sp8 -d /export/home/ips4.0sp8/proxy-acl/
    15527 proxyd -r /export/home/ips4.0sp7 -d /export/home/ips4.0sp7/proxy-adms
    15867 proxyd -r /export/home/ips4.0sp7 -d /export/home/ips4.0sp7/proxy-rpp/

Enter the PID of the web server process to collect data on
(Note that the correct PID for child webservd/ns-httpd process will not be listed correctly here if you have MaxProcs > 1 in your magnus.conf):

(this is already the needed worker process pid from my ptree outputs,

APPLE:/export/home0> ptree 12919
12893 ./webservd-wdog -r /export/home/iws6.1sp10 -d /export/home/iws6.1sp10/htt
  12918 webservd -r /export/home/iws6.1sp10 -d /export/home/iws6.1sp10/https-se
    12919 webservd -r /export/home/iws6.1sp10 -d /export/home/iws6.1sp10/https-


 12919 ?        1:41 webservd

Gathering data now...

Loop 1 for 15 seconds...

collecting pflags...
collecting pmap -x...
collecting pldd...
collecting prstat -L -a...
collecting pstack...
collecting pfiles...
collecting netstat...

Loop 2 for 15 seconds...

collecting pflags...
collecting pmap -x...
collecting pldd...
collecting prstat -L -a...
collecting pstack...
collecting pfiles...
collecting netstat...

Loop 3 for 15 seconds...

collecting pflags...
collecting pmap -x...
collecting pldd...
collecting prstat -L -a...
collecting pstack...
collecting pfiles...
collecting netstat...

collecting 30 seconds of truss...

Collecting gcore...
gcore: /export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/gcore.out.12919 dumped
\* ----------------------------------------------------------------------------------
\* Sun Microsystems RSD pkgapp 3.0 Solaris                               [12/10/2008]
\* ----------------------------------------------------------------------------------
\* OS release                            [5.8]
\* Platform                              [SUNW,Sun-Blade-1000]
\* Checking [-c] is a core or pid        [using pid 12919]
\* Process Root                          [/export/home/iws6.1sp10/https-sess/config]
\* Databin parameter [-s] checks         [reset to /export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3]
\* Creating Databin directories          [/export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/pkgapp_data]
\* Databin used/created is               [/export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/pkgapp_data/pkgapp-121008-01]
\* Creating temp area                    [/tmp/pkgapp.16081/]
\* Pid used, no corefile to check
\* Process binary                        [webservd]
\* Checking usage history                [not recently run]
\* webservd binary bit version           [32]
\* Checking path [-p] to binary name     [failed, path includes binary name]
\* Resetting path [-p] parameter         [/export/home/iws6.1sp10/bin/https/bin]
\* Checking path [-p] is a directory     [success]
\* Locating webservd                     [success]
\* Checking located webservd is 32 bit   [success]
\* Binary located                        [/export/home/iws6.1sp10/bin/https/bin/webservd]
\* Adding binary to pkgapp.pldd          [success]
\* Grabbing pldd                         [success]
\* Grabbing pstack                       [success]
\* Grabbing pmap                         [success]
\* Grabbing pcred                        [success]
\* Grabbing pflags                       [success]
\* Grabbing pargs                        ./pkgapp[103]: /usr/bin/pargs:  not found
./pkgapp[110]: /usr/bin/pargs:  not found
\* Not Including the core/gcore
\* Javatools [-j] not set                [skipped]
\* Grabbing /var/adm/messages            [success]
\* Grabbing uname -a                     [success]
\* Grabbing date/time                    [success]
\* Grabbing showrev -p                   [success]
\* Grabbing pkginfo -l                   [success]
\* Grabbing /etc/release                 [success]
\* Grabbing coreadm                      [success]
\* Grabbing ulimit                       [success]
\* Grabbing libs                         [success]
\* Making lib paths app/                 [success]
\* Making lib paths libs/                [success]
\* Linking libraries                     [success]
\* Libraries linked                      [101 ttl]
\* Using hostid for naming .tar.gz       [83450f78]
\* Writing file                          [pkgapp-83450f78-APPLE-081210-145852.tar.gz]
\* Done gathering files
\* Writing dbxrc & files     [success]
\* Writing manifest-081210-145852.log    [success]
\* Writing pkgapp-args-081210-145852     [success]
\* Creating final tarfile                [success]
\* Compressing tarfile                   [success]
\* End of runtime logging
\* Saving history info                   [/var/tmp/pkgapp-history/history-081210-145852.log]
\* Saving runtime log                    [/var/tmp/pkgapp-history/runtime-081210-145852.log]
\* Removing [-r] temp area/files         [left alone]
\* Operations Complete
Upload the following file(s) to your Cores Directory at Sun

1) File(s) located in directory /export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/pkgapp_data/pkgapp-121008-01

                [ pkgapp-83450f78-APPLE-081210-145852.tar.gz ]

                                Thank you.
                                Sun Software Technology Service Center (STSC)

1) You can check for updates to this script here:
        BigAdmin -
2) Release Notes and Guides located here:
        Docs -
3) GDD information located here:
        Docs -

4) Please send all Bugs and RFE's to the following address:
        Subject "pkgapp bug/rfe" -

5) Please send all other questions etc to:
        Subject "pkgapp feedback" -

Creating a tar file of the results...
a gcore.out.12919 70628K
a gcore_pldd.out 3K
a gcore_pmap.out 14K
a gcore_pstack.out 71K
a netstat.out.1 30K
a netstat.out.2 30K
a netstat.out.3 30K
a pfiles.out.1 4K
a pfiles.out.2 4K
a pfiles.out.3 4K
a pflags.out.1 8K
a pflags.out.2 8K
a pflags.out.3 8K
a pkgapp_data/ 0K
a pkgapp_data/pkgapp-121008-01/ 0K
a pkgapp_data/pkgapp-121008-01/pkgapp-83450f78-APPLE-081210-145852.tar.gz 14849K
a pldd.out.1 3K
a pldd.out.2 3K
a pldd.out.3 3K
a pmap.out.1 20K
a pmap.out.2 20K
a pmap.out.3 20K
a prstat.out.1 2K
a prstat.out.2 2K
a prstat.out.3 2K
a pstack.out.1 71K
a pstack.out.2 71K
a pstack.out.3 71K
a release.out 1K
a showrev.out 70K
a truss.out 37K
a ulimit 1K
a uname.out 1K

The tar file has been created....

Please send the file /export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/11111111-gdd-wshang-.tar.gz to
Sun Microsystems at
Choose cores as the destination.

Then, I checked my output dir,

 APPLE:/export/home0> ls -l /export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/
total 193098
-rw-rw-rw-   1 root     other    25833674 Dec 10 15:00 11111111-gdd-wshang-.tar.gz
-rw-rw-rw-   1 root     other    72322796 Dec 10 14:58 gcore.out.12919
-rw-rw-rw-   1 root     other       2684 Dec 10 14:58 gcore_pldd.out
-rw-rw-rw-   1 root     other      14161 Dec 10 14:58 gcore_pmap.out
-rw-rw-rw-   1 root     other      72266 Dec 10 14:58 gcore_pstack.out
-rw-rw-rw-   1 root     other      30577 Dec 10 14:57 netstat.out.1
-rw-rw-rw-   1 root     other      30577 Dec 10 14:58 netstat.out.2
-rw-rw-rw-   1 root     other      30421 Dec 10 14:58 netstat.out.3
-rw-rw-rw-   1 root     other       3572 Dec 10 14:57 pfiles.out.1
-rw-rw-rw-   1 root     other       3572 Dec 10 14:58 pfiles.out.2
-rw-rw-rw-   1 root     other       3572 Dec 10 14:58 pfiles.out.3
-rw-rw-rw-   1 root     other       8143 Dec 10 14:57 pflags.out.1
-rw-rw-rw-   1 root     other       8143 Dec 10 14:58 pflags.out.2
-rw-rw-rw-   1 root     other       8143 Dec 10 14:58 pflags.out.3
drwxrwxrwx   3 root     other        512 Dec 10 14:58 pkgapp_data
-rw-rw-rw-   1 root     other       2651 Dec 10 14:57 pldd.out.1
-rw-rw-rw-   1 root     other       2651 Dec 10 14:58 pldd.out.2
-rw-rw-rw-   1 root     other       2651 Dec 10 14:58 pldd.out.3
-rw-rw-rw-   1 root     other      19535 Dec 10 14:57 pmap.out.1
-rw-rw-rw-   1 root     other      19535 Dec 10 14:58 pmap.out.2
-rw-rw-rw-   1 root     other      19535 Dec 10 14:58 pmap.out.3
-rw-rw-rw-   1 root     other       1388 Dec 10 14:57 prstat.out.1
-rw-rw-rw-   1 root     other       1390 Dec 10 14:58 prstat.out.2
-rw-rw-rw-   1 root     other       1392 Dec 10 14:58 prstat.out.3
-rw-rw-rw-   1 root     other      72231 Dec 10 14:57 pstack.out.1
-rw-rw-rw-   1 root     other      72235 Dec 10 14:58 pstack.out.2
-rw-rw-rw-   1 root     other      72233 Dec 10 14:58 pstack.out.3
-rw-rw-rw-   1 root     other        184 Dec 10 15:00 release.out
-rw-rw-rw-   1 root     other      71631 Dec 10 15:00 showrev.out
-rw-rw-rw-   1 root     other      37856 Dec 10 14:58 truss.out
-rw-rw-rw-   1 root     other        206 Dec 10 15:00 ulimit
-rw-rw-rw-   1 root     other         66 Dec 10 15:00 uname.out

Then it is all done and just need to do this ,

"Please send the file /export/home/bits/iws/misc/SUN-GDD-webserver-wshang-solaris-v3/data/11111111-gdd-wshang-.tar.gz to
Sun Microsystems at
Choose cores as the destination.

NOTE : Make sure you rename the above xxx.tar.gz  so that your case SR# is part of the filename,

e.g. 71234567-customer_name-wshang-.tar.gz.

This will ensure the uploaded file will be moved to the right location for your case SR#.

Sun support engineers can then find the uploaded file easily.

Tuesday Dec 02, 2008

New wshang.ksh version 3.0 just released


We got a new troubleshooting tool,  wshang.ksh version 3.0 just released. FYI.

You can download it at

There are some user configurable options inside, e.g.

Some notes on usage:  The customer will have to edit the script to
change a few options as needed.  These options will be made interactive
in the next release of wshang.

# The following four settings can be changed as necessary:

# Change this value to set the duration in secs between data collection


# Set the following variable to y or n to allow a gcore to be taken


# Set the following variable to y or n to allow the pkgapp script to run.

# Assumes the pkgapp script is in the same location as wshang script.


# Set the following variable with your Case Number. If no case number
has been

# opened but wish to use wshang.ksh to trap an issue, you can use your

# Name (no spaces) or all 1's.  Once you get a Sun Case number, you can

# the resulting tar file before sending to Sun.


This will be a useful tool to collect troubleshooting data for web and/or proxy server. 

Tuesday Nov 25, 2008

Pkgapp 3.0 for Solaris Available now !

I just know this, so share here,

Pkgapp 3.0 for Solaris Available!

 This is a very useful tool to collect the data for troubleshooting, see more at web GDD , e.g.

Monday Jun 23, 2008

Data collection for Sun Java Web server issues ?

If you hit some issues in Sun Java Web server, then you will likely try to collect some data for more troubleshooting.

One url  that  shows the data collection :


1.5 Types of Web Server Debug Data

This section describes the various kinds of debug data that you need
to provide to the Sun Support Center. The procedure to obtain debug data based
on the kind of problem you are experiencing is described in-detail.

This section contains the following topics:


You can see different types of issues and their appropriate data collection.

The data will be helpful if you open a support case with Sun, or try to do any problem isolation by yourself. 

There are other GDD docs for other products.

Hope this can help you next time when you try to collect data for problem isolation and troubleshooting. 


to share tips and experience in Oracle products, e.g. SunONE / iPlanet Web server, Weblogic server, Web protocols, security and performance.


« June 2016