By user554629 on May 24, 2012
Getting started with Solaris technical support
Browser Alert: Use Firefox, Chrome or Safari to download files in links below (R-Click, Save-As ). Internet Explorer does not work.
The following scripts are located in obisolaris.tar.gz (2013-07-12)
- obimon (0.1) - monitor memory and connection growth.
- pcore.sh (0.1) - miniture version of pstack.sh for Solaris, until we work out access to dbx. Note 1:
- packcore.sh (0.1) - bundle core file and libraries for Oracle Tech Support
- pstack.sh (0.4) - uses dbx to obtain call stacks from a core file.
- unpackDbx.sh.sh (0.2) - extracts dbx from Studio 12.3 download. See instructions below.
Note 1: pcore.sh uses Solaris commands to capture information from a core file.
- pargs captures the environnment variables
- pldd captures libraries in use
- pstack captures pstack for all threads
So what's wrong with pcore.sh output? It's the Solaris pstack command that limits our understanding:
- Doesn't identify the crashing thread
- Doesn't display the registers at the time of the crash
- Doesn't display disassembly of instructions to understand the nature of the crash
- Doesn't demangle C++ object names
Note 2: Submit the pcore output.
If we need more detailed information, we'll request packcore.sh output so we can use dbx at Oracle.
Obtaining Solaris dbx
- Accept the Studio OTN Developer Agreement
- Download the tar file appropriate to your environment:
SPARC (488MB) or x86 (390MB)
Solaris 10 or Solaris 11
Linux x86 (368MB)
- Package Installation requires registration, root install, and a Support Offering.
Tarfile Installation contains these notes:
"This option extracts the installed version of Oracle Solaris Studio software from
a tarfile (no packages). Elevated OS permissions (e.g. root) are not needed."
"Oracle support is not available for this configuration nor can the software be updated with patches."
- Do not unpack the Studio bz2 file. Use unpackDbx.sh
It is 1.6GB unpacked. This script will extract the dbx tools, which are just 22MB
$ time unpackDbx.sh ran less than 5 minutes on my machine ( real 4m43.050s ).
... tar output ...
Add studio12.3/bin to PATH. Add studio12.3/man to MANPATH
Studio commands available:
c++filt dbx dem dwarfdump getmsg version whatdir
unpackDbx.sh script uses md5 to check the download file (SolarisStudio12.3-solaris-sparc-bin.tar.bz2)
With dbx installed, we can obtain much more informative pstack output.
Make sure that pstack.sh and dbx are available by setting $PATH in your .profile at login.
Download pstack.sh, and place it in /opt/studio12.3/bin directory, so it is available from $PATH.
If $obiCollect variable is set to a common directory, pstack output is directed there. Place the scripts from obisolaris.tar.gz into $obiCollect/bin.
$ obiCollect=/net/host-name/obiCollect ; export obiCollect
$ mkdir $obiCollect/bin
$ pstack.sh core_name
$ ls $obiCollect
Core file names
Solaris coreadm is used to change the characteristics of core files from process crashes.
The default name "core" creates a problem when communicating with Tech Support about specific crashes, because we need to relate a specific crash with the obiee server logs to analyze the cause of the server crash. As a starting point, this root command will provide unique core file names, and create a record of the crashes in Solaris log.
coreadm -i core.%f.%p -e log -e global
For more details: man coreadm
It's possible to direct all core files to a specific directory using the -g option. (e.g., -g /var/core/core.%f.%p )
For rapid restart of obiee processes, the directory should be on a fast, local filesystem, and the obiCollect directory should be on a network shared directory available to all nodes in the obiee cluster.
Test drive: coreadm and pstack.sh
Studio 12.3 downloaded and processed with unpackDbx.sh into /opt
Assume bash shell below. pstack.sh in /opt/studio12.3/bin
root: coreadm -i core.%f.%p -e log -e global
sleep 300 &
kill -6 1908
+ Abort (core dumped) sleep 300
dmesg | grep sleep
May 29 15:37:53 ... core_log: sleep no global core file pattern exists
ls -l core.sleep* | cut -c 32-80
1994184 May 29 07:10 core.sleep.29780
2730 May 29 07:10 core.sleep.29780.pstack
2012-05-30: modifications to unpackDbx.sh and pstack.sh
2013-02-11: scripts repackaged into obisolaris.tar.gz
Setup instructions have changed, consolidating the scripts to $obiCollect/bin