Tips | Tuesday, February 21, 2012

Oracle Enterprise Manager Ops Center Doctor Utility - The Administrator's Companion

Oracle Enterprise Manager Ops Center
comes with a very helpful little utility called Ops Center Doctor, or
OCDoctor for short. This utility assists in various stages of
the Ops Center deployment and can be a real life saver. It is updated
on a regular basis with additional knowledge (similar to an antivirus
subscription) to help you identify and resolve known issues or
suggest ways to improve performance. Keep reading to learn how to
leverage it in your environment!




How do I get the latest OCDoctor?

There are 2 ways to get the OCDoctor.
If you did not install Ops Center just yet, you can get the latest
OCDoctor from the following link:

https://updates.oracle.com/OCDoctor/OCDoctor-latest.zip

If you have Ops Center installed,
whether it is the Enterprise Controller, Proxy Controller or Agent
Controller, the OCDoctor is already installed in the following
folder:

#
cd /var/opt/sun/xvm/OCDoctor/

In this folder, run the following:

#
./OCDoctor.sh --update

This will automatically connect to
Oracle and check for a new release. If there is one – it will
automatically download it and update itself. If you need to specify a
proxy, don’t forget to set it beforehand (i.e., export
https_proxy=<proxy_url>:<proxy_port>) .

How can the OCDoctor help me?

The Doctor can help you in the
following areas:

  • Check for prerequisites of the

    Enterprise, Proxy and Agent Controllers

  • Verify connectivity to Oracle

    servers

  • Self Diagnosis/Troubleshoot –

    This is the most popular feature of the Doctor. Search for known

    issues with your current installation. This can save you time and

    resolve issues immediately!

  • Tune and optimize performance

  • Collect logs that can be sent to

    Oracle Support


Checking Prerequisites

As a general rule of thumb, before
using the Doctor, always check for updates first:

#
./OCDoctor.sh --update


Now, if you are about to start a fresh
install of Ops Center, you can check for the Enterprise Controller
requirements. Here’s an example:

#
./OCDoctor.sh --ec-prereq

Ops
Center Doctor version 3.07 (Jan 17 2012 [Build 466]) (Linux)

OK:
Size of Main Memory: 8064 MB

OK:
Size of swap space: 8004 MB

OK:
Found enough disk space on /var/opt/sun/xvm (130G)

OK:
Oracle/RedHat Linux 5.5

OK:
XEN kernel is not loaded

OK:
python-2.4.3 is installed.

OK:
expect-5.43.0 is installed.

OK:
perl-DBD-Pg is installed.

OK:
xinetd is installed.

OK:
tftp-server is installed.

OK:
dhcp is installed.

OK:
gettext is installed.

OK:
perl-XML-Parser is installed.

OK:
ncompress is installed.

OK:
libxml2 is installed.

OK:
gcc is installed.

OK:
httpd is installed.

OK:
wget is installed.

OK:
make is installed.

OK:
bc is installed.

OK:
libxml2 is installed.

OK:
pam is installed.

OK:
root umask is 022

OK:
/etc/profile has the umask of 022

OK:
/etc/profile.d has the umask of 022

OK:
/etc/bashrc has the umask of 022

OK:
/etc/default/init has the umask of 022

OK:
All ports are available (not in use)

OK:
All SSH requirements are met

OK:
SELinux doesn't show up in /etc/selinux/config

OK:
SELinux is not enable based on /usr/sbin/selinuxenabled

OK:
glibc-devel 32 bit is installed

OK:
xvm UID/GID 60 is not in use

OK:
Directory /guests does not exist.

OK:
Locales are OK ("en_US.UTF-8")

OK:
RPM sun-uce-agent not installed

OK:
RPM sun-uce-server not installed

OK:
RPM sun-uce-engine not installed

OK:
Number of configured interfaces is not exceeding maximum.


Other available prerequisites options:

#
./OCDoctor.sh --proxy-prereq

Use this on a machine before installing
a remote Proxy. You do not need to run this if your Proxy will be
installed on the same machine as the Enterprise Controller. The
Enterprise Controller requirements will also include the proxy tests.


#
./OCDoctor.sh --agent-prereq

Use this before installing an agent, or
if an agent installation failed. An agent is deployed when you choose
to “Manage” an operating system through the user interface. An
agent can also be deployed manually.


Verifying Connectivity

Unless you are in a dark environment,
you will likely want to use Ops Center to download patches and
updates automatically for you. Before installing, you can use the
Doctor to verify that your system can reach all the proper Oracle
servers. To do so, run the following:

#
./OCDoctor.sh --check-connectivity


You will then be asked to provide your
My Oracle Support (MOS) username and password. The Doctor will then
try to download 2 different links. This will verify 2 things for you:

  1. That you can reach different

    required servers

  2. That your MOS account is entitled

    to use Ops Center downloads

Here’s an example:

#
./OCDoctor.sh --check-connectivity

Ops
Center Doctor version 3.07 (Jan 17 2012 [Build 466]), OC Version
11.1.0.1571 (SunOS)

========================
Checking Network Connectivity ==============================

Using
Ops Center Proxy server:www-proxy.us.oracle.com Port:80 (you may
overwrite it by setting https_proxy)

Please
enter the My Oracle Support (MOS) Username: eran.steiner@oracle.com

Password:

Test
1/2: Downloading Knowledge Channels file from updates.oracle.com

===========================================================================

OK:
Knowledge download was successful!

Test
2/2: Downloading patch 108437-06 from updates.oracle.com

===========================================================================

OK:
Patch download was successful!

Self Diagnosis / Troubleshoot

The OCDoctor’s most popular feature
is its ability to search your installation for known issues. As
mentioned earlier, the OCDoctor is constantly being updated with new
issues and suggestions gathered from Oracle Support and Field
Engineers.

Having a problem? Try running the
Doctor before calling support, it may be able to provide you with a
hint of what’s wrong or perhaps even offer an automated fix.

To diagnose your installation, run the
following command on the Enterprise Controller, Proxy Controller or
Agent. The Doctor will automatically detect what is installed and run
the relevant tests for this specific machine.

To diagnose your machine, use the
“--troubleshoot” flag. Here’s an example:

#
./OCDoctor.sh --troubleshoot

Ops
Center Doctor version 3.07 (Jan 17 2012 [Build 466]), OC Version
11.1.0.1571 (SunOS)

[Read
only mode] [17-Feb-2012 11:15AM PST]

==================
Checking Enterprise Controller... =========================

OK:
Total number of agents: 2

OK:
Number of agents with inventories: 2

OK:
Enterprise Controller requirements are met

OK:
5240 URN is already in the DB

OK:
auth.cgi returns XML

OK:
No broken indexes in the Management DB

OK:
No broken indexes in the Report DB

OK:
Apache logs are smaller than 2 GB

OK:
User scncon is not locked

OK:
scncon Password is configured properly

OK:
UCE server is running

OK:
n1gc folder has the right permissions

OK:
satadm timeouts were increased

OK:
All agent RPMs are installed properly

OK:
All Enterprise Controller RPMs are installed properly

OK:
Enterprise Controller status is online

OK:
the version is the latest one (11.1.0.1571)

OK:
scndb user exists

OK:
scndb user does not have any messages printed when logged in

OK:
Logs folder has the right permissions

OK:
Found the server .uce.rc

OK:
Server .uce.rc has the correct file permissions

OK:
Server .uce.rc has the correct ownership

OK:
.uce.rc has ddbp entry

OK:
Connectivity to the KB public servers works properly

OK:
No lockfile from a previous update found.

OK:
Grouplock file doesn't exist

OK:
Locales are OK ("en_US.UTF-8")

OK:
found /usr/sbin/rotatelogs

OK:
Basic statistic was uploaded to Oracle (touch
/var/tmp/no-oracle-stats to skip)

OK:
system-auth does not require pam_shells.so

OK:
No update_tmp folder found

OK:
No credential issues found

==================
Checking LOCAL Proxy Controller... ===================

OK:
Enterprise Controller requirements are met

OK:
No cacao SMF issue known on Linux ECs/PCs

OK:
ipmiflash points to a binary

OK:
No hidden servers due to proxy communication

OK:
Proxy status is online

OK:
Found a proxy connection listed in 'sc-console list-connections' -
the local proxy is properly configured

You
may run this on remote proxies as well

OK:
no ssh 'Read timed out' messages found

OK:
Default route is set properly

========================
Checking agent... ==============================

OK:
Agent requirements are met

OK:
All agent RPMs are installed properly

OK:
Memory usage of agent is fine.

OK:
agent version is the latest one (11.1.0-1571)

OK:
no core file found in seekers folder

OK:
Agent services are running

OK:
stclient command works properly

OK:
LD_LIBRARY_PATH is not set

OK:
.uce.rc exists

OK:
.uce.rc symbolic link exists

Here are a few lines from a machine
that the Doctor found an issue with:

=================
Checking LOCAL Proxy Controller... ===================

OK:
Enterprise Controller requirements are met

OK:
Verified a unique Cacao SMF instance.

ERROR:
'ipmiflash' config points to an incorrect path (CR 6920989)

To
fix, edit /opt/sun/n1gc/os/platform.properties and change:

platformproperties.ipmiflashpath:
/opt/sun/n1gc/pkgs/opt/ipmitool/bin/ipmiflash

to

platformproperties.ipmiflashpath:
/opt/ipmitool/bin/ipmiflash

Then
restart the Proxy services:

#
/opt/SUNWxvmoc/bin/proxyadm stop -w -v

#
/opt/SUNWxvmoc/bin/proxyadm start -w -v


or
use '# ./OCDoctor.sh --troubleshoot --fix' to automatically fix this


To
fix this issue automatically you can simply run the following
command:

#
./OCDoctor.sh --troubleshoot --fix

TIP:
Running “--troubleshoot” will also verify that the prerequisites
are met, so there’s no need to run both!


Tune your installation for best performance

Over time you may have added so many
machines that the performance may seem like it’s suffering a bit.
To check for possible tunings and performance tips, try running the
Doctor with the following flag:


#
./OCDoctor.sh --tuning

Ops
Center Doctor version 3.07 (Jan 17 2012 [Build 466]), OC Version
11.1.0.1571 (SunOS)

[Read
only mode] [17-Feb-2012 01:27PM PST]

=====================
Checking for possible tuning... ========================

------------------
Tuning script 1 / 3 ------------------
(tuning/11/clear_old_notifications.sh)

OK:
Old Generated notifications: 14 (lower or equal to 1500)

------------------
Tuning script 2 / 3 ------------------
(tuning/11/hd_persistent_mbean.sh)

OK:
Index is already assigned to hd_persistent_mbean table

------------------
Tuning script 3 / 3 ------------------ (tuning/11/scn_task_logs.sh)

OK:
Index is already assigned to scn_task_logs


Note that if you find a suggestion
here, many times you can use the “--fix” flag to ask the Doctor
to automatically perform the tuning for you:


#
./OCDoctor.sh --tuning --fix


Collecting Logs

OCDoctor can be used to collect all the
relevant logs and information from the box. Typically, if you open a
service request with Oracle Support, they will ask you for those
logs.


The OCDoctor will detect what
components are installed on the machine and based on that collect all
the relevant information including OS files, parameters and Ops
Center log and config files.


To collect the logs, simply run:


#
./OCDoctor.sh --collectlogs

Ops
Center Doctor version 3.07 (Jan 17 2012 [Build 466]), OC Version
11.1.0.1571 (SunOS)

2012-02-20-10-01
INFO: Logging to /var/tmp/GDD-OC.sh.log

2012-02-20-10-01
INFO: User: uid=0(root) gid=0(root)

…..

2012-02-20-10-03
INFO: Collecting OCDoctor troubleshoot output...

2012-02-20-10-04
INFO: Compressing the output directory
/var/tmp/11111111-gdd-oc-x4470-sca11-c-2012-02-20-10-01 ...

2012-02-20-10-04
INFO: ... done

2012-02-20-10-04
INFO: The following file needs to be sent to Oracle Support

2012-02-20-10-04
INFO: case number: 11111111, file size: 7.5M

2012-02-20-10-04
INFO: Output file:
/var/tmp/11111111-gdd-oc-x4470-sca11-c-2012-02-20-10-01.tar.gz

All you have to do next is attach the
above tar.gz file (in /var/tmp) to the service request.

Recording of the Live Webcast

https://oracleconferencing.webex.com/oracleconferencing/ldr.php?AT=pb&SP=MC&rID=66908567&rKey=1bdeee561c8c92c7

Join the discussion

Comments ( 2 )
  • dsc Thursday, February 23, 2012

    When will Solaris 11 support be released? If I try to run the OCDoctor.sh with --ec-prereq I get the following message. The --troubleshoot will run, but some of the output looks like it may be related to Solaris 10.

    ERROR: Solaris 11 is not supported


  • Anand Akela Friday, February 24, 2012

    Solaris 11 is not supported in the Ops Center 11g release. It will be supported with an upcoming Ops Center release. Please check with your sales rep if you need more information now !


Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha
 

Visit the Oracle Blog

 

Contact Us

Oracle

Integrated Cloud Applications & Platform Services