Monday May 23, 2016

Oracle Enterprise Manager 13c and Always-On Monitoring

 
   
  

One of the common concerns of the Administrators of an Oracle Enterprise Manager (EM) environment is that when you have to take EM down for planned maintenance, you are blind to the status of some of your company’s most critical assets. Well, as of EM 13c, Oracle now has a solution for you! Introducing Always-On Monitoring – or AOM for short. Always-On Monitoring provides the capability of monitoring all or a specific list of targets using the same EM agents already deployed to these targets during complete downtime of the EM environment. It is especially useful for monitoring critical targets during planned EM downtime such as patching and/or upgrades. AOM will synchronize with EM for data such as notification contacts and is able to take the alerts from the EM agents and send email notifications to the identified downtime contacts. AOM can be configured and left running all the time so that it is ready to “take over” the alert notification for critical targets once notifications have been enabled. This blog contains the first of a couple of posts on AOM starting with the setup and configuration of the tool.

The setup/configuration of the AOM tool consists of the following steps:

1.  AOM Installation - The AOM application is included with the EM 13c software distribution under the sysman/ems directory and is also available via the Self-Update function in EM. The installation of the application is as simple as unzipping the zip file in the location chosen for the AOM install. An AOM installation consists of a database to hold the AOM repository, and a server to run the AOM Instance. Chapter 12 of the Oracle Enterprise Manager Cloud Control Administrator’s Guide contains the details steps required for installing AOM. This server must have JAVA version 1.7 installed. At this time, the AOM instance/application must run on the EM OMS but the AOM repository database can be created on any Oracle database server.  With the EM 13.2 release, the AOM instance/application will be able to be installed on any host within the monitored environment.


2.  Configure AOM Communication to EM - The configuration is pretty simple and is all handled via the script called emsca found under the AOM installation home in the scripts directory. (Fun Fact: the name of this script and the command line utility all start with ems rather than aom due to a pre-release name change of the product). This script will handle all of the necessary steps to configure AOM including the creation of the user and schema for the AOM repository. Once this configuration is complete, make sure that EM has an Email Server configured. At this time, AOM can only send email messages for the alerts and it uses the email server configuration that it gets from EM during synchronization. See "Configuring Email Servers in Enterprise Manager" for more details on how to setup Email Servers in EM.


3.  Configuration of downtime contacts - The downtime contacts are those that will receive the email notifications in the event of an alert on the identified targets. Setting up the downtime contacts is done in your EM environment and can be done in a couple of different ways and is documented in the section “Configuring Downtime Contacts in Enterprise Manager”.

4.  Synchronizing AOM and EM - This synchronization copies over the notification configuration and downtime contacts for the EM targets over to the AOM application. This should be done before starting AOM for the first time. It is a simple command using the AOM command line utility called emsctl. Here is a sample of the command:

% $AOM_HOME/scripts/emsctl sync

5.  Note that AOM will run a synchronization job every 24 hours by default although this is configurable. It is a good practice to run a manual sync after any changes to downtime contacts.

6.  Configure EM for AOM - The final step in the configuration process is to tell EM to include the upload URL for the AOM service. Once this is done, EM will send this URL to all existing agents and any future agents. This makes this a one-time step that will then be handled for you as your number of agents change in the future. To get the AOM upload URL, issue the command below (the upload URL is in bold below):

% $AOM_HOME/scripts/emsctl status

Oracle Enterprise Manager Cloud Control 13c Release 1

Copyright (c) 1996, 2015 Oracle Corporation. All rights reserved.

------------------------------------------------------------------

Always-On Monitoring Version : 13.1.0.0.0

Always-On Monitoring Home : /u01/app/oracle/aom

Started At : January 13, 2016 4:11:01 PM PST

Last Repository Sync : February 2, 2016 1:41:17 PM PST

Upload URL : https://aomserver.domain:8081/upload

Always-On Monitoring Process ID : 15399

Always-On Monitoring Repository : (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=aomRepo.domain)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=aom)))

Enterprise Manager Repository : (DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=emServer.domain)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=emdb)))

Notifications Enabled : false

Total Downtime Contacts Configured : 29

Then issue the following emctl command to set the property in EM (note this is emctl and not emsctl) – substituting the information in bold below with the correct upoad url and sysman password for your environment:

% emctl set property -name "oracle.sysman.core.events.ems.emsURL" -value " https://aomserver.domain:8081/upload " -sysman_pwd sysman

7.  Starting AOM and Enabling notifications - Starting AOM is done via the emsctl command and specifying the start option:

% $AOM_HOME/scripts/emsctl start

Even though the setup of AOM is complete and it has been started, you will not receive any alerts when EM is down unless you enable notifications. As you probably guessed by now, this is done via the emsctl command:

% $AOM_HOME/scripts/emsctl enable_notification

A sync with EM is done each time you enable_notification unless you use the option –nosync. To disable the notification, use the same sort of command with the disable_notification option:

% $AOM_HOME/scripts/emsctl disable_notification

It is a good practice to leave AOM started but make sure you disable the notifications. Only enable them just before you shutdown EM for planned maintenance. REMEMBER that you will get more notification than you probably want if you leave this enabled all of the time because AOM has no knowledge of the notifications that EM is already doing.

To summarize, Always-On Monitoring offers a solution to the long-time problem of who monitors our critical targets when EM is down. The next blog will provide the details on how to configure AOM for High Availability and Disaster Recovery.

Friday Apr 22, 2016

April 2016 EM Recommended Patch List

 
   
  

As part of the recent April 2016 PSU Patch release for Enterprise Manager (EM), a list of recommended patches has been compiled for EM 12c.  This list contains recommended patches for the EM infrastructure including the OMS, WLS, Plugins, Repository and Agent. These patches are also included in the Exadata Quarterly Full Stack Download patch.

For more details and to review these EM recommended patches for Exadata monitoring, refer to My Oracle Support Note titled Patch Requirements for Setting up Monitoring and Administration for Exadata [1323298.1].   This note was to provide a list of the recommended patches for monitoring an Exadata Machine but can be applicable for any EM environment.  Just keep in mind that your EM environment may require other agent plugin bundle patches depending on what target types are managed in your EM environment.

Also, the information in the note contains not only a recommended list of patches for both EM versions 12.1.0.5 and 12.1.0.4 but also provides a link to a My Oracle Support Note that provides the steps for applying these patches. This note is titled Applying Enterprise Manager 12c Recommended Patches [1664074.1].  The patch apply process in this note provides maximum availability for the EM environment by structuring the patch apply steps so that the primary OMS server is up as long as possible.

Friday Jan 29, 2016

January 2016 EM Recommended Patch List

 
   
  

As part of the recent January 2016 PSU Patch release for Enterprise Manager (EM), a list of recommended patches has been compiled for EM 12c.  This list contains recommended patches for the EM infrastructure including the OMS, WLS, Plugins, Repository and Agent. These patches are also included in the Exadata Quarterly Full Stack Download patch.

For more details and to review these EM recommended patches for Exadata monitoring, refer to My Oracle Support Note titled Patch Requirements for Setting up Monitoring and Administration for Exadata [1323298.1].   This note was to provide a list of the recommended patches for monitoring an Exadata Machine but can be applicable for any EM environment.  Just keep in mind that your EM environment may require other agent plugin bundle patches depending on what target types are managed in your EM environment.

Also, the information in the note contains not only a recommended list of patches for both EM versions 12.1.0.5 and 12.1.0.4 but also provides a link to a My Oracle Support Note that provides the steps for applying these patches. This note is titled Applying Enterprise Manager 12c Recommended Patches [1664074.1].  The patch apply process in this note provides maximum availability for the EM environment by structuring the patch apply steps so that the primary OMS server is up as long as possible.

Thursday Nov 19, 2015

October 2015 EM Recommended Patch List

 
   
  

As part of the recent October 2015 PSU Patch release for Enterprise Manager, a list of recommended patches has been compiled for EM 12c.  This list contains recommended patches for the EM infrastructure including the OMS, WLS, Plugins, Repository and Agent. 

For more details and to review these EM recommended patches, refer to My Oracle Support Note titled Patch Requirements for Setting up Monitoring and Administration for Exadata [1323298.1]

Also, the information in the note contains not only a recommended list of patches for both EM versions 12.1.0.5 and 12.1.0.4 but also provides a link to a My Oracle Support Note that provides the steps for applying these patches. This note is titled Applying Enterprise Manager 12c Recommended Patches [1664074.1].  The patch apply process in this note provides maximum availability for the EM environment by structuring the patch apply steps so that the primary OMS server is up as long as possible.

Monday Jul 27, 2015

July 2015 EM Recommended Patch List

 
   
  

As part of the recent July 2015 PSU Patch release for Enterprise Manager, a list of recommended patches has been compiled for EM 12c to be included in the Exadata Quarterly Full Stack Download patch.  This list contains recommended patches for the EM infrastructure including the OMS, WLS, Plugins, Repository and Agent. 

For more details and to review these EM recommended patches for Exadata monitoring, refer to My Oracle Support Note titled Patch Requirements for Setting up Monitoring and Administration for Exadata [1323298.1]

Also, the information in the note contains not only a recommended list of patches for both EM versions 12.1.0.4 and 12.1.0.3 but also provides a link to a My Oracle Support Note that provides the steps for applying these patches. This note is titled Applying Enterprise Manager 12c Recommended Patches [1664074.1].  The patch apply process in this note provides maximum availability for the EM environment by structuring the patch apply steps so that the primary OMS server is up as long as possible.

Wednesday Apr 22, 2015

April 2015 EM Recommended Patch List

 
   
  

As part of the recent April 2015 PSU Patch release for Enterprise Manager, a list of recommended patches has been compiled for EM 12c to be included in the Exadata Quarterly Full Stack Download patch.  This list contains recommended patches for the EM infrastructure including the OMS, WLS, Plugins, Repository and Agent. 

For more details and to review these EM recommended patches for Exadata monitoring, refer to My Oracle Support Note titled Patch Requirements for Setting up Monitoring and Administration for Exadata [1323298.1]

Also, the information in the note contains not only a recommended list of patches for both EM versions 12.1.0.4 and 12.1.0.3 but also provides a link to a My Oracle Support Note that provides the steps for applying these patches. This note is titled Applying Enterprise Manager 12c Recommended Patches [1664074.1].  The patch apply process in this note provides maximum availability for the EM environment by structuring the patch apply steps so that the primary OMS server is up as long as possible.

Thursday Feb 27, 2014

Fast Recovery Area for Archive Destination


If you are using Fast Recovery Area (FRA) for the archive destination and the destination is set to USE_DB_RECOVERY_FILE_DEST, you may notice that the Archive Area % Used metric does not trigger anymore. Instead you will see the Recovery Area % Used metric trigger when it hits a Warning threshold of 85% full, and Critical of 97% full. Unfortunately, this metric is controlled by the database and the thresholds cannot be modified (see MOS Note 428473.1 for more information). Thresholds of 85/97 are not sufficient for some of the larger, busier databases. This may not give you enough time to kickoff a backup and clear enough logs before the archiver hangs. If you need different thresholds, you can easily accomplish this by creating a Metric Extension (ME) and setting thresholds to your desired values.  This blog will walk through an example of creating an ME to monitor archive on FRA destinations, for more information on ME's and how they can be used, refer to the Oracle Enterprise Manager Cloud Control Administrator's Guide

[Read More]

Monday Feb 24, 2014

Monitoring Archive Area % Used on Cluster Databases


One of the most critical events to monitor on an Oracle Database is your archive area. If the archive area fills up, your database will halt until it can continue to archive the redo logs. If your archive destination is set to a file system, then the Archive Area % Used metric is often the best way to go. This metric allows you to monitor a particular file system for the percentage space that has been used. However, there are a couple of things to be aware of for this critical metric.

Cluster Database vs. Database Instance

You will notice in EM 12c, the Archive Area metric exists on both the Cluster Database and the Database Instance targets. The majority of Cluster Databases (RAC) are built against database best practices which indicate that the Archive destination should be shared read/write between all instances. The purpose for this is that in case of recovery, any instance can perform the recovery and has all necessary archive logs to do so. Monitoring this destination for a Cluster Database at the instance level caused duplicate alerts and notifications, as both instances would hit the Warning/Critical threshold for Archive Area % Used within minutes of each other. To eliminate duplicate notifications, the Archive Area % Used metric for Cluster Databases was introduced. This allows the archive destination to be monitored at a database level, much like tablespaces are monitored in a RAC database.

In the Database Instance (RAC Instance) target, you will notice the Archive Area % Used metric collection schedule is set to Disabled.

If you have a RAC database and you do not share archive destinations between instances, you will want to Disable the Cluster Database metric, and enable the Database Instance metric to ensure that each destination is monitored individually.


Friday Jul 26, 2013

Operational Considerations and Troubleshooting Oracle Enterprise Manager 12c

Oracle Enterprise Manager (EM) 12c has become a valuable component in monitoring and administrating an enterprise environment. The more critical the application, servers and services that are monitored and maintained via EM, the more critical the EM environment becomes. Therefore, EM must be as available as the most critical target it manages.


There are many areas that need to be discussed when talking about managing Enterprise Manager in a data center. Some of these are as follows:

• Recommendations for staffing roles and responsibilities for EM administration

• Understanding the components that make up an EM environment

• Backing up and monitoring EM itself

• Maintaining a healthy EM system

• Patching the EM components

• Troubleshooting and diagnosing guidelines

The Operational Considerations and Troubleshooting Oracle Enterprise Manager 12c whitepaper available on the  Enterprise Manager Maximum Availability Architecture (MAA) site will help define administrator requirements and responsibilities.  It provides guidance in setting up the proper monitoring and maintenance activities to keep Oracle Enterprise Manager 12c healthy and to ensure that EM stays highly available.

About

bocadmin_ww

Search

Archives
« June 2016
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
  
       
Today