Friday Oct 24, 2014

Get Compliant with Oracle Java VM Database PSU OCT 2014 using EM12c

Check for compliance and automate patching of Oracle Database fleet using EM12c

Oracle along with its regular Quarterly Database PSU/SPU/CPU update this October 2014 released Oracle JAVA VM PSU patch, the patch is recommended to be applied to all databases in your fleet (esp. the ones that uses JAVA).  (For more information, support Note- 1929745.1 explains it in detail).
The mandate primarily is to apply the patches against the databases that use JAVAVM option. Ideally, you would need to apply it against all databases, so in case a new database is created in the ORACLE_HOME it is covered.

Oracle Enterprise Manager 12c provides support features of Compliance Management and automated Patching of databases. Using both these features you could identify the databases in your fleet that needs the patch and automate applying them.  

To get started, download the starter kit here. It contains utilities to jump start on your compliance, the kit contains a readily importable Compliance Standard and a step-by-step guide.

[Read More]

Wednesday Jul 30, 2014

Reducing Downtime While Patching Multi-OMS Environments

Enterprise Manager 12.1.0.4 has now been released for a few weeks, as well as the 12.1.0.4 OMS Bundle patches (also known as System patches). If you plan to apply these bundle patches to your 12.1.0.4 OMS, and you are concerned about the downtime, then, you can reduce the downtime by referring to this whitepaper that contains patching instructions to reduce downtime. 


This whitepaper covers various Enterprise Manager High Availability (EM HA)  usecases (level 1, 2, 3, 4), and contains instructions on how to reduce downtime while applying patches to each of these usecases. It also clearly defines the steps that require downtime and those that do not.

If you have a multi-OMS setup, you can also refer to this whitepaper which covers script creation using the opatchauto command, which automates the substeps and further reduces downtime.During our internal testing of this whitepaper on an EM HA setup, we have noticed a significant reduction in downtime. 

If your customer plans to do an Enterprise Manager Upgrade to 12.1.0.4, then as a post upgrade recommendation, they should patch their OMS with the latest bundle patches by following the instructions outlined in this whitepaper.

White paper on OTN:
http://www.oracle.com/technetwork/oem/install-upgrade/reducedowntime-patchmultioms-2251257.pdf

MOS note for the latest Bundle Patches:
Enterprise Manager 12.1.0.4.0 (PS3) Master Bundle Patch List (Doc ID 1900943.1)

Stay Connected:

Twitter |  Face book |  You Tube |  Linked in |  Google+ Newsletter


Thursday Jul 17, 2014

Patching 101 - The User Friendly Guide to Understanding EM Patches

There was a conversation on twitter last week about available patches for Enterprise Manager (EM) 12.1.0.4, and it got a little deeper than 140 characters will allow.  I've written this blog to give a quick Patching 101 on the types of EM patches you might see and the details around how they can be applied.

OMS Patches

The core Enterprise Manager system is typically patched with the quarterly PSU patches (released Jan, Apr, July, Oct) or a one-off when directed by support for a critical issue.  PSU patches will be cumulative, so you need not apply each of them, just apply the latest.  The OMSes must be shutdown during patching, however some patches are being released with rolling patch instructions for multi-OMS systems.  These patches must be applied at the host level, and cannot be automated via EM.   ALWAYS read the readme, yes every time.  The patching steps can change from patch to patch so it's critical to read the readme. OPatch or OPatchauto will be used to apply these patches.  Did I mention to read the readme for every patch?  It's also important to note that there may be additional steps when patching in a multi-OMS or standby environment, so read the output of OPatchauto carefully.

Always download the latest OPatch release for the appropriate version.  If you read the readme, you already know this!   Download patch 6880880 for 11.1 (the OPatch version used by EM) and unzip into the $ORACLE_HOME.  Most errors in patching are related to not updating OPatch. 

For more information on PSU Patches and patching EM:
Oracle Enterprise Manager Cloud Control Administrators Guide - Chapter 16 Patching Oracle Management Server and the Repository
EM 12c Cloud Control: List of Available Patch Set Updates PSU (Doc ID 1605609.1)
How to Determine the List of Patch Set Update(PSU) Applied to the Enterprise Manager OMS and Agent Oracle Homes? (Doc ID 1358092.1)

Each plug-in has binaries that will require patches as well.  Same downtime requirements apply for plug-in patches as the quarterly PSUs.  Starting in 12.1.0.3, the plug-in patches are being released as a monthly bundle.  This means that if you have 6 plug-ins, you may have 6 OMS side patches to apply - 1 for each plug-in.  Bundles are not always released for every plug-in every month.  They are cumulative, so pick the latest.

Starting with 12.1.0.4, the individual OMS-side plug-in bundles are being grouped into a System  Patch each month. So for example, in June 2014 the System patch includes MOS, Cloud, DB, FA, FMW, SMF, and Siebel plug-ins.  Non-required patches will be skipped.

For more information on the EM Patch Bundles and Patching EM:
Enterprise Manager 12.1.0.4.0 (PS3) Master Bundle Patch List (Doc ID 1900943.1)
Enterprise Manager 12.1.0.3 Bundle Patch Master Note (Doc ID 1572022.1)

Agent Patches

Agent patches are applied to each agent.  They can be applied via EM using the MOS patch plans, which makes it a lot easier when you have 100s or 1000s of Agents to patch!  The Patch Plans will start a blackout, validate prerequisites, check for conflicts, and update OPatch for you.  If you don't use the Patch Plan you can patch manually with OPatch, don't forget to read the readme!  The Agent must be shutdown during the patch application.  There are 4 main types of Agent patches you will see:

  • Core Agent - Starting with 12.1.0.3.0 the core Agent will have monthly patch bundles .  These are also cumulative, so my recommendation is to apply the latest one.  
  • Agent-side Discovery Plug-in - This is the lightweight piece of the plug-in used for target discovery.  Discovery plug-in patches are cumulative with other discovery plug-in patches for that component. 
  • Agent-side Monitoring Plug-in - This is the more detailed monitoring side of the plug-ins for the required components.  Monitoring plug-in patches are cumulative with other monitoring plug-in patches for that component.   So if there's a Discovery and Monitoring patch available for the DB Plug-in, you need to apply both of them.  
  • JDBC patches for the Agent will be JDBC version 11.1.0.7.0.  These patches do get applied to the Agent, and can be applied via the Patch Plans.  

You can apply the latest Agent bundle, JDBC patch and the plug-in bundles in one patch plan.   If there's a conflict, you'll be notified.   If the Agents you've selected don't have specified plug-ins, you'll also receive notice during the analyze step.  As of now, for my 12.1.0.4 agents, I would apply the 12.1.0.4.1 patch (18873338) and the two available plug-in agent patches DB monitoring (19002534) and FMW monitoring (18953219) and the latest JDBC patches (18502187,18721761) all in one patch plan.

I discovered a new feature in 12.1.0.4 while testing this.  Normally you had to have Normal Oracle Home preferred credentials set for all Agent targets to patch, or select Override and specify the Normal Oracle Home credentials.   In 12.1.0.4, the Agent uses it's internal credentials to Patch itself, so setting preferred credentials or specifying at run-time is not required.  The user patching would require the Manage Target Patch and Patch Plan privileges.  

For more details on Agent patching:
Oracle Enterprise Manager Cloud Control Administrators Guide - Chapter 17 Patching Enterprise Manager Agents 
Simplified Agent and Plug-in Patching

Infrastructure

The OMS and Agent are the key components, and my main focus here.  However it's important to keep the infrastructure stack up to date as well.  This includes the Oracle Fusion Middleware and Oracle Database that are used for EM.   The recommendation is to follow the best practices for each of these components, and regularly update with the PSU patches available.   The following reference notes will help in identifying the current PSU patches.   The WebLogic Server version used by EM 12c is 10.3.6. 

Oracle Recommended Patches -- Oracle Database (MOS 756671.1)
Master Note on WebLogic Server Patch Set Updates (PSUs) (MOS 1470197.1)

Summary

Hopefully this will help you understand the various types of components involved with keeping EM up to date.   Obviously, you may not want to patch each month and maybe not every quarter, but the patches are available to keep the software up to date and make things easier to apply in bundles.  You'll want to setup a plan for planned software maintenance in your environment.  There's a whitepaper Oracle Enterprise Manager Software Planned Maintenance that will help guide you through the best practices.  

Monday Jun 23, 2014

Patch automation of Data Guard Databases takes a leap with EM12cR4


Patching has always been one of the many joys (or dreads) of Oracle DBAs. 
The effort compounds as the Database sprawl grows and the architecture complexity increases.  Additionally,  there is an increasing pressure to patch the systems with minimal or zero downtime. EM12c's Database patch automation functionality provides  an end-to-end management starting with proactive patch recommendations (like quarterly PSUs and CPUs), complete pre-flight checker to ensure predictable patching during maintenance window and  automation for patching the comprehensive list of products under the Oracle Database family.  ( EM12c Patch Automation overview

Patching has always been one of the many joys (or dreads) of Oracle DBAs. The effort compounds as the Database sprawl grows and the architecture complexity increases. Additionally, there is an increasing pressure to patch the systems with minimal or zero downtime. 

EM12c's Database patch automation functionality provides an end-to-end management starting with proactive patch recommendations (like quarterly PSUs and CPUs), complete pre-flight checker to ensure predictable patching during maintenance window and  automation for patching the comprehensive list of products under the Oracle Database family.  ( EM12c Patch Automation overview)

With the introduction of "Out of Place" patching in EM12c, the automation feature got a complete overhaul. Over the past 3 years I have seen customers moving to this mode to achieve their automation goals.  ( What is Out of Place?)

Patching Data guard environments (read as Primary and its corresponding Standby Databases) has always been a challenging task for the DBAs. 
In addition to handling the differences in steps needed for the configuration, the very nature of its distributed environment and incorporating any additional custom tasks makes it more demanding. Till now in EM we only supported automation in disparate steps and in “In Place” mode.

Starting 12.1.0.4 (EM12c R4), the support is enhanced to be more tighter, can be done in 'Out of Place' mode, and has new supplemental features to manage the process along with its additional tasks.

In this post we will take a closer look of the data guard stand by patching feature and then dive into a use case, you could try out in your environment.

[Read More]

Tuesday Dec 31, 2013

Database Lifecycle Management for Cloud Service Providers

Adopting the Cloud Computing paradigm enables service providers to maximize revenues while driving capital costs down through greater efficiencies of working capital and OPEX changes. In case of enterprise private cloud, corporate IT, which plays the role of the provider, may not be interested in revenues, but still care about providing differentiated service at lower cost. The efficiency and cost eventually makes the service profitable and sustainable. This basic tenet has to be satisfied irrespective of the type of service-infrastructure (IaaS), platform (PaaS) or software application (SaaS). In this blog, we specifically focus on the database layer and how its lifecycle gets managed by the Service Providers.

Any service provider needs to ensure that:

  • Hardware and software population are in control. As new consumers come in and some consumers retire, there is a constant flux of resources in the data center. The flux has to be managed and controlled
  • The platform for providing the service is standardized, so that operations can be conducted predictable and at scale across a pool of resources
  • Mundane and repeatable tasks like backup, patching, etc are automated
  • Customer attrition does not happen owing to heightened compliance risk

While the Database Lifecycle Management features of Enterprise Manager have been widely adopted, I feel that the applicability of the features with respect to service providers is yet well understood and hence appreciated. In this blog, let me try addressing how the lifecycle management features can be effective in addressing each of the above requirements.

1. Controlling hardware and software population:

Enterprise Manager 12c provides a near real-time view of the assets in a data center. It comes with out-of-box inventory reports that show the current population and the growth trend within the data center. The inventory can be further sliced and diced based on cost center, owner, etc. In a cloud, whether private or public, the target properties of each asset can be appropriately populated, so that the provider can easily figure out the distribution of assets. For example, how many databases are owned by Marketing LOB can be easily answered. The flux within the data center is usually higher when virtualization techniques such as server virtualization and Oracle 12c multitenant option are used. These technologies make the provisioning process extremely nimble, potentially leading to a higher number of virtual machines (VMs) or pluggable databases (PDBs) within the data center and hence accentuating the need for such ongoing reporting. The inventory reports can be also created using BI Publisher and delivered to non-EM users, such as a CIO.


Now, not all reports can always be readily available. There can be situations where a data center manager can seek adhoc information, such as, how many databases owned by a particular customer is running on Exadata. This involves an adhoc query based upon an association, viz. database running on Exadata and target properties, viz. owner being the customer. Enterprise Manager 12c provides a sophisticated Configuration Search feature that lets administrators define such adhoc queries and save them for reuse.

2. Standardization of platform:

The massive standardization of platform components is not merely a nice-to-have for a cloud service provider, it is rather a must-have. A provider may choose to offer various levels of services, tagged with levels such as gold, silver and bronze. However, for each such level, the platform components need to be standardized, not only for ease of manageability but also for ensuring consistency of QOS across all the tenants. So how can the platform be standardized? We can highlight two major Enterprise Manager 12c features here:

The ability to rollout gold images that can be version controlled within Enterprise Manager's Software Library. The inputs of the provisioning process can be "locked down" by the designer of the provisioning process, thereby ensuring that each deployment is a replica of the other.

The ability to compare the configuration of deployments (often referred to as the "Points of Delivery" of the services). This is a very powerful feature that supports 1-n comparisons across multiple tiers of the stack. For example, one can compare an entire database machine from storage cells, compute nodes to databases with one or more of those.

3. Automation of repeatable tasks:

A large portion of OPEX for a service provider is expended while executing mundane and repeatable tasks like backup, log file cleanup or patching. Enterprise Manager 12c comes with an automation framework comprising Jobs and Deployment Procedures that lets administrators define these repetitive actions and schedule them as needed. EMCC’s task automation framework is scalable, carries functions such as ability to schedule, resume, retry which are of paramount importance in conducting mass operations in an enterprise scale cloud. The task automation verbs are also exposed through the EMCLI interface. Oracle Cloud administrators make extensive use of EMCLI for large scale operations on thousands of tenant services.

One of the most popular features of Enterprise Manager 12c is the out-of-box procedures for patch automation. The patching procedures can patch the Linux operating system, clusterware and the database. For minimizing the downtime involved in the patching process Enterprise Manager 12c also supports out-of-place patching that can prepare the patched software ahead of time and migrate the instances one by one as needed. This technique is widely adopted by the service providers to make sure the tenants' downtime related SLAs are respected and adhered to. The co-ordination of such downtime can be instrumented by Enterprise Manager 12c's blackout functionality.

4. Managing Compliance risks:

In a service driven model, the provider is liable in case of security breaches. The consumer and in turn, the customer of the consumer's apps need to be assured that their data is not breached into owing to platform level vulnerabilities. The security breaches often happen owing to faulty configuration such as default passwords, relaxed file permissions, or an open network port. The hardening of the platform therefore, has to be done at all levels-OS, network, database, etc. The security breaches often happen owing to faulty configuration such as default passwords, relaxed file permissions, or an open port. . To manage compliance, administrators can create baselines referred to as Compliance Standard. Any deviations from the baselines triggers compliance violation notifications, alerting administrators to resolve the issue before it creates risk in the environment.

We can therefore see how four major asks from a service provider can be satisfied with the Lifecycle Management features of Enterprise Manager 12c. As substantiated through several third party studies and customer testimonials, these result in higher efficiency with lower OPEX.

Stay Connected:

Twitter |  Face book |  You Tube |  Linked in |  Google+ Newsletter

Monday May 21, 2012

Benefits of using Ops Center to deploy and manage Solaris 11

One of the more significant new features in Oracle Enterprise Manager Ops Center 12c is the ability to install Ops Center on Oracle Solaris 11, and to deploy and manage systems running Solaris 11.  The Solaris 11 capabilities are in addition to the analogous features for Solaris 10 and Linux, which can all be handled from the same Ops Center infrastructure.

When the Ops Center Enterprise Controller (EC) is installed on a system running Solaris 11, the EC can create a Solaris 11 Software Update Library containing Oracle Solaris Image Packaging System (IPS) content that is synchronized with the main Oracle repository at pkg.oracle.com. The Ops Center managed Solaris 11 repository becomes the package (pkg) publisher for downstream Solaris 11 deployments and updates on all Solaris 11 systems being managed by Ops Center.

Ops Center provides the ability to define Solaris 11 OS and Software Profiles comprised of Oracle Solaris 11 packages, user-supplied custom packages, scripts, and other files.  Such Software Profiles profiles can then be used to install and update software on systems already running Solaris 11 in a structured and consistent way.  Ops Center not only caches the main Oracle Solaris IPS repository, but more importantly it gives admins the ability to define their own preferred collection of packages so that systems can easily be kept in sync with each other, running a well-defined, life-cycle-managed Standard Operating Environment (SOE), instead of just whatever the latest content is at pkg.oracle.com.

Ops Center 12c also adds Solaris 11 features for bare-metal OS Provisioning, based on the Solaris 11 Auto Install (AI) facility.  Ops Center configures the Solaris 11 AI in a way that shields admins from needing to write custom AI manifests or custom "first boot" packages.  Solaris 11 deployments using Ops Center follow similar profile-based patterns as for Solaris 10 or Linux, all of which can all be deployed from the same Ops Center infrastructure running on Solaris 11.  The gory details of all these different times of bare-metal OS Provisioning are handled automatically for the user so that he or she does not need to put time and resources into manually creating and maintaining infrastructure for deploying different OS's natively -- Solaris 11 with AI, Solaris 10 JumpStart with JET, or Linux with Kickstart or AutoYast.  All of those OS's are handled by Ops Center under the covers, based on whatever network boot capability the OS requires (PXE/DHCP, WANBOOT, or AI), and all from the same Ops Center infrastructure running on Solaris 11.

Specific to Solaris 11 OS Provisioning (via AI), Ops Center provides its own "first-boot" package+scripts to customize the Solaris 11 deployment, and in particular this approach automatically installs the Ops Center agent.  With the Ops Center Agent in place right from the start, it is easy to handle post-install steps using the Ops Center features for handling Solaris 11 OS and Software Profiles containing additional packages, scripts, and content from the Solaris 11 Software Library, described above.

Tying bare-metal Solaris 11 deployment and post-install customization together is a key way that Ops Center simplifies the overall life-cycle management for Solaris 11 (in addition to Solaris 10 and Linux).   For example, a top-level plan based on "Configure and Install Logical Domains" can create numerous Logical Domains into an Oracle VM Server for SPARC "Server Pool" and provision Solaris 11 into each LDom Guest based on a powerful multi-step "Install Server" plan.  Such a plan can cover the end-to-end steps for installing and updating the OS, running scripts and adjusting monitoring parameters, etc.

Here is an example of the kinds of activities that can be performed in conjunction an OS Deployment, or separately, depending on the need:

Oracle Enterprise Manager Ops Center 12c Deployment Plan

NOTE: In the above example, the last step "Create Guests" can be used to create one or more Solaris Containers within the LDom Guest, rounding out the end-to-end deployment all the way from LDom Guest to Solaris 11 Global Zone to multiple Solaris Containers, if so desired.

One of the nicest aspect of deploying and managing Solaris 11 using Ops Center Plans and Profiles is that the same content can be applied as updates to existing Solaris 11 systems, aligned to the same content as chained off a bare-metal OS Provisioning.  It is up to the user which steps they want to include in a deployment plan -- whether they are updating Software Profiles on systems deployed 6 months ago to match the latest standard, or they are deploying new systems based on that same standard, Ops Center provides the means to insure that the outcome is consistent.

Here is an example of the kinds of activities that can be performed on an existing Solaris 11 OS -- either ad hoc, or immediately after a Solaris 11 OS Provisioning step, so that whether the life-cycle started with a new system, or the intent is to update a system deployed six months ago, the outcome can be the same:

S11 Update Multi-Step Plan

In short, Ops Center running on Solaris 11 can manage Solaris 10 and Linux systems, all from a common infrastructure, and all based on a simplified, consistent, profile- and plan-based way to do the OS and Software deployments and updates.  The net effect is an easy to use way to managing the life-cycle of heterogeneous systems, in a very consistent way through automation and re-use.

Please let us know what you think?  Until next time...
\Leon
--

Leon Shaner | Senior IT/Product Architect
Systems Management | Ops Center Engineering @ Oracle

The views expressed on this [blog; Web site] are my own and do not necessarily reflect the views of Oracle.



For more information, please go to Oracle Enterprise Manager  web page or  follow us at : 

Twitter | Facebook | YouTube | Linkedin | Newsletter

About

Latest information and perspectives on Oracle Enterprise Manager.

Related Blogs




Search

Archives
« July 2015
SunMonTueWedThuFriSat
   
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
 
       
Today