Patching 101 - The User Friendly Guide to Understanding EM Patches
By Courtney Llamas on Jul 17, 2014
The core Enterprise Manager system is typically patched with the quarterly PSU patches (released Jan, Apr, July, Oct) or a one-off when directed by support for a critical issue. PSU patches will be cumulative, so you need not apply each of them, just apply the latest. The OMSes must be shutdown during patching, however some patches are being released with rolling patch instructions for multi-OMS systems. These patches must be applied at the host level, and cannot be automated via EM. ALWAYS read the readme, yes every time. The patching steps can change from patch to patch so it's critical to read the readme. OPatch or OPatchauto will be used to apply these patches. Did I mention to read the readme for every patch? It's also important to note that there may be additional steps when patching in a multi-OMS or standby environment, so read the output of OPatchauto carefully.
Always download the latest OPatch release for the appropriate version. If you read the readme, you already know this! Download patch 6880880 for 11.1 (the OPatch version used by EM) and unzip into the $ORACLE_HOME. Most errors in patching are related to not updating OPatch.
For more information on PSU Patches and patching EM:
Oracle Enterprise Manager Cloud Control Administrators Guide - Chapter 16 Patching Oracle Management Server and the Repository
EM 12c Cloud Control: List of Available Patch Set Updates PSU (Doc ID 1605609.1)
How to Determine the List of Patch Set Update(PSU) Applied to the Enterprise Manager OMS and Agent Oracle Homes? (Doc ID 1358092.1)
Each plug-in has binaries that will require patches as well. Same downtime requirements apply for plug-in patches as the quarterly PSUs. Starting in 18.104.22.168, the plug-in patches are being released as a monthly bundle. This means that if you have 6 plug-ins, you may have 6 OMS side patches to apply - 1 for each plug-in. Bundles are not always released for every plug-in every month. They are cumulative, so pick the latest.
Starting with 22.214.171.124, the individual OMS-side plug-in bundles are being grouped into a System Patch each month. So for example, in June 2014 the System patch includes MOS, Cloud, DB, FA, FMW, SMF, and Siebel plug-ins. Non-required patches will be skipped.
For more information on the EM Patch Bundles and Patching EM:
Enterprise Manager 126.96.36.199.0 (PS3) Master Bundle Patch List (Doc ID 1900943.1)
Enterprise Manager 188.8.131.52 Bundle Patch Master Note (Doc ID 1572022.1)
Agent patches are applied to each agent. They can be applied via EM using the MOS patch plans, which makes it a lot easier when you have 100s or 1000s of Agents to patch! The Patch Plans will start a blackout, validate prerequisites, check for conflicts, and update OPatch for you. If you don't use the Patch Plan you can patch manually with OPatch, don't forget to read the readme! The Agent must be shutdown during the patch application. There are 4 main types of Agent patches you will see:
- Core Agent - Starting with 184.108.40.206.0 the core Agent will have monthly patch bundles . These are also cumulative, so my recommendation is to apply the latest one.
- Agent-side Discovery Plug-in - This is the lightweight piece of the plug-in used for target discovery. Discovery plug-in patches are cumulative with other discovery plug-in patches for that component.
- Agent-side Monitoring Plug-in - This is the more detailed monitoring side of the plug-ins for the required components. Monitoring plug-in patches are cumulative with other monitoring plug-in patches for that component. So if there's a Discovery and Monitoring patch available for the DB Plug-in, you need to apply both of them.
- JDBC patches for the Agent will be JDBC version 220.127.116.11.0. These patches do get applied to the Agent, and can be applied via the Patch Plans.
You can apply the latest Agent bundle, JDBC patch and the plug-in bundles in one patch plan. If there's a conflict, you'll be notified. If the Agents you've selected don't have specified plug-ins, you'll also receive notice during the analyze step. As of now, for my 18.104.22.168 agents, I would apply the 22.214.171.124.1 patch (18873338) and the two available plug-in agent patches DB monitoring (19002534) and FMW monitoring (18953219) and the latest JDBC patches (18502187,18721761) all in one patch plan.
I discovered a new feature in 126.96.36.199 while testing this. Normally you had to have Normal Oracle Home preferred credentials set for all Agent targets to patch, or select Override and specify the Normal Oracle Home credentials. In 188.8.131.52, the Agent uses it's internal credentials to Patch itself, so setting preferred credentials or specifying at run-time is not required. The user patching would require the Manage Target Patch and Patch Plan privileges.
The OMS and Agent are the key components, and my main focus here. However it's important to keep the infrastructure stack up to date as well. This includes the Oracle Fusion Middleware and Oracle Database that are used for EM. The recommendation is to follow the best practices for each of these components, and regularly update with the PSU patches available. The following reference notes will help in identifying the current PSU patches. The WebLogic Server version used by EM 12c is 10.3.6.
Hopefully this will help you understand the various types of components involved with keeping EM up to date. Obviously, you may not want to patch each month and maybe not every quarter, but the patches are available to keep the software up to date and make things easier to apply in bundles. You'll want to setup a plan for planned software maintenance in your environment. There's a whitepaper Oracle Enterprise Manager Software Planned Maintenance that will help guide you through the best practices.