Over last few months, the Oracle Enterprise Manager team has released Enterprise Manager 12c Bundle Patch1 (a.k.a. BP1) for Oracle Enterprise Manager Cloud Control 12c (EM 12c) on all the supported platforms (Customer announcement). BP1 is a mandatory patch because all future patches will assume the presence of this patch. BP1 includes several critical fixes. Therefore this patch touches almost all the components of Enterprise Manager and applying it on existing environment is a multi step process that can be tricky. From our recent experience of applying BP1 on an internal production demo site with over a thousand targets, we would like to share following tips and tricks.
1. Applying incremental Bundle Patch or using full install?
Unless you have an EM 12c running in production environments where you can’t afford to lose the existing management repository, it is highly recommended to re-install EM using the patched EM Base Platform 18.104.22.168 Full Installer (With BP1), instead of applying BP1. For instance, if you have a test or a “sandbox” environment even with substantial number of targets, it might be easier to reinstall EM environment instead of manually applying patches (including agent patches) and upgrading plug-ins. Note that reinstalling EM will require reinstallation of target agents as well. Therefore you’ll need to do careful consideration before choosing this option.
2. Plan it well
- Given that the BP1 process involves agent side changes like patching and plug-in upgrades, you might want to consider splitting the agent side changes into phases, especially in large sized environment. In such cases, it is critical to review the Compatibility Matrix and ensure that at no stage during the BP1 upgrade process, your environment is in an unsupported configuration.
- Never break the sequence of steps given in the BP1 application guide or you may risk your entire environment. For example -- as an admin, after upgrading the plug-ins on OMS, you may be tempted to upgrade plug-ins of agents as well even before patching the agent. As the documentation says -- resist the temptation to avoid issues while patching agents later.
- Keep sufficient free space for backup. It might be helpful to take keep multiple backups. See the next tip.
3. EM 12c Backup is critical
- Never underestimate the importance of environment backup during the BP1. It is must to backup your environment before any steps that involve change in repository data. e.g. OMS patching, plug-in upgrade on OMS, etc.
- Additionally it is recommended to take backup after every significant step to minimize rework in case of any failure. For instance in our environment a total of four backups were taken at following stages.
- Before applying BP1 on OMS
- After applying BP1 on OMS, JDeveloper, WSM
- Before upgrading plug-ins on OMS
- After upgrading first few plug-ins on OMS
4. Upgrade plug-ins in bulk using emcli
BP1 comes with the 22.214.171.124 release of plug-ins and it is highly recommended to upgrade plug-ins together with the application of BP1. In a typical environment you may need to upgrade more than 15 plug-ins. For some plug-ins, the upgrade process requires a restart of OMS(s) and therefore can take a good deal of time (almost fifteen minutes for single plug-in upgrade). Hence upgrading plug-ins one by one via the UI can be time consuming process requiring manual intervention after every few minutes.
Use EM command line ‘deploy_plugin_on_server’ to deploy multiple plug-ins in one go in an automated manner. Since multiple plug-ins are upgraded in single downtime window for OMS, it is a far more efficient process. In our environment, using command line we could upgrade eight plug-ins in under 30 minutes which otherwise would have taken more than two hours. Make sure to run the emcli in a pre-requisite check mode before doing the actual deployment such as shown in the screenshot below.
Also note that if your OMS is on Linux platform, you’ll need to apply one-off patch 13638422 after applying BP1 on the OMS to get the ‘deploy_plugin_on_server’ emcli verb. For BP1 on other platforms (e.g. Windows) this patch is included in BP1 itself.
5. From Linux OMS don’t push agents on other platforms without necessary patches
A very common use case is the deployment of agents on non-Linux platform from a Linux OMS with BP1. Since you have Windows/Solaris agent software (with BP1) available via self update, you might tend to assume that, all you need to do is simply push the Windows/Solaris agents from BP1 Linux OMS but that’s not the right way to do. You have to apply few patches on OMS and plug-ins before pushing the agent, follow the instructions from the Oracle® Enterprise Manager Bundle Patch 1 Application Guide.
6. Read the right documentation
Last but surely not the least, it is vital that you go through the Oracle® Enterprise Manager Bundle Patch 1 Application Guide, before you begin with the BP1 process. This guide provides step by step instructions for applying BP1 including pre-requisite checks, recommendations, and troubleshooting steps.
In addition to the documentation, we suggest to refer the following key resources published by Oracle.
These few simple tips can make your experience with BP1 a lot smoother. In future entries, we’ll discuss the best practices for major updates for Oracle Enterprise Manager 12c as and when those are available.