Patching Oracle Database is an essential task that customers should regularly accomplish to avoid functional and security bugs. But coping with a release update per quarter is not an easy task. Customers with a considerable amount of databases have to dedicate precious DBA resources to patching activities. Customers in specific industries must abide by strict regulations that force them to apply the security patches within a short delay. Small customers struggle as well, often lacking knowledge or resources, which sometimes lead to leaving the databases not patched and unprotected. Dealing with patches is a hard job: no one likes doing it. How not agree?
Patching some environments, like Oracle Database Real Application Clusters (RAC), might be even more complicated. One of the great benefits of RAC is the ability to do rolling patches without application downtime (especially if combined with session draining or Application Continuity). But how many tools are involved? runInstaller, opatch, opatchauto, datapatch. What about Grid Infrastructure? Or Exadata software? gridSetup.sh and patchmgr are also part of the list.
There are a few patterns that can help to simplify this activity.
The first one is out-of-place patching.
For the last few years, Oracle's recommended approach has been to use in-place patching for urgent one-offs only. For quarterly patches (RUs and RURs), we recommend out-of-place patching. Prepare the new binaries, install them, and move the resources from the old binaries to the new ones.
That reduces the points of failure, and it has been a strategy that I have frequently used myself for my employers before joining Oracle.
The second pattern is automation.
The automation of patching operations is possible, but it is not effortless. We have customers doing it properly, with shell scripts or more sophisticated automation frameworks like Ansible. But maintaining such a solution is a full-time job per se: the database environment must be uniform and standardized to the extreme, or exceptions will show up and break the whole patching process.
Standardization is indeed a crucial part of automation. Without it, every attempt at automation will fail at some point. And it is our third pattern.
Oracle Database development has already created a product based on these three patterns. It is Oracle Fleet Patching and Provisioning (FPP).
FPP stores all the versions of the binaries in a central repository on the FPP server, and from there centralizes and automates the operations on the targets:
- Installing and removing Oracle Homes on the targets (FPP knows them as "working copies")
- Create, remove, patch out-of-place, and upgrade databases with DBCA, datapatch, DBUA or Autoupgrade
- Deploy, patch, or upgrade Oracle Grid Infrastructure environments (either single-node or multi-node, full GI stack or Oracle Restart)
- Update Exadata Software for DB nodes, InfiniBand switches, storage cells.
Some operations can be combined (for example, Grid Infrastructure and Database patching, or Exadata DB Node and Grid Infrastructure patching) so the overall number of relocations and brownouts stays low.
FPP uses out-of-place patching as its only patching method. Deploying the binaries from the central repository using a repeatable procedure guarantees that all the environments comply with the company's standards. That makes FPP the ideal solution for database lifecycle management.
What about existing automation frameworks, such as Ansible? FPP does not replace them, but rather it fills the gap if the maintenance of Ansible modules for database patching becomes too complex. Indeed, the big customers implementing FPP often use it in conjunction with Ansible, having simple modules calling FPP on the target nodes and delegating to it all the complexity of the patching operations.
We have big customers happily using FPP in medium-sized environments (>100 databases) and large ones (>1000 databases). That includes many of our internal customers: some of our Global Business Units, our Fusion Apps services, and our flagship Autonomous Database on dedicated infrastructure (both on-premises and in the cloud). Other customers include financial institutes, retail companies, public governments, etc. All of them were struggling to cope with quarter patching cycles, and they are now relying on FPP's capabilities to patch hundreds of targets in parallel.
Would you like to try it? You can practice with FPP on your laptop using Vagrant and Virtualbox, or you can try it in OCI using the LiveLabs workshop, either in your paid tenancy or for free in the LiveLabs' one.
Start your journey with Fleet Patching and Provisioning: visit the product page at https://oracle.com/goto/fpp.
From there you can access videos, workshops, whitepapers, and official documentation.