X

News, tips, partners, and perspectives for Oracle’s virtualization offerings

Upgrading to Oracle Private Cloud Appliance Software Release 2.3.2

Jeff Savit
Product Management Senior Manager

Zero downtime upgrade architecture

Oracle recently announced Private Cloud Appliance Software Release 2.3.2, which adds support for X7-2 compute nodes, upgrades Oracle VM to release 3.4.4, and adds several other features. This article gives tips for upgrading to this new release. Don't worry - it's all documented. I just like to add my own .02 and enjoy over-explaining.

Important: PCA is designed for zero downtime upgrades. It uses redundant management, network, storage, and compute resources to avoid single points of failure for both normal operation and during upgrades. Rolling upgrades let us take components offline to upgrade them, while redundant partner components carry on providing service. Much of the effort is automated to reduce errors and administrative effort. While I'm a cynical old-school datacenter guy who says "block off a maintenance window for the most trivial thing", a PCA upgrade can be done without service interruption, permitting 24/7 operation even during upgrade. 

Preparation

  • First, read the Administration Guide chapter Updating Private Cloud Appliance.
  • Download the updated software release from My Oracle Support (MOS). Release 2.3.2 is contained in patch ID 26982346 and consists of two zip files. Don't go to MOS and look for upgraded firmware or software for PCA components (ZFS appliance, InfiniBand switches, etc).unless directed to do so by Oracle PCA support. The components have been tested with each other with the versions that are shipped.
  • Read the patch README. Yes, really! Always a good idea, and it gives explicit instructions for the next steps.
    • Check the pre-requisites and update matrix (from which release, to which release) in the README file.
    • PCAs at a version before 2.1.1 must be updated to an intermediate release before going to 2.3.2.
  • The next few steps can be done locally on your own Linux workstation, or you can scp the zip files to the active PCA management node and do them there.
  • Unzip the two zip files, which contains the pieces that make up the install image, another copy of the README, a pre-upgrade check script, and a script cleverly called RUN_ME_FIRST.sh.
  • Run the RUN_ME_FIRST.sh script, which
    • prints the contents the upgrade matrix and points to the MOS note "PCA 2.3.X Upgrade Checklist and Prerequisites (Doc ID 2242177.1)". This MOS note contains mandatory steps for pre- and post-upgrade checks and administrative actions
    • combines component parts into the install image iso.zip file.
    • Recommends the administrator open an proactive Service Request with Oracle so we can be aware of the upgrade and provide assistance if needed.
  • If you did the above steps locally, preferably using a fresh filesystem directory, move this directory's contents to the active PCA management node. Steps below are done on the PCA.
  • Follow the instructions in MOS note 2242177.1. This includes running pre-checks, which comprehensively test component health.
  • If Enterprise Manager is being used to administer the PCA, the checklist directs you to place the PCA in blacklist mode, so it won't think that it's in trouble when it's doing expected upgrade and reboots. This is documented in MOS note "[ PCA ] Specific steps for customers making use of Enterprise Manager 13c when upgrading Private Cloud Appliance to release 2.3.1 (Doc ID 2280818.1)" and Simon Hayler's blog "Friday Spotlight: Upgrade to Oracle Private Cloud Appliance 2.3.2 from 2.3.1 with Oracle Enterprise Manager 13.2"

PCA controller software upgrade

Once the preparation is done, proceed by upgrading the PCA controller software on the management nodes.

Remember that the management nodes run as an active/passive server pair for resiliency. It's an optional but good idea to first reboot the passive node and then the active node, so we know they're in clean state and cluster status and to verify that failover worked correctly. Determine which node is the master by logging into each management node and typing the command pca-check-master, which will display True or False as needed.

The update is started on the currently active management node, which copies install image iso contents into target directories, and then reimages the passive management node using those contents (the active management node acts as a kickstart server for the passive node).  I like to watch the process running by logging into the management nodes ILOMs and watching the consoles, as described in section 3.3.3. of the Administration Guide.

The pca-admin command is used to perform the upgrade. Issue pca-admin, and then enter update appliance get_image FILEURL where FILEURL can be an http URL: or a file URL on the PCA. A complete command (assuming you place the image in a directory on /nfs/shared_storage) could be pca-admin update appliance get_image file:///nfs/shared_storage/2.3.2/ovca-2.3.2-b286.iso.zip    If you previously unzipped the iso.zip file, that's fine too - the command handles it whether you previously zipped or not.  Administration Guide section 3.3.4 shows how to monitor the progress of this task. When it finishes, issue pca-admin update appliance install_image which starts the reimaging and reboots of each management .mode.

When the passive node finishes upgrading to the new release, it takes over the active role, and then reboots and acts as a kickstart server for the previously active management node. Compute nodes and virtual machines continue operating without disruption to business or applications.

When the process completes, the PCA controller software is upgraded to PCA 2.3.2, and the previously inactive one is the active one. If you like, you can switch their roles again by rebooting (init 6) the active one. You can flip flop them all day long if you're easily entertained :)  It's instructive to see how quickly the passive management node becomes that active one during a failover. Again: VMs continue to operate without being affected by any of this.

Upgrading the Virtualization Platform

The virtualization platform (on the compute nodes running Oracle VM Server) is upgraded after the controller software. You can operate the PCA with upgraded controller software while the compute nodes continue to operate at the prior software level but we recommend upgrading them shortly after in a timely fashion.

Virtualization platform upgrade is done on a one at a time compute node basis. If there are multiple tenant groups, upgrade each tenant group completely before moving to the next one.

  • Log into Oracle VM Manager, select a tenant group (server pool). If the system is currently on PCA 2.2. or earlier, edit the server pool and see which server has the "master server" role. Upgrade that server last. Once the pool is upgraded, it no longer even needs the "master server" role.
  • For each server in the pool, edit it in the Oracle VM Manager user interface to put it in maintenance mode (set the checkbox). That evacuates the server by live migrating all running VMs to other servers in the pool. They continue provide services without interruption.
  • From the active management node, issue the command pca-admin update compute-node ovcacnXXr1 where ovcacnXXr1 is the server selected in the previous step.
  • When the upgrade is complete, go back to Oracle VM Manager user interface, edit the server, and remove the maintenance mode checkbox. Go on to the next server.
  • Lather, rinse, repeat. The server pool and VMs are continuously available.

Post-install

MOS note 2242177.1 has several administrative steps and health checks to perform after the upgrade completes. Perform them, and you're all done.

Summary

This article provides tips and links for upgrading the Oracle Private Cloud Appliance to the latest version. It should be emphasized that virtual machines - where the applications run - continue to operate without any interruption of service, providing continuous application availability for applications.

 

 

 

 

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha