Friday Jun 19, 2009

Zones Parallel Patching versus Update On Attach: When to use which one ?

The Zones Parallel Patching enhancement for the Solaris 10 patch utilities was released this week giving customers a choice of how to improve zones patching performance.

In the Zones "Update On Attach" section of a previous blog posting, I mentioned that the Zones "Update On Attach" feature could also be used to improve Zones patching perfomance.

Zones Parallel Patching is a true patching solution utilizing the 'patchadd' utility.  

Whereas Zones "Update On Attach" uses zones functionality similar to that used during zones creation to provide a pseudo-patching solution that does not utilize 'patchadd'. 

So which one to choose ?

Let's look at the two options in more detail:

Zones Parallel Patching

Zones Parallel Patching is an enhancement to the standard Solaris 10 patch utilities and is delivered in the patch utilities patch, 119254-66 (SPARC) and 119255-66 (x86).

Simply install this patch, set the maximum number of non-global zones to be patched in parallel in the config file /etc/patch/pdo.conf, and away you go.

It works for all Solaris 10 systems. 

It also works well in conjunction with higher level patch automation tools such as xVM Ops Center. 

It can dramatically improve zones patching performance by patching non-global zones in parallel.  The global zone is still patched first.

While the performance gain is dependent on a number of factors, including the number of non-global zones, the number of on-line CPUs, the speed of the system, the I/O configuration of the system, etc., a performance gain of ca. 300% can typically be expected for patching the non-global zones - e.g. On a T2000 with 5 sparse root non-global zones.

See my previous Zones Parallel Patching blog entry for further information.

Since it's a pure enhancement to 'patchadd', it's normal 'patchadd' functionality.  You can subsequently remove patches using 'patchrm', etc.  Nothing has changed except that it's now much faster to patch non global Zones with Zones Parallel Patching invoked.

Zones "Update On Attach"

The primary purpose of Zones "Update on Attach" is Zones migration from one server to another.  

For example, a database instance in a non-global zone hosted on a server has grown to the extent that the Sys Admin wants to transfer it to a better spec'd server which can better handle the workload.   The Sys Admin can detach it from the old server (e.g. a Sun4u) and reattach it to the new server (e.g. a Sun4v) using Zones "Update On Attach".   This will bring the OS Software level on the non-global zone up to the same level as the new server's global zone.

Zones "Update On Attach" can certainly be used for patching but there are limitations you need to be aware of as outlined below.

For example, detach the non-global zones from a system, apply a bunch of patches to the global zone, reattach the non-global zones using "Update On Attach" and viola, the non-global zones will be brought up to the same software level as the global zone (for OS type packages), effectively patching the non-global zones without using 'patchadd' at all.   This is typically even faster than using Zones Parallel Patching.  But there are limitations to this approach which users must be aware of (see below).

My senior engineer, Enda O'Connor, has just published an interesting article on The Zones Update on Attach Feature and Patching in the Solaris 10 OS

Zones "Update On Attach" limitations as a patching aid

Zones "Update On Attach" only works for packages which are SUNW_PKG_ALLZONES=true - i.e. typically OS level packages, and not application packages.

So when to use Zones Parallel Patching in 'patchadd' and when to use Zones "Update On Attach" ?

Here's what my senior engineer, Enda O'Connor, says:

"The Zones Update on Attach Feature and Patching in the Solaris 10 OS document may help customers understand how the technology works, applying a cluster via patching and via zones Update On Attach is not quite the same really.

It really depends on the patches being applied, i.e. applying a firefox patch via Update On Attach would not work if you wanted it to apply to the global zone and all non-global zones as well.

One has to understand how Update On Attach works and then apply that to the list of patches to see if it gets them to a desirable state.

There is no black or white answer here.

I'd recommend Zones Parallel Patching using 'patchadd' as it has a known outcome all the time, whereas Update On Attach makes it's own internal determination based on a number of things, that can vary from system to system ( e.g. inherited directories ).

But if time to patch is critical then if the customer does proper testing to validate things, and are happy with the results, then by all means use Update On Attach.

But using Update On Attach without:

1. Understanding how it determines what packages to update

2. Not inspecting the patches being applied.

...will most likely lead to grief at some point."

And my other senior engineer, Ed Clark, says:

"In terms of giving guidance on which technology to use, there are a number of considerations -- two of these considerations are:

1. Using Update On Attach to update sparse zones can require significantly more disk storage space than would be needed by applying patches with 'patchadd' (3-4 times as much space would not be uncommon i think), due to Update On Attach copying fully populated global zone 'undo' files into the non-global zones, as opposed to having patchadd build sparsely populated 'undo' files in the non-global zones.

2. If a customer is really concerned about the ability to back out patches reliably, then 'patchadd' is a lower risk option than Update On Attach -- 'patchrm' of a patch from a non-global zone that has a copy of the global zones 'undo' pkg data (as is the case after Update On Attach) may potentially have unexpected side effects." [although we have yet to see any actual cases of negative results from this.]

Conclusion

In general, we recommend using the Zones Parallel Patching enhancement in the patch utilities rather than the Zones "Update On Attach" feature as Zones Parallel Patching is standard patching functionality, only faster, whereas Zones "Update On Attach" is really designed for migrating zones from one server as another and was not primarily designed to speed up patching.  

Because Zones "Update On Attach" uses Zones functionality similar to the zone creation functionality, rather than 'patchadd' functionality, limitations exist on what will be patched (typically the OS but not applications) and there's the potential for anomalies around things like the "undo" files which would be used by 'patchrm' if patches applied using Zones "Update On Attach" were subsequently removed from the non-global zones using 'patchrm' (although we have yet to see any actual cases of serious issues resulting from this).

So in patching situations where time is absolutely critical, Zones "Update On Attach" may provide a good option, as long as it's well tested in the customer environment prior to deployment on production systems.

Remember too, Live Upgrade is also your friend in such situations, enabling you to patch an inactive boot environment while the system is still in production.   So a combination of Live Upgrade and Zones Parallel Patching would be ideal.

I hope you find this helpful!

Best Wishes,

Gerry.

Wednesday Jun 17, 2009

Zones Parallel Patching feature now available!

The Zones Parallel Patching feature is now available in the latest Solaris 10 patch utilities patch, 119254-66 (SPARC) and 119255-66 (x86).

This is available for use on all Solaris 10 systems. 

Simply install this patch, set the maximum number of non-global zones to be patched in parallel in the config file /etc/patch/pdo.conf, and away you go.

Prior to this feature, each non-global zone was patched sequentially, leading to unnecessarily long patching times for zones systems.  (Sequential patching remains the default behavior unless the config file is edited to enable Zones Parallel Patching.)

With this feature invoked, the global zone continues to be patched first, but then the non-global zones can be patched in parallel, leading to significant performance gains in patching operations on Zones systems.

While the performance gain is dependent on a number of factors, including the number of non-global zones, the number of on-line CPUs, the speed of the system, the I/O configuration of the system, etc., a performance gain of ca. 300% can typically be expected for patching the non-global zones - e.g. On a T2000 with 5 sparse root non-global zones.

Here's the relevant note from the patch README file:

NOTE 10: 119255-66 is the first revision of the patch utilities to deliver "zones parallel patching".
          This new functionality allows multiple non-global zones to be patched in parallel by patchadd.
          Prior to revision 66, patchadd would patch all applicable non-global zones sequentially,
          that is one after another. With zones parallel patching, a sysadmin can now set the number
          of zones to patch in parallel in a new configuration file for patchadd called /etc/patch/pdo.conf.

         The two factors that affect the number of non-global zones that can be patched in parallel are
         1. Number of on-line CPUs
         2. The value of num_proc in /etc/patch/pdo.conf

          If the value of num_proc is less than or equal to 1.5 times the number of on line CPUs,
          then patchadd limits the maximum number of non-global zones that will be patched in
          parallel to num_proc. If the value of num_proc is greater than 1.5 times the number of on line CPUs,
          then patchadd limits the maximum number of non-global zones that will be patched in parallel
          to 1.5 times the number of on line CPUs. Note that patchadd will patch all applicable non-global
          zones on a system, the above description outlines only how patchaadd determines the
          maximum number of job slots to be used during parallel patching of non-global zones.

          An example of this in operation would be where:
          num_proc=8
          and number of on line CPU's is 4

          In this case the maximum setting for num_proc would be 6, that is the maximum number
          of zones that could be patched in parallel is 6.  If there are more than this number of non-global zones on the
          system, the first 6 will be patched in parallel, then the remaining non-global zones will be patched
          as processes finish patching the first 6 non-global zones.   Only one patch process will be used for each
          non-global zone, so if there are less than 6 non-global zones on the system, then only the number of processes
          equal to the number of non-global zones will be initiated.

          Please see comments in /etc/patch/pdo.conf for more details on setting num_proc.

I would like to thank Ed Clark and Enda O'Connor from my own team for all their work in developing and testing Zones Parallel Patching.

I would also like to thank Jon Bowman, Arindam Sarkar, and the rest of the RPE (Sustaining) Install team for all their work in getting this feature integrated into the patch utilities and delivered to production.

I would also like to thank our selected key customers who kindly Beta tested the feature for us.

I believe this feature is an important milestone in improving our customers' patching experience in a Zones environment as it addresses a long standing customer complaint on Zones patching performance.

Enjoy!

Friday May 08, 2009

Patch News Update

Solaris 10 5/09 (Update 7) and subsequent Kernel PatchIDs 

The Kernel patch included in Solaris 10 5/09 (Update 7) has now been released to SunSolve.  The PatchIDs are 139555-08 (SPARC) and 139556-08 (x86).  The rest of the patches included in Solaris 10 5/09 are either released or in the process of being released over the next week or so. 

I've updated the Solaris Kernel PatchID sequence listed in http://blogs.sun.com/patch/date/20080416 to reflect this, including the PatchIDs of the post Solaris 10 5/09 (Update 7) sustaining Kernel PatchID and the Solaris 10 Update 8 Kernel PatchID.

We will be releasing a patch bundle containing the set of patches included in Solaris 10 5/09 in the next couple of weeks.  This will be available from the "Solaris Updates Patch Bundle" section on the new look SunSolve Patch Cluster and Patch Bundle page, http://sunsolve.sun.com/show.do?target=patches/patch-access, which now includes a description of the purpose, contents, and update frequency of each patch cluster and bundle.

As always, customers need to have a valid support contract in order to download Solaris patch clusters and bundles.

New PatchFinder tool coming

We plan to release a new PatchFinder tool on SunSolve at the end of May.   This leverages the patch metadata in our release database to provide a much richer customer experience to search for patches.  Further enhancements are planned after the initial deployment.  The old PatchFinder tool will remain available for a transition period.

Zones Parallel Patching Performance Enhancement

The Zones Parallel Patching performance enhancement continues on schedule.  It has been successfully beta tested by a number of key customers who confirm a 3x performance improvement patching zones.   It is on schedule to be released in a revision of the patch utilities patch (119254 SPARC / 119255 x86) in June.

Solaris 10 "Recommended" and Sun Alert Patch Clusters

Improvements to the Solaris 10 "Recommended" and Sun Alert Patch Clusters are on schedule to be deployed before the end of June.  The improvements include an improved install_cluster script (currently available in the Solaris 10 Live Upgrade Zones Starter Patch Bundle and the Solaris 10 Updates Patch Bundles) and other process improvements designed to improve quality.

Solaris 8 Vintage Patch Support Program

The Solaris 8 Vintage Patch Support Program is now up and running.  The first Vintage Solaris 8 patches have been released.

Patches which fix SunAlerts which were issued prior to the April 1 start date of the Solaris 8 Vintage Patch Support program will be released as "normal" (i.e. non-vintage) Solaris 8 patches. 

But all other Solaris 8 patches created after April 1, including patches which fix security issues, require customers to have a Solaris 8 Vintage Patch Support contract to use them.

See http://www.sun.com/bigadmin/topics/vintagepatch/  for further information.

Thursday Apr 02, 2009

Patching Zones goes Zoom!

My colleague, Jeff Victor, has written an excellent and informative blog posting on how to improve patching performance, "patching zones goes zoom!".   Enjoy!
About

This blog is to inform customers about patching best practice, feature enhancements, and key issues. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. The Documents contained within this site may include statements about Oracle's product development plans. Many factors can materially affect these plans and the nature and timing of future product releases. Accordingly, this Information is provided to you solely for information only, is not a commitment to deliver any material code, or functionality, and SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. The development, release, and timing of any features or functionality described remains at the sole discretion of Oracle. THIS INFORMATION MAY NOT BE INCORPORATED INTO ANY CONTRACTUAL AGREEMENT WITH ORACLE OR ITS SUBSIDIARIES OR AFFILIATES. ORACLE SPECIFICALLY DISCLAIMS ANY LIABILITY WITH RESPECT TO THIS INFORMATION. ~~~~~~~~~~~~ Gerry Haskins, Director, Software Lifecycle Engineer

Search

Categories
Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today