Wednesday Sep 14, 2011

Useful Oracle Sun patch download options, including metadata & READMEs

(Updated May 14, 2013)

Here are some Oracle Sun patch download options which my colleague Don O'Malley and I believe you may find useful:

You can download an Oracle Sun patch README simply by using an URI of the following form:

https://updates.oracle.com/readme/120068-02

Just replace the PatchID in the URI above with the PatchID you are interested in.

If you are logged on to MOS, and have a valid support contract associated with your account, you can download patches using an URI of the following form for an individual patch:

https://updates.oracle.com/all_unsigned/120068-02.zip

XML metadata for a patch is available using a URI of the form:

https://updates.oracle.com/Orion/Services/search?bug=120068-02

This XML metadata contains useful information like:

  • The MD5 and SHA-1 checksums, see <digest type=...>.  Getting MD5 and SHA-1 checksums directly from MOS or this XML metadata file is the most accurate way to get checksum information. 
  • The latest PatchID in this lineage which obsoletes (supersedes) this patch revision , see <patch_replacements> - in this example 127127-11
  • What bug fixes (CRs) are delivered in the Patch - note if <fixes_bugs truncated="yes">, then the list of CRs fixed in truncated, so see the patch README for the full list of CRs
  • What access entitlement is needed to download this patch - in this example "OS" (Operating System) which means you need a support contract which covers Solaris in order to download it.  Other common access entitlements are "FMW" (Firmware) and "SW" ([other] Software), which means you need a support contract which covers Hardware or other Software respectively.  If multiple access entitlements are shown, then a support contract which covers any of them is sufficient to download the patch.
  • The Oracle BugDB Bug number reference to this patch which can be used as an alternative way to access it (see example below) - in this example 9615556
  • The Oracle BugDB Bug number reference to the README of this patch which can be used as an alternative way to access it (see example below) - in this example 12450076

Note, there are two nearly identical <patch> entries in the XML Metadata file in this example, one for 32-bit and one for 64-bit.  This is common and occurs for the vast majority of Oracle Sun patches.  Java patches are the main exception to this multiple <patch> entries rule for Oracle Sun patches, as they produce a separate 64-bit patch which will have a separate metadata file.  Where multiple <patch> entries exist in a metadata file, they always refer to one and the same patch, so either metadata entry can be parsed.  So while the "aru" <request id> references in the URIs differ for each in addition to <platform>, it's the identical patch which is downloaded in each case.

It's also possible to access a nice landing page using the Oracle BugDB Bug number reference to a patch (taken from the XML Metadata file above) to construct a URI of the form:

https://updates.oracle.com/download/9615556.html

The "View Digest" button on the landing page shows the MD4 and SHA-1 Checksums for the patch.  The landing page also facilitates viewing of the README and download of the patch.

The "HTML version" of the patch README can be accessed two ways: 

https://updates.oracle.com/Orion/Services/download?type=readme&bugfix_name=120068-02 (using the PatchID) or

https://updates.oracle.com/Orion/Services/download?type=readme&aru=12450076 (using the Oracle BugDB Bug number reference to the README taken from the XML Metadata file above)

Both of the above URIs resolve to the same patch README.  The "HTML version" of the README contains a download link for the patch at the top of the page.  It also provides links to two key resources for Oracle Sun patching information:

It's also possible to directly access the MOS Flash-based download page using a URI of the form:

https://support.oracle.com/CSP/ui/flash.html#tab=PatchHomePage(page=PatchHomePage&id=gnrgyece()),(page=PatchDetailPage&id=gnrgyece(patchId=120068-02&patchType=Patch&patchName=120068-02))

Since patchsets are named a little differently, here's a table showing the relevant URIs for key patchsets:
Patchset Name
Landing Page
README
Download
XML Metadata
Checksums (subset of XML Metadata)
Recommended OS Patchset for Solaris 10 SPARC
Landing Page README Download XML Metadata

Click "View Digest" on Landing Page or extract from XML Metadata

Recommended OS Patchset for Solaris 10 x86
Landing Page README Download XML Metadata

Click "View Digest" on Landing Page or extract from XML Metadata

Critical Patch Update (CPU) Patchset for Solaris 10 SPARC, Apr 2013
Landing Page README Download XML Metadata Checksums
Critical Patch Update (CPU) Patchset for Solaris 10 x86, Apr 2013
Landing Page README Download XML Metadata Checksums
Solaris 10 1/13 (Update 11) SPARC Patchset
Landing Page README

See Landing Page

XML Metadata Checksums
Solaris 10 1/13 (Update 10) x86 Patchset
Landing Page README See Landing Page XML Metadata Checksums
Here are some other useful links:
Sun Alerts - Knowledge article with summary of, and links to, all published Sun Alerts alerting customers to known Security (through the link to the "Critical Patch Update (CPU) and Security Alerts" page), Availability and Data Corruption issues
patchdiag.xref - metadata file listing latest available revision of all Oracle Sun 6-2 digit patches
withdrawn_patch_report - list of all Oracle Sun patches withdrawn from release in the last 12 months
weekly_patch_report - list of all Oracle Sun patches released in the last week

You can be proactively notified daily of Sun Alert issues (and other knowledge articles) by configuring the "Hot Topics" option in My Oracle Support:

   1. Go to url https://support.oracle.com/CSP/ui/flash.html
   2. Sign in
   3. Select the tab "More..." --> Settings
   4. Select "Hot Topics E-Mail" on the left
   5. Update the Hot Topics Settings
         1. Toggle the E-Mail to 'On'
         2. Ensure set "Send Every 1 Days"
         3. Select desired format (text or HTML)
         4. Set the item limit to some number larger than 5 (suggest 25)
         5. Set Service Request to "None"
         6. leave "Product Bugs Marked as Favorites" deselected
   6. Add the needed Sun Alert Filter(s) ** Note: To receive all Sun Alerts, use the following filter **
   7. Select  "Add..." (new window will pop up)
         1. Add the Product "Solaris SPARC Operating System"
         2. Add the Platform "GENERIC (All Platforms)"
         3. Check the "Knowledge Articles" box
         4. Check the "Alerts" box
         5. Select "OK" (selection window closes)
   8. Select "Save"
         1. You should be able to see your Hot Topics filter you just set up.
   9. Log out of MOS

Finally, for details on how to script access to resources such as the URIs listed above, check out:

MOS - Using 'wget' to Automate Sun Patch Downloads

I'd like to thank my colleague, Don O'Malley, for researching much of the above. 

I hope you find this helpful.

Best Wishes,

Gerry.

Tuesday Feb 15, 2011

Using My Oracle Support for Hardware Products

My colleagues in Services are running Best Practice Webinars on knowledge searching and how to find Firmware, Storage updates, and Oracle Solaris patchsets.

The next sessions for patching are this Thursday, Feb 18th, at 9AM MT (U.S. Mountain Time) and 5PM MT.  If you miss these, don't worry, there's more being hosted through to the end of April 2011.  See below.

Log into MOS and see https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&doctype=SYSTEMDOC&id=1282218.1 for details.

Here's the blurb:

Our new seminar series titled "Using My Oracle Support for Hardware Products" covers a variety of topics focused on using My Oracle Support to service your hardware products. The seminar series includes five Advisor Webcast topics for customers using Solaris or Sun Hardware. The topics are repeated throughout the schedule. The schedule  runs from 8 February through 28 April.  To see the schedule and to register for a session, please review Advisor Webcasts: My Oracle Support - Best Practices for Hardware Customers

    \* Best practice on knowledge searching and how to find Firmware, Storage updates, and Oracle Solaris patch sets
    \* Hardware Asset and Automated Service Request (ASR) Management
    \* Automatic Service Request from end to end
    \* Proactive Hardware Service Portal
    \* My Oracle Support - Hardware Service Request

Oracle support experts will be on hand to present these topics and answer questions about best practices in using My Oracle Support for servicing your supported hardware products.

Monday Jan 10, 2011

Searching for Oracle Sun patches in My Oracle Support (MOS)

My colleague, Mike Brown, has published this knowledge article which you may wish to bookmark, pointing to frequently accessed Oracle Solaris patch deliverables, including the Recommended patch clusters, quarterly Critical Patch Updates (CPUs), Solaris Update Patch Bundles, patch utilitiy patches, patchdiag.xref, the checksums file, and the LU Zones Starter Patch Bundle.

Also, here's a cut-and-paste of my response to some comments posted regarding finding Oracle Sun patches on My Oracle Support (MOS) which I hope you'll find useful.

To get the Solaris patch clusters and patch bundles, use the "Product or Family (Advanced Search)" option on the "Patches & Updates" tab. Select:

  • Product is Solaris Operating System
  • Release is Solaris 10 Operating System
  • select "Type" instead of "Platform" and Type is "Patchset"

...and it'll return all Solaris 10 patch clusters and patch bundles. This includes the Solaris OS Recommended Patch Clusters, the Solaris Update Patch Bundles, the Solaris OS Critical Patch Updates (CPUs), Live Upgrade (LU) Zones Patch Bundle, etc.

You can add further search filters, e.g. Platform is Oracle Solaris on SPARC (64-bit), to further refine the results.

Using "Platform" is useful to eliminate the double-entries for 32-bit and 64-bit. These dual returns are a pet peeve of mine and I'm continuing to work with the MOS team to get this "fixed" in a future release. They are an historical artifact from Oracle DB platform porting and are not relevant to the Solaris OS.

Note that the alternative option for "Type" is "Patch", which can be used to search for individual patches.

Please note that you can see all revisions of a patch by searching with the format 119254-% . The "-" (dash) is required in the current version of MOS.

You can also search for words included in the Patch Synopsis by using the Description field. For example:

  • Product is Solaris Operating System
  • Release is Solaris 10 Operating System
  • Type is Patch
  • Platform is Oracle Solaris on SPARC (64-bit)
  • Description contains patch utilities

In the example above, the Description option searches for the phrase "patch utilities" in the Synopsis line of patches. This returns the Solaris 10 SPARC patch utility patches.

Since the synopsis line of patches is free format, some guesswork is involved in searching using this method. For example "patch utility" returns nothing. "IP" returns more than just TCP/IP related patches.

Alternatively, you can use "Classification", which can be set to "Security" to return Security patches.

Click on the "Updated" column in the search returns to get these listed from earliest to latest or vice versa.

Firmware updates are also available from My Oracle Support.

  • Click on "Produce or Family (Advanced Search)".
  • Select the hardware product you are interested in.   For example, type "x6" and select, Product is Sun Blade X6440 Server 
    Module.
  • Select the Release(s) you are interested in, e.g. Release is X6440 SW 2.2.0
  • Click Search.

My understanding is that MOS currently limits search results to 100 entries in the current version and again I'm discussing "fixing" this with the MOS team in a later release.

Searches can be edited and saved for re-use at a later date.  MOS also remembers selections you've made in previous sessions which is a useful feature.

A "Classification" of "Other Recommended" rather logically will give other non-security recommended patches included in the Solaris OS Recommended Patch Cluster. (In MOS terminology, "Security" and "Other Recommended" together are equivalent of the old Sun "Recommended" term.) But if you want to know exactly what's in the Solaris OS Recommended Patch Cluster, it's easier to simply look at the patch  list in the Cluster README.

As discussed in the http://blogs.sun.com/patch/entry/solaris_10_recommended_patching_strategy which I published last week, we're really trying to encourage customers to move away from selecting unique patch combinations and to instead use the Solaris OS patch clusters and patch bundles as the core of your patching strategy.

But there is still occasionally the need to search for individual patches to address specific issues.

If you are looking for individual patches to address a specific CR, then use "Patch Name, Number, or Sun CR ID" search option instead of "Product or Family (Advanced Search)". For example, enter Sun CR ID 6927931 and patch 119254-78 is returned which is the patch in which the CR is fixed. A CR which was fixed a long time ago, e.g. 6486471, will return all patch revisions which contain the fix, so you can decide whether you want to take the latest patch revision which fixes it or the earliest.

As I say, I'm continuing to work with the MOS team to enhance the customer experience further, but I hope you find the above tips helpful.

A colleague in MOS has kindly forwarded a link to a tutorial on the PowerView feature in MOS which you may find useful.

Best Wishes,

Gerry.

Thursday Aug 05, 2010

Updated Customer Patching Presentation (and other stuff)

I've updated my Patching Presentation for customers, see http://blogs.sun.com/patch/entry/patch_presentation_for_customers

I hope you find it useful.

Also, I forgot to blog about an enhancement we made in March 2010 to the Solaris Update Patch Bundles.  The Solaris Update Patch Bundles now add a line to /etc/release when they are installed to make it easier to determine that they've been applied - i.e. that all pre-existing packages on the system have been patched up to the same software level as the corresponding Solaris Update.

On a related note, Oracle 11gR2 requires customers to have Solaris 10 10/08 (Update 6) installed.  From Version 11.2.0.3 it will accept the corresponding Solaris Update Patch Bundle as being sufficient to meet this requirement.  The modification of /etc/release by the Solaris Update Patch Bundle is partially to help support this.

Thursday May 20, 2010

Merging the Solaris Recommended and Sun Alert Patch Clusters

The Solaris "Recommended" and Sun Alert Patch Clusters have been merged (June 4th 2010). 

The merged clusters are called the "Recommended OS Cluster Solaris <release> <architecture>", for example "Recommended OS Cluster Solaris 10 SPARC". 

The old "Recommended" and Sun Alert Patch Clusters only ever contained Solaris OS patches (with rare exceptions), so we've added "OS" to the new merged cluster name to make this a little clearer.

The merged Recommended OS Clusters have the same access entitlement as the old clusters - namely, you need a support contract which covers Solaris to access them.

The old "Recommended" patch cluster contains the latest revision of Solaris OS patches which fix Sun Alert issues (i.e. Security, Data Corruption, or System Availability issues).  That is, the top-of-tree patches which fix Sun Alert issues.

The Sun Alert patch cluster contains the minimum revision of Solaris OS patches which fix Sun Alert issues.  Thus, the Sun Alert patch cluster provides the minimum amount of change required to get all available Solaris OS fixes for Security, Data Corruption, and System Availability issues.

The contents for the two clusters are very similar, which causes unnecessary confusion as to which one to use.  When the Sun Alert Cluster was released several years ago, it should have replaced the older "Recommended" Cluster, and this merging of the Clusters is to correct that omission.

The inclusion criteria for the Sun Alert cluster is more logically correct, as in the Recommended Cluster there's no more value in adding the latest revision of a patch whose earlier revision provided a fix to a Sun Alert issue than in adding any other random patch.  Many folks assume "latest is greatest", and Oracle Sun wouldn't release a patch unless it is important, but this is slightly simplistic.  Change implies risk, and as many patches address issues which are only seen in very specific configurations, and while Oracle Sun patches are thoroughly tested prior to release, there is little advantage in taking more change than is necessary in minor maintenance windows or reactive patching situations.  Therefore, providing a minimal patch cluster which provides all available fixes for Solaris OS Sun Alert issues for use in minor maintenance windows makes sense.

The old "Recommended" Clusters were often updated several time a week, simply because a later revision of a patch whose earlier revision fixed a Sun Alert issue was released, even though the later revision didn't fix any additional Sun Alert issues.  Since the "Recommended" flag on SunSolve and in the patchdiag.xref metadata file matches the contents of the old "Recommended" Cluster, we were releasing many more patches which were flagged as "Recommended" than customers really needed to apply.

After the merge, new patches added to the Recommended OS Cluster and hence the "Recommended" flag on SunSolve and in the patchdiag.xref metadata file will be the specific revision of patches which address Sun Alert issues.  Only when an obsoleting patch provides a new fix to a Sun Alert issue will it be included and the obsolete patch removed.  The merged Recommended OS Clusters will update on the same cadence as the old Sun Alert clusters, which is typically about once a week for Solaris 10 (5.5 times a month, on average).  We will continue to update the merged Recommended OS Cluster whenever a patch matching the inclusion criteria is released.

To avoid the potential confusion which may be caused if we were to remove the "Recommended" flag from any patches, we will take the "Recommended" Cluster at the beginning of June 2010 as the basis for the merged cluster and then apply the Sun Alert Cluster inclusion criteria going forward.

The merged Recommended OS Cluster was initially released on June 4th, 2010.  The download link (target) file name of the merged cluster will be the same as the old "Recommended" Cluster, e.g. 10_Recommended.zip, to minimize the changes users need to make to automated download scripts.

Customers who have traditionally downloaded the Sun Alert cluster will need to update download scripts to use the merged cluster file download names as the old Sun Alert cluster are no longer available.

In major maintenance windows, the Best Practice recommendation is to upgrade to the latest available Solaris Update release or at least to apply the equivalent Solaris Update Patch Bundle available from the patch cluster download page.  In both cases, the latest Recommended OS Cluster should also be applied as it will contain any additional Solaris OS Security, Data Corruption, and System Availability fixes released since the Solaris Update contents were finalized.  Solaris Updates are intensely tested, and hence this strategy provides a well tested, stable, and feature rich baseline for production systems.  In between major maintenance windows, the Best Practice recommendation is to try to keep as up to date as possible with the contents of the merged Recommended OS Cluster during minor maintenance windows.

Let's look at an example, to make the rationale for the change clearer: 

In the old model, if a security vulnerability in /usr/bin/ls is fixed in patch 123456-03, then both the old Recommended and Sun Alert clusters will initially include it.  If code interdependencies caused by subsequent code putbacks - e.g. the major Trusted Solaris Extensions feature - result in the contents of the "/usr/bin/ls" patch 123456-07 being accumulated into a feature Kernel patch associated with a Solaris 10 Update, e.g. 234567-14, then the old "Recommended" Cluster would include 234567-14 instead of 123456-03, even if 234567-14 contained no additional fixes for Sun Alert issues (i.e. Security, Data Corruption, or System Availability issues) compared to 123456-03.  The "Recommended" flag on SunSolve, in patchdiag.xref, and elsewhere would be updated every time a patch revision obsoletes the original patch, even though these later patch revisions contain no additional fixes to Sun Alert issues.  This can lead to customers who try to stay up to date with "Recommended" patches patching more content and potentially more often than is really necessary.  In contrast, 123456-03 would remain in the Sun Alert cluster for as long as no additional fixes for Sun Alert issues are contained in obsoleting patches.

In the new merged Recommended OS patch cluster model, while the starting point will be the old "Recommended" Cluster as of the start of June 2010 (to avoid dropping the "Recommended" from any patches, which might cause confusion), further changes to the cluster will follow the old Sun Alert cluster inclusion criteria - that is, the merged Recommended OS patch cluster contents and corresponding Recommended flag in SunSolve and patchdiag.xref will only be updated if a new patch delivers a new fix for a Sun Alert issue.   This means that only patches which we really recommend will be included in the Recommended OS patch cluster and flagged as Recommended in SunSolve and patchdiag.xref.  Since the rate of change will be less, it'll be easier for customers to see what's really recommended and allow more informed decisions regarding when to apply such patches.

Please note that this change has nothing whatsoever to do with the integration into Oracle.  This is an enhancement I've been looking to do for some time to avoid the confusion caused by having two very similar patch clusters and a corresponding "Recommended" flag which was updated much more frequently than was necessary.

My team has been working with known consumers of the "Recommended" patch flag such as TLP, Ops Center, 'smpatch', Update Manager, SRAS, EIS, and 'pca' to ensure that the transition goes smoothly.  

For example, TLP and 'pca' consume the patchdiag.xref file which up to now typically only contained entries for top-of-tree (latest) patch revisions.  From June 4th 2010, patchdiag.xref will contain whatever revision of a patch is flagged as "Recommended" as well as the top-of-tree patch revision.  Hence, a single base PatchID, e.g. 123456, may have two entries in the file, e.g. 123456-03 marked "R" for Recommended and "O" for Obsolete and 123456-08 which is the latest revision of that patch but which won't carry the "R" flag as it contains no additional Sun Alert fixes over rev-03.  

From my discussion with Martin Paul, author of 'pca', my understanding is that initially, he plans to propagate the "R" flag forward to the latest patch revision in his 'pca' metadata as currently 'pca' only handles the latest revision of patches, but he'll look at some stage in the future to leverage the more precise "Recommended" flag data we'll be providing with this change.

Friday Mar 26, 2010

Interesting article on Sun's collaboration with Intel to improve performance

This isn't specifically to do with patching, but here's an interesting article from Intel detailing some of the work Intel and Sun have been doing to maximize Solaris performance on "Nehalem" Xeon 5500 series systems, enabling Solaris to leverage the advanced features of the chip set.

Please note that this does not in any way imply anything negative about SPARC.  Solaris is committed to supporting both x86/x64 and SPARC architectures to give you choice.

Monday Sep 07, 2009

IBM's X-Force Report Praises Sun for Fast Fixes

Internetnews.com has an interesting article on IBM's X-Force Report which praises Sun for fast fixes and being best for patching the highest percentage of reported security vulnerabilities:  http://www.internetnews.com/security/article.php/3836436/IBMs+XForce+Report+Praises+Sun+for+Fast+Fixes.htm

Friday Aug 14, 2009

Improvements to Solaris 10 Recommended and Sun Alert Patch Clusters released

My colleague, Ed Clark, has made significant improvements to the Solaris 10 Recommended and Sun Alert patch clusters.  These improvements have just been released and are in the current clusters available to contract customers from the Patch Cluster & Patch Bundle Downloads on SunSolve.

Ed's improvements include:

  • Filtering out "false negatives" from the patch utility return codes, so that if the cluster install script returns "1", you know you've got a real problem which needs investigating.   As you may know, the Solaris patch utility, 'patchadd', can return errors for some acceptable situations - for example, if the patch is already applied to the system, or a later revision of the patch or a patch which obsoletes it is already applied to the system, or none of the packages in the patch are on the target system (e.g. because a reduced Install Metacluster was used to install it or the system has been security hardened by package removal), etc.   Such conditions are acceptable "errors" which do not usually require further investigation by the user.  By filtering these conditions out, if the 'installcluster' script returns "1", you know it isn't because of one of these acceptable "errors", and therefore you need to look at the logfiles to find out what's gone wrong.  For further information, please see the cluster README and Analyzing a patchadd or patchrm Failure in the Solaris OS.
  • The new 'installcluster' script will exit as soon as it encounters an unexpected failure - i.e. not one of the acceptable "errors" mentioned above.  This prevents potentially compounding issues by attempting to apply further patches.
  • The new 'installcluster' script includes context intelligence for patching operations.   It informs the user when zones need to be halted, and it provides phased installation to handle patches which absolutely require an immediate reboot before further patches can be applied.  Such interim reboots are only needed when patching a live boot environment on a system below Kernel patch 118833-36 (SPARC) / 118855-36 (x86) and well as the earlier interim reboot required on x86 related to 'libc.so' patches and Kernel patch 118844-14.  On systems below these patch levels, the 'installcluster' will stop at the appropriate point when patching the live boot environment, and inform the user to reboot and re-invoke the 'installcluster' script.  (In the old cluster install script, it simply tried to carry on blindly past such interim reboots, spewing out error messages, although code in the relevant patches prevented any harm from being done).  These interim reboots, when required, are dealt with relatively early in the cluster install sequence so that once completed, the Sys Admin can leave the rest of the installation to finish unattended and move onto other systems.
  • The new 'installcluster' script provides better integration with Solaris Live Upgrade as the user can now specify the Live Upgrade alternate boot environment to patch by name.
  • The new 'installcluster' script performs space checking prior to installing each patch, and will halt if it believes there is insufficient space to complete the installation successfully.  For example, this helps avoid non-global zones getting out of sync regarding patch levels with respect to the global zone.  This is an important enhancement as running out of space during patching can potentially leave the system in an inconsistent state and is to be avoided.  Even removing a patch requires space, so immediate removal of a patch which has failed to apply correctly due to space issues should be avoided until sufficient space is freed up and potential issues caused by its partial installation investigated - for example, was the undo.Z file successfully created to enable backout ? (Tip: It may be better to retry the patch installation once space has been freed up rather than patch removal in such circumstances.  Contact Sun Support for instructions if you encounter such issues.).   The space checking enhancements in the 'installcluster' script are designed to prevent such problems occurring.
  • The messages and log files produced by the 'installcluster' script are clear and well structured.  For example, a "failed" log is created if a patch fails to apply.  See the Cluster README for further information.
  • The 'patch_order' places patches in an optimal order for installation to avoid known issues - for example, the patch utilities patches are installed as early in the sequence as possible to avoid hitting patch installation bugs which are fixed in the patch utility patches, and the Kernel patch procedural script override patch, 125555 (SPARC) / 125556 (x86), is ordered prior to 137137-09 (SPARC) / 137138-09 (x86) to resolve some known issues.  When patching an alternate boot environment (which is recommended), a small sub-set of pre-requisite patches, primarily the patch utility patches, need to be applied to the live boot environment to ensure correct patching operation.  The 'installcluster' script will check for these pre-requisite patches are halt installation if they are not present, advising the user of the 'installcluster' script option to use to install these pre-requisite patches.   Further patches may need to be installed on the live boot environment to support Live Upgrade.  See the cluster README for further information.
  • The patches have been moved to a 'patches' sub-directory, to de-clutter the top level directory of the unzipped cluster.
  • Please see the cluster README file for further information.  Customers should read the cluster README file and look at the Special Install Instructions in the patches within the cluster prior to installation.

I really want to thank Ed Clark for the enormous amount of thought and effort he has put into improving the cluster installation experience.   The work he's done on the Solaris 10 Recommended and Sun Alert patch cluster is a continuation of his previous work on the Solaris Update Patch Bundles and the Solaris 10 Live Upgrade Zones Starter Patch Bundle.  Nice work, Ed!

While the 'installcluster' script is copyrighted, I am happy for customers to use it, and the 'patch_order' file, as a starting point for their own customized patch bundles, so long as it is for their own use and is not to be given to a 3rd party or used for commercial gain (e.g. by a 3rd party maintainer or 3rd party commercial automation tool).

We have also made significant improvements to the back end processes to ensure higher and more consistent cluster quality. 

Originally, the clusters were created by the Patch Operations and Distribution (POD) team after patch release.  The POD Cluster QA process left a lot to be desired, resulting in inconsistent cluster quality.   To plug this gap, my Patch System Test team have been testing the clusters for several years, but the old process only allowed us to test them in parallel with their release, which meant that we found issues at the same time that early downloaders of the cluster encountered them.  Although we ensured such issues were fixed as quickly as possible, it still obviously compromised our customers' experience.

In the new process, the clusters are routed to Patch System Test (PST) prior to release.  PST run a transformation script on them to optimize the patch installation order, etc.  The clusters will only be released once they have passed PST testing.  This should ensure higher and more consistent quality for customers.  Work is continuing to move the entire patch cluster generation process to PST, although these future backend enhancements in this regard should be invisible to customers.

Thursday Jun 25, 2009

Training for Solaris 10 Patching

Sun Learning Services are in the process of creating a number of patch related training lessons.

They've launched a blog, which contains the initial introductory videos.

Future lessons will be much more detailed, concentrating for example on Live Upgrade.   These lessons will be available on the Sun Open Learning Center (SOLC) website: https://learning.sun.com/solc/smartstart.

Friday Jun 19, 2009

Zones Parallel Patching versus Update On Attach: When to use which one ?

The Zones Parallel Patching enhancement for the Solaris 10 patch utilities was released this week giving customers a choice of how to improve zones patching performance.

In the Zones "Update On Attach" section of a previous blog posting, I mentioned that the Zones "Update On Attach" feature could also be used to improve Zones patching perfomance.

Zones Parallel Patching is a true patching solution utilizing the 'patchadd' utility.  

Whereas Zones "Update On Attach" uses zones functionality similar to that used during zones creation to provide a pseudo-patching solution that does not utilize 'patchadd'. 

So which one to choose ?

Let's look at the two options in more detail:

Zones Parallel Patching

Zones Parallel Patching is an enhancement to the standard Solaris 10 patch utilities and is delivered in the patch utilities patch, 119254-66 (SPARC) and 119255-66 (x86).

Simply install this patch, set the maximum number of non-global zones to be patched in parallel in the config file /etc/patch/pdo.conf, and away you go.

It works for all Solaris 10 systems. 

It also works well in conjunction with higher level patch automation tools such as xVM Ops Center. 

It can dramatically improve zones patching performance by patching non-global zones in parallel.  The global zone is still patched first.

While the performance gain is dependent on a number of factors, including the number of non-global zones, the number of on-line CPUs, the speed of the system, the I/O configuration of the system, etc., a performance gain of ca. 300% can typically be expected for patching the non-global zones - e.g. On a T2000 with 5 sparse root non-global zones.

See my previous Zones Parallel Patching blog entry for further information.

Since it's a pure enhancement to 'patchadd', it's normal 'patchadd' functionality.  You can subsequently remove patches using 'patchrm', etc.  Nothing has changed except that it's now much faster to patch non global Zones with Zones Parallel Patching invoked.

Zones "Update On Attach"

The primary purpose of Zones "Update on Attach" is Zones migration from one server to another.  

For example, a database instance in a non-global zone hosted on a server has grown to the extent that the Sys Admin wants to transfer it to a better spec'd server which can better handle the workload.   The Sys Admin can detach it from the old server (e.g. a Sun4u) and reattach it to the new server (e.g. a Sun4v) using Zones "Update On Attach".   This will bring the OS Software level on the non-global zone up to the same level as the new server's global zone.

Zones "Update On Attach" can certainly be used for patching but there are limitations you need to be aware of as outlined below.

For example, detach the non-global zones from a system, apply a bunch of patches to the global zone, reattach the non-global zones using "Update On Attach" and viola, the non-global zones will be brought up to the same software level as the global zone (for OS type packages), effectively patching the non-global zones without using 'patchadd' at all.   This is typically even faster than using Zones Parallel Patching.  But there are limitations to this approach which users must be aware of (see below).

My senior engineer, Enda O'Connor, has just published an interesting article on The Zones Update on Attach Feature and Patching in the Solaris 10 OS

Zones "Update On Attach" limitations as a patching aid

Zones "Update On Attach" only works for packages which are SUNW_PKG_ALLZONES=true - i.e. typically OS level packages, and not application packages.

So when to use Zones Parallel Patching in 'patchadd' and when to use Zones "Update On Attach" ?

Here's what my senior engineer, Enda O'Connor, says:

"The Zones Update on Attach Feature and Patching in the Solaris 10 OS document may help customers understand how the technology works, applying a cluster via patching and via zones Update On Attach is not quite the same really.

It really depends on the patches being applied, i.e. applying a firefox patch via Update On Attach would not work if you wanted it to apply to the global zone and all non-global zones as well.

One has to understand how Update On Attach works and then apply that to the list of patches to see if it gets them to a desirable state.

There is no black or white answer here.

I'd recommend Zones Parallel Patching using 'patchadd' as it has a known outcome all the time, whereas Update On Attach makes it's own internal determination based on a number of things, that can vary from system to system ( e.g. inherited directories ).

But if time to patch is critical then if the customer does proper testing to validate things, and are happy with the results, then by all means use Update On Attach.

But using Update On Attach without:

1. Understanding how it determines what packages to update

2. Not inspecting the patches being applied.

...will most likely lead to grief at some point."

And my other senior engineer, Ed Clark, says:

"In terms of giving guidance on which technology to use, there are a number of considerations -- two of these considerations are:

1. Using Update On Attach to update sparse zones can require significantly more disk storage space than would be needed by applying patches with 'patchadd' (3-4 times as much space would not be uncommon i think), due to Update On Attach copying fully populated global zone 'undo' files into the non-global zones, as opposed to having patchadd build sparsely populated 'undo' files in the non-global zones.

2. If a customer is really concerned about the ability to back out patches reliably, then 'patchadd' is a lower risk option than Update On Attach -- 'patchrm' of a patch from a non-global zone that has a copy of the global zones 'undo' pkg data (as is the case after Update On Attach) may potentially have unexpected side effects." [although we have yet to see any actual cases of negative results from this.]

Conclusion

In general, we recommend using the Zones Parallel Patching enhancement in the patch utilities rather than the Zones "Update On Attach" feature as Zones Parallel Patching is standard patching functionality, only faster, whereas Zones "Update On Attach" is really designed for migrating zones from one server as another and was not primarily designed to speed up patching.  

Because Zones "Update On Attach" uses Zones functionality similar to the zone creation functionality, rather than 'patchadd' functionality, limitations exist on what will be patched (typically the OS but not applications) and there's the potential for anomalies around things like the "undo" files which would be used by 'patchrm' if patches applied using Zones "Update On Attach" were subsequently removed from the non-global zones using 'patchrm' (although we have yet to see any actual cases of serious issues resulting from this).

So in patching situations where time is absolutely critical, Zones "Update On Attach" may provide a good option, as long as it's well tested in the customer environment prior to deployment on production systems.

Remember too, Live Upgrade is also your friend in such situations, enabling you to patch an inactive boot environment while the system is still in production.   So a combination of Live Upgrade and Zones Parallel Patching would be ideal.

I hope you find this helpful!

Best Wishes,

Gerry.

Thursday Jun 18, 2009

Solaris 10 5/09 (Update 7) patch bundle now available!

The Solaris 10 5/09 (Update 7) patch bundle is now available for download from the SunSolve Patch Cluster & Patch Bundle Download Page.  Click on the "Solaris Update Patch Bundles" link.

As with previous patch bundles, it contains the patches which are included in the corresponding Solaris Update, in this case Solaris 10 5/09 (Update 7).

This is useful for Sys Admins who wish to bring all their systems up to the same patch level as the Solaris Update without wanting to upgrade to the release - for example, due to change control policy restrictions in their organizations.

See previous blog entries for previous Solaris Update patch bundles for further information.

Friday May 08, 2009

How to analyze patch failures and other useful info

My colleague, Enda O'Connor, has published 3 more patching articles on Big Admin which I hope you will find useful:

I think the first article is particularly useful to help customers and support engineers understand what data to gather to enable analysis of a patching issue.  Even if you are not able to analysis the issue yourself, providing this data to Sun Support when you log a call will help speed up the issue analysis by Sun.

Tuesday Mar 10, 2009

Improvements to Patch Cluster pages on SunSolve

My team and I have been working with the SunSolve team on improvements to the Patch Cluster pages on SunSolve.  These improvements went live on April 20, 2009.

The old "Recommended Patch Clusters" and "Recommended and Security Patches" pages have been combined into a single Patch Cluster & Patch Bundle Downloads page.

A Notice Board section at the top of the page will be used to alert customers to current issues.

Click on the cluster headings to see a brief description of the purpose of the cluster, with links to view the cluster README as well as a download link.   The date the cluster was last updated and the size of the cluster are also shown.

No change has been made to the underlying cluster file names, so scripts using 'wget' to access the patch clusters should be unaffected.

This is part of an ongoing effort to improve our patch presentation to customers.

As before, customers need a valid support contract in order to be able to access patch clusters. 

If you are not registered in Member Support Center, simply log into SunSolve and associated one or more support contracts with your Sun Online Account using the "Change Contract" option in the top right hand menu.

If you are registered in Member Support Center, your contracts will be automatically associated with your account (and the "Change Contract" option will not be shown when you log into SunSolve).

Thursday Mar 05, 2009

Need unzip fix available in patch utilities patch to unzip Solaris 10 Clusters

A fix to the unzip utility is available in recent patch utility patch revisions.  This fix is required in order to be able to successfully unzip very large files such as the Solaris 10 Recommended and Sun Alert Patch Clusters.

Please download the latest revision of the patch utilities patch first and install it, before attempting to unzip the Solaris 10 Recommended or Sun Alert Patch Clusters.

The fix was incorporated in the putback to CRs 6344676 and 6464056.

The following are the earliest revisions of the patch utilities containing the fix:

  • Solaris 10 SPARC: 119254-46 or above
  • Solaris 10 x86:        119255-46 or above
  • Solaris 9 SPARC:   112951-14 or above
  • Solaris 9 x86:          114194-11 or above
  • Solaris 8 SPARC:   108987-19 or above
  • Solaris 8 x86:          108988-19 or above

Without the fix to unzip provided by the above patches, the following error will be seen when attempting to unzip the Solaris 10 Patch Clusters:

# unzip -q 10_Recommended.zip

note:  didn't find end-of-central-dir signature at end of central dir.
  (please check that you have transferred or created the zipfile in the
  appropriate BINARY mode and that you have compiled UnZip properly) 

In addition, do not unzip Solaris patch clusters on Windows. Solaris patch clusters, and solaris patches more generally, can contain case-sensitive file names. Consequently clusters and patches must be unzipped on a case-sensitive filesystem (corruption can occur if unzipping on filesystems that are not case-sensitive). 

The above information is now published in document 1020109.1 available from MOS.

Thursday Mar 06, 2008

Sun Alert Notifications

You can sign up to receive a weekly notification advising of new and updated SunAlerts .

Sun Alerts inform customers of the most critical issues affecting Sun's hardware and software.

They cover Security, Data Corruption, and System Availability issues.

Customers with a valid support contract will be able to access all Sun Alerts and patches which fix Sun Alert issues, including the Sun Alert patch clusters available on SunSolve which contain all Solaris OS patches which address Sun Alert issues.

Customer without a valid support contract will be able to access Sun Alerts and Patches only for Security related issues when they log onto SunSolve.

Tuesday Jan 22, 2008

Patch Automation Tools

First of all, let me say that my personnel experience of Sun's patch automation tools is limited.  I work upstream from the SysNet and Services groups who produce most of Sun's patch automation tools, so I and my team mostly patch from first principles using the basic Solaris patch utilities, patchadd and patchrm.

My team does have some experience of working with some of the patch automation tools.  I've supplemented this with information from SysNet and Services folk.

Sun Connection 1.1.1 Satellite (a.k.a. UCE) and xVM Ops Center 1.0

The official Sun patching tool of choice is now xVM Ops Center, which contains an enhanced version of Sun Connection 1.1.1 Satellite Edition.

Sun acquired Aduva a couple of years ago.  Aduva has a track record of providing patch and update automation tools for multiple Operating Systems.

The next-generation Aduva-based tools are coming on stream. Sun Connection 1.1.1 Satellite is based on Aduva.  Note, "Satellite" has a completely different back end to the Sun Update Connection Hosted edition and Solaris Update Manager, which are based on PatchPro (see below).

I'm hearing good things about the Satellite.  I understand that its initial target market is customers with 50+ systems.

Sun Connection 1.1.1 Satellite Edition is based on Aduva Onstage and Update Connection Enterprise.   A central server (Satellite) at the customer site is used to analyze and update all attached client systems in a fully automated manner.  It builds upon a central Knowledge Database fed by Sun.  It covers the provisioning of patches, packages, config files and scripts.  It is available to customers who pay for it.

Sun Connection 1.1.1 Satellite provides a solution for customers primarily interested in patch and package provisioning.  There is a 10 minute demo introducing you to some of the key features of Sun Connection Satellite at http://frsun.downloads.edgesuite.net/sun/07D01031/SunConnectSatellite.html, or alternatively there is a more detailed 32 minute demo at http://frsun.downloads.edgesuite.net/sun/07D01032/SunConnect.html

Sun Connection Satellite is a component of the xVM Ops Center

xVM Ops Center is a merge of Sun Connection and N1SM.  Here's a BigAdmin article on Patching Solaris using Sun xVM Ops Center.  The monthly patch baselines referred to are the patch sets in the monthly EIS DVD release (see below).

For further information, please see the Sun Connection hub's Product Tour page on BigAdmin.

EIS

EIS stands for Enterprise Installation Standards and originated from Sun field personnel wanting to develop best practice installation standards for systems installed on customer sites.

EIS has proved extremely popular with Sun field personnel and approved partners.  It's widespread adoption was due to it successfully addressing a real need.  I view it's widespread adoption among field personnel and OEMs as proof positive of its efficacy.

The EIS patch baseline goes through QA testing prior to release.  The images installed by Sun's manufacturers on servers are also based upon the EIS patch baseline.  Additional testing by Sun's manufacturers plus feedback from the EIS user community raises confidence in the EIS patch baseline content further.  Since many system installations world-wide use the EIS methodology, any inherent problems will quickly appear and can be dealt with.  In the event of there being issues with the EIS patch baseline recommendations are communicated to the EIS community.

This same EIS set of patches which are considered by Sun Field Engineers as best practice to install on a new system, can also be used to patch existing systems to the same patch level.  The EIS set of patches is based on the Recommended Patch Cluster for the Solaris OS with additional patches included by the Field Engineers for additional products and to address irritating issues which do not meet the criteria for inclusion in the Recommended Patch Cluster.

The EIS patch baseline covers Solaris and other products such as SunCluster, SunVTS, SSP, SMS, QFS, SAM-FS, and includes patches which provide firmware updates.

EIS has traditionally only been available via Sun Services personnel but is now available direct to customers via Sun Connection Satellite.  This provides a good option to customers to patch to a defined and tested patch baseline.  See the Sun Connection blog for further information.

pca

pca is a popular 3rd party tool developed by Martin Paul.  I've only ever heard positive feedback about pca.

pca is available from http://www.par.univie.ac.at/solaris/pca/

To try out pca, just run this on any Solaris machine:

  $ wget http://www.par.univie.ac.at/solaris/pca/pca
  $ chmod +x pca
  $ ./pca

pca is a good solution for customers interested in a simple, easy to use, patch automation tool.

smpatch, Update Manager, and Sun Connection Hosted Edition

smpatch is a command line tool and part of Solaris.   It allows customers to analyze and update Solaris with current patches.  For customers without a valid support contract,  only security and driver patches are available.  For customers with a valid support contract, all patches are available.

updatemanager is a GUI wrapper around smpatch and is also part of Solaris.  It can be used to see what patches/updates are available and to easily select the patches, which the customer wants to install. Again, for customers without a valid support contract,  only security and driver patches are available.  For customers with a valid support contract,  all patches are available.

Sun Connection - Hosted Edition is the internet portal version of  updatemanager.  The customer can register all their servers and  can schedule and review the installation of patches from a central portal.  This is only available to customers who pay for it.

The above tools rely on the "PatchPro" analysis engine to recommend patches to customers.

PatchPro utilizes what are called "Realizations".  These are listed in the patchinfo file in the top directory of a patch.  This allows the patch developer to associate a patch with one or more "Realization Detectors", which determines whether or not it is appropriate to apply a patch to a particular customer environment.  For example, a Realization Detector might only recommend a particular patch if the target system utilizes a particular piece of hardware or software, or if a particular service is enabled.  This provides fine-grained control on patch recommendations.

The vast majority of Realizations simply associate a patch to packages installed on the target system, in the same way patchadd determines whether or not to apply a patch.  That is, if the package name, package version, and platform architecture in the pkginfo file(s) in the patch match at least one package name, package version, and platform architecture on the target system, the patch can be applied, else not.

Errors in writing Realization Detectors cause patch automation tools which utilize the PatchPro analysis engine to occasionally recommend inappropriate patches.  This has impacted the reliability of PatchPro based tools.

Work is underway to write a generic realization detector to match patches to packages.  This will save patch creators from writing their own realization detectors for the common case, simplifying the process and reducing error opportunity.  Patch creators will still be able to write specific Realization Detectors where necessary.

See Instructions for Getting Started with Sun Connection's Update Manager and Sun Hosted solutions and Patch Manager 2.0 FAQs for further information.

TLP

TLP stands for Traffic Light Patching, and is another tool which was developed by Sun Service folk for Sun Service folk to address the need for Patch Automation.

TLP is not directly available to customers.  It's used by Sun Service personnel to determine the appropriate patches to be installed on a customer's system, including things like firmware patches.

TLP has a modular design.   It utilizes the concept of a "baseline" of patches chosen by the user, from the Recommended Patch Set, to the EIS patch set, to a user defined set of patches.  TLP allows a number of different patch analysis engines to be used to determine which patches from the "baseline" to apply to a particular target system.

TLP is popular with customers who use it, as it's reliable and works well. 

TLP was End-Of-Lifed (EOL'd) in September of 2006 and reached End-Of-Service-Life (EOSL) in December 2007.  However, a number of customers have been given extensions on TLP support for transition purposes. 

Sun Services Patch Recommendations 

Most European countries provide a service where customers can submit Explorer logs and the local Sun office provides back a patch bundle.  These services may use SRAS and TLP in the background.

Please contact your local Sun Services office for further details.

I believe the plan will be to consolidate these services into a consistent official worldwide service.

About

This blog is to inform customers about patching best practice, feature enhancements, and key issues. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. The Documents contained within this site may include statements about Oracle's product development plans. Many factors can materially affect these plans and the nature and timing of future product releases. Accordingly, this Information is provided to you solely for information only, is not a commitment to deliver any material code, or functionality, and SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. The development, release, and timing of any features or functionality described remains at the sole discretion of Oracle. THIS INFORMATION MAY NOT BE INCORPORATED INTO ANY CONTRACTUAL AGREEMENT WITH ORACLE OR ITS SUBSIDIARIES OR AFFILIATES. ORACLE SPECIFICALLY DISCLAIMS ANY LIABILITY WITH RESPECT TO THIS INFORMATION. ~~~~~~~~~~~~ Gerry Haskins, Director, Software Lifecycle Engineer

Search

Categories
Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today