Tuesday Jun 09, 2015

Warts and All!

A customer once said to me that "bad news, delivered early, is relatively good news, as it enables me to plan for contingencies". 

That need to manage expectations has stuck with me over the years.

And in that spirit, we issue Docs detailing known issues with Solaris 11 SRUs (Doc ID 1900381.1) and Solaris 10 CPU patchsets (Doc ID 1943839.1).

Many issues only occur in very specific configuration scenarios which won't be seen by the vast majority of customers.

A few will be subtle issues which have proved hard to diagnose and hence may impact a number of releases.

But providing the ability to read up on known issues before upgrading to a particular Solaris 11 SRU or Solaris 10 CPU patchset enables customers to make more informed and hence better decisions.

BTW: The Solaris 11 Support Repository Update (SRU) Index (Doc ID 1672221.1) provides access to SRU READMEs summarizing the goodness that each SRU provides.  (As do the bugs fixed lists in Solaris 10 patch and patchset READMEs.)

For example, from the Solaris 11.2 SRU10.5 (11.2.10.5.0) README:

Why Apply Oracle Solaris 11.2.10.5.0

Oracle Solaris 11.2.10.5.0 provides improvements and bug fixes that are applicable for all the Oracle Solaris 11 systems. Some of the noteworthy improvements in this SRU include:

  • Bug fix to prevent panics when using zones configured with exclusive IP networking, and DR has been used to add and remove CPUs from the domain (Bug 19880562).
  • Bug fix to improve NFS stability when under stress (Bug 20138331).
  • Bug fix to address the generation of FMA events on the PCIEX bus on T5-2 (Bug 20245857).
  • Bug fix to improve the performance of the zoneadm list command for systems running a large number of zones (Bug 20386861).
  • Bug fix to remove misleading warning messages seen while booting the Oracle VM Server for SPARC guests (Bug 20341341).
  • Bug fix to address NTP security issues, which includes the new slew always mode for leap second processing (Bug 20783962).
  • OpenStack components have been updated to Juno. For more information, see OpenStack Upgrade Procedures.
  • The Java 8, Java 7, and Java 6 packages have been updated. For more information, see Java 8 Update 45 Release Notes, Java 7 Update 80 Release Notes, and Java 6 Update 95 Release Notes.

Best Wishes,

Gerry

Thursday Apr 09, 2015

Getting fixes faster

Time is money.

I remember my first unplanned downtime as a Sys Admin on-site at a major Aluminum Mill in up-state New York.  The Operations Manager was literally poking me in the back of the neck asking me "Don't you know downtime costs us $250,000 per hour ?  How long will it take to get back up ?", to which I replied "It'll be faster if you stop poking me in the neck!".  I had the Systems back up in 20 minutes.

For Solaris and other Oracle Sun products, we try to release bug fixes as fast as possible, balancing the need for speed with the need for quality.

Since an Operating System performs many disparate functions for many disparate workloads, testing that a fix isn't toxic in any supported scenario is complex and takes time.

But we can and do provide faster relief to the customer(s) who raised the specific issue as it's easier to ensure the fix is correct for their specific environments. 

We do this by supplying Interim Diagnostics and Relief (an IDR).  As the name suggests, it provides relief for the issue until the final fix is available in a Support Repository Update (SRU) or Solaris Update release (for example, Solaris 11.3).  For hard to diagnose issues, an IDR may also provide additional diagnostic instrumentation to get to the root cause of an issue.

Like many things in Solaris 11, the IDR mechanism is far smoother thanks to the Image Packaging System (IPS) than it was in Solaris 10 and earlier releases.

SRUs for Solaris 11 and patches for Solaris 10 are released on a monthly cadence. These are tested as a unit to ensure quality.

In Solaris 11, IDRs are automatically superseded by later SRUs or Solaris Updates which include fixes for all the bugs the IDR addresses.  An IDR terminal package is included in the SRU Repo for superseded IDRs.  This tells IPS it's OK to overwrite the IDR on the target system.  Therefore, it is no longer necessary to manually remove such IDRs before updating to a later SRU or Solaris Update.

This automatic superseding typically saves customers the need for an additional reboot, since it's no longer necessary to remove an IDR, reboot, apply an SRU, reboot.  Instead, simply 'pkg update' to the desired SRU, reboot once to activate it, and you're done.

If the issues addressed by an IDR are not yet fixed in the later SRU or Solaris Update, IPS will warn the user and a Service Request (SR) should be filed requesting a new IDR at the later software version for the outstanding issues.

Normally, IDRs are provided to the specific customers who have filed Service Requests (SRs) for a specific bug. 

To accelerate the release of fixes for public security vulnerabilities, we intend to release Security IDRs to the SRU Repo and My Oracle Support (MOS) so that all customers can get relief from such vulnerabilities quicker.  Customers should continue to file Service Requests (SRs) for such bugs, so we know there's demand for a Security IDR.

These security fixes will be included into the next SRU to be released, which will automatically obsolete the Security IDRs, so customers need have no concern about installing such Security IDRs in advance of the SRU being available. The Security IDR simply provides a faster delivery mechanism.

As mentioned in a previous post, there's now a security Critical Patch Update (CPU) package which can be installed and updated on Solaris 11 systems to provide all available Criticial Vulnerabilities and Exposures (CVE) security fixes in the minimum amount of change to satisfy security compliance requirements.  This package automagically pulls in the security fixes via IPS dependencies.

There are also significant new security compliance features in Solaris 11.2.

Also in Solaris 11.2 is support for a new Package Group install option: solaris-minimal-server, which provides the minimum useful bootable environment.  Use this and install additional packages as required to support your applications.  This is useful for security compliance as if the vulnerable software isn't installed, you ain't vulnerable, and you don't need to expend unnecessary time and effort applying fixes. 

There's lots of other new stuff in Solaris 11.2 including Open Stack and the Oracle 12c Database Prerequisite Package.  Check it out!

Friday Sep 26, 2014

Solaris SRUs, patches, and IDRs available on MOS for bash vulnerabilities CVE-2014-6271, CVE-2014-7169, CVE-2014-7186, CVE-2014-7187

SRUs, Patches, and IDRs (Interim Diagnostics & Relief) are available from My Oracle Support, support.oracle.com for all supported Solaris releases to address the recent critical bash vulnerabilities, CVE-2014-6271, CVE-2014-7169.

Newer IDR revisions are available on MOS which additionally address the less critical "mop up" vulnerabilities, CVE-2014-7186, CVE-2014-7187.  Patches and SRUs will follow for these too.

See MOS Doc ID 1930090.1 for details.

Many thanks to the folks around the globe who have been working tirelessly over the last 48 hours to code, test, and release these SRUs, patches, and IDRs - from Australia to India to the Czech Republic to Ireland and the US.

I sincerely apologise for the delay in proactively communicating these fixes to you.   That was outside of my control.

Best Wishes,

Gerry.

Tuesday Jun 03, 2014

ORAchk version 2.2.5 is now available for download

Those awfully nice ORAchk folks have asked me to let you know about their latest release...

ORAchk version 2.2.5 is now available for download, new features in 2.2.5:

  • Running checks for multiple databases in parallel
  • Ability to schedule multiple automated runs via ORAchk daemon
  • New "scratch area" for ORAchk temporary files moved from /tmp to a configurable $HOME directory location
  • System health score calculation now ignores skipped checks
  • Checks the health of pluggable databases using OS authentication
  • New report section to report top 10 time consuming checks to be used for optimizing runtime in the future
  • More readable report output for clusterwide checks
  • Includes over 50 new Health Checks for the Oracle Stack
  • Provides a single dashboard to view collections across your entire enterprise using the Collection Manager, now pre-bundled
  • Expands coverage of pre and post upgrade checks to include standalone databases, with new profile options to run only these checks
  • Expands to additional product areas in E-Business Suite of Workflow & Oracle Purchasing and in Enterprise Manager Cloud Control
  • ORAchk has replaced the popular RACcheck tool, extending the coverage based on prioritization of top issues reported by users, to proactively scan for known problems within the area of:

    • Oracle Database
      • Standalone Database
      • Grid Infrastructure & RAC
      • Maximum Availability Architecture (MAA) Validation
      • Upgrade Readiness Validation
      • Golden Gate
    • Enterprise Manager Cloud Control
      • Repository
    • E-Business Suite
      • Oracle Payables (R12 only)
      • Oracle Workflow
      • Oracle Purchasing (R12 only)
    • Oracle Sun Systems
      • Oracle Solaris

    ORAchk features:

    • Proactively scans for the most impactful problems across the various layers of your stack
    • Streamlines how to investigate and analyze which known issues present a risk to you
    • Executes lightweight checks in your environment, providing immediate results with no configuration data sent to Oracle
    • Local reporting capability showing specific problems and their resolutions
    • Ability to configure email notifications when problems are detected
    • Provides a single dashboard to view collections across your entire enterprise using the Collection Manager

    ORAchk will expand in the future with high impact checks in existing and additional product areas. If you have particular checks or product areas you would like to see covered, please post suggestions in the ORAchk subspace in My Oracle Support Community.

    For more details about ORAchk see Document 1268927.2

    Monday Mar 10, 2014

    ORAchk Health Checks for the Oracle Stack (including Solaris)

    My colleagues, Susan Miller and Erwann Chénedé, have been working with the nice people behind the ORAchk tool (formerly RACcheck) to add Solaris health checks to the tool.

    ORAchk 2.2.4, containing the initial 8 Solaris health checks, is now available:

    ORAchk includes EXAchks functionality and replaces the popular RACcheck tool, extending the coverage based on prioritization of top issues reported by users, to proactively scan for known problems within:

    • E-Business Suite Financials Accounts Payables
    • Oracle Database
    • Sun Systems

    ORAchk features:

    • Proactively scans for the most impactful known problems across your entire system as well as various layers of your stack
    • Simplifies and streamlines how to investigate and analyze which known issues present a risk to you
    • Lightweight tool runs within your environment; no data will be sent to Oracle
    • High level reports show your system health risks with the ability to drill down into specific problems and understand their resolutions
    • Can be configured to send email notifications when it detects problems
    • Collection Manager, a companion Application Express web app, provides a single dashboard view of collections across your entire enterprise

    ORAchk will expand in the future with more high impact checks in existing and additional product areas. If you have particular checks or product areas you would like to see covered, please post suggestions in the ORAchk community thread accessed from the support tab on the below document.

    For more details about ORAchk see Document 1268927.1

    Tuesday Sep 17, 2013

    Top Tips for Updating Solaris 11 Systems

    We now have quite a bit of experience of IPS and Repositories under our belt. 

    Feedback from customers has been extremely positive.  I recently met a customer with 1000+ Solaris servers who told me that with Solaris 10 it took them 2 months to roll out a new patchset across their enterprise.  With Solaris 11, it takes 10 days.

    That really helps lower TCO.

    As with anything, experience teaches us how to optimize things.  Here's a few Top Tips around IPS / Repo management which I'd like to share with you from my experience with SuperCluster:

    • To avoid most IPS dependency resolution errors, keep your main local Repository populated with all Solaris Updates and SRUs up to and including the version you wish to apply.  A sparsely populated Repo is much more likely to result in copious IPS dependency resolution errors.
    • Keep any IDRs (Interim Diagnostics or Relief) in a separate Repo local to the Boot Environments (BEs) for which they are relevant.  For example, if you have an IDR to address an issue with 11gR2 RAC on Solaris 11.1.7.5.0 (Solaris 11.1 SRU7.5), keep it local to the relevant BEs running 11gR2.  This avoids IDRs being unnecessarily propagated to LDoms or Zones for which they are irrelevant.
    • Before upgrading, check to ensure that the issues addressed in any IDRs you are using are fixed in the Solaris version to which you are updating.  If they are, IPS will automatically supersede them - that is, unlike in Solaris 10, there's no need to manually remove them.  You can check this by looking in the Support repository, or the relevant Repo ISO image, for packages whose base name is the IDR number, that is 'idr<number>'.  If such a package exists, then the IDR has been superseded and the issues it addresses are fixed in that SRU.  If the issues are not fixed in the Solaris version to which you are updating, you may need to ask Support for new IDR(s) for that Solaris version.
    • Zone creation in Solaris 11 works differently to how it did in Solaris 10.  In Solaris 11, effectively a manifest is taken of the Global Zone and then Non-Global Zones (NGZs) are constructed from that using the Repo(s).  Therefore, your Repo(s) must be up to date with all Solaris software installed on your global zone, including any IDRs.  You can have multiple Publishers specified, so that multiple Repos can be used (e.g. main local Repo for the Solaris Updates / SRUs, BE specific Repo for IDRs).

    I hope you find these tips useful.

    My colleagues, Glynn Foster and Bart Smaalders, will be presenting on "Oracle Solaris 11 Best Practices for Software Lifecycle Management [Con3889]" @ Oracle OpenWorld next week.  The Oracle Sun "Systems" sessions are in the Westin this year.  This particular session is on Tuesday, Sept 24 @ 5:15pm in the "City" meeting room in the Westin and will have lots more tips and best practices.

    Other colleagues, Rob Hulme and Colin Seymour, are presenting on "Best Practices for Maintaining and Upgrading Oracle Solaris [CON8255]" on Monday, Sept 23 @ 10:45am in the Westin San Francisco, also in the "City" meeting room.

    And there's lots of other good stuff on Solaris and SuperCluster.  For example, the "Deep Dive into Oracle SuperCluster [CON8632]" on Tuesday, Sept 24 @ 5:15pm in the Westin, Metropolitan II.

    I'm not presenting this year, but if you would like to meet up with me @ OpenWorld to discuss anything about Solaris / Systems / SuperCluster Lifecycle Maintainence, whether it's ideas you'd like to see implemented, what's keeping you awake at night, issues you want me to look at, etc., I am more than happy to do so.  Just ping me at Gerry.Haskins@oracle.com.

    Best Wishes,

    Gerry.

    Friday Apr 12, 2013

    Solaris 11 SRU naming convention change

    We're tweaking the naming convention used by Oracle Solaris SRUs (Support Repository Updates) to use a 5-digit taxonomy.

    For example, Oracle Solaris 11.1.6.4.0

    The digits represent Release.Update.SRU.Build.Respin

    For the above example, the old name would have been Oracle Solaris 11.1 SRU 6.4. 

    As with Oracle Solaris 10 and below, all bug fixes are putback to the tip of the source tree for Solaris 11, which is currently Solaris 11.1.x.y.z. 

    Therefore, these same SRUs are also the way to get fixes for systems installed with Oracle Solaris 11 11/11, in exactly the same way that Solaris 10 Kernel patches included code from all preceding Solaris 10 Updates.

    As discussed in previously postings, systems should be updated to a later SRU, for example from Oracle Solaris 11 11/11 SRU13.4  to Oracle Solaris 11.1.6.4.0.

    If you maintain a local Solaris Repository behind your firewall, both Solaris 11.1 and whichever subsequent SRUs you are interested in should be added to your Repo.  This is because SRUs only contain the change delta relative to the preceding Solaris Update.

    Solaris's long standing Binary Compatibility Guarantee coupled with the technical benefits of Image Packaging System (IPS) help to ensure a smooth update experience.

    Thursday Apr 12, 2012

    How To Update Oracle Solaris 11

    My colleague, Glynn Foster, has published a nice article on how to update Oracle Solaris 11 which I think you may find interesting.

    Monday Nov 28, 2011

    Solaris 11 Customer Maintenance Lifecycle

    Hi Folks,

    Welcome to my new blog http://blogs.oracle.com/Solaris11Life which is all about the Customer Maintenance Lifecycle for Image Packaging System (IPS) based Solaris releases, such as Solaris 11.

    It'll include policies, best practices, clarifications, and lots of other stuff which I hope you'll find useful as you get up to speed with Solaris 11 and IPS.  

    Let's start with an updated version of my Solaris 11 Customer Maintenance Lifecycle presentation which I originally gave at Oracle Open World 2011 and at the 2011 Deutsche Oracle Anwendergruppe (DOAG - German Oracle Users Group) conference in Nürnberg.

    Some of you may be familiar with my Patch Corner blog, http://blogs.oracle.com/patch , which fulfilled a similar purpose for System V [five] Release 4 (SVR4) based Solaris releases, such as Solaris 10 and below.

    Since maintaining a Solaris 11 system is quite different to maintaining a Solaris 10 system, I thought it prudent to start this 2nd parallel blog for Solaris 11.

    Actually, I have an ulterior motive for starting this separate blog. 

    Since IPS is a single tier packaging architecture, it doesn't have any patches, only package updates. 

    I've therefore banned the word "patch" in Solaris 11 and introduced a swear box to which my colleagues must contribute a quarter [$0.25] every time they use the word "patch" in a public forum.  From their Oracle Open World presentations, John Fowler owes 50 cents, Liane Preza owes $1.25, and Bart Smaalders owes 75 cents. 

    Since I'm stinging my colleagues in what could be a lucrative enterprise, I couldn't very well discuss IPS best practices on a blog called "Patch Corner" with a URI of http://blogs.oracle.com/patch.  I simply couldn't afford all those contributions to the "patch" swear box. :)

    Feel free to let me know what topics you'd like covered - just post a comment in the comment box on the blog.

    Best Wishes,

    Gerry.


    About

    This blog is to inform customers about Solaris 11 maintenance best practice, feature enhancements, and key issues. The views expressed on this blog are my own and do not necessarily reflect the views of Oracle. The Documents contained within this site may include statements about Oracle's product development plans. Many factors can materially affect these plans and the nature and timing of future product releases. Accordingly, this Information is provided to you solely for information only, is not a commitment to deliver any material code, or functionality, and SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISIONS. The development, release, and timing of any features or functionality described remains at the sole discretion of Oracle. THIS INFORMATION MAY NOT BE INCORPORATED INTO ANY CONTRACTUAL AGREEMENT WITH ORACLE OR ITS SUBSIDIARIES OR AFFILIATES. ORACLE SPECIFICALLY DISCLAIMS ANY LIABILITY WITH RESPECT TO THIS INFORMATION. Gerry Haskins, Director, Software Lifecycle Engineering

    Search

    Categories
    Archives
    « July 2015
    SunMonTueWedThuFriSat
       
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
     
           
    Today