Tuesday Aug 13, 2013

Of updates and errata.

A frequently discussed topic inside Oracle and also outside with customers and partners is Oracle Linux versions and how to treat updates and support and certifications and minimum levels. Here's our take on it, from the Oracle Linux side.

When talking about Oracle Linux and versions, there really are 3 major components :

-1- A major new release, such as Oracle Linux 5, Oracle Linux 6,...

A major new release is an update of the entire OS, kernel, userspace, all the 1000's of packages that make up Oracle Linux. A major release is significant in change compared to the previous version. You will see pretty much every package(RPM) updated to a whole new version, like Apache, MySQL, GCC, glibc, X-windows, gnome, etc etc... In a number of cases, the owners of the packages are not so careful about maintaining backward compatibility and introduce different style config files that could make upgrades difficult or sometimes impossible. This is one of the reasons why it's not easy to 'upgrade' from, say, Oracle Linux 5 to Oracle Linux 6. While it would be ok for a good number of RPMs, it's not guaranteed to work for everything. A config file might get overwritten or the older edited config file is not compatible with the new version...etc. So upgrading here, very often is a new/fresh install of the OS on a server. We see most customers go to new versions at time of a hardware refresh and use that as a good opportunity, and that makes total sense.

Because a major version has signifcant changes (new kernel, with new features, new glibc,... so core components), unfortunately sometimes changes that affect or really need change not just testing (upstart from OL5 to OL6,... potentially interesting changes due to systemd in future versions), there's typically a good reason to do certification testing of userspace applications on top of the new version. The way we work here is that we build on the lowest common version of the OS we want to support and run certification against newer versions. So we build an Oracle product on Oracle Linux 5, we won't support anything on a version that's older than Oracle Linux 5, and we do extras testing and certification for any new major version, like Oracle Linux 6. This can require us to do some changes to our application (like the database) in order for us to be able to complete that certifcation (or OS bugfixes), this was a big reason as to why it took a long time to be able to consider Oracle Linux 6 certified for the database. (due to, for instance, things like upstart)

ISV's typically will do the same thing, it takes some time for development teams to add a new major version of the OS, again, because sometimes application changes might be required or OS changes might be required, like bugfixes due to testing.

A major OS release happens every few years, not more frequently. OL4 -> (for us 2006 since we started with update 4), OL5 -> mid 2006 OL6 -> early 2011

-2- A minor update, really just a point in time current snapshot of a given major release with bugfixes and security updates applied (Oracle Linux 5 update 9, Oracle Linux 6 update 4...)

A minor update, released on a regular basis (several months), is really just a snapshot in time of the major OS release. As a version is out, on a regular basis, there are bugfixes, security updates introduced into that release, so out of the 1000's of packages, a number of those will have a minor bugfix update every now and then. These updates really are focused on fixing bugs and fixing security vulnerabilities. New features are normally not introduced into packages as part of a certain major OS release. Sometimes there are some new things added but they are only introduced if it doesn't break compatibility or doesn't change the understood use of that package. Because a major OS release is only every few years and to make it easy to provide a good snapshot of fixes within the release cycle, updates are done on a regular basis. The update is literally a snapshot at a given date and then the latest version of all packages within the release are bundled together and put onto an updated media (iso).

This makes it convenient for users in a number of ways : 1) As mentioned in the first point, sometimes we need to create OS bugfixes for an application to work on a given version of the OS, these fixes go into the OS at some point or another. It is very convenient to use the update releases to point customers to the minimum version as a starting point. 2) a very important component that gets updated regularly (bugfixes, updated device drivers/hardware support) is the kernel. So if new hardware is released, you typically need a new boot kernel to recognize newer hardware, an update release pretty much always have a newer version of the linux kernel with updated device drivers on the installation media and that's required to be able to install a given OS version on newer hardware. So it might be that you need a minimum level of Oracle Linux 5.9 to install on the latest version of a given server... but it's still "Oracle Linux 5" as a product.

So, look at update releases as minimum patch levels of an OS release, not as a product version. Oracle Linux 5.9 is not a product version, Oracle Linux 5 is the product, update 9 just implies a recent point in time of the fixes made in the product. Too often a customer (or product team) certifies on very specific update versions, and make the mistake of implying really just that version and not that version and newer. We all stand by the fact that if a minimum version required is, let's say Oracle Linux 5 update 6, then that always implies Oracle Linux 5, starting from the snapshot point update 6 and anything newer since then, as part of Oracle Linux 5.

There is no point in sticking to a given update version and consider that a release, there are -always- important fixes, whether it be potential crashes or security vulnerabilities released after that update and it really is not a best practice to stick at a certain point in time. We always do a lot of testing on any bugfixes update or security vulnerability and there's no breakage or introduction of incompatibilities within a given OS version. Look at Oracle Linux X update Y as running Oracle Linux X, update Y is just a point in time. ISVs should point out a starting update of what is considered supported or certified but with the understanding that it will imply anything changed from that point on for the same major version X. So if 5.6 is the base certification level, then anything post 5.6 as part of Oracle Linux 5 should be OK and it makes sense to try to remain as current as possible.

-3- An errata update, an update of a given package, either because of a bugfix or a security vulnerability fix.

At any point in time during the lifetime of a major OS version (OL5, OL6,...) we obviously fix bugs or address a security issue. These fixes are introduced in a continous stream of updates for each major OS. They always update the minor digits of the package version, for instance the kernel, 2.6.39-300.0.1 to 2.6.39-300.0.2 etc... They do not introduce behavior change or impact how an application runs, they make things more stable and secure. We try to not do this more often than necessary. It would be highly recommended to apply these updates soon after they are released. Most critically the kernel and glibc ones as they are under every application. Of course, with ksplice updates we make kernel updates a breeze since there's no reboot involved. And as mentioned in -2-, these errata are on a regular basis bundled into a newer snapshot of the OS.

What's the take-away here?

We recommend you look at an update release as a starting point, not as a product version, and we highly recommend customers and partners to be as current as possible in applying errata packages on their OS. It makes things more stable... it contains fixes... it contains patched security vulnerabilities... those all seem rather important to keep in mind.

So often, a customer service request comes into the support organization and it's a problem that's known and fixed in an errata, a downtime that could've been prevented by being more current...

About

Wim Coekaerts is the Senior Vice President of Linux and Virtualization Engineering for Oracle. He is responsible for Oracle's complete desktop to data center virtualization product line and the Oracle Linux support program.

You can follow him on Twitter at @wimcoekaerts

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
9
10
11
12
13
14
15
16
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today