Which Are Better: Family Packs or Consolidated Updates?

Boy, is my inbox going to get hammered because of this article.  But when have I shied from controversy?  (Always, a cynical voice mutters).  Nevertheless, into the breach I charge.

11.5.10.CU2 Download screenshot: Screenshot of Metalink download site for E-Business Suite Release 11i 11.5.10 Consolidated Update #2 download

A recurring question raised at OpenWorld came from customers debating between applying individual Family Packs or the much larger Consolidated Updates to their E-Business Suite environments.  Which is better?

If I Were An IT Manager

I understand your dilemma:  maintenance windows grow tighter by the day.  Staffing levels don't keep pace with rising workloads.  Risk-averse stakeholders pressure you to keep the system unchanged -- all the while clamoring for their own personally-reported bugs to be fixed immediately.  Given this climate, anything that looks like it makes your life more complex is going to be summarily rejected.

If I had the choice of applying a random and arbitrary combination of individual family packs (e.g. FWK.H + AD.I + XDO.H) or applying a single Consolidated Update (e.g. 11.5.10.CU2), I'd choose the latter without hesitation.  If I had had the choice, Consolidated Updates are unquestionably what I'd choose.

Pulling Back the Curtain

The recommendation to apply a much larger patchset with broader impact might seem irrational, especially given what I just acknowledged is the typical IT environment.  But bear with me:  this may make more sense if I give you a glimpse into our development and testing processes.

How Consolidated Updates are Tested

Consolidated Updates (CU) are tested by the entire E-Business Suite development division in a central testing environment.  In fact, Consolidated Updates are repeatedly tested in multiple iterations of these centralized testing environments, some built for automated regression testing, others for cross-product integration flows (e.g. "Order to Cash"), and even others built for internationalization testing with NLS character sets and localizations. 

We also test multiple migration paths to a given Consolidated Update.  For example, the 11.5.10.CU2 Consolidated Update was tested on top of the 11.5.9 Rapid Install, 11.5.9.CU2, the 11.5.10 Rapid Install, and 11.5.10.CU1.

On top of all that, Consolidated Updates are tested in a variety of so-called "advanced architectural configurations" that include load-balancers, DMZs, Single Sign-On, Discoverer, SSL, RAC, different JInitiator versions, different native Sun Java (JRE) plug-ins, different desktop platforms (e.g. Windows and Mac OS X), and so on. 

And then, just because we have too much free time on our hands, the Consolidated Updates are tested on various operating system platforms, including Solaris, HP-UX, IBM AIX, and so on.

How Individual Family Packs are Tested

Individual Family Packs are tested by their respective product teams.  They receive product team manual testing and automated regression testing.  Depending on the product, they may receive some additional central testing by multiple product teams or in with advanced configurations, but this is relatively rare.

You can see that there are fundamental and profound differences in the depth and range of Consolidated Updates vs. Family Pack testing.  A Consolidated Update receives massive, intensive, coordinated testing across all E-Business Suite products.

A reasonable analogy might be:  Family Packs are to Consolidated Updates as emergency one-time patches are to Family Packs. 

The Sum of the Parts Does Not Equal the Whole

Here's something else to consider:  let's say that FIN_PF.G is in 11.5.10.CU2 (this is just an example -- I don't know if this is actually true).  Even if you install FIN_PF.G on top of 11.5.9, you aren't officially on 11.5.10.CU2 since you haven't applied the actual 11.5.10.CU2 patchset.  If you haven't explicitly applied 11.5.10.CU2 itself, Oracle Support does not consider you to be on that code level, regardless of the sum of the individual Family Packs that you might have applied.

So, when 11.5.9 is desupported, your calls to Oracle Support will prompt the usual  discussions about being on a desupported release, even though your individual family packs may be up-to-date to 11.5.10.CU2. 

Sometimes "More" is Better

Knowing all that, I always recommend applying the latest Consolidated Update instead of individual Family Packs.  This guarantees an end-run around the hassles of possible family pack incompatibilities and desupport issues.  In my view, the initial larger overhead of applying a bigger patch is far outweighed by the benefits.

I know that this will be a controversial position for some of you, so I welcome your comments. Let the debate begin...

Related

Comments:

In general I agree with this article. But what about the poor fellows of us, which are already beyond 11.5.10.2? Once you've applied the CU2 patch (which is already more than 2 years old), you face an unstructured labyrinth of mini-packs, family-packs, "consolidated rollups" and others beyond - it seems that every product development team has its own strategy how to deploy patches beyond 11.5.10.2.
I plead for the next CU patch, which (hopefully) will help us to navigate through this labyrinth.

Posted by Thomas Toepfer on November 27, 2007 at 11:51 PM PST #

Steven,

Agree with Thomas. CU2 is well over 2 years old. What about the 8 missing Family Packs (_FP)?

Also, I'm pretty sure that CU2 did not include a lot of the _FPs or mini-packs due to timings / release dates.

I tell everyone that cares that they *should* apply the CU2 AND all _FP (Family Packs as I thought they were called?) and then any additional module mini-packs from the fully installed and pseudo products list (from patchsets.sh).

This also goes back to asking about a 11.5.10.3 CU. How goes that battle?

Good document and you're right, you're going to get hammered... :-)

thx,
John

Posted by John Stouffer on November 28, 2007 at 02:47 AM PST #

Steven,

Your comparison is right on the money. Patch application is much like using a band-aid. A band-aid benefits the healing process, much like a patch benefits my EBS environment. Ripping off the band-aid quickly and suffering the pain (overhead) over a short time period (as in applying a CU) is much better than incurring smaller increments of pain over a longer time period (applying individual family packs).

As Thomas and John have pointed out, there is an issue with the timeliness of CU patch releases...and therein lies the rub with putting your opinion into practice. CU2 is now so old that customers often have no choice but to begin applying individual FPs in order to resolve their issues (or just to stay current). So customers are ripping off their respective band-aids slowly because we have no other options. Ouch!

So, we're with you: better to apply CUs than FPs. Now, if those of us limping along on 11.5.10.2 just had a CU to apply...

Posted by Floyd Teter on November 29, 2007 at 02:41 AM PST #

Here at our site, we always prefer Consolidated Updates over family packs.

From the DBA's point of view, it makes the patching easier too.
Instead of searching pre-req's for multiple family packs, applying the Consolidated Updates is comparatively easy and less error prone.

And let me repeat the same question that Thomas and John raised - What are your plans for CU3?

October to December is our annual patch apply season.
As there is no CU3 at this time, we applied all the family packs for the installed products, which is a pain.

Since, the 11i will be desupported in 2009, we may go to Rel 12 next year skipping any CU3 - unless Oracle extends the support for 11i.

Thanks again for your wonderful blog.

Rama Nalam

Posted by Rama Nalam on November 29, 2007 at 02:49 AM PST #

I agree that CU's, when available (yeah, where is CU3??) are the way to go if you can. In some cases, you end up with a humongous, riskier patch set anyway by the time you put together all the pre-reqs. And even if you merge as many of the patches as possible, I would assume that the CU is optimized for reduced downtime to a much better extent than you could do by yourself on 2 FP's and 56 one-offs...

What would really help with those "risk-averse stakeholders" would be if you could be as transparent as possible with the test scenarios and scripts that you have already run in Oracle QA. Even better if they let customers submit scenarios to add to your testing (hmm... cross-post to Mix?)

Posted by Jim Cassella on November 29, 2007 at 03:10 AM PST #

Funny, I should have anticipated that this article would reignite the discussion about a third 11.5.10 Consolidated Update (aka 11.5.10 CU3).  Chalk it up to my post-Thanksgiving haze as I plow through my enormous email backlog.Jim,Good suggestion about our test scenarios and scripts run by Apps QA.  We make all of our automated Apps 11i & 12 regression testing scripts and infrastructure available free-of-charge via Metalink.  These scripts are built with HP Mercury's WinRunner, and are released for each major E-Business Suite release.  You can find more details about them (and a pointer to the master Metalink documentation for them) in this article:Automated Testing for the E-Business Suite  http://blogs.oracle.com/schan/2006/06/automated_testing_for_the_ebus.htmlAll,It's true that those of you already on 11.5.10.CU2 still face additional patching requirements.  It's also true that the number of those patches continues to grow.In his executive Q&A at OpenWorld 2007, Cliff Godwin noted that there are currently no plans for creating a new 11.5.10.CU3 release.   As you can tell from my description of the activities associated with putting out Consolidated Updates, decisions to release these updates aren't made lightly.  CUs require an enormous concerted mobilization of resources across the Apps Division. As time goes on, the gap between 11.5.10.CU2 and the latest released patches grows, so I continue to present your feedback to our management team about the need for a third CU (I suspect that people are starting to find my repeated carping about this a touch annoying, but I persist).  You should know that a collective customer request carries more weight than individual ones.  One possible avenue for ensuring that your voices are combined in chorus would be to work with OAUG on polling the membership on this issue.  I would be curious to see what weight a petition would carry.Regards,Steven 

Posted by Steven Chan on November 29, 2007 at 03:55 AM PST #

Steven,

OAUG poll - Done. I will be drafting up a note from the OAUG Upgrade SIG Chair (me :-)) and will submit it to the user base to see what their responses will be...

Whose email address would you like me to use? In all seriousness, I think the response would be overwhelming and my gmail account wouldn't cut it and I don't want to saturate your account any more than it is already.

Suggestions?

thx,
John

Posted by John Stouffer on November 29, 2007 at 12:45 PM PST #

John, Steven, et al,

Why not set up a EBS customer poll on the need for a CU3 on Mix as a question to be answered by EBS customers? It's an easy way to gather feedback and the Mix team would love to have such a large group "kicking the tires".

--Floyd--

Posted by Floyd Teter on November 30, 2007 at 01:23 AM PST #

John, Floyd,I'm all in favor of multiple lines of convergence when it comes to assessing customer demand.  It's likely that different groups of customers will respond to an OAUG survey vs. an Oracle Mix poll, so it's useful to have both avenues of communication.I'll monitor the Mix poll regularly.  John, feel free to use me as your contact for the OAUG responses.  Fear not for my inbox -- it's tough; it can take it.Regards,Steven

Posted by Steven Chan on November 30, 2007 at 03:16 AM PST #

In the spirit of Web 2.0, I decided that action was better than just yaking about it. Those that are interest in expressing their support or disapproval for a CU3 can vote on the idea at https://mix.oracle.com/ideas/13656-cu3-patch-for-ebs-11510 (you need to join Oracle Mix in order to vote).

--Floyd--

Posted by Floyd Teter on November 30, 2007 at 04:56 AM PST #

Floyd's approach is exactly what we'd hoped people would use Mix to do, i.e. poll the community. I'm actually surprised that 50% of the people who commented (Jim and John) are registered in Mix.

Floyd's idea has 3 votes, which I doubt is representative of its support, i.e. 0.1% of the community wants this.

Let's not give up yet. Adoption takes time. I urge everyone who reads here with an opinion to share it on Floyd's idea on Mix as well.
Jake

Posted by Jake Kuramoto on December 01, 2007 at 03:46 AM PST #

Steve - I looked at the test scripts you mentioned (thanks for the reminder!) but they are only for the CU level, and as many have pointed out, CU2 is a good baseline, but most people need CU2+ minipacks, etc.

I would assume that there are no test cases for the CU2+ known bugs in those test scripts, since if they were known before release they (probably) would have been fixed. So what we would really need is the test plans from the mini-packs as well, and ideally the one-offs also. Obviously these tests would all pass, but at least we could focus our test time on the stuff that you aren't already testing.

Are these other tests not done by WinRunner, or is there some other reason they are not released?

Posted by Jim Cassella on December 04, 2007 at 05:48 AM PST #

Jim,We don't provide automated test scripts for specific bugs.  These types of bug fixes would be manually tested, so there's nothing that can be released.  Our WinRunner-based automated regression tests are for the standard intra- and inter-product flows, screens, modules, etc. Regards,Steven 

Posted by Steven Chan on December 06, 2007 at 04:45 AM PST #

After 18 days, our idea on Oracle Mix for a new 11.5.10 CU has 33 votes. Personally, I've learned two things from this experience:

1) 33 votes in favor of a new CU is not even close to the critical demand we need to get Oracle's attention. A new CU does not seem to be on the radar screen for most EBS customers.

2) Probably the more important point here - Oracle Mix works incredibly well for determining interest in an Oracle-centric idea.

--Floyd--

Posted by Floyd Teter on December 19, 2007 at 03:00 AM PST #

Floyd,John Stouffer organized an email-bombing campaign via the OAUG SIG mailing list in parallel with your Oracle Mix poll.  My inbox cringed in fearful anticipation of the responses.  However, his email campaign only resulted in 23 votes for 11.5.10.CU3, some of which I suspect overlap with the Oracle Mix respondents and commenters on these threads.  You're right, I'm afraid.  I don't believe that this comes close to the level of customer demand to revive interest in producing a third Consolidated Update.Regards,Steven 

Posted by Steven Chan on December 19, 2007 at 05:50 AM PST #

Floyd,

The challenge with the Mix is that it's linked to officially recognized email addresses (which I understand).

What I'm not sure is represented is all of the clients who are still running 11.5.10.2 and letting them know about this "straw" poll.

Would you mind putting this out in your "Fusion Corner" of the next SB Newsletter? With well over 2000+ EBS specific subscribers, it would be a good test.

Alternatively, Steven could post this question on his blog which might be a far more accurate indicator but then the expectation would be that 11.5.10.3 is a given which it's not.

Posted by John Stouffer on December 19, 2007 at 07:46 AM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
4
5
6
7
8
9
10
11
12
13
14
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today