Top 7 Ways of Reducing Patching Downtimes for Apps

[June 20 Update:  Our blogging software temporarily lost its mind and somehow misnumbered the list at the heart of this article.  There are only seven tips here, not ten.]

I've previously discussed the Top 5 Myths About Patching Apps Environments.  One of the most commonly-cited reasons for not patching E-Business Suite environments is that it takes too much downtime.  If you're relatively new to Apps system administration, here's a primer on the key techniques compiled by our Applications Release Engineering group for reducing patching downtimes.  Although some of the linked documents are written specifically for Release 11i, these techniques may be used for both Apps 11i and 12.

  1. Use a staged applications system

    This major time-saver hinges on a key principle:  all of your applications filesystem patches are applied to a clone of your production Apps environment.  This can be done while your production system is still running.  Your production system is down only for the time needed to apply database patches.  For details, see:
  2. Use a shared application-tier file system

    If you have a pool of application-tier servers set up for load-balancing, make sure that all of the individual servers share a single application filesystem.  Patches applied to this central shared filesystem are instantly available to all application-tier servers.  I've previously given an overview of this technique in this article.

  3. Distribute worker processes across multiple servers

    When applying a patch that includes a large number of processes, you can reduce the downtime even further by distributing the worker processes across multiple servers on multiple nodes. Using the Distributed AD feature of AutoPatch and AD Controller, you can assign workers to run on the primary node and on other nodes that share the filesystem. See:

    Distributed AD (Metalink Note 236469.1)

  4. Merge multiple patches using AD Merge Patch

    Merging patches saves time because the AutoPatch overhead of starting a new session is eliminated for those patches that are consolidated.  Duplicate linking, generating or database actions are run once only.  If two patches update the same file, AD Merge Patch will save time by applying only the latest one.  Patches can -- and should -- be merged with their listed prerequisite patches. 

    For more details about this AD utility, see the Oracle Applications Maintenance Procedures guide for your Apps release.

  5. Run AD Patch in non-interactive mode

    Applying a set of patches using AD Patch in non-interactive mode eliminates the delay between successive tasks.

  6. Defer system-wide database tasks until the end

    Using adpatch options=nocompiledb,nomaintainmrc defers system-wide database tasks such as "Compile APPS schema" and "Maintain MRC" until after all patches have been applied.  As of AD.H, AutoPatch automatically compiles the APPS schema and maintains MRC when applying standard patches.
  7. Avoid resource-related bottlenecks

    Patching can grind to a halt if you bump into the ceiling on your system.  Before patching, make sure that you've enabled automatic tablespace management, and that you have sufficient hardware and free disk and temp space.
The cumulative downtime reductions of all of these techniques can be quite significant.  I've touched on some of the biggest timesavers, but this short article isn't comprehensive, by any means.  The linked Notes in this article and below discuss a number of other tips for shrinking your patching downtimes.  A small investment in learning these techniques can pay off in large reductions in patching times, and is well worth your time.

References

Comments:

Top 10? I see only 7!

Posted by Nitin on June 19, 2007 at 06:13 PM PDT #

>> Top 10? I see only 7!
LOL :)
Other 3 is Learn constantly , Get expireince, have a good backup & recovery plan
:)

Posted by Yury Velikanov on June 19, 2007 at 09:25 PM PDT #

Steven all other TOP 10 ways sounds good to me, but:
>> Distribute worker processes across multiple servers
This feature looks attractive. The Idea behind it is logical.
Just wonder if you somebody from this blog readers list can give us Real Live study case there this feature helped to reduce patching time for at least 10%?
From my experience it doesn稚' help that much as bottleneck most of the cases are DB or Apps nodes IO.

DB: This feature doesn稚 help in DB field at all.
APPS IO: DWP requires Shared FS, if we implement Shared FS then it is difficult to reduce Io time if files are read from network FS in concurrent mode (network might be a problem).

Just curious,
Jurijs

PS If we are talking about Ways (not EBS Features) then i would add the following:
- tuning patching in a Test environment might help tremendously to reduce time in a production

Posted by Yury Velikanov on June 19, 2007 at 09:32 PM PDT #

Ah, an excellent way of rounding this out to 10 items, Yury.Thanks are due to you, Randy, Nitin, and all others who noticed the bug in our blogging software's <OL> tag numbering.  I should have noticed that, myself.Unlike the modern-day music industry and its current handling of albums, I don't pad out lists arbitrarily with filler items, so I've taken the expedient approach of simply retitling the article.  ;-)Regards,Steven

Posted by Steven Chan on June 20, 2007 at 02:32 AM PDT #

Great list Stephen. Working with our corporate partners, we are often asked to deliver tips on best practices. I will be adding some of these to our list. This blog has been a great resource.

Chuck

Posted by Chuck Speaks on June 20, 2007 at 01:46 PM PDT #

Jurijs,Like you, I'd be interested in hearing from readers about actual benefits from some of these tips.  Our Release Engineering team did a presentation at OpenWorld a few years ago that compared the differences between a "normal" patching process vs. an "optimized" patching process using these (and other) tips.  The reduction in elapsed time using the "optimized" method was significant -- 50% or more.  I can't find that presentation, alas.Thanks for your comments.Regards,Steven

Posted by Steven Chan on June 21, 2007 at 04:44 AM PDT #

Thanks for the feedback, Chuck.  I look forward to seeing what you come up with on your new blog.Regards,Steven

Posted by Steven Chan on June 21, 2007 at 05:19 AM PDT #

Steven,

Very instructional, crisp, to-the-point and as always very practical and helpful. You might want to consider the following related to Point 7: The presence of a larger number of workers is not necessarily an advantage - there are times when the workers step on each others' toes to acquire locks (both internal / external) and sessions wait for each other while showing up as 'running'. This also occurs when a previous 'runaway' session holds a lock and the worker is waiting on this hung session to complete (it never does and DBAs sometimes bounce the DB to clear it!) A simple method of monitoring this is to use the following query:

select event, count(*) from v$session_wait group by event;

This will expose any waiters and allow the DBA to troubleshoot holders/waiters. V$ACCESS is also another not-so-well known view that can show who is using what object.... I deal with the parsing of the SQL above as a way of monitoring and alerting the DBA in my script at http://www.geocities.com/john_sharmila/links.htm - see article on Low overhead monitoring.

John Kanagaraj

Posted by John Kanagaraj on June 30, 2007 at 02:21 PM PDT #

John,Excellent points.  There are, indeed, some situations where multiple workers may be sub-optimal.  Thanks, also, for the pointer to your script and article.Regards,Steven

Posted by Steven Chan on July 02, 2007 at 01:39 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
4
5
6
7
8
9
10
11
12
13
14
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today