Five Errors Customers Make When Patching E-Business Suite 12 (Part 3)

This is the third in a series of short articles about common technical myths or misunderstandings about the E-Business Suite. Other articles have covered installations, cloning, migrations, upgrades, and maintenance. This article discusses patching. Subsequent articles will cover migrating, upgrading, and general maintenance. Each article is absolutely not definitive and I’d be interested in comments from readers who think there are other misconceptions about the day to day maintenance and upgrade of a typical E-Business Suite environment.


COMMON ERROR #1: Assuming that all patches can be applied using the hotpatch and nocheckfile parameters.

ORACLE’S RECOMMENDATION: The adpatch ‘hotpatch’ parameter should be used very sparingly and is not intended to be an easy way of applying patches to your system without shutting down the application tier services. Most patches should only be applied during scheduled downtimes when all users are logged off the system, application tier services have been shut down and Maintenance Mode enabled. From the adpatch documentation:

“If an emergency patch can be applied without shutting down services, the customized instructions generated by PAA will explicitly state so. If the customized instructions do not explicitly state this, assume that you need to shut down services and enable Maintenance mode before applying the patch.”

Similarly, the ‘nocheckfile’ parameter should not be used on all patches. Nocheckfile forces the patch to run all jobs in a patch regardless of whether the jobs have been run before. It degrades patch performance and should only be used in specific circumstances where it is genuinely needed. It should not be used as the default option for applying all patches. There are very few cases in which you would want to run with nocheckfile.


COMMON ERROR #2: Assuming that patches can be applied “manually” or backed out of “manually”.

ORACLE’S RECOMMENDATION: There is no manual alternative to using adpatch to apply a patch. It is not supported to selectively choose only certain files from a patch and manually copy or execute these files in your E-Business Suite environment. Similarly, there is no simple method of reversing a patch.

COMMON ERROR #3: Increasing the number of patch workers in an attempt to make patches run faster.

ORACLE’S RECOMMENDATION: Do not assume more workers automatically improves patch performance. When you first test a patch, accept the default number of workers. Use this as the performance benchmark. If you think a different number of workers will improve patch performance then try this once you have your default benchmark, but do not exceed the default number of workers without good justification for doing so. Once a patch is started, the number of workers cannot be changed mid-patch.


COMMON ERROR #4: Ignoring or skipping patch workers that fail on products that are not used or needed.

ORACLE’S RECOMMENDATION: You should aim to have zero errors when applying a patch. All E-Business Suite products are installed in the E-Business Suite database whether you are licenced to use them or not. Patches will update all products in the database whether they are licenced or not. If you find yourself having to skip jobs or ignore errors during patching then this indicates an underlying problem in your system that should always be investigated and addressed. Review the patch prerequisites and ensure all steps have been completed successfully.


COMMON ERROR #5: Using adpatch default log file names during patching.

ORACLE’S RECOMMENDATION: Identify each new patch log file with a unique patch log file name. If you always choose the default log file name in adpatch, your adpatch.log quickly becomes enormous. This makes it very difficult to identify individual patch information in the log file and makes it difficult to keep an accurate record of your patch history. Use a simple naming convention, e.g. choose a unique adpatch log name for each patch (<patchnumber>.log. Archive and then delete your main patch log and worker files after each patch so each patch has new log and worker files.

References

Related Articles


Comments:

I have some concerns with some of these common errors
"COMMON ERROR #1: Assuming that all patches can be applied using the hotpatch and nocheckfile parameters."

If I were to take down a production ebs every time I needed to apply a patch that didn't explicitly say hotpatch was supported, the companies I work with would find another DBA. For example, My main client has an EBS system used in 6 countries spanning North America, Europe, and APAC regions. We look at (using OAM and other tools) what changes a patch makes in a test environment and then decide if/when we need to wait until scheduled downtime. I'd say roughly 80% of our patches go in with hotpatch. There are of course some patches that should definitely not be applied in hotpatch mode - but it's more the exception than the rule, imho.

"COMMON ERROR #5: Using adpatch default log file names during patching."

I'll actually put this down to more of a "style" thing vs being an error. Yes, putting everything in the default patch log will make a large patch log file. But besides the file becoming large, it's not that big of a deal. Need to just get the log for a patch application to Oracle Support? tail -n XXXXXXX is your friend. One of the points you brought up "makes it difficult to keep an accurate record of your patch history." - isn't correct. I could argue having everything written to 1 file sequentially vs 100 files sequentially is much *easier* for keeping an accurate record. Regardless, instead of awk-ing thru text files, you should be using something like OAM and the tables it queries against for an accurate record of your patch history - its because of those patch history tables that many SRs and MOS Notes have sql to run to see exactly what patches have been applied.

Let's think about doing things your way - a customized filename containing the patch number for each patch. Simple question: what would you then suggest for merged patches? "patch1.patch2.patch3.log" ? No, put it all in one standard file and rotate (archive / zip) the old file as necessary.

This does bring up something I do wish Oracle would do though - have a patch backup cleanup tool. With most EBS patches, the files being replaced go into the backup/<INSTANCE>/ directory tree. That's good and I like it. Now imagine we've got one NFS dump spot containing say 100 applied patches (and remember, these would have multiple directories under backup - ideally DEV,UAT,PROD, etc). Ok, it's time to clean up this space. It'd be great if Oracle provided a tool that quickly would strip out the backup directory trees to a specified location so that the backed up files can then be archived for audit or even just packrat purposes. I've already written scripts to do this, but this would be something Oracle could provide that would help other customers.

This leads to what I would call "Common Error #6" - storing downloaded patches (and their backups) under the $APPL_TOPs ". Sure, the vast majority of Oracle EBS databases dwarf the space used for patches, but remember, the $APPL_TOPs (and anything in them) then get cloned from PROD to other environments, thereby wasting that space multiple times... and generally that storage is enterprise class, since this is an Enterprise Resource Planning application. You're (hopefully) thinking "who would store patches under $APPL_TOP?" I've been sadly surprised at the number of companies that have done this.

Even after my quibbling, I do want to thank you for putting this series of articles together.

Posted by Jay Weinshenker on July 21, 2012 at 06:31 PM PDT #

Hi Jay

Thanks for taking the time to respond so comprehensively. Your points are extremely valid.

Regarding hotpatch, yes, you may find you are able to apply some patches using hotpatch. If you can do the OAM analysis and test thoroughly then you may conclude a particular patch is safe to apply hot but we want to make sure the message is of caution which I see you are practising. What I want to dispel is they myth of applying every patch (including Family Parks or Release Update Packs) using hotpatch simply because the hotpatch option exists. This is a common misunderstanding and does cause issues.

I should have mentioned in the article that all translation/NLS patches can be applied using hotpatch. In a system with several additional languages, this means many patches can be applied with hotpatch even if the main US English patch cannot. You may also be aware of plans to include online patching in 12.2 - https://blogs.oracle.com/stevenChan/entry/glimpses_of_e_business_suite .

Regarding adpatch log files, as I was writing the article I realised there was a very good chance your points would be raised. Adpatch does not really manage its log files and choosing the defaults will raise the issues I describe. Constant appending to exisiting log files is, as you say, *easier* and also guarantees completeness, but if a question is raised about a particular patch that was applied in the past and you need to find the log and worker files for only that patch, then having its log and worker files stored separately is a lot easier than trying to trying to match the contents one of enormous adaptch log file with the contents of several equally enormous worker log files. I've never seen adpatch fail because the log files have become unmanageable because of their size but I have seen log and worker files which are so big that I would happily describe them as unmanageable. I agree, you would need a more refined naming convention for merged patch log files.

Your point about where to store downloaded patches is also valid. Keeping them within the APPL_TOP and copying them every time you clone would quickly have an effect on the time it takes to clone and as you rightly say, the backup directories in adpatch can also have an effect on the time to create a clone. An adpatch backup cleanup tool to manage this sort of thing would be helpful.

Regards

Nick

Posted by Nick Quarmby on July 23, 2012 at 01:54 AM PDT #

COMMON ERROR #4: Ignoring or skipping patch workers that fail on products that are not used or needed

1)Does this mean, we have to apply pathces of products which we have not using (licensed) also.

Could someone plese suggest

Posted by guest on September 27, 2012 at 04:51 PM PDT #

Hi Guest

All products are installed in your system regardless of whether you use them or not. All major patches will contain updates to products you will not use or have not licensed.

In the case of one-off patches for products that you do not use, it is unlikely you would find a need to apply these however these patches might later be included in a major Family Pack patch you would need to apply and therefore the changes would made on your system.

Regards

Nick

Posted by Nick Quarmby on September 28, 2012 at 12:17 AM PDT #

Traditionally Oracle recommended that patches are only applied if you have encountered the problem that the Patch will fix.

Patch Wizard seems to contradict this, it recommends patches based on your codeline regardless if you have a problem.

Your thoughts are appreciated.

Thank you

Posted by guest on September 25, 2013 at 02:44 AM PDT #

Hi Guest

Patch Wizard will recommend all patches it thinks are appropriate for your system. You can review the one-off patches shown and decide if you would like to proactively apply some/all of these patches in order to eliminate the possibility of encountering a known issue.

If you have not encountered the issue but think it is possible you may do so in the future then we advise you apply the patch in your next appropriate downtime.

Regards

Nick

Posted by guest on September 26, 2013 at 12:58 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
4
5
6
7
8
9
10
11
12
13
14
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today