[Editor: A follow-up article with corrections and additional information has been posted here]
Quality Assurance: everybody's favourite whipping boy. As I mentioned in my Blue Bridge of Death article, there exists an infinite number of testing scenarios that must somehow be covered by a limited amount of resources. The best solution is to automate as many tests as possible.
Internally here in E-Business Suite Development, we use Mercury WinRunner to create automated regression tests for each 11.5.x release. We've been using this tool for years -- it's extremely effective. Passing all automated regression tests is a mandatory exit criteria for all product teams for all releases.
There's an up-front investment cost to create the testing scripts, and there are ongoing costs to maintain them as screens and user flows change. The benefits far outweigh the costs.
WinRunner is used to create an automatic testing script, which can then be repeated as many times as you'd like using Mercury LoadRunner. [Editor: Correction -- WinRunner and LoadRunner are not integrated; see this follow-up article]
LoadRunner can be used to simulate hundreds of simultaneous users on your system. This is particularly useful for testing whether load-balancers are routing traffic correctly, or whether database traffic is automatically being rerouted between RAC nodes in failover conditions. Our Performance Group uses LoadRunner for scalability testing, performance benchmarking, and trapping those notoriously hard-to-find JVM memory leaks.
Using Oracle's Regression Tests in Your Environment
The entire suite of regression tests that we use to test every E-Business Suite release is available for download from Metalink. These tests represent millions of dollars worth of development time, and I'm personally amazed that they're available for free.
We call this a Test Starter Kit since you'll need to customize the scripts for your specific purposes. You might find the investment in training and development to be costly, but having your own battery of automated tests can potentially ease the anxiety around applying big Apps patches. Worth a look if you're considering new options for streamlining your Apps upgrade and QA process.
Reference:
- Test Starter Kit Information Center (Note 239746.1)
Comments (8)
Great post as usual, I'm hoping that Mercury is on Oracle's list of company's to acquire, that way we shouldn't have to worry too much about having to purchase Mercury software in addition to what's already been outlaid for Oracle.
Posted by Richard Byrom | June 2, 2006 6:24 AM
Posted on June 2, 2006 06:24
I feel the terrifying spectre of Oracle Legal reviewing this thread, especially since we're in a Quiet Period prior to our latest earnings announcement.I know that the above comments were tongue-in-cheek, but even so, unauthorized Oracle staff like me can't make any statements or even speculations about such matters. Regards,Steven
Posted by Steven Chan | June 2, 2006 12:45 PM
Posted on June 2, 2006 12:45
Great stuff. Will take a look at this as my organization moves towards Oracle E-biz.
Let me buy shares in Mercury first before Oracle do a take over :D
Posted by Peter K | June 2, 2006 3:34 PM
Posted on June 2, 2006 15:34
We spoke with Mercury yesterday and they recommended QuickTestPro as their best tool for functional automation of an Oracle 11i implementation. I asked if WinRunner scripts could be used with LoadRunner and they said the scripts were not compatible. They said that you could run functional scripts and watch their behavior while generating load with LoadRunner, but that there was no advantage of WinRunner over QTP for doing this. Any comment?
Posted by Yvette Francino | June 14, 2006 10:22 AM
Posted on June 14, 2006 10:22
Yvette, thanks for your comments. A follow-up article with corrections and additional information has been posted here:Errata: Automated Testing, ReloadedRegards,Steven
Posted by Steven Chan | June 15, 2006 8:21 AM
Posted on June 15, 2006 08:21
I am looking to hear from anyone who has used Automated functional testing (QTP) tools for Oracle Financials,HR,Payroll,SCM, Assetmanagement. If you have information on the application of the scripts available and any issues that need to be considered during the planning phase will be most useful. I am specifically looking at testing Oracel ebusiness 11.5.12.
Thank
Bipin
Posted by Bipin Mistry | October 30, 2006 7:33 PM
Posted on October 30, 2006 19:33
G'day Steven,
Can you please let us know if Jmeter is certified for 11i.?
Haven't seen much of hits in metalink/Apache on using this tool against 11i.
-Pavan
Posted by Pavan Kumar | March 16, 2009 10:05 AM
Posted on March 16, 2009 10:05
G'day, Pavan,
We haven't experimented with JMeter in EBS environments. It would fall under our general policy for third-party support. For more details about that, see this blog article:
Certification and Support for Third-Party Products - http://blogs.oracle.com/stevenChan/2006/05/certification_support_for_thir.html
Regards,
Steven
Posted by Steven Chan | March 18, 2009 10:59 AM
Posted on March 18, 2009 10:59