It's possible to scale up your E-Business Suite environment with multiple application tier servers to improve fault tolerance and performance. It's also possible to share a single filesystem between them: all application tier files are installed on a single shared disk resource that's mounted from each application tier node. In Release 12, that would look like this:
This allows you to apply patches once to the central filesystem, rather than maintaining each application tier server node individually. We recommend this approach; it reduces maintenance overheads for those multiple servers and shortens your patching downtimes.
Beginning with Oracle E-Business Suite Release 12, we also allow you to share an applications tier file system between multiple E-Business Suite database instances, too. For more details about this advanced option, see this article.
Customers embarking upon this path inevitably ask, "Which shared filesystem do you recommend?" The short answer is that we don't recommend any specific filesystem, but there's more to it than just that.
Does Oracle Certify Storage Systems?
Not any more. Our Server Technologies division used to run an Oracle Storage Compatibility Program (OSCP) to validate specialized storage products. At one time, Oracle and its partners worked together to validate specialized storage technology including NFS file servers, remote mirroring, and snapshot products. The storage industry matured over time, and this program was ended in January, 2007.
The successor to this program is the Oracle Certification Environment (OCE) group. This group provides resources for third-party vendors to certify their own products with Oracle technology. The OCE team works with Oracle Partner Management and third party vendors for approving support statements published by third party vendors with respect to certification projects with Oracle.
It's important to note that these certifications are performed by the third-party vendors themselves and not the E-Business Suite Development division. Certification statements made by third-party vendors partnering with the Oracle Certification Environment group are not reviewed or endorsed by the E-Business Suite division.
Does the E-Business Suite Division Certify Storage Systems?
No, I'm afraid not. EBS Development doesn't have the resources to certify or compare even a subset of the leading filesystems. Since we don't have hands-on experience with different filesystems in a controlled test environment, we can't make any informed recommendations for a given product. We generally suggest that customers either perform their own product testing or consult a trusted consultancy that compares the relative merits of each product against a consistent set of criteria.
What are the EBS Requirements for a Shared Filesystem?
Shared filesystems must be transparent to the calling application, in this case, the E-Business Suite. In other words, no modifications to the E-Business Suite should be necessary to ensure compatibility with the shared filesystem.
Our Frequently Asked Questions: Sharing the Application Tier File System in Oracle Applications 11i (Note 243880.1) states:
... your shared application tier file system can reside on any type of shared disk resource. Examples of shared disk resources include an NFS mounted disk or a disk array. The shared disk resource does not have to be local to the machine, and it can also be a standalone disk array. Usual tuning considerations apply.
The same thing applies to Oracle E-Business Suite Release 12, too.
What About OCFS2 or GFS?
There are many different shared filesystems out there, too many to list here. The general statements about EBS requirements for a shared filesystem above apply to all third-party file system products.
However, we get a lot of questions about two specific products due to their close relationship and packaging with Oracle's own operating system releases:
- Oracle Clustered File System (OCFS2)
- Red Hat Global File System (GFS)
Here's the E-Business Suite position on these two shared file systems:
Oracle Clustered File System (OCFS2)
The E-Business Suite's database tier is built on the Oracle Database. The Oracle Database is certified with OCFS2. Therefore, OCFS2 is supported for the E-Business Suite database tier, too.
The E-Business Suite's application tier is built on Oracle Application Server. Oracle Application Server is not yet certified to run on OCFS2.
If our Fusion Middleware group ever certifies Oracle9i Application Server 1.0.2.2.2 (used by Apps 11i) or Oracle Application Server 10g (used by Apps 12) to run on OCFS2, then the E-Business Suite's application tier will be certified on that file system.
Red Hat Global File System (GFS)
Specific versions of the Oracle Database are certified with GFS running on specific Red Hat and Oracle Enterprise Linux releases. Therefore, those GFS combinations are supported for the E-Business Suite database tier, too.
Sadly, I haven't been able to locate any externally-published statements about Oracle Application Server compatibility with GFS. This usually means that these two products haven't been tested together. If you want an explicit statement of support for GFS for Fusion Middleware products, your best bet would be to log a Service Request against the Oracle Application Server product in question.
Back to the database and GFS: there are some special support provisions for this database configuration. See the "Support Process for GFS 6.0 and 6.1" section of Using Redhat Global File System (GFS) as shared storage for RAC (Note 329530.1), which states:
Oracle's product support teams will not take support calls on Red Hat GFS. All issues known to be related to Red Hat GFS must be opened with Red Hat directly. When an Oracle SR is opened for an Oracle product or a Red Hat Enterprise Linux issue in a configuration that includes GFS, Oracle Support will do their best effort to determine if the issue is GFS software related. In that case, Oracle will hand-off the GFS related issue to Red Hat Support.
It's important to note that the E-Business Suite division does not test the E-Business Suite with GFS. We haven't performed any certification or compatibility tests with that filesystem and don't have any empirical data about how well this particular combination will work.
What Does EBS Development Use Internally?
We're in Development, not marketing, and we're expressly not able to endorse third-party products. What we can do is give you a glimpse of what we use internally within Oracle for the E-Business Suite Development division.
At any given time we have hundreds of E-Business Suite environments running simultaneously within the EBS Development division. These are centrally managed by our terrific EBS/Fusion Operations group. This internal Oracle group has has created some really interesting infrastructure over the years. One of the most useful custom solutions allows developers to get a new EBS environment on demand. Shortly after their request, an automated process instantiates a new Apps environment and the developer is off to the races.
The underpinnings of this are Network File System (NFS) mounted filesystems running on NetApp. Our Operations group has tested ZFS-based filers, which are also NFS-mounted filesystems.
In practical terms, this means that nearly all of our development, testing, and certification environments for the E-Business Suite are all running on NFS mounts. We explicitly assume that our use of NFS generalizes to all shared file systems.
What Does Oracle Use Internally for its Production Global Single Instance?
Our EBS development use of NFS is paralleled by Oracle's own global single instance deployment of Apps 12. Our production EBS instance connects via Gigabit Ethernet to a shared NFS (NAS) NetApp FAS960 clustered storage system running NetApp 7.2.4.
Our four production Sun F25K database servers are equipped with 44 dual core CPUs and over 176 GB RAM, Sun Solaris 9, Sun Cluster 3.1, and Veritas VxVM/VxFS 4.0* mp02. Each of these database nodes has three GigE cards connecting them to the backend database storage, an EMC Symmetrix DMX3000 storage system.
Remember, this isn't an endorsement or a recommendation; it's merely a peek into what we use here internally at Oracle.
References
- Using Redhat Global File System (GFS) as shared storage for RAC (Note 329530.1)
- OCFS2: Supportability as a general purpose filesystem (Note 421640.1)
Related Articles
Comments (22)
Great post, as usual, Steven. It's nice to have all of this information in one place. Clears up a lot of ambiguity. Now when shared file system questions crop up on the OTN forums, we have another resource to which to direct posters. :-)
Regards,
John P.
Posted by John Piwowar | July 11, 2009 9:46 AM
Posted on July 11, 2009 09:46
Steven,
Thanks for the post. One question -- the recommendation in the documentation is that the INST_TOP is located on each host and not shared. Yet it appears from the diagram that our internal apps are sharing the instance_top. Is it safe to presume in this scenario the only thing on the local file system are the lock and pid files?
Tom
Posted by Tom Mullen | July 12, 2009 7:25 PM
Posted on July 12, 2009 19:25
Hello, Tom,
I can't say for certain, but that would be a reasonable assumption. I think the diagram was busy enough that the GSI team was using a bit of shorthand in that tiny little box for the INST_TOP details.
You're right about our standing recommendation: Note 384248.1 states, "... for optimum performance, it is advisable to use a local file system for the Instance Home."
Regards,
Steven
Posted by Steven Chan | July 15, 2009 9:41 AM
Posted on July 15, 2009 09:41
Hello, John,
My pleasure -- and thanks for your recent help on our OTN forums. I saw that you'd fielded questions about this lately. That, along with some discussions on Oracle's internal forums, spurred me to pull all of this together in one place at last.
Regards,
Steven
Posted by Steven Chan | July 15, 2009 9:44 AM
Posted on July 15, 2009 09:44
Steven
We are currently going through an upgrade to R12 and 10g of the database. The Systems Integrator has said that the local disk used to store the INST_TOP directory must be on local INTERNAL disk to the machine (we're using Sun M4000 and M5000) servers. We asked if it would be possible to use dedicated fibre attached SAN and were told that it wasn't supported and had to be internal disk. Is this correct? It seems very unusual that Oracle would insist on using internal disk rather than dedicated SAN storage. I can understand not using shared storage. This will present us with an even bigger problem soon as we are moving everything to SAN including the boot partition. Any advice on this would be appreciated. I guess the question is does local disk really have to be local to the machine or is SAN ok if it's dedicated to that machine.
Many thanks
William
Posted by william butcher | July 29, 2009 1:20 AM
Posted on July 29, 2009 01:20
Hello, William,
Thanks for the insight into your deployment architecture.
My understanding is that our guidance on the INST_TOP being installed on a local filesystem is to avoid issues with aliases, collisions, or conflicts that might occur in a shared filesystem.
I don't think our architects have any opinions about dedicated fibre-attached SANs, since we don't do any explicit testing with that particular configuration. From a support perspective, if the fibre-attached SAN is transparent to the application server (i.e. appears to the server as local storage), I can't see any issues with this configuration.
I'll run this by our architects, just to be sure. I'll post any follow-ups here, if I learn something new that contradicts what I just outlined.
Regards,
Steven
Posted by Steven Chan | August 3, 2009 11:28 AM
Posted on August 3, 2009 11:28
Steven
Thanks for your reply, this is very useful. I have been having a chat with a colleague at a different company and they have tried to address this issue with support. A SR was logged (7473036.994) in which support stated the following
QUESTION #1
===========
Implemented a shared filesystem but have noticed errors in the logs that
are consistent with incorrect disk usage. it is needed to get verification and clarification of what is meant by local disk in this context - does it
exclude any network/san sourced disk even when used exclusively for the
server to which it is made available? Does it expected to be "internal"
server disk storage?
ANSWER #1
==========
Local is internal disk storage in this context. See also the following
from the R12 SHARED APPL_TOP NOTE:
* Oracle HTTP Server Lock Files
Oracle HTTP Server processes create temporary lock files for its internal operation. The location of these lock files is specified in httpd.conf by
the LockFile and OpmMtxFile respectively. You must ensure that value of
the AutoConfig variable s_lock_pid_dir is set to a location on the local
file system to avoid file locking issues on the network file system.
So this suggests that support require the lock and pid files to be on internal disk rather than SAN which does seem rather strange.
thanks for your help in resolving this issue it has given us a real headache with disk configuration.
William
Posted by william butcher | August 4, 2009 8:10 AM
Posted on August 4, 2009 08:10
Hi, William,
One of our EBS architects was inspired by your question to write the following blog article:
Is It Safe to Use SANs for EBS R12 Instance Tops? - http://blogs.oracle.com/stevenChan/2009/08/local_disk_vs_san_for_inst_tops.html
Let me know if you still have any outstanding questions after reviewing that new article.
Regards,
Steven
Posted by Steven Chan | August 5, 2009 11:59 AM
Posted on August 5, 2009 11:59
Hi steven,
do you know if the application techstack team will be in a position to pursue certification of shraed appl_top and ASm cluster filesystems ? I notice that certification against 11g R2 is already underway and I am very interested in whether this includes certification against the new cluster filesystem for sahring the application top...
Thanks
Posted by Robin Chatterjee | September 10, 2009 6:07 AM
Posted on September 10, 2009 06:07
Hi, Robin,
I'm looking into this. I'll post an update here as soon as I get a confirmation about Oracle Application Server's certification on ASM.
Regards,
Steven
Posted by Steven Chan
|
September 11, 2009 2:26 PM
Posted on September 11, 2009 14:26
Hi, Robin,
I have gotten some weak confirmation that the Fusion Middleware group was planning to certify Oracle Application Server with ASM at some point. I haven't been able to get details on where they stand on that effort, though.
Your best bet would be to log a formal Service Request via My Oracle Support (formerly Metalink) to get a definitive answer from one of the Fusion Middleware specialists.
Regards,
Steven
Posted by Steven Chan | October 1, 2009 9:35 AM
Posted on October 1, 2009 09:35
Hi Williams,
I am designing the architecture which looks more or less similar to yourr but i will be using 2 node Sun T-series servers for shared application tier, and we have SAN storage.
I have not yet finalised on the use of shared filesystem, as per oracle doc and stevens blog we can use any NFS mounted disk or disk array, could you please share if you have used NFS or disk array(please explain) .
thanks
Pravin
Posted by Pravin | November 19, 2009 4:58 PM
Posted on November 19, 2009 16:58
Very nice post, keep it up buddy....
cheers
Posted by Mehmood | December 4, 2009 2:21 PM
Posted on December 4, 2009 14:21
is certification on OCFS2 underway? any dates?
Posted by anthony | December 17, 2009 2:03 AM
Posted on December 17, 2009 02:03
Anthony,
Yes, a certification project is underway. The Fusion Middleware division has revisited their support position for Oracle Application Server 10g on OCFS2, so we're following up by testing that in Oracle E-Business Suite Release 12 environments.
Oracle's Revenue Recognition rules prohibit us from discussing certification and release dates, but you're welcome to monitor or subscribe to this blog for updates, which I'll post as soon as soon as they're available.
Regards,
Steven
Posted by Steven Chan
|
December 17, 2009 9:37 AM
Posted on December 17, 2009 09:37
Hi
I need to install EBS Appa R12 11g, using shared file system in RHEL5 is GFS supported for this config, can I use OCFS2 instead or any other solution avilable.
S Chandy
Posted by S Chandy | January 6, 2010 12:02 AM
Posted on January 6, 2010 00:02
Hi, S Chandy,
Please refer to the article above for our support positions on both GFS and OCFS2 for shared filesystem use today.
We're in the midst of a certification project that may change our current position on OCFS2. Oracle's Revenue Recognition rules prohibit us from discussing certification and release dates, but you're welcome to monitor or subscribe to this blog for updates, which I'll post as soon as soon as they're available.
Regards,
Steven
Posted by Steven Chan
|
January 6, 2010 9:39 AM
Posted on January 6, 2010 09:39
Hi Steven
I am having a situation in which we are running 2 IBM P series server, the AIX version is 5300-11-01-0944m below is the configuration:
Oracle Application Release 12.0.4
Server A -- Database and Concurrent
Server B -- Forms and Web services
The above 2 servers are on SAN using AIX HACMP. I am having a lot of problems in patching and cloning stuff because everything has to be done twice which increases the downtime, so sharing application tier is a very good option. I believe that i have to run the NFS service on AIX Machines and mount the Concurrent File system of Server A to that of Server B using NFS service on AIX.
I found a metalink id 359515.1 of how to use nfs on different O/S Platforms. My question is:
Is it posible to use only a NFS service between 2 machines and share the application on SAN using HACMP?
Thank You
Kind Regards
Shahrukh Yasin
Posted by Shahrukh Yasin | January 26, 2010 11:56 PM
Posted on January 26, 2010 23:56
Hi, Shahrukh,
Our Applications Platforms Group does not test or certify the IBM high availability solution HACMP with E-Business Suite. Without formal testing, we don't have any informed opinions about whether this would work or not.
Our requirements for shared APPL_TOP are outlined in Note 384248.1, which is mostly platform-generic, assuming that there is a means of sharing filesystems transparently between nodes (via NFS, for example). The question of compatibility between this and an IBM-specific feature like HACMP may be best-answered by IBM themselves.
I'd be interested in hearing about any guidance that IBM gives you on this front. Feel free to drop me a private email with the outcome of your discussions with IBM.
Regards,
Steven
Posted by Steven Chan
|
January 28, 2010 10:34 AM
Posted on January 28, 2010 10:34
Hi Steven
Thank you for the prompt reply., i have discussed this with IBM and they will get back to me with the solution for sharing filesystem using HACMP, meanwhile i have started the testing of sharing application tier on my test servers which are on local disks and not clustered.
i am having issues regarding the sharing of application tier, below are the steps i followed following note id 384248.1 section 7:
Source instance: - The filesystem we want to share
Target Instance: - Share the source filesystem to target
The source and target instances are already created before and are used as a multi node environment, i am here migrating non shared to shared application tier where source is the primary instance and target is the secondary instance.
1) Run adpreclone on source instance
2) NFS mount the source filesystem to target.
3) Copy the source context file to target
4) Run adclonextx.pl addnode contextfile=
** adclonectx is being run from the target filesystem
** After running adclonectx on target instance it creates the 'inst' directory of target instance in the share file system (INST_TOP of the source filesystem is on the same location where apps directory and which is shared) because it has the path of inst_top of the source filesystem.
Please correct this step if applied incorrectly.
5) perl $AD_TOP/bin/adconfig.pl on the target instance using the new context file created by the above step.
*** adconfig.pl was run from the previous target syste filesystem.
Please correct the step if applied incorrectly.
6) Run autoconfig on the source tier as well
After completing all the steps above the problem was the target system inst_top was created under the share directory. Please correct the steps if applied incorrrectly
The filesystem was shared successfully but the main problem was it created the inst top folder on the same location where source filesystem inst top was. Please help me on this.
Thank You
Kind Regards
Shahrukh Yasin
Posted by Shahrukh Yasin | January 30, 2010 11:09 PM
Posted on January 30, 2010 23:09
Hi, Shahrukh,
I'm afraid that this blog isn't the best place to get technical support for specific issues like the one that you've encountered. Your best bet is to log a formal Service Request via My Oracle Support (formerly Metalink) to get one of our our ATG specialists to help you out with this.
Feel free to forward your SR number to me if it gets stuck in the Support pipeline for some reason.
Regards,
Steven
Posted by Steven Chan
|
January 30, 2010 11:16 PM
Posted on January 30, 2010 23:16
Hi steven
I had raised a service request also for this issue, and will update you as soon as it resolves
Thank You
Kind Regards
Shahrukh Yasin
Posted by Shahrukh Yasin | January 31, 2010 4:38 AM
Posted on January 31, 2010 04:38