Oracle Support Master Note for Real Application Clusters (RAC), Oracle Clusterware and Oracle Grid Infrastructure (Doc ID 1096952.1)
By Scott.Jesse@oracle.com on Oct 04, 2010
Are you implementing Oracle RAC or Oracle Grid Infrastructure for the first time? Are you an experienced RAC or Oracle Clusterware user, but want feedback on the latest knowledge and Best Practices? This blog is intended to give you insights into the quickest way to find information about RAC, Oracle Clusterware and Oracle Grid Infrastructure. The contents of this blog replicate the data found in the My Oracle Support Master Note#1096952.1 for Oracle RAC, Oracle Clusterware and Oracle Grid Infrastructure.
This Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes with respect to Oracle Real Application Clusters, Oracle Clusterware and Oracle Grid Infrastructure implementations. This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest, within the RAC, Clusterware and Grid Infrastructure spaces. This includes the following categories:
- RAC Assurance RAC Starter Kits - Installation and Best Practices Information
- Additional Notes on RAC Installation and Frequently asked Questions:
- RAC and Clusterware Monitoring, Troubleshooting & Data Collection:
- Troubleshooting and Data Collection
- My Oracle Support references
- RAC and Clusterware Testing and Performance Tuning
- RAC and Clusterware Maintenance and Patching References
- General Maintenance References
- Patching Best Practices and References
- Upgrade and Platform Migration for RAC
- RAC and Oracle Applications
- RAC ASM Documentation
- Maximum Availability Architecture References
Scope and Application
Oracle Real Application Clusters (RAC) is a cluster database with a shared cache architecture that overcomes the limitations of traditional shared-nothing and shared-disk approaches to provide a highly scalable and available database solution for all your business applications. Oracle RAC provides the foundation for enterprise grid computing.
Oracle Clusterware is a portable cluster software that allows clustering of single servers so that they cooperate as a single system. Oracle Clusterware provides the required infrastructure for Oracle Real Application Clusters while also enabling high availability protection of any Oracle or third party application.
Oracle Automatic Storage Management (ASM) provides a virtualization layer between the database and storage. It treats multiple disks as a single disk group, and lets you dynamically add or remove disks while keeping databases online. ASM greatly simplifies the storage complexities often associated with an Oracle database while increasing availability and performance of the database.
With Oracle grid infrastructure 11g release 2 (11.2), Oracle Automatic Storage Management (Oracle ASM) and Oracle Clusterware are installed into a single home directory, which is referred to as the Grid Infrastructure home. The installation of the combined products is called Oracle Grid Infrastructure. However, Oracle Clusterware and Oracle Automatic Storage Management remain separate products.
This note applies to the following versions of these products:
Oracle Clusterware - Version 10.1.0.2 to 220.127.116.11
Oracle Grid Infrastructure - Version 11.2.0.x
Oracle Real Application Clusters - Version 10.1.0.2 to 11.2.0.x
Master Note for Real Application Clusters (RAC) Oracle Clusterware and Oracle Grid Infrastructure
RAC Assurance RAC Starter Kits - Installation and Best Practices
The goal of the Oracle Real Application Clusters (RAC) Starter Kits are to provide the latest information on generic and platform specific best practices for implementing an Oracle cluster using Oracle Clusterware or Oracle Grid Infrastructure (11gR2) and Oracle Real Application Clusters. These documents are compiled and maintained based on Oracle Support's ongoing experience with its global RAC customer base. The process used to install, configure, and create an Oracle Cluster and an Oracle Real Application Clusters (RAC) database shares much in common across all supported operating system platforms. Despite these commonalities, many aspects of the deployment process are O/S specific. As such, there is a Generic Starter Kit, as well as a separate Starter Kit for each supported platform. These Starter Kits provide the latest information in terms of installation help, as well as best practices for ongoing/existing implementations.
- <Document ID 810394.1> RAC Assurance Support Team: RAC and Oracle Clusterware Starter Kit and Best Practices (Generic)
- <Document ID 811306.1> RAC Assurance Support Team RAC Starter Kit (Linux)
- <Document ID 811280.1> RAC Assurance Support Team RAC Starter Kit (Solaris)
- <Document ID 811271.1> RAC Assurance Support Team RAC Starter Kit (Windows)
- <Document ID 811293.1> RAC Assurance Support Team RAC Starter Kit (AIX)
- <Document ID 811303.1> RAC Assurance Support Team RAC Starter Kit (HP-UX)
The Generic Starter Kit has attached documents with sample test plan outlines for system testing and basic load-testing to validate the infrastructure is setup correctly, as well as a sample project plan outline for your RAC implementation. In addition, best practices for Storage and networking and other areas are included. The Platform-specific Starter Kit notes complement this with Step-by-Step Installation Guides, for each particular platform, as well as best practices for the specific platform in question.
Additional Notes on RAC and Clusterware Installation and Frequently asked Questions
While the Starter Kits referenced above will contain much of what you need to get started, additional notes and resources that are important to take heed of, particularly for a new implementation, can be found here in the following FAQ's and other references:
- <Document ID 220970.1> RAC: Frequently Asked Questions
- <Document ID 316817.1> CLUSTER VERIFICATION UTILITY FAQ
- <Document ID 332257.1> Using Oracle Clusterware with Vendor Clusterware FAQ
- <Document ID 397460.1> Oracle's Policy for Supporting RAC 10g with Symantec SFRAC
- <Document ID 391771.1> OCFS2 - Frequently Asked Questions
- <Document ID 787420.1> Cluster Interconnect in Oracle 10g and 11g
Key notes related to OS Installation requirements for a RAC or Oracle Clusterware environment
- <Document ID 359515.1> Mount Options for Oracle files when used with NAS devices
- <Document ID 401132.1> How to install Oracle Clusterware with shared storage on block devices
- <Document ID 357472.1> Configuring device-mapper for CRS/ASM
- <Document ID 169706.1> Oracle Database on AIX,HP-UX,Linux,MacOSX,Solaris,Tru64 Operating Systems Installation and Configuration Requirements Quick Reference (8.0.5 to 11.1)
Key notes introducing differences in Oracle 11gR2
- <Document ID 1053147.1> 11gR2 Clusterware and Grid Home - What You Need to Know
- <Document ID 948456.1> Pre 11.2 Database Issues in 11gR2 Grid Infrastructure Environment
- <Document ID 1050908.1> How to Troubleshoot Grid Infrastructure Startup Issues
- <Document ID 887522.1> 11gR2 Grid Infrastructure Single Client Access Name (SCAN)Explained
RAC and Clusterware Monitoring, Troubleshooting and Data Collection Notes
To effectively manage your RAC cluster, it is important to know how to monitor, troubleshoot and collect data, both for your own diagnosis, and also to provide supporting information when logging a service request. The following Notes will give you information on how to do both:
Tools for Monitoring RAC and Clusterware Environments
- <Document ID 301137.1> OS Watcher User Guide
- <Document ID 459694.1> Procwatcher: Script to Monitor and Examine Oracle and CRS
- <Document ID 736752.1> Introducing Cluster Health Monitor
- <Document ID 811151.1> How to install Oracle Cluster Health Monitor (former IPD/OS) on Windows
- <Document ID 314422.1> Remote Diagnostics Agent (RDA)
- <Document ID 422893.1> 11g Understanding Automatic Diagnostic Repository
- <Document ID 559365.1> Using Diagwait as a Diagnostic.
References needed for Troubleshooting and Data Collection
- <Document ID 556679.1> Data Gathering for Troubleshooting RAC Issues
- <Document ID 289690.1> Data Gathering for Troubleshooting CRS issues
- <Document ID 330358.1> CRS 10gR2/ 11gR1/ 11gR2 Diagnostic Collection Guide
- <Document ID 557204.1> Crsd Ocssd Evmd Log Rotation Policy
- <Document ID 563566.1> gc lost blocks diagnostics
- <Document ID 219361.1> Troubleshoot ORA-29740 errors in a RAC Environment.
- <Document ID 745960.1> 11g How to Unpack a Package in to ADR
- <Document ID 443529.1> 11g Quick Steps to Package and Send Critical Error Diagnostic Information to Support
- <Document ID 726446.1> What Is The Difference Between RDA and Configuration Manager
- <Document ID 77483.1> External Support FTP site: Information Sheet
RAC and Clusterware Testing and Performance Tuning
Much of RAC performance tuning is no different from "single instance" tuning. However, there are some additional areas to check in a RAC environment when tuning or troubleshooting performance related issues. The following notes will make you aware of some of the areas where performance tuning and diagnosis has a specific RAC flavor:
- <Document ID 390374.1> Oracle Performance Diagnostic Guide (OPDG)
- <Document ID 181489.1> Tuning Inter-Instance Performance in RAC and OPS
- <Document ID 561414.1> Transactional Sequences in Applications in a RAC environment
- <Document ID 395314.1> RAC Hangs due to small cache size on SYS.AUDSES$
- <Document ID 563566.1> gc lost blocks diagnostics
- <Document ID 361323.1> HugePages on Linux: What It Is... and What It Is Not...
- <Document ID 280939.1> Checklist for Performance Problems with Parallel Execution
- <Document ID 759565.1> Turning NUMA on can cause database hangs
RAC and Clusterware Certification, Maintenance, and Patching References
Maintenance of a RAC environment is similar in many respects to maintaining a single-instance environment. However, with the Clusterware and Grid Infrastructure stacks, there are some additional tasks to be aware of regarding ongoing maintenance, as well as maintenance that may be brought on by changes to your environment. Below are some references speaking to the areas of general maintenance that may be needed or affect RAC environments in a particular way, as well as references to patching, which is a key part of an ongoing maintenance strategy.
Certification References for RAC and Clusterware Environments
- <Document ID 184875.1> How To Check The Certification Matrix for Real Application Clusters
- <Document ID 1065024.1> Database 11g Release 2 Certification Highlights
- <Document ID 337737.1> Oracle Clusterware - ASM - Database Version Compatibility
General Maintenance References for RAC and Clusterware Environments
- <Document ID 428681.1> How to ADD/REMOVE/REPLACE/MOVE Oracle Cluster Registry (OCR) and Voting Disk
- <Document ID 276434.1> Modifying the VIP or VIP Hostname of a 10g or 11g Oracle Clusterware Node.
- <Document ID 283684.1> How to Change Interconnect/Public Interface IP or Subnet in Oracle Clusterware
- <Document ID 754594.1> Potential disk corruption when adding a node to a cluster
- <Document ID 403743.1> VIP Failover Take Long Time After Network Cable Pulled
- <Document ID 563905.1> Implementing LIBUMEM for CRS on Solaris 64 with 3rd Party Clusterware
- <Document ID 92602.1> How to Password Protect the Listener
Patching Best Practices and References for RAC and Oracle Clusterware
As part of an overall maintenance strategy, it is critical that customers have a formal strategy to stay in front of known issues and bugs. To make it easier for customers to obtain and deploy fixes for known critical issues please refer to the following list of references.
- <Document ID 209768.1> Database, FMW, Em Grid Control and OCS Software Error Correction Support Policy.
- <Document ID 316900.1> Oracle 10g Release 2 (10.2) Support Status and Alerts
- <Document ID 454507.1> Oracle 11g Release 1 (11.1) Support Status and Alerts
- <Document ID 742060.1> Release Schedule of Current Database Releases.
- <Document ID 756671.1> Oracle Recommended Patches -- Oracle Database.
- <Document ID 854428.1> Introduction to Patch Set Updates (PSUs)
- <Document ID 850471.1> Oracle Announces First Patch Set Update For Oracle Database Release 10.2
- <Document ID 405820.1> 10.2.0.X CRS Bundle Patch Information.
- <Document ID 810663.1> 11.1.0.X CRS Bundle Patch Information.
- <Document ID 1082394.1> 11.2.0.X Grid Infrastructure Bundle Patch Information.
- <Document ID 363254.1> Applying one-off Oracle Clusterware patches in a mixed version home environment
- <as yet unpublished> Note 844983.1 Apply CRS Bundle Patch Using opatch auto Option.
- <Document ID 761111.1> Online Patches (Hot Patching).
- <Document ID 438314.1> Critical Patch Update - Introduction to Database n-Apply CPUs.
- <Document ID 1082394.1> 11.2.0.X Grid Infrastructure PSU Known Issues
Upgrade and Platform Migration for RAC and Oracle Clusterware
When migrating a RAC cluster to a new platform or upgrading a RAC cluster from an older version of Oracle to the current version of Oracle, the following references will be helpful:
- <Document ID 466181.1> Oracle 10gR2 Upgrade Companion.
- <Document ID 601807.1> Oracle 11gR1 Upgrade Companion.
- <Document ID 785351.1> Oracle 11gR2 Upgrade Companion.
- <Document ID 429825.1> Complete Checklist for Manual Upgrades to 11gR1.
- <Document ID 747457.1> How to Convert 10g Single-Instance database to 10g RAC using Manual Conversion procedure.
- <Document ID 338706.1> Cluster Ready Services (CRS) rolling upgrade
References for running RAC with Applications
- <Document ID 790189.1> Oracle Clusterware and Application Failover Management
RAC and Oracle E-Business
- <Document ID 362135.1> Configuring Oracle Applications Release 11i with Oracle10g Release 2 Real Application Clusters and Automatic Storage Management.
- <Document ID 380489.1> Using Load-Balancers with Oracle E-Business Suite Release 12
- <Document ID 727171.1> Implementing Load Balancing On Oracle E-Business Suite - Documentation For Specific Load Balancer Hardware
- <Document ID 217368.1> Advanced Configurations and Topologies for Enterprise Deployments of E-Business Suite 11i
- <Document ID 403347.1> MAA Roadmap for the E-Business Suite
- <Document ID 388577.1> Using Oracle 10gR2 RAC and Automatic Storage Management with Oracle E-Business Suite Release 12
RAC ASM Documentation
Automatic Storage Management (ASM) is an evolution in file system and volume management functionality for Oracle database files. ASM further enhances automation and simplicity in storage management that is critical to the success of the Oracle grid architecture. ASM also improves file system scalability and performance, manageability and database availability for RAC environments. Use of ASM with RAC is an Oracle Best Practice.
- <Document ID 1187723.1> Master Note for Automatic Storage Management (ASM)
- <Document ID 1187674.1> Master Note for Oracle Database Machine and Exadata Storage Server
Data Guard provides the management, monitoring, and automation software infrastructure to create and maintain one or more standby databases to protect Oracle data from failures, disasters, errors, and data corruptions. As users commit transactions at a primary database, Oracle generates redo records and writes them to a local online log file.
- <Document ID 1101938.1> Master Note for Data Guard
Using My Oracle Support Effectively
- <Document ID 374370.1> New Customers Start Here
- <Document ID 736737.1> My Oracle Support - The Next Generation Support Platform
- <Document ID 747242.5> My Oracle Support Configuration Management FAQ
- <Document ID 868955.1> My Oracle Support Health Checks Catalog
- <Document ID 166650.1> Working Effectively With Global Customer Support
- <Document ID 199389.1> Escalating Service Requests with Oracle Support Services
Keywords and search terms: RAC, ASM, CRS, Cluster, Clusterware, DB Machine, Exadata, GI, Grid, Infrastructure, OCR, Oracle, RAC, Votedsk, votedisk, 'Voting Disk', ohasd, ocssd, crsd, evmd, srvctl, crsctl, 'Grid Infrastructure', 'Real Application Clusters', HA, Scalability, ACFS, ADVM
*** - For reference to the content in this blog, and for the latest and most up to date information, refer to Note:1096952.1 Master Note for Real Application Clusters (RAC) Oracle Clusterware and Oracle Grid Infrastructure