RAC Survival Kit: Real Application Clusters Troubleshooting and Information

Is your configuration certified?  The certification matrix can be found using Note 184875.1.

Before opening any RAC issues, customers should review the certification matrix.  To eliminate any problems on unsupported configurations and any certification inquiries, the certification matrix for RAC/OPS should be checked before opening an issue to verify a certified database, OS, and cluster are being used.  Please note that the OS vendor certifies all hardware unless otherwise noted.

Yes

No     Please use a certified configuration. 

Have you installed all required OS patches?

Often problems are discovered that turn out to be operating system issues that are resolved in the latest OS patch.  Since Real Application Clusters gets installed on top of the OS, we should verify that the OS version is up to date to ensure that RAC is running in a stable operating environment.

Patch information is available in the Release Notes at http://otn.oracle.com/documentation/content.html.

1.  Click on the appropriate platform for your OS.

2.  Choose the latest cd pack for your version.

3.  Click on the release notes for your platform.

4.  Verify the system requirements.

Yes

No    Please apply all mandatory OS patches.

Please choose the category that best describes your issue:

Concepts and Design  -  RAC architecture, features, implementation considerations, and HA questions and issues

Setup and Configuration  -  OS configuration, installation, relinking, database creation, and DBCA questions and issues

SRVCTL and GSD Issues  -  Questions and issues when using SRVCTO and/or GSD to adminster a RAC database.

Database Administration  -  ORA-29702 Errors, Administrative questions and issues for existing clustered databases

Internal Errors  -  Troubleshooting ORA-600 or ORA-7445 errors

Backup and Recovery  -  RMAN and manual backup and recovery with raw and clustered filesystems

Performance Issues and Hangs  -  Diagnosing and resolving performance and/or hang issues 

Upgrade and Migration  -  Performing software and database upgrades, patch installations, and migration with a clustered database

Other

[Back to top]

Concepts and Design

The answers to most Concepts and Design questions can be found within the documentation.  

The RAC documentation is available via Note 188135.1. 

Please select the category that best describes your issue:

  Introduction to Real Application Clusters - What is RAC?  Learn More About RAC Architecture and New Features

  Implementation Considerations - Things to consider when implementing and maintaining RAC databases

  High Availability and RAC - High Availability features and considerations with RAC databases.

  Other

[Back to top]

Setup and Configuration

Please select the category that best describes your issue:

  OS Configuration  OS specific Cluster Information and Setup

  Installation  Step-By-Step Installation instructions

  Relinking  How to resolve Relinking issues

  Database Creation Manual database creation troubleshooting

  Using DBCA   Using DBCA, common issues and errors

  Turn on/off RAC Option How to disable or enable RAC

  Other

[Back to top]

SRVCTL and GSD Issues

Please select the category that best describes your issue:

  PRK Errors  - Receiving PRK% Errors When Using SRVCTL or GSD

  Usage and Commands - Common SRVCTL commands

  Tracing -  How to trace SRVCTL and GSD

  Other

[Back to top]

Database Administration

Please select the category that best describes your issue:

  ORA-29702 Errors  Receiving ORA-29702 Errors

  Parameter Management  How to set and manage parameters in RAC with a text file or SPFILE

  Enterprise Manager for RAC  How to set up Enterprise Manager to administer RAC

  Recreating Control Files -  How to recreate a controlfile in RAC

  Storage - adding redo, undo, etc  Storage management and issues with RAC

  Changing Archivelog Mode - How to change archivelog mode in RAC

  Other

[Back to top]

Internal Errors

Please select the category that best describes your issue:

  What is an internal error? Information about how and why an internal error is generated

  Troubleshooting ORA-600 and ORA-7445 Errors in RAC - You are receiving an internal error in RAC  

  Other

[Back to top]

Backup and Recovery  

Please select the category that best describes your issue:

  RMAN Backup â�� Using RMAN to backup a RAC database

  RMAN Recovery  Using RMAN to recover a RAC database

  Manual Backup  Backing up a RAC database with cold and hot backups

  Manual Recovery  Recovering a RAC database from cold and hot backups

  Other

[Back to top]

Performance Issues and Hangs

Is the problem due to a slow/hung query or is the entire database affected?  Try running the following SQL statement in SQL*Plus: 

select inst_id, sid, state, event, seconds_in_wait seconds

from gv$session_wait

where seconds_in_wait > 5

and event not in ('rdbms ipc message','smon timer','pmon timer',

'SQL*Net message from client','lock manager wait for remote message',

'ges remote message', 'client message', 'SQL*Net more data from client',

'pipe get', 'Null event', 'PX Idle Wait', 'single-task message',

'PX Deq: Execution Msg', 'KXFQ: kxfqdeq - normal deqeue', 'listen endpoint status',

'slave wait','wakeup time manager');

If the query does not return, you may have a hung database.

If the query returns no rows, you may have database performance issues.

If the query returns rows, you may have hung sessions.

Please select the category that best describes your issue:

  Hung Database All queries against the database hang 

  Database Performance Issues Overall database performance is slower than normal

  Hung Sessions One or more sessions in the database are hung

  Slow Query A particular query is slower than normal (SQL Tuning)

  Other

[Back to top]

Upgrade and Migration

Please select the category that best describes your issue:

  OS Software Upgrades Questions and issues related to OS software upgrades and RAC 

  Database Upgrades and Migrations - Questions and issues related to upgrading a RAC database

  Patch Installation Questions and issues related to installing patches or patchsets on RAC

  Other