X

Recent Posts

Oracle Database Support News - your monthly guide to Database support information

Oracle Database Support News - your monthly guide to Database support informationWant to keep up-to-date with the latest Database Support News as well as accessing interesting technical Database articles?Check out the monthly Oracle Premier Support - Oracle Database Support News (Document 230.1) for content such as Feature Article: discussing a technical topic, best practice, new support tool or new features in a release Database Technology Updates: providing up-to-date information about the current database releases such as release schedule, known Issues, relevant My Oracle Support documents as well as release information about other Database Technology related products. Support Tips: upcoming advisor webcasts, My Oracle Support updates and new or updated My Oracle Support documents. Use Oracle Database Support News Archive (Document 1284265.1) to look up previous versions or find a specific feature articles. Don't delay and bookmark Database Support News today so you always access the latest edition - December 2015: Openworld 2015: Database, Enterprise Manager and Engineered System Support Sessions (replays, presentations etc) Oracle Database Technology Updates  Oracle Database 12c Oracle Database 11g Release 2 Oracle Database 11g Release 1 Support Tips New/Updated Database Support Blogs Upcoming webcasts Other Updates Patch Updates

Oracle Database Support News - your monthly guide to Database support informationWant to keep up-to-date with the latest Database Support News as well as accessing interesting technical Database...

Master Notes

Oracle Resource Manager and DBMS_RESOURCE_MANAGER

The Oracle Database Resource Manager (the Resource Manager) is an infrastructure that provides granular control of database resources allocated to users, applications, and services. The Oracle Database Resource Manager (RM) enables you to manage multiple workloads that are contending for system and database resources. Resources are allocated to users according to a resource plan specified by the database administrator. The plan specifies how the resources are to be distributed among resource consumer groups, which are user sessions grouped by resource requirements. A resource plan directive associates a resource consumer group with a plan and specifies how resources are to be allocated to the group. DBRM (database resource manager) process is responsible for setting resource plans and other resource manager related tasks. Oracle Documentation on Oracle Database Resource Manager  For 12cR1 - Oracle® Database Administrator's Guide 12c Release 1 (12.1) - Part Number E17636-17 Managing Resources with Oracle Database Resource Manager http://docs.oracle.com/database/121/ADMIN/dbrm.htm#ADMIN027 DBMS_RESOURCE_MANAGER - Reference http://docs.oracle.com/database/121/ARPLS/d_resmgr.htm Using Oracle Resource Manager on pluggable databases (PDBs) in a multitenant container database (CDB) Sqlplus: http://docs.oracle.com/database/121/ADMIN/cdb_dbrm.htm#ADMIN13774 EM: http://docs.oracle.com/database/121/ADMIN/cdb_dbrm_em.htm#ADMIN14129 For 11gR2 - Oracle® Database Administrator's Guide 11g Release 2 (11.2) - Part Number E25494-02 Managing Resources with Oracle Database Resource Manager http://docs.oracle.com/cd/E11882_01/server.112/e25494/dbrm.htm#ADMIN027 For 11gR1 - Oracle® Database Administrator's Guide 11g Release 1 (11.1) - Part Number B28310-04 Managing Resource Allocation with Oracle Database Resource Manager http://docs.oracle.com/cd/B28359_01/server.111/b28310/dbrm.htm#i1010776 DBMS_RESOURCE_MANAGER - Reference http://docs.oracle.com/cd/B28359_01/appdev.111/b28419/d_resmgr.htm Please refer the Note 1119407.1 for the Video Tutorial on Resource Manager, which includes both Concepts and Lab practices Note 1119407.1   Resource Manager Training (11.2 features included) [Video]  Setup and Configuration The DBMS_RESOURCE_MANAGER package is to be used to setup and configure resourcemanager. The DBMS_RESOURCE_MANAGER package maintains plans, consumer groups, and plan directives. It also provides semantics so that you may group together changes to the plan schema. Resource Manager Administration Privileges You must have the system privilege ADMINISTER_RESOURCE_MANAGER to administer the Resource Manager. This privilege (with the ADMIN option) is granted to database administrators through the DBA role. GRANT_SYSTEM_PRIVILEGE and REVOKE_SYSTEM_PRIVILEGE procedures of DBMS_RESOURCE_MANAGER_PRIVS PL/SQL package are used to grant & revoke this. The ADMINISTER_RESOURCE_MANAGER system privilege can only be granted or revoked using the DBMS_RESOURCE_MANAGER_PRIVS package. It cannot be granted or revoked through the SQL GRANT or REVOKE statements. begindbms_resource_manager_privs.grant_system_privilege(grantee_name => 'DBADMIN',admin_option => false);end;/  In order to assign users to resource consumer groups other than default, the users must have been granted the system privilege grant_switch_consumer_group in order to be able to switch to that consumer group. GRANT_SWITCH_CONSUMER_GROUP and REVOKE_SWITCH_CONSUMER_GROUP procedures of DBMS_RESOURCE_MANAGER_PRIVS PL/SQL package are used to grant & revoke this. begindbms_resource_manager_privs.grant_switch_consumer_group(grantee_name => 'PUBLIC',consumer_group => `GROUP_NAME');end;/  Resources Managed by the Resource Manager There are many types of resources, which are managed by the RM like CPU, Idle time, Undo pool, etc. Distribute CPU is the widely used RM feature. Complete list of resources managed by the RM is listed in the documentation below http://docs.oracle.com/cd/E11882_01/server.112/e25494/dbrm.htm#ADMIN11851 Elements of the Resource Manager As explained in the concepts section, Resources are allocated to users according to a resource plan, distributed among resource consumer groups and resource plan directive used to specify how resources are to be allocated to the group. So The elements of the Resource Manager are Resource consumer group, Resource plan, Resource plan directive. These elements are stored in tables in the data dictionary. So you can view information about them with Resource manager related data dictionary views. Enabling the Database Resource Manager You can enable the Database Resource Manager by setting the RESOURCE_MANAGER_PLAN initialization parameter.This parameter specifies the top plan, identifying the plan schema to be used for this instance. If noplan is specified with this parameter, the Database Resource Manager is not activated. The followingexample activates the Database Resource Manager and assigns the top plan as mydb_plan. RESOURCE_MANAGER_PLAN = mydb_plan You can also activate or deactivate the Database Resource Manager, or change the current top plan, usingthe ALTER SYSTEM statement. In this example, the top plan is specified as mydb_plan.ALTER SYSTEM SET RESOURCE_MANAGER_PLAN = mydb_plan; An error message is returned if the specified plan does not exist in the data dictionary.To deactivate the Database Resource Manager, issue the following statement:ALTER SYSTEM SET RESOURCE_MANAGER_PLAN = ''; Note 223572.1     Resource Manager Setup and Configuration for the Database  A database can have multiple plans defined, but only one can be active at a given time per instance. Different instance of a RAC can activate different plans, if required. Getting Started with Resource Manager If you use Resource Manager and have up to eight consumer groups and needed only CPU resource allocation to control then simple resource plan is adequate, using the DBMS_RESOURCE_MANAGER.CREATE_SIMPLE_PLAN procedure. When your situation calls for a more complex resource plan, you must create the plan, with its directives and consumer groups, in a staging area called the pending area, and then validate the plan before storing it in the data dictionary (Complex Resource Plan). Resource plans can be associated with scheduler windows or jobs, So Integration with Oracle Database Scheduler (DBMS_SCHEDULER) is possible. Resource plans do not impose a hard limit, meaning if there is no other session is taking resource allocated, then this session can take all the available resource and not just the limit allocated. Please refer Note 1119407.1 for the Video Tutorials on Resource Manager, which contains both Concepts and Lab practices Note 1119407.1   Resource Manager Training (11.2 features included) [Video]  Create Simple Resource Plan using CREATE_SIMPLE_PLAN procedure You can specify up to eight consumer groups with this procedure. The only resource allocation method supported is CPU. The plan uses the EMPHASIS CPU allocation policy (the default) and each consumer group uses the ROUND_ROBIN scheduling policy (also the default). The following PL/SQL block creates a simple resource plan with two user-specified consumer groups: BEGIN DBMS_RESOURCE_MANAGER.CREATE_SIMPLE_PLAN(SIMPLE_PLAN => 'mydb_plan',  CONSUMER_GROUP1 => 'MYGROUP1', GROUP1_PERCENT => 80,  CONSUMER_GROUP2 => 'MYGROUP2', GROUP2_PERCENT => 20);END;/ Executing the preceding statements creates the following plan: Consumer Group Level 1 Level 2 Level 3 SYS_GROUP 100% - - MYGROUP1 - 80% - MYGROUP2 - 20% - OTHER_GROUPS - - 100% Each consumer group specified in the plan is allocated its CPU percentage at level 2. Also implicitly included in the plan are SYS_GROUP (a system-defined group that is the initial consumer group for the users SYS and SYSTEM) and OTHER_GROUPS. The SYS_GROUP consumer group is allocated 100% of the CPU at level 1, and OTHER_GROUPS is allocated 100% of the CPU at level 3. Refer Oracle Documentation for "Creating a Simple Resource Plan" mentioned below http://docs.oracle.com/cd/B28359_01/server.111/b28310/dbrm002.htm#ADMIN11861 Create Complex Resource Plan using CREATE_PLAN procedure A complex resource plan is any resource plan that is not created with the CREATE_SIMPLE_PLAN procedure. Note that any resource plan created by the Simple Resource Plan method using CREATE_SIMPLE_PLAN can also be created by the Complex Resource Plan (this) method, also creating resource plan with this method give more flexibility and can have more granularity. When your situation calls for a more complex resource plan, you must create the plan, with its directives and consumer groups, in a staging area called the pending area, and then validate the plan before storing it in the data dictionary. You can create sub-plans (plan within plans) for more granular resource management. If a consumer-group does not use its CPU-percentage, the remainder falls through to the next level, and the administrator can define explicit what to do with it. Up to eight levels can be specified. Here is a general guideline of the steps in implementing a Database Resource Manager plan: 1. Create resource plans2. Create resource consumer groups3. Create resource plan directives4. Grant switch privilege for resource consumer groups to users or roles5. Assign users to resource consumer groups6. Specify a plan to be used by the instance In addition to the above steps, we need to create these elements in a staging area called the pending area, so the below steps are related to the pending area, which can be repeated each time to create, validate and submit this pending area. i. Create pending areaii. Validate Pending areaiii. Submit pending area Following example assign 80% of the CPU-resources to sessions in the OLTP-group, and 10% to ADHOC- and remaining 10% to BATCH-group each. assuming we have created 3 users, ADHOC, BATCH, and OLTP and assigned a consumer group for each of them. The CPU-resources are allocated across all consumer-groups by the percentages given to each group. Specifying a zero percentage for a consumer-group means that it should not get any processing resources at that level, given a CPU consumption of 100%. This can force sessions for that consumer-group to wait forever in an overloaded system. It does not mean that users belonging to OTHER_GROUPS (having 0%) never are able to run, but for a fully loaded system, sessions for OTHER_GROUPS have to wait for the others to finish work. If this is the only one active session running, it is of course allowed to utilize all available CPU resources up to 100%. But if another session is started in another group, Resource Manager will balance CPU resources according to the plan directives. ---Create pending area for plan, consumer group and directives begin dbms_resource_manager.create_pending_area();end;/  ---1. Create resource plans begin dbms_resource_manager.create_plan(  plan => 'MYDB_PLAN',  comment => 'Resource plan/method for Single level sample');end;/  ---2. Create resource consumer groups begin dbms_resource_manager.create_consumer_group(  consumer_group => 'OLTP_Group',  comment => 'Resource consumer group/method for online users sessions'); dbms_resource_manager.create_consumer_group(  consumer_group => 'BATCH_Group',  comment => 'Resource consumer group/method for users sessions who run batch jobs'); dbms_resource_manager.create_consumer_group(  consumer_group => 'ADHOC_Group',  comment => 'Resource consumer group/method for users sessions who execute Ad-Hoc Queries');end;/  ---3. Create resource plan directives begin dbms_resource_manager.create_plan_directive(   plan => 'mydb_plan',   group_or_subplan => 'OLTP_Group',   comment => 'Online day users sessions at level 1',   cpu_p1 => 80,   parallel_degree_limit_p1 => 0); dbms_resource_manager.create_plan_directive(   plan => 'mydb_plan',   group_or_subplan => 'BATCH_Group',   comment => 'batch day users sessions at level 1',   cpu_p1 => 10,   parallel_degree_limit_p1 => 10); dbms_resource_manager.create_plan_directive(   plan => 'mydb_plan',   group_or_subplan => 'ADHOC_Group',   comment => 'ADHOC day users sessions at level 1',   cpu_p1 => 10,   parallel_degree_limit_p1 => 5); dbms_resource_manager.create_plan_directive(   plan => 'mydb_plan',   group_or_subplan => 'OTHER_GROUPS',   comment => 'OTHER_GROUPS day users sessions at level 1',   cpu_p1 => 0,   parallel_degree_limit_p1 => 0);end;/  ---Validate Pending area for plan, consumer group and directives begin dbms_resource_manager.validate_pending_area();end;/  ---submit pending area for plan, consumer group and directives begin dbms_resource_manager.submit_pending_area();end;/  ---Next, we have to grant privileges in order to assign consumer groups to users:---We need to again create a pending area, till we submit the pending area ---Create pending area for privilages, roles and assign users begin dbms_resource_manager.create_pending_area();end;/  ---4. Grant switch privilege for resource consumer groups to users or roles begin dbms_resource_manager_privs.grant_switch_consumer_group(   grantee_name => 'ADHOC',   consumer_group => 'ADHOC_Group',   grant_option => FALSE); dbms_resource_manager_privs.grant_switch_consumer_group(   grantee_name => 'OLTP',   consumer_group => 'OLTP_Group',   grant_option => FALSE); dbms_resource_manager_privs.grant_switch_consumer_group(   grantee_name => 'BATCH',   consumer_group => 'BATCH_Group',   grant_option => FALSE);end;/  ---5. Assign users to resource consumer groups begin dbms_resource_manager.set_initial_consumer_group(  user => 'ADHOC',  consumer_group => 'ADHOC_Group'); dbms_resource_manager.set_initial_consumer_group(  user => 'OLTP',  consumer_group => 'OLTP_Group'); dbms_resource_manager.set_initial_consumer_group(  user => 'BATCH',  consumer_group => 'BATCH_Group');end;/  ---Validate Pending area for privilages, roles and assign users begin dbms_resource_manager.validate_pending_area();end;/  ---submit pending area for privilages, roles and assign users begin dbms_resource_manager.submit_pending_area();end;/  6. Specify a plan to be used by the instance ALTER SYSTEM SET RESOURCE_MANAGER_PLAN = MYDB_PLAN;  Refer Oracle Documentation for "Creating a Complex Resource Plan" mentioned below and the Note 106948.1 http://docs.oracle.com/cd/B28359_01/server.111/b28310/dbrm003.htm#ADMIN11862 Checking Configuration and Monitoring Resource Manager  We can use dynamic performance views to help us monitor the results of our Oracle Database Resource Manager  Resource Plans: select plan, cpu_method, status from dba_rsrc_plans order by 1; PLAN          CPU_METHOD  STATUS------------- ----------- ---------MYDB_PLAN     EMPHASIS    ACTIVE1 row selected.  Resource Consumer Groups: select consumer_group,cpu_method, status from dba_rsrc_consumer_groups order by 1; CONSUMER_GROUP CPU_METHOD STATUS------------------------------ ------------------------------ ------------------------------ADHOC_GROUP ROUND-ROBIN ACTIVEBATCH_GROUP ROUND-ROBIN ACTIVEDEFAULT_CONSUMER_GROUP ROUND-ROBIN ACTIVEOLTP_GROUP ROUND-ROBIN ACTIVEOTHER_GROUPS ROUND-ROBIN ACTIVE5 rows selected.  Resource Plan Directives: select plan, group_or_subplan, type, cpu_p1, cpu_p2, cpu_p3, cpu_p4, statusfrom dba_rsrc_plan_directives order by 1,2,3,4,5,6; PLAN GROUP_OR_SUBPLAN TYPE CPU_P1 CPU_P2 CPU_P3 CPU_P4 STATUS---------------------- -------------------- -------------- ---------- ---------- ---------- ---------- -------MYDB_PLAN ADHOC_GROUP CONSUMER_GROUP 10 0 0 0 ACTIVEMYDB_PLAN BATCH_GROUP CONSUMER_GROUP 10 0 0 0 ACTIVEMYDB_PLAN OLTP_GROUP CONSUMER_GROUP 80 0 0 0 ACTIVEMYDB_PLAN OTHER_GROUPS CONSUMER_GROUP 0 0 0 0 ACTIVE4 rows selected.  Resource consumer group privileges: select * from dba_rsrc_consumer_group_privs; GRANTEE GRANTED_GROUP GRA INI------------------------------ ------------------------------ --- ---BATCH BATCH_GROUP NO YESOLTP OLTP_GROUP NO YESPUBLIC DEFAULT_CONSUMER_GROUP YES YESADHOC ADHOC_GROUP NO YES4 rows selected.  These above users have the privilege to switch to a consumer group other than default. Default consumer groups for users: select username, initial_rsrc_consumer_group from dba_users; USERNAME INITIAL_RSRC_CONSUMER_GROUP------------------------------ ------------------------------SYS DEFAULT_CONSUMER_GROUPSYSTEM DEFAULT_CONSUMER_GROUPOLTP OLTP_GROUPADHOC ADHOC_GROUPOUTLN DEFAULT_CONSUMER_GROUPDBSNMP DEFAULT_CONSUMER_GROUPBATCH BATCH_GROUP7 rows selected.  Monitor CPU Usage and Waits by Consumer Group: In this example, the server has 16 CPUs which means that a maximum of 16 sessions can run at any time. Refer Note 1338988.1 select to_char(m.begin_time, 'HH:MI') time, m.consumer_group_name, m.cpu_consumed_time / 60000 avg_running_sessions, m.cpu_wait_time / 60000 avg_waiting_sessions, d.mgmt_p1*(select value from v$parameter where name = 'cpu_count')/100 allocation from v$rsrcmgrmetric_history m, dba_rsrc_plan_directives d, v$rsrc_plan p where m.consumer_group_name = d.group_or_subplan and p.name = d.plan order by m.begin_time, m.consumer_group_name; TIME NAME AVG_RUNNING AVG_WAITING ALLOCATION  ----- ------------------ ----------- ----------- ----------13:34 ADHOC_GROUP 1.76 8.4 .813:34 BATCH_GROUP 2.88 .4 2.413:34 ETL_GROUP 0 0 2.413:34 INTERACTIVE_GROUP 10.4 6.8 5.613:34 OTHER_GROUPS .32 .08 .813:34 SYS_GROUP 0 0 4  The Below Articles gives more Scripts/Tips for Monitoring Oracle Resource Manager  Note 1338988.1  Scripts and Tips for Monitoring CPU Resource Manager Note 106948.1     Database Resource Manager samples includes scripts and quires to check/monitor Note 240877.1    Script to monitor PX limits from Resource Manager for active sessions Note 428325.1    How to Find the Sessions Killed by MAX_IDLE_BLOCKER_TIME Plan Directive Monitoring Oracle Database Resource Manager http://docs.oracle.com/cd/B28359_01/server.111/b28310/dbrm009.htm#ADMIN11906 Resource Manager Data Dictionary Views http://docs.oracle.com/cd/B28359_01/server.111/b28310/dbrm011.htm#ADMIN11916 Cleaning the Resource Manager Plan In order to delete the current plan, first it has to be `switched off' or another plan has to be selected as current. alter system set resource_manager_plan='';  ---Create pending area for delete plan and consumer group begindbms_resource_manager.create_pending_area();end;/  --Delete plan (assuming we have created 2 plans - MYDB_PLAN, ANY_OTHER_CREATED_PLAN) begin dbms_resource_manager.delete_plan(   plan => 'MYDB_PLAN'); dbms_resource_manager.delete_plan(   plan => 'ANY_OTHER_CREATED_PLAN');end;/  --Delete consumer group begin dbms_resource_manager.delete_consumer_group(   consumer_group => 'OLTP_GROUP'); dbms_resource_manager.delete_consumer_group(   consumer_group => 'BATCH_GROUP'); dbms_resource_manager.delete_consumer_group(   consumer_group => 'ADHOC_GROUP');end;/  ---Validate Pending area for delete plan and consumer group begin dbms_resource_manager.validate_pending_area();end;/  ---submit pending area for delete plan and consumer group begin dbms_resource_manager.submit_pending_area();end;/  --Clearing only the Pending Area, if you dont want to validate and submit BEGIN DBMS_RESOURCE_MANAGER.CLEAR_PENDING_AREA();END;/  Applications of Resource Manager (Additional Examples/Samples) Resource Manager can be used in various ways to control and distribute the resources in Oracle Database Environment. The below Articles provides more samples/examples using resource manager for the database for different scenarios.       Note 106948.1     Database Resource Manager samples Note 471265.1     Example: How to control CPU Resources using the Resource Manager and Scheduler Note 1266650.1   How To Switch Users Between Resource Plans Based On Schedule Window Note 1168574.1   How to Force RESOURCE_MANAGER_PLAN with the Scheduler window boundaries Note 971991.1  How To Incorporate Pre-Defined jobs into your Resource Manager Plan Note 277419.1     Prevention of long-running queries with RM MAX_EST_EXEC_TIME Plan Directive Note 1359043.1   Configuring Parallel Statement Queuing to Manage Mixed Priority Workloads in a DataWarehouse Note 470764.1     How to Control Resources used by a Statisics Job using the Resource Manager Note 848666.1     How to Limit max number of concurrently active sessions with Resource Manager Note 1321099.1   How to limit the Parallel Degree for PX Queries with Resource Manager Note 1345429.1   Parallel Statement Queuing in 11.2.0.2 Note 1269321.1   Automatic Degree of Parallelism in 11.2.0.2 Note 1388634.1   Reverse Engineering a Resource Manager Plan From Within the Database Note 1674897.1   How to Generate Resource Manager Configuration DDLs Using Dbms_metadata Packages Note 987436.1     How to Identify the Session that Started the Resource Manager Pending Area Note 398985.1     How to set Resource Manager to Disconnect Idle Users after a Predetermined time? Note 233651.1     Why Resource Manager Is Not Distributing CPU As Per Resource Plan Note 1557657.1   Using Resource Manager to Detect and Kill Idle Sessions     Note 786346.1     Resource Manager and Sql Tuning Advisory DEFAULT_MAINTENANCE_PLAN Note 1296086.1   Using PL/SQL To Execute SQL or Stored Procedure/Function Calls for a Set of Users Note 727062.1     Configuring and using Calibrate I/O Note 1393405.1   How To Delete Calibrate I/O In The Instance Note 884082.1     Resource Manager Enhancements in Oracle Database 11g Note 451889.1     Logical Standby and Resource Manager Note 1339769.1   Master Note for Oracle Database Resource Manager for EXADATA Note 1358709.1   Configuring Resource Manager for Mixed Workloads in a Database Note 1362445.1   Configuring and Monitoring Instance Caging Oracle Database Resource Manager Examples http://docs.oracle.com/cd/B28359_01/server.111/b28310/dbrm006.htm#ADMIN11012 12c: Using Oracle Resource Manager on pluggable databases (PDBs) in a multitenant container database (CDB) Oracle Resource Manager can be used to allocate resources to pluggable databases (PDBs) in a multitenant container database (CDB).In a non-CDB, you can use Resource Manager to manage multiple workloads that are contending for system and database resources. However, in a CDB, you can have multiple workloads within multiple PDBs competing for system and CDB resources. Please refer the Oracle Documentation Below: Using Oracle Resource Manager for PDBs with SQL*Plus http://docs.oracle.com/database/121/ADMIN/cdb_dbrm.htm#ADMIN13774 Using Oracle Resource Manager for PDBs with Cloud Control http://docs.oracle.com/database/121/ADMIN/cdb_dbrm_em.htm#ADMIN14129 Oracle Resource Manager and Operating System (OS) Resource manager The Oracle Database Resource Manager solves many resource allocation problems that an operating system does not manage well like Excessive overhead, Inefficient scheduling, Inappropriate allocation of resources, Inability to manage database-specific resources. The Resource Manager helps overcome these problems by giving the database more control over allocation of hardware resources and enabling you to prioritize work within the database. You can classify sessions into groups based on session attributes, and then allocate resources to these groups to optimize hardware utilization. Refer the Note 1102753.1 mentioned below, if you want to enable Oracle Resource Manager and an Operating System Resource Manager together on one system Note 1102753.1    Operating System Resource Manager and Oracle Resource Manager support  Oracle Database Services and Resource Manager Database Resource Manager can map services to consumer groups. Therefore, you can automatically manage the priority of one service relative to others. You can use consumer groups to define relative priority in terms of either ratios or resource consumption. Services are integrated with the Resource Manager, which enables you to restrict the resources that are used by the users who connect to an instance by using a service. The Resource Manager enables you to map a consumer group to a service so that users who connect to an instance using that service are members of the specified consumer group. Troubleshooting/Known Issues The following Troubleshooting Guide (TSG) can assist you for investigating the knownissues and will help you to fix those. Note 975455.1  Troubleshooting Guide: Oracle Resource Manager   Recommended Patches The following Notes will list out the recommended patches for Oracle Database Resource Manager, so that you can use RM without issues. Note 1340184.1   Recommended Patches for Managing Runaway Queries with RM (Supercedes Note 1208133.1)  Note 1339803.1   Recommended Patches for CPU Resource Manager (Supersedes Note 1207483.1) Note 1340172.1   Recommended Patches for Instance Caging (Supersedes Note 1208064.1) Note 1340178.1   Recommended Patches for Resource Manager Utilization Limits(Supercedes Note 1208104.1)  Note 1340180.1   Recommended Patches for Parallel Statement Queuing Note 1340181.1   Recommended Patches for Exadata I/O Resource Manager 

The Oracle Database Resource Manager (the Resource Manager) is an infrastructure that provides granular control of database resources allocated to users, applications, and services. The Oracle...

Master Notes

EM Express 12c Database Administration Page FAQ

EM Express is a database management tool that focuses primarily on performance management and monitoring.  However, it does include basic administration functions to assist the database administrator. When users first log into EM Express, they see the home page which displays information about the database, as well as performance data, such as resource usage, average active sessions, and monitored SQL statements.The three main categories of administrative tasks are: 1- Configuration The four main features available in the Configuration menu are Initialization Parameters, Memory Configuration, Database Feature Usage and Database Properties.Database Administrators can view and set initialization parameters for one or more instances of the database.  They can also determine how memory has been configured, and in particular, the amount of memory allocated to the SGA and PGA, the top processes consuming memory, and whether or not any of the automatic memory management features are in use.  In addition, memory advisor data is also displayed on the page.  To see an example of the Memory page, click here. 2- Storage As part of Storage administration, DBAs can manage redo logs, archive logs, and tablespaces (view space consumption, create and/or alter tablespaces).  In addition, DBAs are also given some visibility into the control file contents.  DBAs can also view the Undo Advisor, as well as undo statistics, such as the undo generation rate and undo space usage.  To see an example of the Undo Page, click here. 3- Security DBAs can use the features available in the Security menu to manage users, roles and profiles by creating, altering or granting and revoking privileges from them.   Frequently Asked Questions How can I see Advisor Data on the Memory Page? The advisor data visible at the top right panel of the Memory page is dependent on the memory configuration settings that are in use. If Automatic Memory Management is used, then only the Memory Advisor is displayed on the page. If Automatic Shared Memory Management is used, then the SGA Advisor, along with the PGA Advisor is displayed.If Manual Shared Memory Management is used, then the Buffer Cache Advisor, along with the PGA Advisor is displayed I'm trying to perform a task that I used to do in DB Control.  Where is it in EM Express? EM Express is not a direct replacement for DB Control.  EM Express is built inside the database server and there are limitations as to what it can do. EM Express is also focused on providing performance management and monitoring functionality, and is not meant to be an all around database administration tool. Oracle Database 12c also introduces Oracle Enterprise Manager Cloud Control (Cloud Control). Cloud Control provides you with complete monitoring across the Oracle technology stack and non-Oracle components. You must install Cloud Control separately after installing Oracle Database 12c , No additional license required .Enterprise Manager Cloud Control supports Oracle Database 12c targets, including multitenant container databases (CDBs), pluggable databases (PDBs), non-CDBs, Oracle Real Application Clusters (Oracle RAC) databases, and Oracle Automatic Storage Management (Oracle ASM) databases.Note:Enterprise Manager Database Control is no longer available in Oracle Database 12c. You can use Enterprise Manager Cloud Control 12c or Enterprise Manager Database Express 12c to manage your Oracle Database 12c databases. Why do I see a "Warning" icon on the Memory Page? The warning icon is displayed if the memory configuration is different across the instances in your RAC cluster.  For example, this may be displayed if one instance uses Automatic Memory Management, while the other instance uses Manual Shared Memory Management. In the Memory Page, why are the sizes in the Configuration region different from the Allocation region? The configuration section shows the parameters that have been specified by the user, while the allocation section shows the amount of memory that has been allocated.  For example, if the system is using sga_target, the user can still specify minimum sizes for the different pools, and these sizes will be displayed in the Configuration section.  However, the allocation section will show what the system has allocated for the pools.For an example, click here.  shared_pool_size=512M in the Configuration section, yet the Allocation breakdown shows 1G has been allocated. In the Memory Page, why does sga_target in the Configuration region show a different value from the Current Setting in the SGA Advisor region? This can happen on RAC.  The Configuration region shows the aggregate across all instances in the RAC target.  However, when showing the SGA Advisor data, if the sga_target is the same on all instances, it shows the advisor data based on the sga_target setting of the instances, with the estimated percentage of time taking into account the time saved over all instances.For an example of a 2-node RAC environment with sga_target=2000M on each node, click here.  The page shows sga_target=3.9G, but the Current Setting in SGA Advisor shows 2G. I'm on a RAC system, and the Home Page shows "Host CPU".  Which host is it showing? This shows cluster CPU utilization.  If you have a symmetric cluster, this is the average of CPU utilization across all the nodes.  If you have an asymmetric cluster, then this shows the utilization over the entire cluster, e.g. sum of CPU used / sum of available CPU.Similarly, the other charts in the Resources region, such as Active Sessions and Memory, also show aggregated values over the entire cluster.  For an example, click here.To see the details per instance, click on the RAC link in the Status region, from the live UI. Why is the Storage menu missing the Tablespaces entry in a CDB? Tablespaces are local to a PDB, so the Tablespaces entry is not available at the root container.  To view and manage tablespaces or datafiles on a PDB, go to the EM Express configured on that PDB. Can I manage and create Common Users with EM Express? Yes, Common Users can be created and managed from the root container.At the PDB container, you can only create and manage local users. Why is the whole Performance tab missing? This Might Be Database Standard Edition which does not support Performance Feature/Option ,You can double check by :SQL> select * from v$version; To interpret the output, please check: How To Find Which Oracle Database Edition Is Currently Running? (Doc ID 417573.1)Please check  Feature Availability for Oracle Database Editions Documentation Link for Why are tabs missing from the Performance Hub? At the PDB container, only the Activity and Monitored SQL tabs are visible because these are the only tabs that display PDB-specific data.At the root container, all tabs are available in the Performance Hub. Can I see the activity breakdown by PDB? Yes, at the root container, you can see the Average Active Sessions breakdown by container using the Home Page and the Performance Hub (Summary tab or Activity tab).This data is taken from ASH, not metrics, so you might see a discrepancy when comparing average active sessions per container against a different breakdown. Why do I see PDB$SEED when viewing Activity breakdown by PDB? Although PDB$SEED is just a seed database, when queries are issued against CDB_* views, the server internally switches to all containers, including PDB$SEED, to retrieve data.  If there are frequent queries against CDB_* views, this could result in activity captured against PDB$SEED. Can I plug and unplug PDBs with EM Express? No, EM Express does not provide full CDB management capabilities in DB 12.1.0.1.0. How can I tell if I'm connected to the root container or to a PDB? You can get this information from the Status region in the home page.  If you are at the root container, the type will show "CDB (# of PDBs)", and if you are at the PDB, the region will include the Container Name to which you're connected. In addition, the blue panel at the top of the page will have an icon on the top left that indicates if the database is a CDB.  The database name will be displayed after the icon:How do I log into EM Express?EM Express can be accessed thorugh the URL to https://<hostname>:<port>/em. A login page will be displayed, and the user can log in with any valid DB credential or use the following query while connected to CDB or PDB to generate the URL :For HTTPS :SQL> SELECT 'https://'||SYS_CONTEXT('USERENV','SERVER_HOST')||'.'||SYS_CONTEXT('USERENV','DB_DOMAIN')||':'||dbms_xdb_config.gethttpsport()||'/em/' from dual;SQL> SELECT 'https://'||SYS_CONTEXT('USERENV','SERVER_HOST')||':'||dbms_xdb_config.gethttpsport()||'/em/' from dual;For HTTP :SQL> SELECT 'http://'||SYS_CONTEXT('USERENV','SERVER_HOST')||'.'||SYS_CONTEXT('USERENV','DB_DOMAIN')||':'||dbms_xdb_config.gethttpport()||'/em/' from dual;SQL> SELECT 'http://'||SYS_CONTEXT('USERENV','SERVER_HOST')||':'||dbms_xdb_config.gethttpport()||'/em/' from dual; How can I find the port on which EM Express is configured? There are two ways to find the HTTP/HTTPS port for EM Express: lsnrctl status | grep HTTP (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=xxx.us.oracle.com)(PORT=5500))(Security=(my_wallet_directory=/u01/oracle/app/oracle/admin/orcl/xdb_wallet))(Presentation=HTTP)(Session=RAW)) --> this is For HTTPS(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xxx.us.oracle.com)(PORT=8080))(Presentation=HTTP)(Session=RAW)) --> this is For HTTP or from SQL*Plus: SQL> select dbms_xdb_config.gethttpsport() from dual;SQL> select dbms_xdb_config.gethttpport() from dual; How can I give users read-only access to EM Express? To grant users read-only access so that they can view the UI but not make any changes, grant them the EM_EXPRESS_BASIC role as follows: SQL> grant EM_EXPRESS_BASIC to <user>;   Does EM Express have support for RAC? Yes.  EM Express is a database management tool that is automatically RAC-aware.  When connected to a RAC system, the information displayed is for the entire database, i.e. for all instances.  For example, the average active session values is aggregated across all instances.  In addition, an extra Instances tab is displayed in the Performance Region on the DB Home Page to show the distribution of average active sessions across instances. When using EM Express on RAC, how can I see instance-specific information? Although EM Express is RAC-aware in general, some pages will allow you to view instance-specific information as well.  There are two ways to get to this instance-specific information.The first way is using the Status Region in the Home Page.  You can click the "RAC - <n> instance(s) up" link and it will take you to a page with the instance details.  You can now select an instance and navigate to the Performance Hub, the Instance Home Page, or the Memory Page for the selected instance.The second way is through the Performance Hub.  If you click the RAC tab, you will see a list of instances.  From this list, you can now select an instance and navigate to the Performance Hub, the Instance Home Page, or the Memory Page for the selected instance.For Undo Details, the information is always instance-specific.  When you navigate to the Undo Management page using the Storage menu, you are taken to a page that lists all the instances and the undo summary information for each instance.  You can now select an instance and navigate to the Undo Details Page for the selected instance. EM Express seems slow when using HTTPS.  Why? Some browsers disable caching if you are using SSL and have a self-signed certificate.  This is known to occur on Chrome and Safari.  To avoid this issue, use a CA certificate when using HTTPS or use a browser that supports caching when using self-signed certificates (Firefox or IE). What is the recommended screen resolution for EM Express? Although the EM Express UI dynamically resizes based on the screen real-estate available, it works best with resolutions of at least 1280x1024 or 1600x900. Can I run EM Express in Safari? Yes, you can run EM Express in Safari on MacOS.  You can also run EM Express in Safari 5.0.* on Windows, but Safari 5.1.* on Windows is known to have an issue with input text fields that prevents users from entering their username and password on the login page How can I replace the default wallet/certificate? The XDB wallet is stored in the $ORACLE_BASE/admin/<db_name>/xdb_wallet directory .  If ORACLE_BASE is not defined, the XDB wallet is stored in $ORACLE_HOME/admin/<db_name>/xdb_wallet, where <db_name> is the unique database name. What should I do if the default XDB wallet expires? If the wallet expires, you can create a new wallet with an existing SSL certificate and replace the existing wallet with the new one.  Alternatively, you can use dbms_xdb.installDefaultWallet() to replace the existing wallet with a new Oracle wallet that has a self-signed certificate. Why am I getting a "This Connection is untrusted" message when I try to log into EM Express?  Is this a security concern? The browser displays this message if you are using a self-signed certificate.  You can either create an exception for it, or you can replace the default wallet/certificate with a trusted certificate.  See the documentation for more information.   Is EM Express secure? Yes, EM Express uses SSL (HTTPS) out of the box. Do I need any database privileges to use EM Express? In order to use EM Express, a database user needs to have been granted the EM_EXPRESS_BASIC or EM_EXPRESS_ALL role.  The DBA role includes both the EM_EXPRESS_BASIC and the EM_EXPRESS_ALL roles. EM_EXPRESS_BASIC grants a user read-only privileges, so that the user can view pages but not perform any actions.  EM_EXPRESS_ALL grants a user all privileges required to perform any action in EM Express. Do I need licenses or packs to use EM Express? You can use the basic administration features offered by EM Express with no additional licenses.  However, to use the performance features, you will need the Oracle Diagnostics Pack.  For the Performance Hub, you will need the Diagnostics Pack and for the SQL Monitor and SQL Tuning Advisor features, you will need the Tuning Pack. Depending on what packs you own, you should set the control_management_pack_access init.ora parameter on your system.  For example, if you have the Diagnostics Pack, but not the Tuning Pack, you should set the control_management_pack_access parameter=DIAGNOSTIC. Is EM Express supported on Oracle Database Standard Edition?  What about Database XE? Yes, EM Express is supported on both Standard Edition (SE) and Express Edition (XE).  However, features that require the Diagnostics and Tuning Packs will not be available on SE and XE.  For example, the Performance Menu won't be available on SE and XE because the features require the Diagnostics or Tuning Packs.  Similarly, certain regions of the Home Page, such as the SQL Monitor List, will not be shown on SE and XE because they require the Tuning Pack. How to configure EM Express on a CDB/PDB? Users can configure EM Express both at the root and the PDB containers, with each container using a different HTTP/HTTPS port.  When connected to the root container, the information displayed is for the entire database including all PDBs.  When connected to a PDB, the information displayed is restricted to data for the PDB  Simply login to the container and set the HTTPS/HTTP port. By default, DBCA picks a free port from the 5500 to 5599 range to use as the EM Express port.If you want a particular port to be used as the EM Express port, specify that port using the DBEXPRESS_HTTPS_PORT operating system environment variable prior to starting OUI or DBCA.To start EM Express, use the EM Express URL provided by DBCA when DBCA configured your database, there are two ways to find the HTTP/HTTPS port for EM Express:   lsnrctl status | grep HTTP(DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=xxx.us.oracle.com)(PORT=5500))(Security=(my_wallet_directory=/u01/oracle/app/oracle/admin/orcl/xdb_wallet))(Presentation=HTTP)(Session=RAW)) --> this is For HTTPS(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=xxx.us.oracle.com)(PORT=8080))(Presentation=HTTP)(Session=RAW)) --> this is For HTTPor from SQL*Plus:SQL> select dbms_xdb_config.gethttpsport() from dual;SQL> select dbms_xdb_config.gethttpport() from dual; or use the following query while connected to CDB or PDB to generate the URL :For HTTPS:SQL> SELECT 'https://'||SYS_CONTEXT('USERENV','SERVER_HOST')||'.'||SYS_CONTEXT('USERENV','DB_DOMAIN')||':'||dbms_xdb_config.gethttpsport()||'/em/' from dual;SQL> SELECT 'https://'||SYS_CONTEXT('USERENV','SERVER_HOST')||':'||dbms_xdb_config.gethttpsport()||'/em/' from dual;For HTTP :SQL> SELECT 'http://'||SYS_CONTEXT('USERENV','SERVER_HOST')||'.'||SYS_CONTEXT('USERENV','DB_DOMAIN')||':'||dbms_xdb_config.gethttpport()||'/em/' from dual;SQL> SELECT 'http://'||SYS_CONTEXT('USERENV','SERVER_HOST')||':'||dbms_xdb_config.gethttpport()||'/em/' from dual; When EM Express prompts you for your username and password, log in as a user with DBA privilege (such as SYS).Note: The first time you enter the URL for EM Express in your web browser, your browser may display warning messages.EM Express is a servlet built on top of Oracle XML DB. The Oracle XML DB default wallet has a self-signed certificate, and some existing browsers consider self-signed certificates as untrusted because they are not signed by a trusted CA (certificate authority). However, the self-signed certificate is still secure, as it ensures that the traffic is encrypted between the Oracle XML DB server and the client (browser).Therefore, enter a security exception for the EM Express URL in your web browser.To manually configure the HTTP/HTTPS port for EM Express:1- If the listener is running on a nonstandard port (for example, not 1521), then the init.ora file for the database you want to manage using EM Express must contain a local_listener entry so that the HTTP/HTTPS port can register with the correct listener. The local_listener entry references a TNSNAMES entry that points to the correct listener. For example:    local_listener=inst1where inst1 is a TNSNAMES entry defined in tnsnames.ora that points to the listener. For example:    inst1= (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=host_name)(PORT=1234))    (CONNECT_DATA=(SERVICE_NAME=service_name)(SERVER=DEDICATED)))2- Enable the TCP dispatcher by adding the following entry to the init.ora file for the database you want to manage using EM Express:    dispatchers="(PROTOCOL=TCP)(SERVICE=<sid>XDB)"    For example, if the database SID is ORCL, then the entry would be:    dispatchers="(PROTOCOL=TCP)(SERVICE=ORCLXDB)"3- Use the PL/SQL procedure DBMS_XDB_CONFIG.SETHTTPSPORT to set the HTTPS port for EM Express. This will update the HTTPS port in the xdbconfig.xml file in the Oracle XML DB Repository. You must connect as SYS / AS SYSDBA to run the procedure. For example: SQL> connect sys/<password>@<container> as sysdbaSQL> exec DBMS_XDB_CONFIG.SETHTTPSPORT(5500);or use the PL/SQL procedure DBMS_XDB_CONFIG.SETHTTPPORT to set the HTTP port for EM Express. This will update the HTTPS port in the xdbconfig.xml file in the Oracle XML DB Repository. You must connect as SYS / AS SYSDBA to run the procedure. For example: SQL> connect sys/<password>@<container> as sysdbaSQL> exec DBMS_XDB_CONFIG.SETHTTPPORT(2200);

EM Express is a database management tool that focuses primarily on performance management and monitoring.  However, it does include basic administration functions to assist the database administrator....

ORAchk New Version 12.1.0.2.1

The new ORAchk release 12.1.0.2.1 is now available to download. Easier to stay up to date with the most current version of ORAchk ORAchk release versioning now aligns with and follows the same format used by the Oracle 12c Database Patch Set Updates (PSUs); this version is 12.1.0.2.1, the next will be 12.1.0.2.2. It’s also now even easier to update ORAchk across multiple machines. Download the latest release of orachk.zip into a shared network staging directory Set the environment variable $RAT_UPGRADE_LOC to the staging directory The next time ORAchk is run it will prompt you to auto update New ORAchk Support for Windows ORAchk is now supported on Windows when run within a Cygwin environment. Instructions for configuring Cygwin can be found from Document 1268927.2. ORAchk now includes hundreds of database and application checks which will run on Windows. There are even more Windows specific checks in the pipeline. Easier execution for environments with role separation You no longer need to have different users execute different ORAchk profiles to workaround your company’s implementation of role separation. ORAchk can now be run once as root to execute all checks. Prior to executing checks that do not require root access, ORAchk will switch user to the lower level accounts. Faster execution time When running against multiple databases, ORAchk can now run database checks in parallel meaning it takes a fraction of the time to complete execution. Parallel database execution is now the default. It can be turned off, if you prefer to run checks serially. Enhanced comparison of result output Quickly find out what has changed on your system between two ORAchk runs. When ORAchk is run with the –diff command it will now not only compare check results but collection data too. Quickly compare and understand differences in kernel parameters or database initialization parameters. Over 70 new EBS checks ORAchk support for EBS has been enriched and broadened, with even more checks for Oracle Payables (R12) and Oracle Workflow and now with release 12.1.0.2.1 introduces new support for Oracle Order Management (R12) and Oracle Process Manufacturing (R12). For more details and to download the latest release of ORAchk see Document 1268927.2

The new ORAchk release 12.1.0.2.1 is now available to download. Easier to stay up to date with the most current version of ORAchk ORAchk release versioning now aligns with and follows the same format...

Master Notes

Master Note for ORA-1555 Errors

ORA-1555 Errors Concepts/Definitions The ORA-1555 errors can happen when a query is unable to access enough undo to builda copy of the data at the time the query started. Committed "versions" of blocks aremaintained along with newer uncommitted "versions" of those blocks so that queries canaccess data as it existed in the database at the time of the query. These are referred to as"consistent read" blocks and are maintained using Oracle undo management. See Document 40689.1 - ORA-1555 "Snapshot too old" - Detailed Explanation for more aboutthese errors. Diagnosing Due to space limitations, it is not always feasible to keep undo blocks on hand for the life of the instance. Oracle Automatic Undo Management (AUM) helps to manage the time frame that undo blocks are stored. The time frame is the "retention" time for those blocks. There are several ways to investigate the ORA-1555 error. In most cases, the error is a legitimate problem with getting to an undo block that has been overwritten due to the undo "retention" period having passed. AUM will automatically tune up and down the "retention period, but often space limitations or configuration of the undo tablespace will throttle back continuous increases to the "retention" period. The error message is reported in the user session and often is not captured in the alert log. The user could see a message like Using rollback segment functionality:ORA-1555: snapshot too old (rollback segment too small) or Using AUM:ORA-01555: snapshot too old: rollback segment number 9 with name "_SYSSMU9$" too small If the error is captured in the alert.log, you would see something like Tue May 26 16:16:57 2009ORA-01555 caused by SQL statement below (SQL ID: 54yn3n36w24ft, Query Duration=922 sec, SCN: 0x0007.8a55f4e3) Initial Investigation Rollback Segments: With Oracle 10g and later versions of Oracle, you can still use a Rollback Segments configuration. ORA-1555 errors in that environment still follow older guidelines as described in Document 10579.1 - How many Rollback Segments to HaveDocument 107085.1 - Tuning Rollback SegmentsDocument 69464.1 - Rollback Segment Configuration & Tips Automatic Undo Management: The database will be self tuning for undo when using Automatic Undo Management. This does not eliminate ORA-1555 completely, but does minimize ORA-1555 as long as there is adequate space in the undo tablespace and workloads tend to follow repeatable patterns. In some cases with periodic changes to workload (large data updates particularly with LOB data) the self tuning of undo can become aggressive and lead to undo issues. Document 461480.1 - FAQ Automatic Undo Management (AUM) / System Managed Undo (SMU)Document 135053.1 - How to Create a Database with Automatic Undo ManagementDocument 268870.1 - How to Shrink the datafile of Undo TablespaceDocument 231776.1 - How to switch a Database from Automatic Undo Management (AUM) back to using Rollback SegmentsDocument 396863.1 - How to Keep All UNDO Segments from Being Offlined in Oracle 10g - Fast Ramp-Up LOB Issues: Out-of-row LOB undo is maintained in the LOB segment. So the UNDO tablespace and undo retention is not associated with most LOB ORA-1555 issues. Instead the LOB column is created using either PCT_VERSION or RETENTION to manage how much space within blocks or time transpires before the LOB undo is overwritten. In environments with high updates, deletes on rows including LOBs, the chances of ORA-1555 on LOB undo is very high. PCT_VERSION and RETENTION are not auto-tuned. To "tune" those configuration settings, you must change the values for PCT_VERSION or RETENTION. Changes to UNDO_RETENTION does not change LOB retention time frames. Document 162345.1 - LOBS - Storage, Read-consistency and RollbackDocument 386341.1 - How to determine the actual size of the LOB segments and how to free the deleted/unused space above/below the HWMDocument 563470.1 'Lob retention not changing when undo_retention is changedDocument 422826.1 How to identify LOB Segment Use PCTVERSION or RETENTION from Data Dictionary Error Tracing Undo error tracing can be done for normal undo operations using the following events: NOTE: Normal undo operations will be indicated in the error message in that the error message includes a segment name like '¦. name "_SYSSMU1$" too small If the error doesn't show a segment name '¦ name "" too small the problem is often related to LOB undoIf using pfile: event="10442 trace name context forever, level 10" If using spfile: Alter system set events '10442 trace name context forever, level 10'; Reproduce the ORA-1555 error and upload the trace file to Oracle Support. LOB undo error tracing is more difficult. Set additional tracing events as follows: Start Session 1Alter session set events '10046 trace name context forever, level 12';Reproduce the errorExit Session 1 Start Session 2Alter session set events '10051 trace name context forever, level 1';Reproduce the errorExit Session 2 Start SessionAlter session set events '1555 trace name errorstack forever, level 3';Reproduce the errorExit Session 3 Additional resources to review: Document 846079.1 - LOBs and ORA-1555 troubleshootingDocument 253131.1 - Concurrent Writes May Corrupt LOB Segment When Using Auto Segment Space ManagementDocument 467872.1 - TROUBLESHOOTING GUIDE (TSG) - ORA-1555 V$UNDOSTAT Analysis The V$UNDOSTAT view holds undo statistics for 10 minute intervals. This viewrepresents statistics across instances, thus each begin time, end time, andstatistics value will be a unique interval per instance. This does not track undo related to LOB Document 262066.1 - How To Size UNDO Tablespace For Automatic Undo ManagementDocument 1112363.1 - When Does Undo Used Space Become Available?Document 240746.1 - 10g NEW FEATURE on AUTOMATIC UNDO RETENTION Diagnostics Scripts Refer to Document 746173.1 : Common Diagnostic Scripts for AUM problemsand Document 877613.1 : AUM Common Analysis/Diagnostic Scripts Common Causes/Solutions Using Rollback Segments functionality: * Problem happening on SYSTEM tablespace that still uses old Rollback Segment functionality even when configured for Automatic Undo Management (AUM). * There are not enough rollback segments to manage the undo needed for long running queries. * Rollback Segments are too small and undo is overwritten before long running queries complete. Reference: Document 69464.1 - Rollback Segment Configuration & TipsDocument 10630.1 - ORA-1555: 'Snapshot too old' - OverviewDocument 862469.1 - ORA-604 & ORA-1555 Rollback Segment 0 with Name "System" Too Small Using Automatic Undo Management (AUM): * TUNED_UNDORETENTION in V$UNDOSTAT around the time of the error is lower than the QUERY DURATION indicated in the error message. This is a legitimate ORA-1555 and if queries are going to run for very long time frames, UNDO_RETENTION may need to be larger. Auto-tuned retention may not be able to keep up with the undo workload and staying within space limitations on the UNDO tablespace. * LOB updates and/or deletes are frequent and a higher PCT_VERSION is required to provide enough space in the LOB Segment to accommodate the LOB undo. RETENTION on LOBs that are updated or deleted frequently can run into problems holding UNDO long enough for queries. * QUERY DURATION shown in the error message is 30+ years and therefore, no amount of undo will satisfy the consistent read blocks. Document 750195.1 - ORA-1555 Shows Unrealistic Query Duration (billions of seconds) * QUERY DURATION shown in the error message is 0. NOTE: This has been filed as a bug on many release levels and has been very difficult to narrow down to a specific problem. Document 761128.1 - ORA-1555 Error when Query Duration as 0 Seconds * QUERY DURATION is lower than TUNED_UNDRETENTION. Undo header information can sometimes get overwritten or you could be seeing a bug. * TUNED_UNDORETENTION stays very high and UNDO tablepsace continues to grow continuously or getting space errors. Document 1112431.1 - Undo Remains Unexpired When Using Non-autoextensible Datafiles for Undo Tablespace. * How to find the complete SQL statement caused ORA-1555 :         If the Database was not restarted after the error ORA-1555 , so the Statement can be obtained from :         select SQL_TEXT from SQL_TEXT where SQL_ID='<sql id from the error message>';         If the Database was restarted after the error ORA-1555 and an AWR snapshot was gathered before the restart , so the Statement can be obtained from :         select SQL_TEXT from DBA_HIST_SQLTEXT where SQL_ID='<sql id from the error message>'; Additional Reference: Database Administration Community

ORA-1555 Errors Concepts/Definitions The ORA-1555 errors can happen when a query is unable to access enough undo to builda copy of the data at the time the query started. Committed "versions" of blocks...

Master Notes

Master Note: Troubleshooting Oracle Scheduler

Master Note: Troubleshooting Oracle Scheduler (Doc ID 1520580.1) To Bottom Troubleshooting Steps Concepts Oracle provides a collection of functions and procedures in the DBMS_SCHEDULER package to help simplify management tasks, as well as providing a rich set of functionality for complex scheduling needs.  Collectively, these functions are called the Scheduler, and they are callable from any PL/SQL program.Prior to Oracle release 10g similar limited functionality was available with DBMS_JOB package.The Scheduler lets database administrators and application developers control when and where various tasks take place in an enterprise environment. For example, database administrators can schedule and monitor database maintenance jobs such as backups or nightly data warehousing loads and extracts. Starting from 12c : The coordinator looks at the root database and all the child PDBs and selects jobs based on the job priority, the job scheduled start time, and the availability of resources to run the job. The latter criterion depends on the consumer group of the job and the resource plan currently in effect. The coordinator makes no attempt to be fair to every PDB. The only way to ensure that jobs from a PDB are not starved is to allocate enough resources to it. Known Issues Involving Oracle Scheduler Problems with the Scheduler can be caused by such things as versions, incorrect setups and/or incorrect usage. The steps that follow will help to clarify where the problem might be and provides information that can be collected to help in diagnosing the issue. Please note that for Oracle Jobs, it is important to distinguish between the usage of the conventional Job queue (DBMS_JOB) and the usage of the Scheduler (DBMS_SCHEDULER). DBMS_JOB is mainly used in Oracle version 9i and lower and has been replaced by the Scheduler, which was introduced in Oracle version 10G. This document will focus mainly on Scheduler (DBMS_SCHEDULER) related issues.Note: It is still possible to use DBMS_JOB in Oracle version 10. For a thorough list of known issues with the Oracle Scheduler, please refer to:NOTE 1311355.1 Known Issues And Bugs Using the Scheduler Diagnosing Scheduler Issues Firstly, to identify the affected job the following can be used:<Note 744645.1> How to find the job name if a scheduled job fails with ORA-12012 Building a Testcase In the area of the Scheduler, a testcase can often help in solving the issue or finding a solution because there are so many ways to build a job structure. Unexpected behavior or errors may be located in the user commands executed by the job and jobs can execute different types of commands, like PLSQL-programs, procedures, external shell scripts and more. Depending on the problem it is useful to test the successful execution of these commands outside the jobs first to confirm if the problem is in fact located in the job usage. Problems occurring after changes in a job setup or with a newly created job setup often benefits showing the behavior using a small testcase where the issue can be reproduced. A guideline for setting up a simple testcase is provided in the following:Note 807159.1 Sample Code and Select Statements to Build A simple Testcase for Scheduler / Jobs According to the situation, combine the SQL commands to build as simple a job structure as possible with only the objects that are needed. Replacing large and complex job setups and minimizing the job structure to only the relevant parts of the problem is important to developing a simple testcase where the behavior / problem can be shown in an easy way and can actually help in finding any possible workarounds. Generic Tracing When the problem in the job structure cannot be found in an easy manner it is often recommended to activate sql tracing by setting the event 10046. To produce a valuable result, set timed_statistics to true and max_dump_file_size to unlimited before executing the job.For example: SQL> alter session set max_dump_file_size=unlimited;SQL> alter session set timed_statistics = true;SQL> alter session set events '10046 trace name context forever, level 4';SQL> BEGIN DBMS_SCHEDULER.run_job (job_name => 'runtest_job',                                   use_current_session => TRUE);end;/ Please note that by setting the event in a parameter file, each process would have produced a trace. In this case, optimal tracing is set at the session level, where the problem can be reproduced and then tracing disabled. Ensure that only matching traces are uploaded.Details on how to diagnose the 10046 trace can be found in Note 376442.1 and Note 39817.1 For the Scheduler, additional tracing has been introduced with setting the attribute logging_level for the jobs and using an event to dump further diagnostics for the job coordinator and the job slaves: Limited Availability Note 461304.1 How to activate tracing for the scheduler[This section is not visible to customers.] Automatic Scheduling Issues Common problems with Scheduler are that jobs do not start at the expected time or won't start automatically.As a first step, confirm that the job runs successfully with an explicit run (dbms_job.run or dbms_scheduler.run_job). This will exclude a configuration problem of the job itself. Once an explicit execution of the affected job is successful, then look at diagnosing the automatic scheduling process: JOB_QUEUE_PROCESSES Until Oracle Database version 11.1 , the setting of the parameter JOB_QUEUE_PROCESSES is not mandatory for the Scheduler. The Job Coordinator starts job slaves on demand, to work on the job queue table. This means setting JOB_QUEUE_PROCESSES to 0 caused DBMS_JOB jobs to not run, but DBMS_SCHEDULER jobs were unaffected and would still run.Starting with Oracle Database 11g Release 2 (11.2), setting JOB_QUEUE_PROCESSES to 0 causes both DBMS_SCHEDULER and DBMS_JOB jobs not to run. This has been implemented to provide a clean method of stopping all jobs. Window Issues Note 742683.1 Scheduled Job That Works Does Not Start In Window Purging Diagnostics Note 443364.1 How To Purge DBA_SCHEDULER_JOB_LOG and DBA_SCHEDULER_WINDOW_LOGNote 749440.1 Dbms_scheduler.Purge Not Removing Entries from dba_scheduler_job_run_details   Schedule Issues Note 312547.1 Advanced use of DBMS_JOB.SUBMIT 'interval' ParameterNote 428872.1 Scheduled Jobs Do Not Run After A Re-Start With Repeat_interval => Null. Issues with Statistic Jobs (10g) Note 430636.1 GATHER_STATS_JOB Is Not RunningNote 377143.1 How to check what automatic statistics collection is scheduled on 10gNote 803191.1 Checklist To Diagnose Issues Related To Automatic Run Of Job GATHER_STATS_JOB  Issues with 11g Autotasks Note 865933.1 DBMS_AUTO_TASK_IMMEDIATE.GATHER_OPTIMIZER_STATS Procedure Requires SYSDBA Privilege Helpful Articles / Examples for Scheduler Note 807159.1 Sample Code and Select Statements to Build A simple Testcase for Scheduler / JobsNote 727499.1 Sample how to create a job running chainsNote 1300444.1  How to create a event based chained jobNote 404238.1 How to Disable an Entry from DBMS_SCHEDULERNote 873691.1 How to Submit a DBMS_SCHEDULER Job to Run Starting with the Nth Day of a MonthNote 461432.1 How To Exclude A Certain Window In A Schedule For A Day Using DBMS_SCHEDULER PackageNote 270256.1 How to create a Job using new 10g Scheduling feature.Note 1431664.1 - How to build up an event based job - an exampleNote 1074141.1 - How to install and setup the Scheduler Agent (schagent) so can run Remote External Jobs Further Diagnostics If you were not able to resolve the issue with the details provided in this or referenced documents, please raise a Service Request for further assistance from Oracle Support. Please provide any relevant tracing and/or a testcase as well as the following information:Check the columns STATE, ENABLED, NEXT_RUN_DATE from DBA_SCHEDULER_JOBS.Collect the information from DBA_SCHEDULER_JOB_LOG set ver offset term offset page 0set markup html on spool onspool scheduler.html --The proceding lines needs to be un commented when executing aganist 12c Database for a CDB (pluggable instance )--alter session set container='<pdp name>'--/ alter session set NLS_TIMESTAMP_TZ_FORMAT='DD-MON-RR HH.MI.SS AM TZR'/alter session set NLS_DATE_FORMAT='DD-MON-RR HH.MI.SS AM'/select WINDOW_NAME,WINDOW_PRIORITY,ENABLED,RESOURCE_PLAN,NEXT_START_DATE,DURATION from DBA_SCHEDULER_WINDOWS/select LOG_DATE,WINDOW_NAME,REQ_START_DATE,ACTUAL_START_DATE,WINDOW_DURATION,ACTUAL_DURATION,ADDITIONAL_INFO from DBA_SCHEDULER_WINDOW_DETAILSwhere rownum<20order by LOG_DATE/select LOG_DATE,STATUS,WINDOW_NAME, OPERATION from DBA_SCHEDULER_WINDOW_LOGorder by LOG_DATE desc/select WINDOW_GROUP_NAME, ENABLED, NEXT_START_DATE from DBA_SCHEDULER_WINDOW_GROUPS/select LOG_DATE, OWNER,JOB_NAME, STATUS,ERROR#,REQ_START_DATE,ACTUAL_START_DATE,ADDITIONAL_INFO from DBA_SCHEDULER_JOB_RUN_DETAILSwhere JOB_NAME in ('')order by LOG_DATE/show parameter job_queue/spool offset markup html off spool off

Master Note: Troubleshooting Oracle Scheduler (Doc ID 1520580.1) To Bottom Troubleshooting Steps Concepts Oracle provides a collection of functions and procedures in the DBMS_SCHEDULER package to help...

ORAchk Health Checks for the Oracle Stack

ORAchk includes EXAchk’s functionality and replaces the popular RACcheck tool, extending the coverage based on prioritization of top issues reported by users, to proactively scan for known problems within: E-Business Suite Financials Accounts Payables Oracle Database Sun Systems ORAchk features: Proactively scans for the most impactful known problems across your entire Engineered System as well as various layers of your stack Simplifies and streamlines how to investigate and analyze which known issues present a risk to you Lightweight tool runs within your environment; no data will be sent to Oracle High level reports show your system health risks with the ability to drill down into specific problems and understand their resolutions Can be configured to send email notifications when it detects problems Collection Manager, a companion Application Express web app, provides a single dashboard view of collections across your entire enterprise ORAchk will expand in the future with more high impact checks in existing and additional product areas. If you have particular checks or product areas you would like to see covered, please post suggestions in the ORAchk community thread accessed from the support tab on the below document. For more details about ORAchk see Document 1268927.1

ORAchkincludes EXAchk’s functionality and replaces the popular RACcheck tool, extending the coverage based on prioritization of top issues reported by users, to proactively scan for known...

Master Notes

Master Note for Database Startup/Shutdown (Doc ID 1270450.1)

Concepts During Startup, the instance is started first (nomount stage) which then mounts the database by accessing the control files specified in the CONTROL_FILES parameter. The next step will be to open the database which includes opening the datafiles and redo log files. Applying the redo (rollforward) and the rollback of uncommitted data (rollback) also happens if the previous shutdown was not a clean one (shutdown normal or immediate). Similarly the database and instance undergoes multiple phases during shutdown. First the database is closed ie, the datafiles and the redolog files are closed. Then the database will be dismounted  to disassociate it from the instance. After a database is unmounted, Oracle Database closes the control files of the database. The next step will be to shutdown the instance by removing the SGA from the memory and terminating the background processes. There are various shutdown modes such are normal, immediate,transactional and abort. When a database is closed as part of a SHUTDOWN with any option other than ABORT, Oracle Database writes data in the SGA to the data files and online redo log files. If a SHUTDOWN ABORT or abnormal termination occurs, then the instance of an open database closes and shuts down the database instantaneously. Oracle Database does not write data in the buffers of the SGA to the data files and redo log files. The subsequent reopening of the database requires instance recovery, which Oracle Database performs automatically. Database startup and shutdown are powerful administrative options that are restricted to users with administrator privileges. For details you can refer Note 1505155.1 - Master Note: Overview of Database Startup and Shutdown Common Issues During Database Startup This Section list out the common issues encountered during startup. The error can be reported in any of the above mentioned phases like nomount,mount or open. Since transaction recovery continues even after opening the database, you may observe issues like hang, high CPU usage etc, even after the database is open. The documents mentioned in the below section can be specific to platform or database versions. Startup upgrade/migrate or setting _system_trig_enabled = FALSE will disable startup triggers to rule out startup trigger problems. 12c Startup/Shutdown Startup and shutdown of the container database is the same as it has always been for regular instances.  For PDBs Using the STARTUP SQL*Plus Command on a PDB Modifying the Open Mode of PDBs with the SQL*Plus STARTUP Command  Shutting Down a CDB Instance Also Refer to the following documents for more information on 12c Startup and Shutdown:  Note 1592247.1 12c: How to Startup/Shutdown PDB's?  Note 1582453.1 When to use startup / shutdown database and alter database command in 12c ORA-27102 Errors ORA-27102 errors normally occurs due to memory issues.The Common causes could be due to Semaphore Kernel misconfigurations,Memory related Ulimit settings,RAM or swap configurations. ORA-27300 Errors These errors are generally reported when the Operating System called for error or when there was a connection killed or a network interconnection failures or an OS configuration issue.The error ORA-27300 will also be accompanied by ora-27301 and ora-27302 Note 579365.1 Troubleshooting ORA-27300 ORA-27301 ORA-27302 errors ORA-64 Errors This error could occur when the database init.ora parameter calling for more resources than the Operating System is configured to provide.The parameters could be PROCESSES,DB_BLOCK_SIZE,SGA and more. Note 1457812.1 ORA-00064 Error Reported After Increasing Processes Parameter value ORA-27123 Errors Note 115753.1 UNIX: Resolving the ORA-27123 error ORA-1081 Errors This error could occur when we try to startup an instance that is already running or if the shared segments/semaphores already exist. Note 1010214.6 ORA-1081: Starting InstanceNote 18657.1 OERR: ORA 1081 cannot start already-running Oracle - shut it down first NFS Related Issues Note 8418190.8 Bug 8418190 - Direct NFS warnings during database startupNote 236794.1 NFS Locking Problems Encountered During Database StartupNote 971406.1 DATABASE STARTUP HANGS AT MOUNTING CONTROLFILE WHEN DNFS IS ENABLEDNote 1430654.1 Database Startup Failed with "Direct NFS: please check that oradism is setuid"Note 430920.1 NetApp: Using 'nolock' NFS Mount Option with non-RAC Systems Results in Database Corruption Transaction Recovery Slowness There could be slowness in the database during the open phase when the database is busy performing transaction recovery. Note 1494886.1 Database Transaction RecoveryNote 414242.1 Database Hangs Because SMON Is Taking 100% CPU Doing Transaction RecoveryNote 12934890.8 Bug 12934890 - Startup hangs waiting for row cache lock due to open transaction against UNDO$ ORA-704 Errors This is a general error reported at startup when there is some problem during processing of bootstrap information.There should be an accompanying error/s. Plug the ORA=704 and the other errors it reports into the Oracle Support search engine for a solution. ORA-9968 Errors There are some client shadow processes hanging. Although the lk< SID> file is deleted the hanging processes still have a lock on the open file handle. This prevents the database to startup although a new lk<SID> file can be created successfully. An oracle process (background or shadow process) that exists while the instance is not started (crashed or not cleanly stopped) can have a lock on a file while this file is actually removed from the system. This is because on UNIX there is still a lock on the open file handle. Note 160395.1 Database Startup Fails with ORA-1102 and ORA-9968Note 1488147.1 Instance Startup Raises Error ORA-09968: unable to lock file (Doc ID 1488147.1) ORA-12547 Errors The error ORA-12547 indicates that the communication channel has been broken. It's most often thrown because the other end of the process went away unexpectedly. Note 1307075.1 Oracle Database Fails to Start with Error ORA-12547Note 381566.1 connect / as sysdba Fails with Ora-12547 And Tns-12514Note 744512.1 Ora-12547: Tns:Lost Contact Creating Database After Clean Installation ORA-1157 Errors  The background process was not able to find one of the datafiles.The database will prohibit access to this file but other files will be unaffected.However, the first instance to open the database will need to access all online datafiles.Accompanying messages from the operating system will describe why the file was not found. Note 184327.1 ORA-1157 Troubleshooting Common Errors / Issues During Database Shutdown The most common issue observed while bringing down the database is shutdown immediate hang. The main reasons for Shutdown immediate hang is:- processes still continue to be connected to the database and do not terminate.- SMON is cleaning temp segments or performing delayed block cleanouts.- Uncommitted transactions are being rolled back. The below section provides the consolidated list of known issues during shutdown. The documents mentioned in the below section can be specific to platform or database versions. Transaction Recovery Note 375935.1 What To Do and Not To Do When 'shutdown immediate' HangsNote 1076161.6 Shutdown Normal or Shutdown Immediate Hangs. SMON disabling TX RecoveryNote 414242.1 Database Hangs Because SMON Is Taking 100% CPU Doing Transaction RecoveryNote 100054.1 Transaction Rollback after a failed operation or during Database Shutdown ORA-24324 Errors Note 794293.1 ORA-24324 During Shutdown DISM Note 1001248.1 On Solaris 9 Systems, Oracle Shutdown May Hang If Utilizing Dynamic Initmate Shared Memory (DISM)  Issues specific to automatic shutdown and startup This section is specific to the automatic shutdown and startup that can be configured with the dbora / dbshut / dbstart scripts. Automatic Startup Failure The key to diagnosing automatic startup failures is to determine where startup fails. This can be done via the following steps: Determine if instance starts manually as Oracle software owner. Determine if instance starts via dbstart command run as Oracle software owner. Determine if instance starts when root runs following dbstart command:su - $ORA_OWNER -c $ORA_HOME/bin/dbstart where $ORA_OWNER is set to Oracle software owner. Determine if instance starts when running as root the OS script which calls dbstart, ie "/etc/init.d/dbora start". NOTE: Running via sh -x command will show each command as it is run from script to better see what is going on. #> sh -x /etc/init.d/dbora start Automatic Shutdown Failure As with automatic startup, the key to diagnosing automatic shutdown failures is to determine where shutdown fails. This can be done via following steps: Determine if instance stops manually as Oracle software owner. Determine if instance stops via dbshut command run as Oracle software owner. Determine if instance stops when root runs command su - $ORA_OWNER -c $ORA_HOME/bin/dbshutwhere $ORA_OWNER is set to Oracle software owner. Determine if instance stops when running as root the OS script which calls dbshut, ie "/etc/init.d/dbora stop". NOTE: Running via sh -x command will show each command as it is run from script to better see what is going on. #> sh -x /etc/init.d/dbora stop Further Diagnostics If you were not able to resolve the issue with the details provided in this document, please raise a Service Request for further assistance from Oracle Support. The diagnostic information required to troubleshoot the issue is listed in the below document: Troubleshooting Database Startup/Shutdown Problems (Doc ID 851057.1). ReferencesNOTE:1505155.1 - Master Note: Overview of Database Startup and Shutdown

Concepts During Startup, the instance is started first (nomount stage) which then mounts the database by accessing the control files specified in the CONTROL_FILES parameter. The next step will be to...

Master Notes

Master Note: Database Performance Overview (Doc ID 402983.1)

Purpose Questions and Answers Pro-Active Problem Avoidance and Diagnostic Collection Performance Information Centres Troubleshooting Guides Common Problem Topics Slow Database Performance Concurrency Issues Database Hangs/Spins Session Hangs/Spins Locking Issues Deadlock Pre-Upgrade Planning Post-Upgrade Performance Issue Resolution Debugging Waits for Various Events Library Cache/Mutex Contention/Cursor Type Events: Other Types: Common Causes of Performance issues Cursor Sharing/High Version Counts for Cursors High CPU usage Issues With waits for 'log file sync' WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! Buffer Busy/Cache Buffers Chains Latch waits SYSAUX Issues Performance Diagnostics References General Diagnostics Overview AWR/Statspack 10046 Trace Systemstates/Hanganalyze Errorstacks for Performance Issues PStack PL/SQL Profiler OS Watcher LTOM Trace/Result Interpretation Performance (and other) Webcasts Performance and Scalability White Papers and Documentation Interacting With Performance Support Community: Database Tuning References Applies to:Oracle Database - Standard Edition - Version 7.0.16.0 and laterOracle Database - Personal Edition - Version 7.1.4.0 and laterOracle Database - Enterprise Edition - Version 6.0.0.0 and laterEnterprise Manager for Oracle Database - Version 8.1.7.4 and laterInformation in this document applies to any platform.PurposeThis Document provides an overview reference to content that pertains to the resolution of Performance issues affecting the whole database.For the Database Performance Frequently Asked Questions FAQ,information regarding Performance Related Features or  issues involving individual SQL statements, see:Document 1360119.1 FAQ: Database Performance Frequently Asked QuestionsDocument 1361401.1 Where to Find Information About Performance Related FeaturesDocument 199083.1 Master Note: SQL Query Performance OverviewDocument 1516494.2 Information Center: SQL Query Performance OverviewDocument 398838.1 FAQ: SQL Query Performance - Frequently Asked QuestionsQuestions and AnswersPro-Active Problem Avoidance and Diagnostic CollectionAlthough some problems may be unforeseen, in many cases problems may be avoidable if signs are detected early enough. Additionally, if an issue does occur, it is no use collecting information about that issue after the event. For information on suggested preparations to avoid issues and in case diagnostics are required, see:Document 1482811.1 Best Practices: Proactively Avoiding Database and Query Performance IssuesDocument 1477599.1 Best Practices Around Data Collection For Performance IssuesPerformance Information CentresPerformance Information Centres provide access to various aspects of performance issues:Document 1512292.2 Information Center: Database Performance OverviewDocument 1516494.2 Information Center: SQL Query Performance OverviewTroubleshooting GuidesThere are a number of troubleshooting guides that can help resolve various issues:Document 1543445.2 Troubleshooting Assistant: Troubleshooting Performance IssuesDocument 1542678.2 Troubleshooting Assistant: SQL Performance IssuesDocument 1528847.1 Troubleshooting: Avoiding and Resolving Database Performance Related Issues After UpgradeDocument 60.1 Troubleshooting Assistant: Oracle Database ORA-00060 Errors on Single Instance (Non-RAC) Diagnosing Using Deadlock Graphs in ORA-00060 Trace FilesCommon Problem TopicsSlow Database PerformanceIf the database is performing slowly see the following:Document 1362329.1 How To Investigate Slow Database Performance Issues Concurrency IssuesIf you have problems with the concurrency of transactions, initially treat this as a "Slow Database Performance" issue and refer to the following:Document 1362329.1 How To Investigate Slow Database Performance Issues Database Hangs/SpinsIf the database encounters what looks like a hang or a spin conditions refer to the following:Document 452358.1 How to Collect Diagnostics for Database Hanging IssuesDocument 68738.1 No Response from the Server, Does it Hang or Spin?Document 392037.1 Database Hangs. Sessions wait for 'resmgr:cpu quantum' Session Hangs/SpinsIf an individual session appears to hang or spin conditions refer to the relevant sections in the following articles:Document 1362329.1 How To Investigate Slow Database Performance IssuesDocument 352648.1 How to Diagnose high CPU usage problems Locking IssuesIf sessions encounter locking conflicts refer to the following:Document 1392319.1 Master Note: Locks, Enqueues and DeadlocksDocument 15476.1 FAQ about Detecting and Resolving Locking ConflictsDocument 62354.1 TX Transaction locks - Example wait scenariosDocument 33453.1 REFERENTIAL INTEGRITY AND LOCKINGDeadlockORA-60 is an application error which usually occurs because a consistent locking strategy has not been followed throughout an application.Please refer to the following articles for more information:Document 1509919.1 Master Note for Database Error ORA-00060Document 18251.1 OERR: ORA 60 "deadlock detected while waiting for resource"Document 62365.1 What to do with "ORA-60 Deadlock Detected" ErrorsPlease ensure that you are encountering none of the conditions in these articles. Pre-Upgrade PlanningFor upgrades to 11g, there is a useful webcast entitled "11G Upgrade Best Practices" :This recorded seminar provides an overview of tips and best practices for upgrading your database to 11G. It starts with general Challenges and Support Best Practices, but moves on to performance specifics such as AWR and STATSPACK, SQL Plan Management, and Real Application Testing.This webcast can be found here:11G Upgrade Best Practices For advice on planning your upgrade see:Document 785351.1 Oracle 11gR2 Upgrade CompanionDocument 601807.1 Oracle 11gR1 Upgrade CompanionDocument 466181.1 10g Upgrade CompanionDocument 465787.1 Managing CBO Stats during an upgrade to 10g or 11gFor more general upgrade related information see:Document 1392633.1 Things to Consider before upgrading to 11.2.0.3 to avoid performance or wrong resultsDocument 1320966.1 Things to Consider before upgrading to 11.2.0.2 to avoid performance or wrong resultsDocument 1464274.1 Master Note for Real Application Testing OptionDocument 762540.1 Consolidated Reference List Of Notes For Migration / Upgrade Service Requests Post-Upgrade Performance Issue ResolutionTo troubleshoot performance issues encountered after upgrading a database see:Document 1528847.1 Troubleshooting: Avoiding and Resolving Database Performance Related Issues After Upgrade Debugging Waits for Various EventsThere are numerous articles to help you debug issues with wait event contention. The most common of these being:Library Cache/Mutex Contention/Cursor Type Events:Document 1356828.1 FAQ: 'cursor: mutex ..' / 'cursor: pin ..' / 'library cache: mutex ..' Type Wait EventsDocument 1377998.1 Troubleshooting: Waits for Mutex Type EventsDocument 1349387.1 Troubleshooting 'cursor: pin S wait on X' waitsDocument 1357946.1 Troubleshooting 'library cache: mutex X' waits.Other Types:Document 1376916.1 Troubleshooting: "Log File Sync" WaitsDocument 1342917.1 Troubleshooting 'latch: cache buffers chains' Wait ContentionDocument 34405.1 WAITEVENT: "buffer busy waits" Reference NoteDocument 223117.1 Troubleshooting I/O-related waitsDocument 1415999.1 Troubleshooting: Virtual Circuit Waits Common Causes of Performance issuesCursor Sharing/High Version Counts for CursorsThe failure to share cursors effectively can have a highly detrimental affect on a database. For example, having hundreds of versions of the same cursor is likely to be wasting space, is unlikely to promote good performance and, in extreme cases, can cause contention issues.For issues linked to Cursor related waits see:Document 1356828.1 FAQ: 'cursor: mutex ..' / 'cursor: pin ..' / 'library cache: mutex ..' Type Wait EventsAlso, see the following articles for more information and help with troubleshooting issues:Document 296377.1Troubleshooting: High Version Count IssuesDocument 438755.1 High SQL Version Counts - Script to determine reason(s)Document 62143.1 Understanding and Tuning the Shared Pool High CPU usageIn cases where CPU usage of individual sessions or the whole database is high, refer to the following:Document 352648.1 How to Diagnose High CPU Usage Problems to the Module LevelDocument 273646.1 How to diagnose the high CPU utilization of ORACLE.EXE in Windows environmentNote that High CPU usage can be caused by many factors including contention for cursors (see above), inefficient SQL and may be associated with excessive buffer reads etc. Issues With waits for 'log file sync'If your system is waiting for the redo log files to synchronise with the disk, the following articles can help:Document 1376916.1 Troubleshooting: "Log File Sync" WaitsDocument 34592.1 WAITEVENT: "log file sync" Reference NoteDocument 857576.1 How to Minimise Waits for 'Log File Sync'?Document 1278149.1 High Log File Sync Wait Events, LGWR Posting Slow Write Times, Low IO WaitsDocument 1064487.1 Script to Collect Log File Sync Diagnostic Information (lfsdiag.sql)WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK!This Issue occurs when the database detects that a waiter has waited for a resource for longer than a particular threshold. The message "WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK!" appears in the alert log and trace and systemstates are dumped.Typically this is caused by two (or more) incompatible operations being run simltaneously. See:Document 278316.1 Troubleshooting: "WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK!" Buffer Busy/Cache Buffers Chains Latch waitsBuffer Busy waits occur when a session wants to access a database block in the buffer cache but it cannot as the buffer is "busy".Cache Buffers Chains Latch waits are caused by contention where multiple sessions waiting to read the same block.Typical solutions are to look for SQL that accesses the blocks in question and determine if the repeated reads are necessary.Document 1342917.1 Troubleshooting 'latch: cache buffers chains' Wait ContentionDocument 34405.1 WAITEVENT: "buffer busy waits" Reference NoteDocument 155971.1 Resolving Intense and "Random" Buffer Busy Wait Performance Problems:Document 163424.1 How To Identify a Hot Block Within The Database Buffer CacheSYSAUX IssuesSYSAUX is a mandatory tablespace in 10g which stores all auxiliary database metadata related to various product options and features. Of particular interest to Performance is the storage of Automatic Workload Repository (AWR) data:Document 552880.1 General Guidelines for SYSAUX Space IssuesDocument 1055547.1 SYSAUX Grows Because Optimizer Stats History is Not PurgedDocument 387914.1 WRH$_ACTIVE_SESSION_HISTORY Does Not Get PurgedDocument 554831.1 How to Control the Set of Top SQLs Captured During AWR Snapshot GenerationDocument 1292724.1 SYSAUX Tablespace Grows RapidlyPerformance Diagnostics ReferencesGeneral Diagnostics OverviewDocument 438452.1 Performance Tools Quick Reference GuideDocument 781198.1 Required Diagnostics for Database Performance Issues AWR/StatspackDocument 1363422.1 AWR Reports - Start PointDocument 94224.1 FAQ- Statspack Complete Reference 10046 TraceDocument 376442.1 Recommended Method for Obtaining 10046 trace for Tuning Systemstates/HanganalyzeDocument 121779.1 - Taking a SYSTEMSTATE dump when you cannot CONNECT to Oracle.Document 175006.1 Steps to generate HANGANALYZE trace filesErrorstacks for Performance IssuesDocument 1364257.1 How to Collect Errorstacks for use in Diagnosing Performance Issues PStackDocument 70609.1  How To Display Information About Processes on SUN Solaris PL/SQL ProfilerDocument 243755.1 Implementing and Using the PL/SQL Profiler OS WatcherOS Watcher (OSW) is a collection of UNIX shell scripts intended to collect and archive operating system and network metrics to aid support in diagnosing performance issues.Document 301137.1 OS Watcher User Guide LTOMThe Lite Onboard Monitor (LTOM) is a java program designed as a proactive, real-time diagnostic platform and provides real-time automatic problem detection and data collection.Document 352363.1 LTOM - The On-Board Monitor User GuideDocument 461050.1 The LTOM Graph (LTOMg) User GuideDocument 461228.1 The LTOM Graph FAQDocument 461052.1 LTOM System Profiler - Sample OutputDocument 977975.1 Diagnose LTOM Connection ProblemsTrace/Result InterpretationThe following articles can help with the interpretation of various diagnostics:Document 390374.1 - Oracle Performance Diagnostic Guide (OPDG)Document 39817.1 Interpreting Raw SQL_TRACE and DBMS_SUPPORT.START_TRACE outputhttp://www.oracle.com/technetwork/database/focus-areas/performance/statspack-opm4-134117.pdfPerformance (and other) WebcastsA series of Performance (and General Datbase topic) related webcasts, including topics such as SQLHC, SQLTXPLAIN and OSWBB, can be found here:Document 1456176.1 Oracle Database Advisor Webcast ArchivesPerformance and Scalability White Papers and DocumentationPerformance and Scalability White Papers:http://www.oracle.com/technetwork/database/focus-areas/performance/index.htmlFor White papers concerning Real Application Testing (RAT) and SQL Performance Analyzer (SPA) see:Document 1546337.1 Real Application Testing (RAT): Recommended White PapersLinks to the main Tuning and Performance documentation:Document 1195363.1 Database Performance and SQL Tuning Documentation on OTNInteracting With Performance SupportDocument 210014.1 How to Log a Good Performance Service RequestDocument 166650.1 Working Effectively With Global Customer SupportCommunity: Database TuningA community has been set up for My Oracle Support (MOS) users moderated by Oracle. The goal of this community is to exchange database related Tuning knowledge and concepts. The community can be found via the following article:Document 1383594.1 Collaborate With MOS Database Tuning Community MembersReferencesNOTE:1320966.1 - Things to Consider Before Upgrading to 11.2.0.2 to Avoid Poor Performance or Wrong ResultsNOTE:461228.1 - The LTOM Graph FAQNOTE:175006.1 - Steps to generate HANGANALYZE trace files (9i and below)NOTE:18251.1 - OERR: ORA 60 "deadlock detected while waiting for resource"NOTE:452358.1 - How to Collect Diagnostics for Database Hanging IssuesNOTE:199083.1 - * Master Note: SQL Query Performance OverviewNOTE:223117.1 - Troubleshooting I/O-related waitsNOTE:68738.1 - No Response from the Server, Does it Hang or Spin?NOTE:387914.1 - WRH$_ACTIVE_SESSION_HISTORY Does Not Get PurgedNOTE:390374.1 - Oracle Performance Diagnostic Guide (OPDG)NOTE:392037.1 - Database 'Hangs'. Sessions Wait for 'resmgr:cpu quantum'NOTE:166650.1 - Working Effectively With Support Best PracticesNOTE:461050.1 - The LTOM Graph (LTOMg) User GuideNOTE:1357946.1 - Troubleshooting 'library cache: mutex X' waits.NOTE:352648.1 - How to Diagnose High CPU Usage Problems to the Module LevelNOTE:554831.1 - How to Control the Set of Top SQLs Captured During AWR Snapshot GenerationNOTE:398838.1 - FAQ: SQL Query Performance - Frequently Asked QuestionsNOTE:465787.1 - How to: Manage CBO Statistics During an Upgrade from 10g or 9i into 11gNOTE:376442.1 - * How To Collect 10046 Trace (SQL_TRACE) Diagnostics for Performance IssuesNOTE:39817.1 - Interpreting Raw SQL_TRACE outputNOTE:62365.1 - Troubleshooting "ORA-00060 Deadlock Detected" ErrorsNOTE:785351.1 - Oracle 11gR2 Upgrade CompanionNOTE:857576.1 - How to Minimise Waits for 'Log File Sync'NOTE:94224.1 - FAQ- Statspack Complete ReferenceNOTE:70609.1 - How To Display Information About Processes on SUN SolarisNOTE:977975.1 - Diagnose LTOM Connection ProblemsNOTE:1055547.1 - SYSAUX Grows Because Optimizer Stats History is Not PurgedNOTE:34592.1 - WAITEVENT: "log file sync" Reference NoteNOTE:352363.1 - LTOM - The On-Board Monitor User GuideNOTE:601807.1 - Oracle 11gR1 Upgrade CompanionNOTE:210014.1 - How to Log a Good Performance Service RequestNOTE:1377446.1 - * Troubleshooting Performance IssuesNOTE:243755.1 - Script to produce HTML report with top consumers out of PL/SQL Profiler DBMS_PROFILER dataNOTE:1528847.1 - Troubleshooting: Avoiding and Resolving Database Performance Related Issues After UpgradeNOTE:1392633.1 - Things to Consider Before Upgrading to 11.2.0.3 to Avoid Poor Performance or Wrong ResultsNOTE:466181.1 - Oracle 10g Upgrade CompanionNOTE:1361401.1 - Where to Find Information About Performance Related FeaturesNOTE:1362329.1 - How To Investigate Slow or Hanging Database Performance IssuesNOTE:1363422.1 - Automatic Workload Repository (AWR) Reports - Start PointNOTE:1364257.1 - How to Collect Errorstacks for use in Diagnosing Performance Issues.NOTE:163424.1 - How To Identify a Hot Block Within The Database Buffer Cache.NOTE:1360119.1 - * FAQ: Database Performance Frequently Asked QuestionsNOTE:552880.1 - General Guidelines for SYSAUX Space IssuesNOTE:1278149.1 - Intermittent Long 'log file sync' Waits, LGWR Posting Long Write Times, I/O Portion of Wait MinimalNOTE:1195363.1 - Database Performance and SQL Tuning Documentation on OTNNOTE:121779.1 - Taking Systemstate Dumps when You cannot Connect to OracleNOTE:62143.1 - Troubleshooting: Tuning the Shared Pool and Tuning Library Cache Latch ContentionNOTE:1415999.1 - Troubleshooting: Virtual Circuit WaitsNOTE:1064487.1 - Script to Collect Log File Sync Diagnostic Information (lfsdiag.sql)NOTE:438452.1 - Performance Tools Quick Reference GuideNOTE:438755.1 - High SQL Version Counts - Script to determine reason(s)NOTE:461052.1 - LTOM System Profiler - Sample OutputNOTE:278316.1 - Troubleshooting: "WAITED TOO LONG FOR A ROW CACHE ENQUEUE LOCK! "NOTE:15476.1 - FAQ: Detecting and Resolving Locking Conflicts and Ora-00060 errorsNOTE:1392319.1 - Master Note: Locks, Enqueues and Deadlocks (ORA-00060)NOTE:155971.1 - Resolving Intense and "Random" Buffer Busy Wait Performance ProblemsNOTE:1376916.1 - Troubleshooting: "log file sync" WaitsNOTE:1292724.1 - Suggestions if your SYSAUX Tablespace grows rapidly or too largeNOTE:62354.1 - Waits for 'Enq: Tx - Row Lock Contention' - Wait Scenario ExamplesNOTE:1377998.1 - Troubleshooting: Waits for Mutex Type EventsNOTE:1383594.1 - Collaborate with MOS Database Tuning Community MembersNOTE:762540.1 - Consolidated Reference List Of Notes For Migration / Upgrade Service RequestsNOTE:1356828.1 - FAQ: 'cursor: mutex ..' / 'cursor: pin ..' / 'library cache: mutex ..' Type Wait EventsNOTE:1509919.1 - Master Note for Database Error ORA-00060 "deadlock detected while waiting for resource"NOTE:781198.1 - Diagnostics for Database Performance IssuesNOTE:273646.1 - How to Diagnose High CPU Utilization of ORACLE.EXE in Windows EnvironmentNOTE:1342917.1 - Troubleshooting 'latch: cache buffers chains' Wait ContentionNOTE:1349387.1 - Troubleshooting 'cursor: pin S wait on X' waits.NOTE:296377.1 - Troubleshooting: High Version Count IssuesNOTE:301137.1 - OSWatcher Black Box (Includes: [Video])NOTE:33453.1 - Locking and Referential IntegrityNOTE:34405.1 - WAITEVENT: "buffer busy waits" Reference Note

Purpose Questions and Answers Pro-Active Problem Avoidance and Diagnostic Collection Performance Information Centres Troubleshooting Guides Common Problem Topics Slow Database Performance Concurrency...

Master Notes

Master Note For Database and Client Certification

For most current information refer:Master" Note For Database and Client Certification [ID 1298096.1]In this Document  Purpose  Scope and Application  Master Note For Database and Client Certification     Main Note     General     Using My Oracle Support EffectivelyApplies to:Information in this document applies to any platform. PurposeThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support notes with respect to Oracle Certify and Oracle Database Server/Client.Scope and ApplicationThis note is applicable to all levels of expertise.Master Note For Database and Client CertificationMain NoteNote 1295603.1 Locate Database Server Certification Information on My Oracle SupportGeneralNote 763087.5 Tips for Finding Certifications in Classic CertifyNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryNote 207303.1 Client / Server / Interoperability Support Between Different Oracle VersionsNote 77627.1 Oracle Database Server product support Matrix for Windows 2000Note 1062972.1 Locate Oracle Database Server Certification Information for Microsoft Windows PlatformsNote 824935.1 Where To Find Oracle Patchset/Software for Specific Server/ Hardware Model?Note 942852.1 Oracle VM and VMWare Certification for Oracle ProductsNote 750215.1 Which OS File Systems Are Certified For Single Instance Oracle Databases?Note 403202.1 Certification of Zeta File System (Zfs) On Solaris 10 for Oracle RDBMSNote 1075717.1 Installing 32-bit RDBMS Client software on x86_64 Linux.Note 870253.1 32-bit Client Install on 64-bit Windows PlatformNote 753601.1 SUPPORT FOR ORACLE DATABASE ON WPARS UNDER AIX 6.1Note 1194734.1 Where do I find that on My Oracle Support (MOS) [Video]Note 971464.1 FAQ - 11gR2 requires Solaris 10 update 6 or greater Using My Oracle Support EffectivelyNote 374370.1 New Customers Start HereNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support Services 

For most current information refer: Master" Note For Database and Client Certification [ID 1298096.1] In this Document  Purpose   Scope and Application   Master Note For Database and Client...

Master Notes

Master Note for Diagnosibility

Master Note for Diagnosibility (Oracle Database) This Master Note is intended to provide a summary, index and references on topics around diagnosibility. This  includes: - What is diagnosibility ?   - What's new with the 11g diagnosibility concept? - How to gather diagnostic data for different releases?  - How to upload diagnostic files to Oracle support?  - How to verify uploads?  - Known Problems  This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest, within diagnosibility.   What is Diagnosibilty? Actually diagnosibility has different meanings:Fault diagnosibility captures diagnostic data, such as dump files or core dump files, on the OCCI client when a problem occurs. In pre 11g databases diagnosibilty based on single files like alert.log and process tracefiles.With 11g 'Diagnosibility' was introduced as a feature that simplifies collecting the diagnostic information and sending this infornation to support. Diagnosibilty covers the whole process from data collection, package creation,  package upload to ORACLE and package investigation at ORACLE support. The overall view is called Diagnosibilty Framework (DFW)RDBMS diagnostic data has been reorganized and are stored in a common directory structure named ADR (Automatic Directory Repository)  ADR contains different diagnostic files of xml format in a database-like structure. Getting started <<Note 1292665.1>> - Oracle Database Support Newsletter - February, 2011 Packaging 1. Package types Packages can be categorized as following: Description base type DB version  file format upload target standard OS package Y all tar, gz, zip GTCR RDA (Remote Diagnostic Agent) Y all zip ADR Repository IPS (11g Incident Packaging Service) Y 11.1 and higher zip ADR Repository IPS + RDA (bundle) N 11.2.0.2 and higher zip ADR Repository OCM (Oracle Configuration Manager) Y 8.1.7 and higher ocmconfig.jar Configuration repository OCM + RDA (bundle) N 8.1.7and higher zip Configuration Repository &ADR Repository 2. Incident Packaging Services (IPS)   There are 2 options to create IPS packages: 2.1 Commandline Interface (ADRCI) For IPS package creation in general and use of command line utility (ADRCI) refer to: <<Note 443529.1>> - 11g Quick Steps to Package and Send Critical Error Diagnostic Information to Support [Video] 2.2. GUI (SWB) For use of Support Workbench - the graphical user interface refer to: <<Note 1091653.1>> - 11g Quick Steps - How to create an IPS package using Support Workbench [Video] Upload Diagnostic Data Diagnostic files and packages can be uploaded to ORACLE using My ORACLE Support (MOS) the graphical customer user interface. This requires a Service Request (SR). The upload should not be interrupted or stopped. Once completed you will receive a completion message and SR becomes automatically updated with different upload status details. These messages can be used by support or customers to check upload status. Verify Uploads in MOS In order to examine upload messages, open your SR in MOS and verify following settings: set [Filter By] to <-- Show all Entries -->uncheck [Hide System Messages] checkbox Package Processing - Step 1 (virus scan) Each incoming package or file will be processed by virus scan first. After completion you will see a message like: Virus Scan successfully completed for RDA.RDA_dwhprod.zip. If you see this message, this will indicate that your file was successfully uploaded and received by ORACLE. Package Processing - Step 2 (package specific actions) After Virus Scan package and file processing depends on package type. Different types have different processing. The following will summarize processing of base package types. Package bundles will be splited into base components and processed accordingly. a) Standalone RDA packages will be moved and extracted to Global ADR Repository. Completion tag: Auto generated message by RDA FUP :RDA.RDA_BBDD1.zip has been uploaded to the ADR Repository. b) Standalone IPS packages will be moved to Global ADR Repository, extracted and processed by ADRCI. Completion tag: Diagnostic file ORA600_20100526101903_COM_1.zip was successfully processed at Oracle by 11g Diagnostics c) Configurations will be moved to Configuration Repository. Completion tag:  Auto generated message by RDA-OCM FUP :ocmconfig.jar has been uploaded to Configuration repository. d) Anything else - different from RDA, IPS or configurations - goes to Global Testcase Repository (GTCR) Health Monitor <<Note 466920.1>> - 11g New Feature: Health monitor<<Note 951022.1 >> - How To Purge or Delete Health Monitor Checks After DBMS_HM.RUN_CHECK ? Diagnostic Internals <<Note 422893.1>> - 11g Understanding Automatic Diagnostic Repository<<Note 750982.1>> - TRM Trace Files Getting Generated in 11g Database FAQs <<Note: Note 453125.1>> - 11g Diagnosability Frequently Asked Questions Troubleshooting  ADR - Purging and Retension <<Note 564269.1>> - Retention Policy for ADR<<Note 751082.1>> - Adrci Purge Does Not Clear the Text-formatted Alert.Log located in the Trace Folder<<Note 975448.1>> - WHICH FILES ARE PART OF SHORTP_POLICY AND LONGP_POLICY IN ADR? Tips and Tricks 1. How to create an incident for testing? Attention: Don't use this in production systems ! <<Note 390293.1>> - Introduction to 600/7445 Internal Error Analysis Diagnostic Tools <<Note 559339.1>> - Diagnostic Tools Catalog<<Note 153788.1>>  - ORA-600/ORA-7445 Error Look-up Tool

Master Note for Diagnosibility (Oracle Database) This Master Note is intended to provide a summary, index and references on topics around diagnosibility. This  includes:  - What is diagnosibility ?  -...

Master Notes

Master Note for Partitioning [ID 1312352.1]

++++++++++++++++++++++++++++++++++++++++++++++++++++The complete and the most recent version of this article can be viewedfrom My Oracle Support Knowledge Section. Master Note for Partitioning [ID 1312352.1]++++++++++++++++++++++++++++++++++++++++++++++++++++In this Document Purpose Scope and Application Master Note for Partitioning Concepts/Definitions Section How-To Section Troubleshooting Section Known issues Additional Resources ReferencesThis document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process and therefore has not been subject to an independent technical review.Applies to:Oracle Database Products > Oracle Database > Data WarehousingInformation in this document applies to any platform.PurposeThis article is intended to give you a single reference point and to assist in quickly finding various information about partitioning. Scope and ApplicationThe document will cover the following topics:Concepts/DefinitionsHow-To SectionTroubleshooting SectionKnown issuesAdditional ResourcesMaster Note for PartitioningConcepts/Definitions SectionOverview of PartitioningPartitioning enables you to decompose very large tables and indexes into smaller and more manageable pieces called partitions. Each partition is an independent object with its own name and optionally its own storage characteristics.Partitioning offers these advantages:Increased availability of mission-critical databases if critical tables and indexes are divided into partitions to reduce the maintenance windows, recovery times, and impact of failures.Easier administration of schema objects reducing the impact of scheduled downtime for maintenance operations.Reduced contention for shared resources in OLTP systemsEnhanced query performance: Often the results of a query can be achieved by accessing a subset of partitions, rather than the entire table. For some queries, this technique (called partition pruning) can provide order-of-magnitude gains in performance.Relevant LinksOverview of Partitioning:Oracle® Database Concepts 11g Release 2 (11.2) Part Number E16508-05 Chapter 4Oracle® Database VLDB and Partitioning Guide 11g Release 2 (11.2) Part Number E16541-08Relevant articles regarding partitions creation and maintenance:Document   69715.1 Creating & Adding Table and Index Partitions Document 165303.1  Examples about Insert into Range Partitioned TablesDocument 164874.1 Example of Script to Create a Range Partition Table Document 166652.1 Example of Script to Maintain Range Partitioned Table Document 149116.1 Oracle9i Partitioning Enhancements, LIST PartitioningDocument 209368.1 Range List Partitioning - Oracle 9.2 Enhancement Document 276158.1 Partitioning Enhancements in Oracle 10g Relevant articles regarding latest features:Document 452447.1 11g Partitioning Enhancements Document 785462.1 11g New Features:System Partitioning Document 805976.1 11g New Features:INTERVAL PARTITIONING Document 943567.1 11g new feature: Extended Composite Partitioning (Overview, Example and Use) Document 761251.1 Oracle 11G Reference Partitioning examples Document 466352.1 11g Feature: Interval Partitioning Example Document 757754.1 Interval Partitioning By WeekRelevant articles regarding partition pruning:Document 179518.1 Partition Pruning and Joins Document 166118.1 Partition Pruning/Elimination (This article provide a very detailed example of how to identify which partitions/subpartitions were accessed during an execution of a statement using event 10128)How-To SectionHow to Partition a Non-partitioned TableDocument 1070693.6 How to Partition a Non-partitioned Table Document 472449.1   How To Partition Existing Table Using DBMS_Redefinition 'How to' relevant articles for partitioning typesDocument 854332.1 How To Introduce Interval Parititioning into a Range Partitioned Table Document 165701.1 How to Implement Hash Partitioning on IOT Tables in 9i & Above Document 1266993.1 How To Use Multicolumn Partitioning Keys Document 74181.1 Partitioning Tables with User-Defined Types and LOBs Document 1304370.1 How to use Partition Names for ref partitioning tables when Split is usedDocument 854332.1 How To Introduce Interval Partitioning into a Range Partitioned Table Document 846405.1 How To Change the Partition Column Of A Partitioned Table Using DBMS_Redefinition'How to' relevant articles for partitioned indexes:Document 69374.1 Partitioned Indexes: Global, Local, Prefixed and Non-Prefixed Document 74224.1 How to Create Primary Key Partitioned Indexes  Document 795854.1 How To Update Both Global and Local Indexes when Moving Table Partition? 'How to' relevant articles for statistics collection when partitions are used:Document 175258.1 How to Compute Statistics on Partitioned Tables and Indexes Document 237538.1 How to Move from ANALYZE to DBMS_STATS on Partitioned Tables - Some Examples Document 111990.1 ORA-14508: ANALYZE PARTITION TABLE VALIDATE STRUCTURE CASCADE Document 1050294.1 STALE_STATS OF SUB PARTITION INDEX REPORTED INCORRECTLY IN DBA_TAB_STATISTICS Document 1302628.1 Collect statistics for a large partitioned table takes a lot when incremental is used Document 1319225.1 Collect incremental statistics for a large partitioned table in versions 10.2 and 11 Troubleshooting SectionRelevant bulletin articles for partitioning related issues:Document 209070.1 Partition Pruning based on Joins to Partitioning Criteria Stored in Dimension Tables Document 378138.1 What to check when the fast split partitioning does not appear as working? Document 232628.1 Fast Split partitioning in 9iR2Relevant articles for troubleshooting import slow issues with partitioningDocument 1073195.1 Data Pump Import (Impdp) slow when importing partitioned table  Document 752904.1 DataPump Export of Partitioned Table is Very Slow and Apparently Hangs  Document 1224663.1 IMPDP Raises ORA-39001, ORA-39203 When Importing A Partition Of A Partitioned Table Over A Network Link OthersDocument 1281826.1 What Types of Partitioning Are Eligible for Partition Change Tracking (PCT) Fast Refresh?Known issuesDocument 165599.1 Top Partitioned Tables IssuesDocument 199623.1 Top Issues Encountered Regarding Split PartitionDocument 166215.1 Top Partition Performance Issues Document 372357.1 Fast Split Partitioning Not Ocurring When It Was Expected Document 272312.1 How to Recreate a Table Partition After Having Dropped the Datafile?Document 959116.1 Interval Partitioning Does Not Inherit Logging Document 198120.1 Exchange Partitions - Common Problems Document 1077819.1 PARTITIONED TABLE: CHANGE IN BUFFER_POOL FOR A PARTITION NEEDS A REBOUNCE Relevant article to handle various errors:Document 887659.1 Getting ORA-01410 for Partitioned Tables Without Any Apparent DDL On The Partitions. Document 1081230.1 INTERVAL RANGE Partition Giving ORA-14400 Document 389804.1 Range Partition Splitting Fails with ORA-01882 Document 405922.1 ORA-07445 [evaopn2] Query With Bit Mapped Indexes And Partitioning Document 1084542.1 Ora-600 Internal Error Code, Arguments: [Kkedsamp: Bad Partitioning Info.], [1290], [663], Document 1050966.1 ORA-14652 When Using Reference Partitioning Document 784989.1 ORA-600 [kdblGetRDBA] During Create Table with Compression and Partitioning Document 790630.1 ORA-600 [Kkpamdgspam1] When Joining Partitioned Tables at Different Partitioning Levels Document 727306.1 Ora-14074 When Trying To Add Partition Dictionary issues introduced with partitioningDocument 1289275.1 OBJ$-PARTOBJ$-TABPART$ mismatch - Dictionary Inconsistency reported for Interval Range Partition Tables Relevant article to handle size for partitioningDocument 729149.1 Table/Index (partition) Growth Is Far More Than Expected Install partitioning optionDocument 434743.1 Can The Partitioning Option Be Deinstalled When System Partitioned Objects Are Used Document 430239.1 How To De-install Partitioning Option From Enterprise Edition? Additional ResourcesCommunity: Database DataWarehousingThis community has been set up for My Oracle Support (MOS) users, and is being moderated by Oracle. The goal of this community is to exchange database related Data Warehousing knowledge and concepts including Partitioning.White PapersPartitioning with Oracle Database 11g Release 2 (September 2009)Partitioning in Oracle Database 11g (June 2007) Partitioning White papers are also accessible via Note 1329441.1ReferencesNOTE:1329441.1 - White Papers for Data Warehousing Components in the Oracle Database

++++++++++++++++++++++++++++++++++++++++++++++++++++ The complete and the most recent version of this article can be viewed from My Oracle Support Knowledge Section. Master Note for Partitioning [ID 13...

Master Notes

Troubleshooting ORA-1555 (Doc ID 1307334.1)

This article is intended to assist in finding tips and techniques to assist with finding solutions to ORA-1555 errors. The document will cover the following topics:Concepts/DefinitionsDiagnosingCommon Causes/SolutionsConcepts/DefinitionsThe ORA-1555 errors can happen when a query is unable to access enough undo to builda copy of the data at the time the query started. Committed “versions” of blocks aremaintained along with newer uncommitted “versions” of those blocks so that queries canaccess data as it existed in the database at the time of the query. These are referred to as“consistent read” blocks and are maintained using Oracle undo management.See Note 40689.1 - ORA-1555 "Snapshot too old" - Detailed Explanation for more aboutthese errors.DiagnosingDue to space limitations, it is not always feasible to keep undo blocks on hand for the life of the instance. Oracle Automatic Undo Management (AUM) helps to manage the time frame that undo blocks are stored. The time frame is the “retention” time for those blocks. There are several ways to investigate the ORA-1555 error. In most cases, the error is a legitimate problem with getting to an undo block that has been overwritten due to the undo “retention” period having passed. AUM will automatically tune up and down the “retention” period, but often space limitations or configuration of the undo tablespace will throttle back continuous increases to the “retention” period. The error message is reported in the user session and often is not captured in the alert log. The user could see a message likeUsing rollback segment functionality:ORA-1555: snapshot too old (rollback segment too small)orUsing AUM:ORA-01555: snapshot too old: rollback segment number 9 with name "_SYSSMU9$" too smallIf the error is captured in the alert.log, you would see something likeTue May 26 16:16:57 2009ORA-01555 caused by SQL statement below (SQL ID: 54yn3n36w24ft, Query Duration=922 sec, SCN: 0x0007.8a55f4e3)Initial InvestigationRollback Segments:With Oracle 10g and later versions of Oracle, you can still use a Rollback Segments configuration. ORA-1555 errors in that environment still follow older guidelines as described in Note 10579.1 - How many Rollback Segments to HaveNote 107085.1 - Tuning Rollback SegmentsNote 69464.1 - Rollback Segment Configuration & TipsAutomatic Undo Management:The database will be self tuning for undo when using Automatic Undo Management. This does not eliminate ORA-1555 completely, but does minimize ORA-1555 as long as there is adequate space in the undo tablespace and workloads tend to follow repeatable patterns. In some cases with periodic changes to workload (large data updates particularly with LOB data) the self tuning of undo can become aggressive and lead to undo issues.Note 461480.1 - FAQ Automatic Undo Management (AUM) / System Managed Undo (SMU)Note 135053.1 -How to Create a Database with Automatic Undo ManagementNote 268870.1 - How to Shrink the datafile of Undo TablespaceNote 231776.1 - How to switch a Database from Automatic Undo Management (AUM) back to using Rollback SegmentsNote 296863.1 - How to Keep All UNDO Segments from Being Offlined in Oracle 10g - Fast Ramp-UpLOB Issues:Out-of-row LOB undo is maintained in the LOB segment. So the UNDO tablespace and undo retention is not associated with most LOB ORA-1555 issues. Instead the LOB column is created using either PCT_VERSION or RETENTION to manage how much space within blocks or time transpires before the LOB undo is overwritten. In environments with high updates, deletes on rows including LOBs, the chances of ORA-1555 on LOB undo is very high.PCT_VERSION and RETENTION are not auto-tuned. To “tune” those configuration settings, you must change the values for PCT_VERSION or RETENTION. Changes to UNDO_RETENTION does not change LOB retention time frames.Note 162345.1 - LOBS - Storage, Read-consistency and RollbackNote 386341.1 - How to determine the actual size of the LOB segments and how to free the deleted/unused space above/below the HWMNote 563470.1 – Lob retention not changing when undo_retention is changedNote 422826.1 – How to identify LOB Segment Use PCTVERSION or RETENTION from Data DictionaryError TracingUndo error tracing can be done for normal undo operations using the following events:NOTE: Normal undo operations will be indicated in the error message in that the error message includes a segment name like…. name "_SYSSMU1$" too smallIf the error doesn’t show a segment name… name "" too smallthe problem is often related to LOB undoIf using pfile:event="10442 trace name context forever, level 10"If using spfile:Alter system set events '10442 trace name context forever, level 10';Reproduce the ORA-1555 error and upload the trace file to Oracle Support.LOB undo error tracing is more difficult. Set additional tracing events as follows:Start Session 1Alter session set events '10046 trace name context forever, level 12';Reproduce the errorExit Session 1Start Session 2Alter session set events '10051 trace name context forever, level 1';Reproduce the errorExit Session 2Start SessionAlter session set events '1555 trace name errorstack forever, level 3';Reproduce the errorExit Session 3Additional resources to review:Note 846079.1 – LOBs and ORA-1555 troubleshootingNote 253131.1 –Concurrent Writes May Corrupt LOB Segment When Using Auto Segment Space ManagementNote 467872.1 – TROUBLESHOOTING GUIDE (TSG) – ORA-1555V$UNDOSTAT AnalysisThe V$UNDOSTAT view holds undo statistics for 10 minute intervals. This view represents statistics across instances, thus each begin time, end time, and statistics value will be a unique interval per instance.This does not track undo related to LOBNote 262066.1 – How To Size UNDO Tablespace For Automatic Undo ManagementNote 1112363.1 – When Does Undo Used Space Become Available?Note 240746.1 – 10g NEW FEATURE on AUTOMATIC UNDO RETENTIONDiagnostics ScriptsRefer to Note 746173.1 : Common Diagnostic Scripts for AUM problemsand Note 877613.1 : AUM Common Analysis/Diagnostic ScriptsCommon Causes/SolutionsUsing Rollback Segments functionality:* Problem happening on SYSTEM tablespace that still uses old Rollback Segment functionality even when configured for Automatic Undo Management (AUM).* There are not enough rollback segments to manage the undo needed for long running queries.*Rollback Segments are too small and undo is overwritten before long running queries complete.Reference:Note 69464.1 – Rollback Segment Configuration & TipsNote 10630.1 – ORA-1555: “Snapshot too old” – OverviewNote 862469.1 – ORA-604 & ORA-1555 Rollback Segment 0 with Name “System” Too SmallUsing Automatic Undo Management (AUM):*TUNED_UNDORETENTION in V$UNDOSTAT around the time of the error is lower than the QUERY DURATION indicated in the error message. This is a legitimate ORA-1555 and if queries are going to run for very long time frames, UNDO_RETENTION may need to be larger. Auto-tuned retention may not be able to keep up with the undo workload and staying within space limitations on the UNDO tablespace.* LOB updates and/or deletes are frequent and a higher PCT_VERSION is required to provide enough space in the LOB Segment to accommodate the LOB undo. RETENTION on LOBs that are updated or deleted frequently can run into problems holding UNDO long enough for queries.* QUERY DURATION shown in the error message is 30+ years and therefore, no amount of undo will satisfy the consistent read blocks.Note 750195.1 – ORA-1555 Shows Unrealistic Query Duration (billions of seconds)* QUERY DURATION shown in the error message is 0. NOTE: This has been filed as a bug on many release levels and has been very difficult to narrow down to a specific problem.Note 761128.1 – ORA-1555 Error when Query Duration as 0 Seconds * QUERY DURATION is lower than TUNED_UNDRETENTION. Undo header information can sometimes get overwritten or you could be seeing a bug.* TUNED_UNDORETENTION stays very high and UNDO tablepsace continues to grow continuously or getting space errors.Note 1112431.1 – Undo Remains Unexpired When Using Non-autoextensible Datafiles for Undo Tablespace.Additional Reference:Database Administration Community

This article is intended to assist in finding tips and techniques to assist with finding solutions to ORA-1555 errors. The document will cover the following topics: Concepts/Definitions DiagnosingCommo...

Master Notes

Oracle Support Master Note for AQ Queue Monitor Process (QMON) (Doc ID 305662.1)

Master Note for AQ Queue Monitor Process (QMON) (Doc ID 305662.1)Copyright (c) 2011, Oracle Corporation. All Rights Reserved.In this Document  Purpose  Scope and Application  Master Note for AQ Queue Monitor Process (QMON)     Queue Monitor - QMON     QMON coordinator     QMON tasks     QMON Server Processes      Pre-10g QMON Architecture      10g/11g QMON Architecture      Significance of AQ_TM_PROCESSES Parameter     Common Observations / Issues linked to QMON       - PROCESSED Messages not being removed      - TM Operations : Delay, Expiration, Retention not working as expected      - Delay / WAIT Period Incorrect after Daylight Saving Time change      - High CPU usage from QMON Coordinator process      - Unexpected Growth in Queue Table Objects     QMON Space Reclamation / Coalesce Queues     Collecting Diagnostic Information for Troubleshooting QMON issues  ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 8.1.7.0 to 11.2.0.2 - Release: 8.1.7 to 11.2Oracle Server - Standard Edition - Version: 9.2.0.1 to 11.2.0.1   [Release: 9.2 to 11.2]Information in this document applies to any platform.PurposeIn this article, we will discuss the following 1. Queue Monitor Coordinator (QMNC) , Qmon Server Processes (QXXX) and Task Operations which can be assigned to Servers. Collectively named QMON.2. Known issues which affect these processes.3. How to collect useful diagnostic information when problems arise with them.Scope and ApplicationDatabase administrators of Advanced Queueing (AQ) and Streams databases. Master Note for AQ Queue Monitor Process (QMON)Queue Monitor - QMONQMON is connected with Oracle Streams Advanced Queueing (AQ), Streams and a variety of other Database products which monitor and maintain system and user-owned AQ persistent and buffered objects. For example, the Oracle job scheduler uses AQ and serves as a client to various database components to allow operations to be coordinated at scheduled times and intervals. Similarly, Oracle Grid Control relies on AQ for its Alerts and Service Metrics and database server utilities such as datapump now use AQ. Furthermore, Oracle Applications has been using AQ for a significant period of time and this will continue. QMON is primarily associated with the mechanisms for message expiration, retry, delay, maintaining queue statistics, removing PROCESSED messages from a queue table and updating the dequeue IOT as necessary.QMON has a part to play in permanent and buffered message processing. If a qmon process should fail, this should not cause the instance to fail. This is also the case with job queue processes. QMON itself operates on queues but does not use a database queue for its own processing of tasks and time based operations.QMON can be envisaged as a number of discrete tasks which are run by Queue Monitor processes or servers.QMON coordinatorThe coordinator was introduced in 10.1 and is responsible for allocating tasks to QMON processes. Some of these tasks are scheduled, time based activities whereas others are event driven.In the case of buffered messaging in a RAC environment, if a RAC instance should fail, an existing QMON process will move ownership of the queues, where necessary to a new owning instance. This would be relevant in a Streams configuration for example where a primary / secondary instance is defined. As an aim of Streams is to maintain messages in memory when an instance is down, the processing of buffered messages has to be done on a surviving instance (a related Capture or Apply process would also need to be relocated); once the owning instance has been changed, QMON can then resume activity on the buffered queue on this instance. Starting with 11.2.0.1, the coordinator is visible in GV$QMON_COORDINATOR_STATS.QMON tasksTasks relate to a specific action which will be allocated to a QMON server process. In 11.2.0.1, the view : GV$QMON_TASK_STATS shows all the tasks available at this version in addition to whether any errors may have been encountered in the task processing. The view shows details relating to the following tasks (based on columns : task_name and remark - as detailed in the Oracle Reference Guide) :Task NameRemarkQMON_PERSISTENT_TMPersistent messages time manager activityQMON_SPILL Buffered messages spillingQMON_DEALLOC_SPILLEDSpilled messages memory deallocationQMON_DELETE_SPILLEDDequeued spilled messages deletionQMON_PURGENot specifiedQMON_COMPUTE_ACKSAcknowledgement update for a queue locallyQMON_FLUSH_STATS Replay info table updateQMON_PROCESS_IPCIPC message send and receive for queue operationsQMON_RECOVER_SPILLED Spilled messages recovery on startupQMON_PROP_MSGDELETE Acknowledged buffered messages deletionQMON_JOBCACHE_REPARTITION  Queue table ownership changeQMON_PURGE_SPILLED Purge spilled messages at startupQMON_BUFFERED_TM_COORD Buffered messages time manager activity checkQMON_BUFFERED_TM Buffered messages time manager activityQMON_QUEUE_SERVICE_START Start queue services at startupStart queue services at startupQMON_PURGE_REGISTRATION Notification registration purgeQMON_RECOVER_EMON EMON recovery at startupQMON_ORPHANED_MSGDELETE Orphaned messages deletionOrphaned messages deletionQMON_SEND_ALTEROWNER Non-owner persistent time manager activity send to ownerQMON_NONDURSUB_SESS_DELSession end nondurable subscriber deleteQMON_NONDURSUB_INST_DELInstance end Nondurable subscriber deleteQMON_DELETE_DEADREG Notification delete registrations of dead locationsNote : Earlier versions may not have implemented all of the above . The task list gives an impression of those operations the QMON process is responsible for. It can be gleaned that a significant number of the above are associated with activities such as cleanout of messages and housekeeping activities, i.e. it is more efficient on the performance of the foreground Application AQ process which is performing enqueue / dequeue operations that cleanout operations be handled in the background . TM (Time Management : delay , retry delay, expiration , retention) related activity is also handled by QMON server processes. e.g when an application enqueues a message with a delay period the message will only become available for dequeue once the delay period has elapsed and QMON has changed the state of the message to READY.In 11.2.0.1, the view : GV$QMON_TASKS shows an indication of the tasks which are running or have been scheduled by QMON. Some tasks can only be run on a single instance for a queue such as might be the case with buffered messaging ; others can be run (not at the same time) across multiple instances by different qmon processes. Some tasks are categorised as repeatable operations and are scheduled to run periodically; others are viewed as one time operations with no schedule - as detailed in GV$QMON_TASKS.QMON Server Processes These are Processes or Servers at the OS level which are associated with task work activities scheduled by the coordinator.In 11.2.0.1, the view : GV$QMON_SERVER_STATS presents an indication of the server processes which are active. Pre-10g QMON Architecture The number of queue monitor processes was controlled via the dynamic initialisation parameter AQ_TM_PROCESSES. If this parameter is set to a non-zero value X, Oracle creates that number of QMNX processes starting from ora_qmn0_<SID> (where <SID> is the identifier of the database) up to ora_qmnX_<SID> ; if the parameter is not specified or is set to 0, then QMON processes are not created. There can be a maximum of 10 QMON processes running on a single instance. For example the parameter can be set in the init.ora as follows :aq_tm_processes=1 or set dynamically via alter system set aq_tm_processes=1; 10g/11g QMON Architecture Beginning with release 10.1, the architecture of the QMON processes was changed to an automatically controlled coordinator / slave architecture. The Queue Monitor Coordinator, ora_qmnc_<SID>, dynamically spawns slaves named, ora_qXXX_<SID>, depending on the system load up to a maximum of 10 per instance. These server processes are outlined above.Significance of AQ_TM_PROCESSES ParameterFor version 10.1 onwards it is no longer necessary to set AQ_TM_PROCESSES when Oracle Streams AQ or Streams is used. However, if you do specify a value, then that value is taken into account but the number of processes can still be auto-tuned and so the number of running qXXX processes can be different from what was specified by AQ_TM_PROCESSES. It should be noted that if AQ_TM_PROCESSES is explicitly specified then the process(es) started will only maintain persistent messages. For example if aq_tm_processes=1 then at least one queue monitor slave process will be dedicated to maintaining persistent messages. Other process can still be automatically started to maintain buffered messages. If you explicitly set aq_tm_processes = 10 then there will be no processes available to maintain buffered messages. This should be borne in mind in environments which use Streams replication and from 10.2 onwards user enqueued buffered messages.In addition you should never disable the Queue Monitor processes by setting aq_tm_processes=0 on a permanent basis. As can be seen above, disabling will stop all related processing in relation to tasks outlined. This will likely have a significant affect on operation of queues - PROCESSED messages will not be removed and any time related, TM actions will not succeed ; AQ objects will grow in size. To check whether auto-tuning is enabled or aq_tm_processes=0 do the following:connect / as sysdba set serveroutput on declare  mycheck number;  begin   select 1 into mycheck from v$parameter where name = 'aq_tm_processes' and value = '0'   and (ismodified <> 'FALSE' OR isdefault='FALSE');   if mycheck = 1 then   dbms_output.put_line('The parameter ''aq_tm_processes'' is explicitly set to 0!');   end if;   exception when no_data_found then   dbms_output.put_line('The parameter ''aq_tm_processes'' is not explicitly set to 0.'); end; / The parameter should not be set to 0 explicitly. If it is , then it is recommended to unset the parameter. However, this requires bouncing the database. In the meantime, if the database cannot be immediately bounced, the recommended value to set it to is '1', and this can be done dynamically: connect / as sysdba alter system set aq_tm_processes = 1; To unset the parameter: When using a pfile: Comment out or remove the aq_tm_processes entry, and restart the database. When using a spfile: connect / as sysdba alter system reset aq_tm_processes scope=spfile sid='*'; and restart the database Common Observations / Issues linked to QMON The following outlines a number of commonly observed issues attributable to certain aspects of QMON operation or which may have an affect on QMON. Some cases outline specific steps to resolve and issue or detail steps to run to avoid issues connected with the issue. Pertinent references are detailed with the intention of providing relevant context into what is being discussed. - PROCESSED Messages not being removedIf processed messages are not being cleaned out of queues once all subscribers have dequeued the message, this would suggest that QMON is not operating as expected : is the operation occurring at all or is it taking considerably longer than expected for this to occur.This consequence of this may be the growth of queue table related objects.Useful related references are : Note:251737.1 PROCESSED Messages remain in Queue Table after a Successful Dequeue , Note:378247.1 PROCESSED Messages not removed from Queue Table in a RAC database after Reconfiguration, Note:752708.1 Intermittently PROCESSED Messages are not removed from Queue Tables by the QMON Processes. - TM Operations : Delay, Expiration, Retention not working as expectedAre any of these Time Manager related features being used . The deferred processing of messages in these cases may require more processing than necessary . Is high CPU being observed which might suggest that something else is behind the problem. Something to consider as a general rule is that high CPU from a process might be typically connected with high buffer gets suggesting that a large object is being accessed, possibly with a Full Table Scan. In such a situation an AWR report and or 10046 / level 12 trace (as detailed below can identify the object) can identify the object. tkprof can then be used to summarise the exection plan as well as statistic information such as buffers accessed. Using retention has the affect of keeping messages for a longer period than they would be otherwise with the obvious knock on affect that queue table related objects will be larger. Useful related references are as follows : Note:341133.1 Messages not changed from Wait To Ready State in a RAC database , Note:343282.1 CPU Consumption Of Queue Monitor Processes Increases when using Retention, Note:464514.1 Messages Enqueued With a Delay Specified to an Advanced Queue in a RAC Database Are Not Dequeued Immediately After the Delay Expires, Note:732743.1 Qmon Processes Are Not Removing Processed Messages or changing the state of WAITING messages. - Delay / WAIT Period Incorrect after Daylight Saving Time changeFollowing a change in DST, TM based activities may not occur when expected. The enq_time may not as expected and given that the wait time or delay is calculated relative to the enq_time this will have affect on the operation. The related fix referenced in the notes below does correct QMON activity.This is outlined in Note:429630.1 - A Dequeue Condition fails to work properly after a Daylight Savings Time Change and Note:429681.1 - Casting AQ$QUEUE_TABLE Enqueue and Dequeue Time Values To SESSIONTIMEZONE causes Reporting and Message Processing issues.  - High CPU usage from QMON Coordinator processEnsure that aq_tm_processes is not set to 10. All the following refer to this same type of issue which manifests itself as high CPU from the Coordinator : Note:393781.1 , Note:604246.1 and Note:738873.1 all are linked to this issue. - Unexpected Growth in Queue Table ObjectsFirst of all please refer to section : QMON Space Reclamation / Coalesce Queues. QMON should perform periodic clean out of single consumer queue table indexes and coalesce multi consumer IOTs to ensure that space is reclaimed for AQ objects. If this does not work as expected, this can cause growth in these objects when there are actually few messages in the associated queues.An initial analysis would be to consider enqueue / dequeue activity as well as how many references there are to messages in the queue before then determining the space used by the related objects :- what is the throughput of messages in the queue - X messages per hour; - are any of the TM features : delay, retry delay , expiration or retention being used; - how many messages are currently in the queue (refer to queue table) :select count(*), msg_state from aq$<queue_table> group by msg_state;select count(*) from aq$_<queue_table>_i;select count(*) from aq$_<queue_table>_l; (new in 11.2.0.1)select count(*) from aq$_<queue_table>_h;select count(*) from aq$_<queue_table>_t;select count(*) from aq$_<queue_table>_p; (optional / spill / Streams related)select count(*) from aq$_<queue_table>_d; (optional / spill / Streams related)- then, for each of the above and their associated IOTs, determine the related space usage :select sum(bytes)/1024/1024 MB from user_segments where segment_name='<object_name>';The above is for a multi consumer queue ; a single consumer queue is simpler to look at as there is only the queue table and related indexes.Note : If Streams related objects are large , this might be a valid Application issue , suggesting for example that Streams has spilled due to memory pressure possibly indicating some other problem. If an IOT is particular are large, the following references may be useful : Note:394713.1 Index SYS_IOT_TOP_<N> on History IOT is very large / Qmn uses high CPU, Note:267137.1 QMON does not perform space management operations on the dequeue IOT in Locally Managed Tablespaces using ASSM or when using FREELIST GROUPs, Note:238272.1 Procedure to Manually Purge Messages from a Single-Consumer Queue when QMON fails to do it efficiently, Note:271855.1 Procedure to manually coalesce all the IOTs/indexes associated with Advanced Queueing tables to maintain Enqueue/Dequeue performance and reduce QMON CPU usage and Redo generation.QMON Space Reclamation / Coalesce QueuesThis is linked directly to the potential growth in AQ related objects in section Unexpected Growth in Queue Table Objects. As discussed in Note:271855.1 Procedure to manually coalesce all the IOTs/indexes associated with Advanced Queueing tables to maintain Enqueue/Dequeue performance and reduce QMON CPU usage and Redo generation , QMON does not service all related queue objects correctly until 11.2 Please consult this note and implement the script in your environment since it is probable that queues will have been created in ASSM tablespaces. As well as the space usage implications of this issue, the effect of implementing this procedure will likely be to improve the performance and effectiveness of QMON. Collecting Diagnostic Information for Troubleshooting QMON issuesIf the issue is not one which can be easily understood and addressed in the section : Common Observations / Issues linked to QMON then in an ideal situation troubleshooting any issue is easier to progress with a testcase. In the absence of this the following are some useful diagnostic steps for troubleshooting QMON issues. Typically this will be a situation in which the QMON process(es) are consuming a large amount of CPU or processed messages are not being removed. 1. For CPU consumption issues sql trace the QMON process in question by doing the following Determine the pid of the Queue Monitor process (either qmnc or q00*), call it X sqlplus "/ as sysdba" oradebug setospid X oradebug unlimit oradebug Event 10046 trace name context forever, level 12 --Generate trace for 20 minutes oradebug Event 10046 trace name context off Tkprof the raw sql trace file by following Note 232443.1. Provide both the raw trace file and tkprof output to Oracle Support. 2. For issues where a queue table is not being serviced in some way then the following may be useful: Determine the pid of the Queue Monitor processes (either qmnc or q00*), call them X, Y, etc. sqlplus "/ as sysdba" oradebug setospid X oradebug unlimit oradebug Event 10046 trace name context forever, level 12 oradebug Event 10850 trace name context forever, level 10 --10852 only applies to 10.1 onwards oradebug Event 10852 trace name context forever, level 32 --Generate trace for 20 minutes oradebug Event 10046 trace name context off oradebug Event 10850 trace name context off oradebug Event 10852 trace name context off Repeat this tracing for all the running Queue Monitor Coordinator and Queue Monitor slave processes. Tkprof the raw sql trace file by following Note 232443.1. Provide both the raw trace file and tkprof output to Oracle Support. 3. For investigating issues with QMON processes in a RAC environment then the following trace events are also useful oradebug Event 10852 trace name context forever, level 128 this traces queue table ownership changes and Event = '26700 trace name context forever, level 256' which traces inter-instance IPC communication. Note that event 26700 has a different meaning in 9.2 and should not be used. ReferencesNOTE:232443.1 - * How to Identify Resource Intensive SQL for TuningNOTE:47318.1 - Init.ora Parameter "AQ_TM_PROCESSES" Reference Note

Master Note for AQ Queue Monitor Process (QMON) (Doc ID 305662.1) Copyright (c) 2011, Oracle Corporation. All Rights Reserved. In this Document  Purpose   Scope and Application   Master Note for AQ...

Master Notes

Master Note For Database and Client Certification

For most current information refer:Master" Note For Database and Client Certification [ID 1298096.1]In this Document  Purpose  Scope and Application  Master Note For Database and Client Certification     Main Note     General     Using My Oracle Support EffectivelyApplies to:Information in this document applies to any platform.PurposeThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support notes with respect to Oracle Certify and Oracle Database Server/Client.Scope and ApplicationThis note is applicable to all levels of expertise.Master Note For Database and Client CertificationMain NoteNote 1295603.1 Locate Database Server Certification Information on My Oracle SupportGeneralNote 763087.5 Tips for Finding Certifications in Classic CertifyNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryNote 207303.1 Client / Server / Interoperability Support Between Different Oracle VersionsNote 77627.1 Oracle Database Server product support Matrix for Windows 2000Note 1062972.1 Locate Oracle Database Server Certification Information for Microsoft Windows PlatformsNote 824935.1 Where To Find Oracle Patchset/Software for Specific Server/ Hardware Model?Note 942852.1 Oracle VM and VMWare Certification for Oracle ProductsNote 750215.1 Which OS File Systems Are Certified For Single Instance Oracle Databases?Note 403202.1 Certification of Zeta File System (Zfs) On Solaris 10 for Oracle RDBMSNote 1075717.1 Installing 32-bit RDBMS Client software on x86_64 Linux.Note 870253.1 32-bit Client Install on 64-bit Windows PlatformNote 753601.1 SUPPORT FOR ORACLE DATABASE ON WPARS UNDER AIX 6.1Note 1194734.1 Where do I find that on My Oracle Support (MOS) [Video]Note 971464.1 FAQ - 11gR2 requires Solaris 10 update 6 or greater Using My Oracle Support EffectivelyNote 374370.1 New Customers Start HereNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support Services 

For most current information refer: Master" Note For Database and Client Certification [ID 1298096.1] In this Document  Purpose   Scope and Application   Master Note For Database and...

Master Notes

Master Note for Generic Data Warehousing

++++++++++++++++++++++++++++++++++++++++++++++++++++The complete and the most recent version of this article can be viewedfrom My Oracle Support Knowledge Section.Master Note for Generic Data Warehousing [ID 1269175.1]++++++++++++++++++++++++++++++++++++++++++++++++++++In this Document  Purpose  Master Note for Generic Data Warehousing     Components covered      Oracle Database Data Warehousing specific documents for recent versions     Technology Network Product Homes     Master Notes available in My Oracle Support     White Papers     Technical PresentationsPlatforms: 1-914CU; This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process and therefore has not been subject to an independent technical review.Applies to: Oracle Server - Enterprise Edition - Version: 9.2.0.1 to 11.2.0.2 - Release: 9.2 to 11.2Information in this document applies to any platform.PurposeProvide navigation pathMaster Note for Generic Data WarehousingComponents covered Read Only Materialized ViewsQuery RewriteDatabase Object PartitioningParallel Execution and Parallel QueryDatabase CompressionTransportable TablespacesOracle Online Analytical Processing (OLAP)Oracle Data MiningOracle Database Data Warehousing specific documents for recent versions11g Release 2 (11.2)11g Release 1 (11.1)10g Release 2 (10.2)10g Release 1 (10.1)9i Release 2 (9.2)9i Release 1 (9.0)Technology Network Product HomesOracle Partitioning Advanced CompressionOracle Data MiningOracle OLAPMaster Notes available in My Oracle SupportThese technical articles have been written by Oracle Support Engineers to provide proactive and top level information and knowledge about the components of thedatabase we handle under the "Database Datawarehousing".Note 1166564.1 Master Note: Transportable Tablespaces (TTS) -- Common Questions and IssuesNote 1087507.1 Master Note for MVIEW 'ORA-' error diagnosis. For Materialized View CREATE or REFRESHNote 1102801.1 Master Note: How to Get a 10046 trace for a Parallel QueryNote 1097154.1 Master Note Parallel Execution Wait Events Note 1107593.1 Master Note for the Oracle OLAP OptionNote 1087643.1 Master Note for Oracle Data MiningNote 1215173.1 Master Note for Query RewriteNote 1223705.1 Master Note for OLTP Compression Note 1269175.1 Master Note for Generic Data WarehousingWhite PapersTransportable Tablespaces white papers Database Upgrade Using Transportable Tablespaces:Oracle Database 11g Release 1 (February 2009)Platform Migration Using Transportable Database Oracle Database 11g and 10g Release 2 (August 2008)Database Upgrade using Transportable Tablespaces: Oracle Database 10g Release 2 (April 2007)Platform Migration using Transportable Tablespaces: Oracle Database 10g Release 2 (April 2007)Parallel Execution and Parallel Query white papersBest Practices for Workload Management of a Data Warehouse on the Sun Oracle Database Machine (June 2010)Effective resource utilization by In-Memory Parallel Execution in Oracle Real Application Clusters 11g Release 2 (Feb 2010)Parallel Execution Fundamentals in Oracle Database 11g Release 2 (November 2009)Parallel Execution with Oracle Database 10g Release 2 (June 2005)Oracle Data Mining white paper Oracle Data Mining 11g Release 2 (March 2010)Partitioning white papers Partitioning with Oracle Database 11g Release 2 (September 2009)Partitioning in Oracle Database 11g (June 2007)Materialized Views and Query Rewrite white papers Oracle Materialized Views  and Query Rewrite (May 2005)Improving Performance using Query Rewrite in Oracle Database 10g (December 2003)Database Compression white papers Advanced Compression with Oracle Database 11g Release 2 (September 2009)Table Compression in Oracle Database 10g Release 2 (May 2005)Oracle OLAP white papers On-line Analytic Processing with Oracle Database 11g Release 2 (September 2009)Using Oracle Business Intelligence Enterprise Edition with the OLAP Option to Oracle Database 11g (July 2008)Generic white papers Enabling Pervasive BI through a Practical Data Warehouse Reference Architecture (February 2010)Optimizing and Protecting Storage with Oracle Database 11g Release 2 (November 2009)Oracle Database 11g for Data Warehousing and Business Intelligence (August 2009)Best practices for a Data Warehouse on Oracle Database 11g (September 2008)Technical PresentationsA selection of ObE - Oracle by Examples documents:Generic Using Basic Database Functionality for Data Warehousing (10g) PartitioningManipulating Partitions in Oracle Database (11g Release 1) Using High-Speed Data Loading and Rolling Window Operations with Partitioning (11g Release 1) Using Partitioned Outer Join to Fill Gaps in Sparse Data (10g)Materialized View and Query RewriteUsing Materialized Views and Query Rewrite Capabilities (10g) Using the SQLAccess Advisor to Recommend Materialized Views and Indexes (10g)Oracle OLAPUsing Microsoft Excel With Oracle 11g Cubes (how to analyze data in Oracle OLAP Cubes using Excel's native capabilities)Using Oracle OLAP 11g With Oracle BI Enterprise Edition (Creating OBIEE Metadata for OLAP 11g Cubes and querying those in BI Answers)Building OLAP 11g CubesQuerying OLAP 11g CubesCreating Interactive APEX Reports Over OLAP 11g CubesSelection of presentations from theBIWA website:Extreme Data Warehousing With Exadata  by Hermann Baer (July 2010) (slides 2.5MB, recording 54MB)Data Mining Made Easy! Introducing Oracle Data Miner 11g Release 2 New "Work flow" GUI   by Charlie Berger (May 2010) (slides 4.8MB, recording 85MB )Best Practices for Deploying a Data Warehouse on Oracle Database 11g  by Maria Colgan (December 2009)  (slides 3MB, recording 18MB, white paper 3MB )

++++++++++++++++++++++++++++++++++++++++++++++++++++ The complete and the most recent version of this article can be viewed from My Oracle Support Knowledge Section.Master Note for Generic Data...

Master Notes

Enterprise Manager Database Control Configuration - Recovering From Errors Due to CA Expiry on Oracle Database 10.2.0.4 or 10.2.0.5 from 31-Dec-2010 onwards

This is applicable for Database Control and not Grid Control.If you are looking for Certificate Expiry related information for Grid Control, please refer to post here.DescriptionWhat is the Issue?In Enterprise Manager Database Control with Oracle Database 10.2.0.4 and 10.2.0.5, the root certificate used to secure communications via the Secure Socket Layer (SSL) protocol will expire on 31-Dec-2010 00:00:00. The certificate expiration will cause errors if you attempt to configure Database Control on or after 31-Dec-2010. Existing Database Control configurations are not affected by this issue.Likelihood of OccurrenceWhat Versions Are Affected?The issue impacts configuration of Database Control with Oracle Database 10.2.0.4 and 10.2.0.5 only. It does not impact database creation or upgrade.The issue does not impact existing Database Control configurations.What Happens During Database Control Configuration Failure?Database Configuration Assistant (DBCA) and Database Upgrade Assistant (DBUA) ErrorsDatabase Configuration Assistant (DBCA) and Database Upgrade Assistant (DBUA) will report the following error in the console:Could not complete the Enterprise Manager configuration.Enterprise manager configuration failed due to the following error -Error starting Database ControlEnterprise Manager Configuration Assistant (EMCA) ErrorsEnterprise Manager Configuration Assistant (EMCA) will write errors similar to those below to the emca.log file:CONFIG: Securing Database Control completed successfully .Jan 2, 2011 7:22:47 PM oracle.sysman.emcp.ParamsManager getParamCONFIG: No value was set for the parameter ORACLE_HOSTNAME.Jan 2, 2011 7:22:47 PM oracle.sysman.emcp.util.DBControlUtil startOMSINFO: Starting Database Control (this may take a while) ...Jan 2, 2011 7:22:47 PM oracle.sysman.emcp.util.PlatformInterface addEnvVarToListCONFIG: Value for env var 'ORACLE_HOSTNAME' is '', discarding the sameCONFIG: Returning env array from cacheJan 2, 2011 7:22:47 PM oracle.sysman.emcp.util.PlatformInterface executeCommandCONFIG: Starting execution: /myhost/bin/emctl start dbconsoleJan 2, 2011 7:27:26 PM oracle.sysman.emcp.util.PlatformInterface executeCommandCONFIG: Exit value of 1Jan 2, 2011 7:27:26 PM oracle.sysman.emcp.util.PlatformInterface executeCommandCONFIG: Oracle Enterprise Manager 10g Database Control Release 10.2.0.4.0Copyright (c) 1996, 2007 Oracle Corporation. All rights reserved.https://myhost:5501/em/console/aboutApplicationStarting Oracle Enterprise Manager 10g Database Control............................................................................................. failed.------------------------------------------------------------------Logs are generated in directory /myhost/sysman/logJan 2, 2011 7:27:26 PM oracle.sysman.emcp.util.PlatformInterface executeCommandWARNING: Error executing /myhost/bin/emctl start dbconsoleJan 2, 2011 7:27:26 PM oracle.sysman.emcp.EMConfig performSEVERE: Error starting Database ControlRefer to the log file at /myhost/dbua/d4/upgrade/emConfig.log for more details.Jan 2, 2011 7:27:26 PM oracle.sysman.emcp.EMConfig performCONFIG: Stack Trace:oracle.sysman.emcp.exception.EMConfigException: Error starting Database Controlat oracle.sysman.emcp.EMDBPostConfig.performUpgrade(EMDBPostConfig.java:763)at oracle.sysman.emcp.EMDBPostConfig.invoke(EMDBPostConfig.java:232)at oracle.sysman.emcp.EMDBPostConfig.invoke(EMDBPostConfig.java:193)at oracle.sysman.emcp.EMConfig.perform(EMConfig.java:184)at oracle.sysman.assistants.util.em.EMConfiguration.run(EMConfiguration.java:436)at oracle.sysman.assistants.util.em.EMConfigStep.executeImpl(EMConfigStep.java:140)at oracle.sysman.assistants.util.step.BasicStep.execute(BasicStep.java:210)at oracle.sysman.assistants.util.step.BasicStep.callStep(BasicStep.java:251)at oracle.sysman.assistants.dbma.backend.EMConfigStep.executeStepImpl(EMConfigStep.java:104)at oracle.sysman.assistants.dbma.backend.SummarizableStep.executeImpl(SummarizableStep.java:175)at oracle.sysman.assistants.util.step.BasicStep.execute(BasicStep.java:210)at oracle.sysman.assistants.util.step.Step.execute(Step.java:140)at oracle.sysman.assistants.util.step.StepContext$ModeRunner.run(StepContext.java:2488)at java.lang.Thread.run(Thread.java:534)The EMCA console will display output similar to the following:aime@myhost09 db_1]$ bin/emca -config dbcontrol db -repos recreate -clusterSTARTED EMCA at Jan 11, 2011 4:11:01 PMEM Configuration Assistant, Version 10.2.0.1.0 ProductionCopyright (c) 2003, 2005, Oracle. All rights reserved.Enter the following information:Database unique name: catestDatabase Control is already configured for the database catestYou have chosen to configure Database Control for managing the database catestThis will remove the existing configuration and the default settings and perform a fresh configurationDo you wish to continue? [yes(Y)/no(N)]: YListener port number: 1521Cluster name: myclusterPassword for SYS user:Password for DBSNMP user:Password for SYSMAN user:Email address for notifications (optional):Outgoing Mail (SMTP) server for notifications (optional):........Jan 11, 2011 4:18:05 PM oracle.sysman.emcp.util.DBControlUtil secureDBConsoleINFO: Securing Database Control (this may take a while) ...Jan 11, 2011 4:19:31 PM oracle.sysman.emcp.util.DBControlUtil startOMSINFO: Starting Database Control (this may take a while) ...Jan 11, 2011 4:28:38 PM oracle.sysman.emcp.EMConfig performSEVERE: Error starting Database ControlRefer to the log file at /myhost/oracle/product/10.2.0/db_1/cfgtoollogs/emca/catest/emca_2011-01-11_04-11-01-PM.log for more details.Could not complete the configuration. Refer to the log file at /myhost/oracle/product/10.2.0/db_1/cfgtoollogs/emca/catest/emca_2011-01-11_04-11-01-PM.log for more details.At the end of the database installation on non-Windows platforms, both Database Control and the Management Agent will be up and running, even though the status of both components will be shown as not running, because EMCTL will be unable to connect to the dbconsole process. In addition, Database Control will fail to connect to the Agent.Note for Windows Platform Only:On Windows, the dbconsole process will be stopped after the failed configuration attempt. Note that the tool used to perform Database Control configuration (DBUA, DBCA or EMCA) will also wait for 15 minutes for Database Control to start, then time out.The output of the "emctl status dbconsole" command incorrectly returns the status of Database Control, as shown below:$ ./emctl status dbconsoleOracle Enterprise Manager 10g Database Control Release 10.2.0.1.0Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.https://myhost:1158/em/console/aboutApplicationOracle Enterprise Manager 10g is not running.The output of the "emctl status agent" command incorrectly returns the status of the Agent, as shownbelow:$ ./emctl status agentOracle Enterprise Manager 10g Database Control Release 10.2.0.1.0Copyright (c) 1996, 2005 Oracle Corporation. All rights reserved.---------------------------------------------------------------Agent is Not RunningFor Solution, refer to Note: 1222603.1 Enterprise Manager Database Control Configuration - Recovering From Errors Due to CA Expiry on Oracle Database 10.2.0.4 or 10.2.0.5Note: 1217493.1 ATTENTION - Enterprise Manager Database Control 10.2.0.4 Or 10.2.0.5 - Patch Required from 31-Dec-2010 onwards

This is applicable for Database Control and not Grid Control. If you are looking for Certificate Expiry related information for Grid Control, please refer to post here. Description What is the Issue? In...

Master Notes

AIX Grid Control 10.2.0.5 Communication and Monitoring Issue since 31-DEC-2010

Detailed symptoms for Oracle Management Server (OMS) 10.2.0.5 on AIXOracle Management Service 10.2.0.5 instances on AIX 5L remain active and functional, but the OMS instances fail to communicate with the Grid Control Management Agents.An SSLPeerUnverified exception will be reported in the file $ORACLE_HOME/sysman/log/emoms.trc when OMS attempts to connect with an Agent:Javax.net.ssl.SSLPeerUnverifiedException: peer not authenticatedat com.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificateChain(DashoA12275)at oracle.sysman.emSDK.emd.comm.EMDClient.authenticateHTTPConnection(EMDClient.java:2002)at oracle.sysman.emSDK.emd.comm.EMDClient.getConnection(EMDClient.java:1877)at oracle.sysman.emSDK.emd.comm.EMDClient.getConnection(EMDClient.java:1810)at oracle.sysman.emSDK.emd.comm.EMDClient.verifyHttpConnection(EMDClient.java:2540)at oracle.sysman.emSDK.emd.comm.EMDClient.getResponseForRequest(EMDClient.java:2323)at oracle.sysman.emSDK.emd.comm.EMDClient.getUploadManagerStatus(EMDClient.java:4853)at oracle.sysman.eml.admin.rep.emdConfig.EmdConfigTargetsData.getEmdUploadData(EmdConfigTargetsData.java:1640)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)This error may be reported when:- Accessing the Agent home page in Grid Control- Setting preferred credentials for a target monitored by the Agent- Managing metrics for a target monitored by the AgentThe jobs scheduled to be run by Agents can become non-responsiveThe OMS log file $ORACLE_HOME/sysman/log/emoms.trc can show:2010-12-31 00:06:58,204 [JobWorker 430:Thread-34] DEBUG emSDK.comm getStreamResponse.4015 - oracle.sysman.emSDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticatedoracle.sysman.emSDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticatedat oracle.sysman.emSDK.emd.comm.EMDClient.getStreamResponse_(EMDClient.java:4088)at oracle.sysman.emSDK.emd.comm.EMDClient.getStreamResponse(EMDClient.java:4009)at oracle.sysman.emSDK.emd.comm.EMDClient.remoteOperation(EMDClient.java:3404)at oracle.sysman.emdrep.jobs.CommandManager.requestRemoteCommand(CommandManager.java:765)at oracle.sysman.emdrep.jobs.commands.RemoteOp.executeCommand(RemoteOp.java:434)at oracle.sysman.emdrep.jobs.commands.RemoteOp.executeCommand(RemoteOp.java:491)at oracle.sysman.emdrep.jobs.BaseJobWorker.runStep(BaseJobWorker.java:614)at oracle.sysman.emdrep.jobs.BaseJobWorker.doOneOperation(BaseJobWorker.java:738)at oracle.sysman.emdrep.jobs.JobWorker.doOneOperation(JobWorker.java:306)at oracle.sysman.emdrep.jobs.JobWorker.run(JobWorker.java:288)at oracle.sysman.util.threadPoolManager.WorkerThread.run(Worker.java:261)Detailed symptoms for Grid Control Management Agent 10.2.0.5 on AIXBeginning 31-DEC-2010 00:00:00, 10.2.0.5 Management Agents running on the AIX 5L operating system will fail to monitor Oracle Application Server targets. As a result, the Availability Status for the Oracle Application Server targets will be in the "Metric Error" state.NOTE: The 10.2.0.5.0 Agents would experience these errors regardless of the version/platform of the OMS.The following metric error is seen in the console for the Oracle Application Server targets monitored by a Grid Control Management Agent 10.2.0.5 installed on AIX and experiencing a Root Certificate Authority issue:Message oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated The Grid Control Management Agent log file $ORACLE_HOME/sysman/log/emagentfetchlet.log (or $ORACLE_HOME/hostname/sysman/log/emagentfetchlet.log for a clustered Agent) includes the following errors:2010-12-31 00:01:03,626 [nmefmgr_getJNIFetchlet] ERROR ias.ResponseMetric getResponseMetric.154 - Unable tocompute application server statusoracle.sysman.emSDK.emd.fetchlet.FetchletException: oracle.sysman.emSDK.emd.comm.CommException: java.io.IOException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticatedat oracle.sysman.ias.ias.ResponseMetric.getResponseMetric(ResponseMetric.java:108)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:618)at oracle.sysman.emd.fetchlets.JavaWrapperFetchlet.getMetric(JavaWrapperFetchlet.java:217)at oracle.sysman.emd.fetchlets.FetchletWrapper.getMetric(FetchletWrapper.java:382)Beginning 31-DEC-2010, 10.2.0.5 Management Agents on the AIX 5L platform will fail to secure or re-secure with Oracle Management Service (OMS). This failure will cause installation of 10.2.0.5 Agents on the AIX 5L platform to fail.NOTE: The 10.2.0.5.0 Agents would experience these errors regardless of the version/platform of the OMS.The "emctl secure agent" command will fail with the following error, which will be written to the $ORACLE_HOME/sysman/log/secure.log file (or $ORACLE_HOME/hostname/sysman/log/secure.log for a clustered Agent) :2011-01-03 21:06:11,941 [main] ERROR agent.SecureAgentCmd main.207 - Failedto secure the Agent:javax.net.ssl.SSLPeerUnverifiedException: peer not authenticatedatcom.sun.net.ssl.internal.ssl.SSLSessionImpl.getPeerCertificateChain(DashoA6275)atoracle.sysman.emctl.secure.agent.SecureAgentCmd.checkUpload(SecureAgentCmd.java:478)atoracle.sysman.emctl.secure.agent.SecureAgentCmd.secureAgent(SecureAgentCmd.java:249)atoracle.sysman.emctl.secure.agent.SecureAgentCmd.main(SecureAgentCmd.java:200) For solution, refer to AIX Grid Control 10.2.0.5 SSL Communication and Monitoring Issue since 31-DEC-2010 (Doc ID 1275070.1)Note: If you are looking for DBControl, please refer to the posting here

Detailed symptoms for Oracle Management Server (OMS) 10.2.0.5 on AIX Oracle Management Service 10.2.0.5 instances on AIX 5L remain active and functional, but the OMS instances fail to communicate with...

Master Notes

Oracle Support Master Note for Troubleshooting Advanced Queuing and Oracle Streams Propagation Issues (Doc ID 233099.1)

Master Note for Troubleshooting Advanced Queuing and Oracle Streams Propagation Issues (Doc ID 233099.1)Copyright (c) 2010, Oracle Corporation. All Rights Reserved.  In this Document  Purpose  Last Review Date  Instructions for the Reader  Troubleshooting Details     1. Scope and Application      2. Definitions and Classifications     3. How to Use This Guide     4. Basic AQ Propagation Troubleshooting     5. Additional Troubleshooting Steps for AQ Propagation of User-Enqueued and Dequeued Messages     6. Additional Troubleshooting Steps for Propagation in an Oracle Streams Environment     7. Performance Issues  ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 8.1.7.0 to 11.2.0.2 - Release: 8.1.7 to 11.2Information in this document applies to any platform.PurposeThis document presents a step-by-step methodology for troubleshooting and resolving problems with Advanced Queuing Propagation in both Streams and basic Advanced Queuing environments. It also serves as a master reference for other more specific notes on Oracle Streams Propagation and Advanced Queuing Propagation issues.Last Review DateDecember 20, 2010 Instructions for the ReaderA Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.Troubleshooting Details1. Scope and ApplicationThis note is intended for Database Administrators of Oracle databases where issues are being encountered with propagating messages between advanced queues, whether the queues are used for user-created messaging systems or for Oracle Streams. It contains troubleshooting steps and links to notes for further problem resolution.It can also be used a template to document a problem when it is necessary to engage Oracle Support Services. Knowing what is NOT happening can frequently speed up the resolution process by focusing solely on the pertinent problem area. This guide is divided into five parts:Section 2: Definitions and Classifications (discusses the different types and features of propagations possible - helpful for understanding the rest of the guide)Section 3: How to Use this Guide (to be used as a start part for determining the scope of the problem and what sections to consult)Section 4. Basic AQ propagation troubleshooting (applies to both AQ propagation of user enqueued and dequeued messages as well as Oracle Streams propagations)Section 5. Additional troubleshooting steps for AQ propagation of user enqueued and dequeued messagesSection 6. Additional troubleshooting steps for Oracle Streams propagationSection 7. Performance issues2. Definitions and ClassificationsGiven the potential scope of issues that can be encountered with AQ propagation, the first recommended step is to do some basic diagnosis to determine the type of problem that is being encountered.2.1. What Type of Propagation is Being Used?2.1.1. Buffered MessagingFor an advanced queue, messages can be maintained on disk (persistent messaging) or in memory (buffered messaging). To determine if a queue is buffered or not, reference the GV_$BUFFERED_QUEUES view. If the queue does not appear in this view, it is persistent.2.1.2. Propagation mode - queue-to-dblink vs queue-to-queueAs of 10.2, an AQ propagation can also be defined as queue-to-dblink, or queue-to-queue:queue-to-dblink: The propagation delivers messages or events from the source queue to all subscribing queues at the destination database identified by the dblink. A single propagation schedule is used to propagate messages to all subscribing queues. Hence any changes made to this schedule will affect message delivery to all the subscribing queues. This mode does not support multiple propagations from the same source queue to the same target database.queue-to-queue: Added in 10.2, this propagation mode delivers messages or events from the source queue to a specific destination queue identified on the database link. This allows the user to have fine-grained control on the propagation schedule for message delivery. This new propagation mode also supports transparent failover when propagating to a destination Oracle RAC system. With queue-to-queue propagation, you are no longer required to re-point a database link if the owner instance of the queue fails on Oracle RAC. This mode supports multiple propagations to the same target database if the target queues are different.The default is queue-to-dblink. To verify if queue-to-queue propagation is being used, in non-Streams environments query DBA_QUEUE_SCHEDULES.DESTINATION - if a remote queue is listed along with the remote database link, then queue-to-queue propagation is being used. For Streams environments, the DBA_PROPAGATION.QUEUE_TO_QUEUE column can be checked.See the following note for a method to switch between the two modes:Document 827473.1 How to alter propagation from queue-to-queue to queue-to-dblink2.1.3. Combined Capture and Apply (CCA) for StreamsIn 11g Oracle Streams environments, an optimization called Combined Capture and Apply (CCA) is implemented by default when possible. Although a propagation is configured in this case, Streams does not use it; instead it passes information directly from capture to an apply receiver. To see if CCA is in use:COLUMN CAPTURE_NAME HEADING 'Capture Name' FORMAT A30COLUMN OPTIMIZATION HEADING 'CCA Mode?' FORMAT A10SELECT CAPTURE_NAME, DECODE(OPTIMIZATION,0, 'No','Yes') OPTIMIZATIONFROM V$STREAMS_CAPTURE;Also, see the following note:Document 463820.1 Streams Combined Capture and Apply in 11g2.2. Queue Table CompatibilityThere are three types of queue table compatibility. In more recent databases, queue tables may be present in all three modes of compatibility:8.0 - earliest version, deprecated in 10.2 onwards8.1 - support added for RAC, asynchronous notification, secure queues, queue level access control, rule-based subscribers, separate storage of history information10.0 - if the database is in 10.1-compatible mode, then the default value for queue table compatibility is 10.02.3. Single vs Multiple Consumer Queue TablesIf more than one recipient can dequeue a message from a queue, then its queue table is multiple consumer. You can propagate messages from a multiple-consumer queue to a single-consumer queue. Propagation from a single-consumer queue to a multiple-consumer queue is not possible.3. How to Use This Guide3.1. Are Messages Being Propagated at All, or is the Propagation Just Slow?Run the following query on the source database for the propagation (assuming that it is running):select TOTAL_NUMBER from DBA_QUEUE_SCHEDULES where QNAME='<source_queue_name>';If TOTAL_NUMBER is increasing, then propagation is most likely functioning, although it may be slow. For performance issues, see Section 7.3.2. Propagation Between Persistent User-Created QueuesSee Sections 4 and 5 (and optionally Section 6 if performance is an issue).3.3. Propagation Between Buffered User-Created QueuesSee Sections 4, 5, and 6 (and optionally Section 7 if performance is an issue).3.4. Propagation between Oracle Streams Queues (without Combined Capture and Apply (CCA) Optimization)See Sections 4 and 6 (and optionally Section 7 if performance is an issue).3.5. Propagation between Oracle Streams Queues (with Combined Capture and Apply (CCA) Optimization)Although an AQ propagation is not used directly in this case, some characteristics of the message transfer are inferred from the propagation parameters used. Some parts of Sections 4 and 6 still apply.3.6. Messaging Gateway PropagationsThis note does not apply to Messaging Gateway propagations.4. Basic AQ Propagation Troubleshooting4.1. Double-check Your CodeMake sure that you are consistent in your usage of the database link(s) names, queue names, etc. It may be useful to plot a diagram of which queues are connected via which database links to make sure that the logical structure is correct.4.2. Verify that Job Queue Processes are Running4.2.1. Versions 10.2 and Lower - DBA_JOBS PackageFor versions 10.2 and lower, a scheduled propagation is managed by DBMS_JOB package. The propagation is performed by job queue process background processes. Therefore we need to verify that there are sufficient processes available for the propagation process. We should have at least 4 job queue processes running and preferably more depending on the number of other jobs running in the database. It should be noted that for AQ specific work, AQ will only ever use half of the job queue processes available.An issue caused by an inadequate job queue processes parameter setting is described in the following note:Document 298015.1 Kwqjswproc:Excep After Loop: Assigning To Self4.2.1.1. Job Queue Processes in Initalization Parameter FileThe parameter JOB_QUEUE_PROCESSES in the init.ora/spfile should be > 0. The value can be changed dynamically viaconnect / as sysdbaalter system set JOB_QUEUE_PROCESSES=10;4.2.1.2. Job Queue Processes in MemoryThe following command will show how many job queue processes are currentlyin use by this instance (this may be different than what is in the init.ora/spfile): connect / as sysdbashow parameter job;4.2.1.3. OS PIDs Corresponding to Job Queue ProcessesIdentify the operating system process ids (spids) of job queue processes involved in propagation viaselect p.SPID, p.PROGRAM from V$PROCESS p, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j where s.SID=jr.SID and s.PADDR=p.ADDR and jr.JOB=j.JOBand j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%';and these SPIDs can be used to check at the operating system level that they exist.In 8i a job queue process will have a name similar to: ora_snp1_<instance_name>.In 9i onwards you will see a coordinator process: ora_cjq0_ and multiple slave processes: ora_jnnn_<instance_name>, where nnn is an integer between 1 and 999.4.2.2. Version 11.1 and Above - Oracle SchedulerIn version 11.1 and above, Oracle Scheduler is used to perform AQ and Streams propagations. Oracle Scheduler automatically tunes the number of slave processes for these jobs based on the load on the computer system, and the JOB_QUEUE_PROCESSES initialization parameter is only used to specify the maximum number of slave processes. Therefore, the JOB_QUEUE_PROCESSES initialization parameter does not need to be set (it defaults to a very high number), unless you want to limit the number of slaves that can be created. If JOB_QUEUE_PROCESSES = 0, no propagation jobs will run.See the following note for a discussion of Oracle Streams 11g and Oracle Scheduler:Document 1083608.1 11g Streams and Oracle Scheduler4.2.2.1. Job Queue Processes in Initalization Parameter FileThe parameter JOB_QUEUE_PROCESSES in the init.ora/spfile should be > 0, and preferably be left at its default value. The value can be changed dynamically viaconnect / as sysdbaalter system set JOB_QUEUE_PROCESSES=10;To set the JOB_QUEUE_PROCESSES parameter to its default value, run:connect / as sysdbaalter system reset JOB_QUEUE_PROCESSES;and then bounce the instance.4.2.2.2. Job Queue Processes in MemoryThe following command will show how many job queue processes are currently in use by this instance (this may be different than what is in the init.ora/spfile):connect / as sysdbashow parameter job;4.2.2.3. OS PIDs Corresponding to Job Queue ProcessesIdentify the operating system process ids (SPIDs) of job queue processes involved in propagation viacol PROGRAM for a30select p.SPID, p.PROGRAM, j.JOB_namefrom v$PROCESS p, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j where s.SID=jr.SESSION_ID and s.PADDR=p.ADDRand jr.JOB_name=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%';and these SPIDs can be used to check at the operating system level that they exist.You will see a coordinator process: ora_cjq0_ and multiple slave processes: ora_jnnn_<instance_name>, where nnn is an integer between 1 and 999.4.3. Check the Alert Log and Any Associated Trace FilesThe first place to check for propagation failures is the alert logs at all sites (local and if relevant all remote sites). When a job queue process attempts to execute a schedule and fails it will always write an error stack to the alert log. This error stack will also be written in a job queue process trace file, which will be written to the BACKGROUND_DUMP_DEST location for 10.2 and below, and in the DIAGNOSTIC_DEST location for 11g. The fact that errors are written to the alert log demonstrates that the schedule is executing. This means that the problem could be with the set up of the schedule. In this example the ORA-02068 demonstrates that the failure was at the remote site. Further investigation revealed that the remote database was not open, hence the ORA-03114 error. Starting the database resolved the problem. Thu Feb 14 10:40:05 2002 Propagation Schedule for (AQADM.MULTIPLEQ, SHANE816.WORLD) encountered following error:ORA-04052: error occurred when looking up Remote object AQADM.DBMS_AQADM@SHANE816.WORLDORA-00604: error occurred at recursive SQL level 4ORA-02068: following severe error from SHANE816ORA-03114: not connected to ORACLEORA-06512: at "SYS.DBMS_AQADM_SYS", line 4770ORA-06512: at "SYS.DBMS_AQADM", line 548ORA-06512: at line 1Other potential errors that may be written to the alert log can be found in the following notes:Document 827184.1 AQ Propagation with CLOB data types Fails with ORA-22990 (11.1)Document 846297.1 AQ Propagation Fails : ORA-00600[kope2upic2954] or Ora-00600[Kghsstream_copyn] (10.2, 11.1)Document 731292.1 ORA-25215 Reported on Local Propagation When Using Transformation with ANYDATA queue tables (10.2, 11.1, 11.2)Document 365093.1 ORA-07445 [kwqppay2aqe()+7360] Reported on Propagation of a Transformed Message (10.1, 10.2)Document 219416.1 Advanced Queuing Propagation Fails with ORA-22922 (9.0)Document 1203544.1 AQ Propagation Aborted with ORA-600 [ociksin: invalid status] on SYS.DBMS_AQADM_SYS.AQ$_PROPAGATION_PROCEDURE After Upgrade (11.1, 11.2)Document 1087324.1 ORA-01405 ORA-01422 reported by Advanced Queuing Propagation schedules after RAC reconfiguration (10.2)Document 1079577.1 Advanced Queuing Propagation Fails With "ORA-22370 incorrect usage of method" (9.2, 10.2, 11.1, 11.2)Document 332792.1 ORA-04061 error relating to SYS.DBMS_PRVTAQIP reported when setting up Statspack (8.1, 9.0, 9.2, 10.1)Document 353325.1 ORA-24056: Internal inconsistency for QUEUE <queue_name> and destination <dblink> (8.1, 9.0, 9.2, 10.1, 10.2, 11.1, 11.2)Document 787367.1 ORA-22275 reported on Propagating Messages with LOB component when propagating between 10.1 and 10.2 (10.1, 10.2)Document 566622.1 ORA-22275 when propagating >4K AQ$_JMS_TEXT_MESSAGEs from 9.2.0.8 to 10.2.0.1 (9.2, 10.1)Document 731539.1 ORA-29268: HTTP client error 401 Unauthorized Error when the AQ Servlet attempts to Propagate a message via HTTP (9.0, 9.2, 10.1, 10.2, 11.1)Document 253131.1 Concurrent Writes May Corrupt LOB Segment When Using Auto Segment Space Management (ORA-1555) (9.2)Document 118884.1 How to unschedule a propagation schedule stuck in pending stateDocument 222992.1 DBMS_AQADM.DISABLE_PROPAGATION_SCHEDULE Returns ORA-24082Document 282987.1 Propagated Messages marked UNDELIVERABLE after Drop and Recreate Of Remote QueueDocument 1204080.1 AQ Propagation Failing With ORA-25329 After Upgraded From 8i or 9i to 10g or 11g.Document 1233675.1 AQ Propagation stops after upgrade to 11.2.0.1 ORA-307574.3.1. Errors Related to Incorrect Network ConfigurationThe most common propagation errors result from an incorrect network configuration. The list below contains common errors caused by tnsnames.ora file or database links being configured incorrectly:- ORA-12154: TNS:could not resolve service name- ORA-12505: TNS:listener does not currently know of SID given in connect descriptor- ORA-12514: TNS:listener could not resolve SERVICE_NAME - ORA-12541: TNS-12541 TNS:no listener4.4. Check the Database Links Exist and are Functioning CorrectlyFor schedules to remote databases confirm the database link exists via.SQL> col DBLINK for a45SQL> select QNAME, NVL(REGEXP_SUBSTR(DESTINATION, '[^@]+', 1, 2), DESTINATION) dblink2 from DBA_QUEUE_SCHEDULES3 where MESSAGE_DELIVERY_MODE = 'PERSISTENT';QNAME DBLINK------------------------------ ---------------------------------------------MY_QUEUE ORCL102B.WORLDConnect as the owner of the link and select across it to verify it works and connects to the database we expect. i.e.select * from ALL_QUEUES@ ORCL102B.WORLD;You need to ensure that the userid that scheduled the propagation (using DBMS_AQADM.SCHEDULE_PROPAGATION or DBMS_PROPAGATION_ADM.CREATE_PROPAGATION if using Streams) has access to the database link for the destination.4.5. Has Propagation Been Correctly Scheduled? Check that the propagation schedule has been created and that a job queue process has been assigned. Look for the entry in DBA_QUEUE_SCHEDULES and SYS.AQ$_SCHEDULES for your schedule. For 10g and below, check that it has a JOBNO entry in SYS.AQ$_SCHEDULES, and that there is an entry in DBA_JOBS with that JOBNO. For 11g and above, check that the schedule has a JOB_NAME entry in SYS.AQ$_SCHEDULES, and that there is an entry in DBA_SCHEDULER_JOBS with that JOB_NAME. Check the destination is as intended and spelled correctly.SQL> select SCHEMA, QNAME, DESTINATION, SCHEDULE_DISABLED, PROCESS_NAME from DBA_QUEUE_SCHEDULES;SCHEMA QNAME DESTINATION S PROCESS------- ---------- ------------------ - -----------AQADM MULTIPLEQ AQ$_LOCAL N J000AQ$_LOCAL in the destination column shows that the queue to which we are propagating to is in the same database as the source queue. If the propagation was to a remote (different) database, a database link will be in the DESTINATION column. The entry in the SCHEDULE_DISABLED column, N, means that the schedule is NOT disabled. If Y (yes) appears in this column, propagation is disabled and the schedule will not be executed. If not using Oracle Streams, propagation should resume once you have enabled the schedule by invoking DBMS_AQADM.ENABLE_PROPAGATION_SCHEDULE (for 10.2 Oracle Streams and above, the DBMS_PROPAGATION_ADM.START_PROPAGATION procedure should be used). The PROCESS_NAME is the name of the job queue process currently allocated to execute the schedule. This process is allocated dynamically at execution time. If the PROCESS_NAME column is null (empty) the schedule is not currently executing. You may need to execute this statement a number of times to verify if a process is being allocated. If a process is at some time allocated to the schedule, it is attempting to execute.SQL> select SCHEMA, QNAME, LAST_RUN_DATE, NEXT_RUN_DATE from DBA_QUEUE_SCHEDULES;SCHEMA QNAME LAST_RUN_DATE NEXT_RUN_DATE------ ----- ----------------------- ----------------------- AQADM MULTIPLEQ 13-FEB-2002 13:18:57 13-FEB-2002 13:20:30In 11g, these dates are expressed in TIMESTAMP WITH TIME ZONE datatypes. If the NEXT_RUN_DATE and NEXT_RUN_TIME columns are null when this statement is executed, the scheduled propagation is currently in progress. If they never change it would suggest that the schedule itself is never executing. If the next scheduled execution is too far away, change the NEXT_TIME parameter of the schedule so that schedules are executed more frequently (assuming that the window is not set to be infinite). Parameters of a schedule can be changed using the DBMS_AQADM.ALTER_PROPAGATION_SCHEDULE call. In 10g and below, scheduling propagation posts a job in the DBA_JOBS view. The columns are more or less the same as DBA_QUEUE_SCHEDULES so you just need to recognize the job and verify that it exists. SQL> select JOB, WHAT from DBA_JOBS where WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%';JOB WHAT---- ----------------- 720 next_date := sys.dbms_aqadm.aq$_propaq(job);For 11g, scheduling propagation posts a job in DBA_SCHEDULER_JOBS instead:SQL> select JOB_NAME from DBA_SCHEDULER_JOBS where JOB_NAME like 'AQ_JOB$_%';JOB_NAME------------------------------AQ_JOB$_41If no job exists, check DBA_QUEUE_SCHEDULES to make sure that the schedule has not been disabled. For 10g and below, the job number is dynamic for AQ propagation schedules. The procedure that is executed to expedite a propagation schedule runs, removes itself from DBA_JOBS, and then reposts a new job for the next scheduled propagation. The job number should therefore always increment unless the schedule has been set up to run indefinitely. 4.6. Is the Schedule Executing but Failing to Complete?Run the following query:SQL> select FAILURES, LAST_ERROR_MSG from DBA_QUEUE_SCHEDULES;FAILURES LAST_ERROR_MSG------------ -----------------------1 ORA-25207: enqueue failed, queue AQADM.INQ is disabled from enqueueingORA-02063: preceding line from SHANE816The failures column shows how many times we have attempted to execute the schedule and failed. Oracle will attempt to execute the schedule 16 times after which it will be removed from the DBA_JOBS or DBA_SCHEDULER_JOBS view and the schedule will become disabled. The column DBA_QUEUE_SCHEDULES.SCHEDULE_DISABLED will show 'Y'. For 11g and above, the DBA_SCHEDULER_JOBS.STATE column will show 'BROKEN' for the job corresponding to DBA_QUEUE_SCHEDULES.JOB_NAME. Prior to 10g the back off algorithm for failures was exponential, whereas from 10g onwards it is linear. The propagation will become disabled on the 17th attempt. Only the last execution failure will be reflected in the LAST_ERROR_MSG column. That is, if the schedule fails 5 times for 5 different reasons, only the last set of errors will be recorded in DBA_QUEUE_SCHEDULES. Any errors need to be resolved to allow propagation to continue. If propagation has also become disabled due to 17 failures, first resolve the reason for the error and then re-enable the schedule using the DBMS_AQADM.ENABLE_PROPAGATION_SCHEDULE procedure, or DBMS_PROPAGATION_ADM.START_PROPAGATION if using 10.2 or above Oracle Streams. As soon as the schedule executes successfully the error message entries will be deleted. Oracle does not keep a history of past failures. However, when using Oracle Streams, the errors will be retained in the DBA_PROPAGATION view even after the schedule resumes successfully. See the following note for instructions on how to clear out the errors from the DBA_PROPAGATION view:Document 808136.1 How to clear the old errors from DBA_PROPAGATION view?If a schedule is active and no errors are being reported then the source queue may not have any messages to be propagated.4.7. Do the Propagation Notification Queue Table and Queue Exist?Check to see that the propagation notification queue table and queue exist and are enabled for enqueue and dequeue. Propagation makes use of the propagation notification queue for handling propagation run-time events, and the messages in this queue are stored in a SYS-owned queue table. This queue should never be stopped or dropped and the corresponding queue table never be dropped. 10g and belowThe propagation notification queue table is of the format SYS.AQ$_PROP_TABLE_n, where 'n' is the RAC instance number, i.e. '1' for a non-RAC environment. This queue and queue table are created implicitly when propagation is first scheduled. If propagation has been scheduled and these objects do not exist, try unscheduling and rescheduling propagation. If they still do not exist contact Oracle Support.SQL> select QUEUE_TABLE from DBA_QUEUE_TABLES2 where QUEUE_TABLE like '%PROP_TABLE%' and OWNER = 'SYS';QUEUE_TABLE------------------------------AQ$_PROP_TABLE_1SQL> select NAME, ENQUEUE_ENABLED, DEQUEUE_ENABLED2 from DBA_QUEUES where owner='SYS'3 and QUEUE_TABLE like '%PROP_TABLE%';NAME ENQUEUE DEQUEUE------------------------------ ------- -------AQ$_PROP_NOTIFY_1 YES YESAQ$_AQ$_PROP_TABLE_1_E NO NOIf the AQ$_PROP_NOTIFY_1 queue is not enabled for enqueue or dequeue, it should be so enabled using DBMS_AQADM.START_QUEUE. However, the exception queue AQ$_AQ$_PROP_TABLE_1_E should not be enabled for enqueue or dequeue.11g and aboveThe propagation notification queue table is of the format SYS.AQ_PROP_TABLE, and is created when the database is created. If they do not exist, contact Oracle Support.SQL> select QUEUE_TABLE from DBA_QUEUE_TABLES2 where QUEUE_TABLE like '%PROP_TABLE%' and OWNER = 'SYS';QUEUE_TABLE------------------------------AQ_PROP_TABLESQL> select NAME, ENQUEUE_ENABLED, DEQUEUE_ENABLED2 from DBA_QUEUES where owner='SYS'3 and QUEUE_TABLE like '%PROP_TABLE%';NAME ENQUEUE DEQUEUE------------------------------ ------- -------AQ_PROP_NOTIFY YES YESAQ$_AQ_PROP_TABLE_E NO NOIf the AQ_PROP_NOTIFY queue is not enabled for enqueue or dequeue, it should be so enabled using DBMS_AQADM.START_QUEUE. However, the exception queue AQ$_AQ$_PROP_TABLE_E should not be enabled for enqueue or dequeue.4.8. Does the Remote Queue Exist and is it Enabled for Enqueueing?Check that the remote queue the propagation is transferring messages to exists and is enabled for enqueue:SQL> select DESTINATION from USER_QUEUE_SCHEDULES where QNAME = 'OUTQ';DESTINATION-----------------------------------------------------------------------------"AQADM"."INQ"@M2V102.ESSQL> select OWNER, NAME, ENQUEUE_ENABLED, DEQUEUE_ENABLED from USER_QUEUES@M2V102.ES;OWNER NAME ENQUEUE DEQUEUE-------- ------ ----------- -----------AQADM INQ YES YES4.9. Do the Target and Source Database Charactersets Differ?If a message fails to propagate, check the database charactersets of the source and target databases. Investigate whether the same message can propagate between the databases with the same characterset or it is only a particular combination of charactersets which causes a problem.4.10. Check the Queue Table Type AgreementPropagation is not possible between queue tables which have types that differ in some respect. One way to determine if this is the case is to run the DBMS_AQADM.VERIFY_QUEUE_TYPES procedure for the two queues that the propagation operates on. If the types do not agree, DBMS_AQADM.VERIFY_QUEUE_TYPES will return '0'.For AQ propagation between databases which have different NLS_LENGTH_SEMANTICS settings, propagation will not work, unless the queues are Oracle Streams ANYDATA queues.See the following notes for issues caused by lack of type agreement:Document 1079577.1 Advanced Queuing Propagation Fails With "ORA-22370: incorrect usage of method"Document 282987.1 Propagated Messages marked UNDELIVERABLE after Drop and Recreate Of Remote QueueDocument 353754.1 Streams Messaging Propagation Fails between Single and Multi-byte Charactersets when using Chararacter Length Semantics in the ADT4.11. Enable Propagation Tracing4.11.1. System LevelThis is set it in the init.ora/spfile as follows: event="24040 trace name context forever, level 10"and restart the instanceThis event cannot be set dynamically with an alter system command until version 10.2:SQL> alter system set events '24040 trace name context forever, level 10';To unset the event:SQL> alter system set events '24040 trace name context off';Debugging information will be logged to job queue trace file(s) (jnnn) as propagation takes place. You can check the trace file for errors, and for statements indicating that messages have been sent. For the most part the trace information is understandable. This trace should also be uploaded to Oracle Support if a service request is created. 4.11.2. Attaching to a Specific ProcessWe can also attach to an existing job queue processes that is running a propagation schedule and trace it individually using the oradebug utility, as follows:10.2 and belowconnect / as sysdbaselect p.SPID, p.PROGRAM from v$PROCESS p, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j where s.SID=jr.SID and s.PADDR=p.ADDR and jr.JOB=j.JOB and j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%';-- For the process id (SPID) attach to it via oradebug and generate the following traceoradebug setospid <SPID>oradebug unlimitoradebug Event 10046 trace name context forever, level 12oradebug Event 24040 trace name context forever, level 10-- Trace the process for 5 minutesoradebug Event 10046 trace name context offoradebug Event 24040 trace name context off-- The following command returns the pathname/filename to the file being written tooradebug tracefile_name11gconnect / as sysdbacol PROGRAM for a30select p.SPID, p.PROGRAM, j.JOB_NAMEfrom v$PROCESS p, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j where s.SID=jr.SESSION_ID and s.PADDR=p.ADDR and jr.JOB_NAME=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%';-- For the process id (SPID) attach to it via oradebug and generate the following traceoradebug setospid <SPID>oradebug unlimitoradebug Event 10046 trace name context forever, level 12oradebug Event 24040 trace name context forever, level 10-- Trace the process for 5 minutesoradebug Event 10046 trace name context offoradebug Event 24040 trace name context off-- The following command returns the pathname/filename to the file being written tooradebug tracefile_name4.11.3. Further TracingThe previous tracing steps only trace the job queue process executing the propagation on the source. At times it is useful to trace the propagation receiver process (the session which is enqueueing the messages into the target queue) on the target database which is associated with the job queue process on the source database.These following queries provide ways of identifying the processes involved in propagation so that you can attach to them via oradebug to generate trace information.In order to identify the propagation receiver process you need to execute the query as a user with privileges to access the v$ views in both the local and remote databases so the database link must connect as a user with those privileges in the remote database. The <DBLINK> in the queries should be replaced by the appropriate database link.The queries have two forms due to the differences between operating systems. The value returned by 'Rem Process' is the operating system identifier of the propagation receiver on the remote database. Once identified, this process can be attached to and traced on the remote database using the commands given in Section 4.11.2.10.2 and below - Windowsselect pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from v$PROCESS pl, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SID and s.PADDR=pl.ADDR and jr.JOB=j.JOB and j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%' and pl.SPID=substr(sr.PROCESS, instr(sr.PROCESS,':')+1);10.2 and below - Unixselect pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from V$PROCESS pl, DBA_JOBS_RUNNING jr, V$SESSION s, DBA_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SID and s.PADDR=pl.ADDR and jr.JOB=j.JOB and j.WHAT like '%sys.dbms_aqadm.aq$_propaq(job)%' and pl.SPID=sr.PROCESS;11g - Windowsselect pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from V$PROCESS pl, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SESSION_ID and s.PADDR=pl.ADDR and jr.JOB_NAME=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%%' and pl.SPID=substr(sr.PROCESS, instr(sr.PROCESS,':')+1);11g - Unixselect pl.SPID "JobQ Process", pl.PROGRAM, sr.PROCESS "Rem Process" from V$PROCESS pl, DBA_SCHEDULER_RUNNING_JOBS jr, V$SESSION s, DBA_SCHEDULER_JOBS j, V$SESSION@<DBLINK> sr where s.SID=jr.SESSION_ID and s.PADDR=pl.ADDR and jr.JOB_NAME=j.JOB_NAME and j.JOB_NAME like '%AQ_JOB$_%%' and pl.SPID=sr.PROCESS;5. Additional Troubleshooting Steps for AQ Propagation of User-Enqueued and Dequeued Messages5.1. Check the Privileges of All Users InvolvedEnsure that the owner of the database link has the necessary privileges on the aq packages.SQL> select TABLE_NAME, PRIVILEGE from USER_TAB_PRIVS;TABLE_NAME PRIVILEGE------------------------------ ----------------------------------------DBMS_LOCK EXECUTEDBMS_AQ EXECUTEDBMS_AQADM EXECUTEDBMS_AQ_BQVIEW EXECUTEQT52814_BUFFER SELECTNote that when queue table is created, a view called QT<nnn>_BUFFER is created in the SYS schema, and the queue table owner is given SELECT privileges on it. The <nnn> corresponds to the object_id of the associated queue table.SQL> select * from USER_ROLE_PRIVS;USERNAME GRANTED_ROLE ADM DEF OS_------------------------------ ------------------------------ ---- ---- ---AQ_USER1 AQ_ADMINISTRATOR_ROLE NO YES NOAQ_USER1 CONNECT NO YES NOAQ_USER1 RESOURCE NO YES NOIt is good practice to configure central AQ administrative user. All admin and processing jobs are created, executed and administered as this user. This configuration is not mandatory however, and the database link can be owned by any existing queue user. If this latter configuration is used, ensure that the connecting user has the necessary privileges on the AQ packages and objects involved. Privileges for an AQ Administrative userExecute on DBMS_AQADMExecute on DBMS_AQGranted the AQ_ADMINISTRATOR_ROLEPrivileges for an AQ userExecute on DBMS_AQExecute on the message payloadEnqueue privileges on the remote queueDequeue privileges on the originating queuePrivileges need to be confirmed on both sites when propagation is scheduled to remote destinations. Verify that the user ID used to login to the destination through the database link has been granted privileges to use AQ. 5.2. Verify Queue Payload TypesAQ will not propagate messages from one queue to another if the payload types of the two queues are not verified to be equivalent. An AQ administrator can verify if the source and destination's payload types match by executing the DBMS_AQADM.VERIFY_QUEUE_TYPES procedure. The results of the type checking will be stored in the SYS.AQ$_MESSAGE_TYPES table. This table can be accessed using the object identifier OID of the source queue and the address database link of the destination queue, i.e. [schema.]queue_name[@destination]. Prior to Oracle 9i the payload (message type) had to be the same for all the queue tables involved in propagation. From Oracle9i onwards a transformation can be used so that payloads can be converted from one type to another. The following procedural call made on the source database can verify whether we can propagate between the source and the destination queue tables.connect aq_user1/aq_user1@M1V102.ESset serverout onDECLARErc_value number;BEGINDBMS_AQADM.VERIFY_QUEUE_TYPES(src_queue_name => 'AQ_USER1.Q_1', dest_queue_name => 'AQ_USER2.Q_2',destination => 'dbl_aq_user2.es',rc => rc_value);dbms_output.put_line('rc_value code is '||rc_value);END;/If propagation is possible then the return code value will be 1. If it is 0 then propagation is not possible and further investigation of the types and transformations used by and in conjunction with the queue tables is required. With regard to comparison of the types the following sql can be used to extract the DDL for a specific type with' %' changed appropriately on the source and target. This can then be compared for the source and target.SET LONG 20000 set pagesize 50 EXECUTE DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM, 'STORAGE',false); SELECT DBMS_METADATA.GET_DDL('TYPE',t.type_name) from user_types t WHERE t.type_name like '%'; EXECUTE DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM, 'DEFAULT'); 5.3. Check Message State and DestinationThe first step in this process is to identify the queue table associated with the problem source queue. Although you schedule propagation for a specific queue, most of the meta-data associated with that queue is stored in the underlying queue table. The following statement finds the queue table for a given queue (note that this is a multiple-consumer queue table).SQL> select QUEUE_TABLE from DBA_QUEUES where NAME = 'MULTIPLEQ';QUEUE_TABLE --------------------MULTIPLEQTABLE For a small amount of messages in a multiple-consumer queue table, the following query can be run:SQL> select MSG_STATE, CONSUMER_NAME, ADDRESS from AQ$MULTIPLEQTABLE where QUEUE = 'MULTIPLEQ';MSG_STATE CONSUMER_NAME ADDRESS-------------- ----------------------- -------------READY AQUSER2 AQADM.INQ@M2V102.ESREADY AQUSER1READY AQUSER3 AQADM.INQIn this example we see 2 messages ready to be propagated to remote queues and 1 that is not. If the address column is blank, the message is not scheduled for propagation and can only be dequeued from the queue upon which it was enqueued. The MSG_STATE column values are discussed in Document 102330.1 Advanced Queueing MSG_STATE Values and their Interpretation. If the address column has a value, the message has been enqueued for propagation to another queue. The first row in the example includes a database link (@M2V102.ES). This demonstrates that the message should be propagated to a queue at a remote database. The third row does not include a database link so will be propagated to a queue that resides on the same database as the source queue. The consumer name is the intended recipient at the target queue. Note that we are not querying the base queue table directly; rather, we are querying a view that is available on top of every queue table, AQ$<queue_table_name>.A more realistic query in an environment where the queue table contains thousands of messages is8.0.3-compatible multiple-consumer queue table and all compatibility single-consumer queue tablesselect count(*), MSG_STATE, QUEUE from AQ$<queue_table_name>  group by MSG_STATE, QUEUE;8.1.3 and 10.0-compatible queue tablesselect count(*), MSG_STATE, QUEUE, CONSUMER_NAME from AQ$<queue_table_name>group by MSG_STATE, QUEUE, CONSUMER_NAME;For multiple-consumer queue tables, if you did not see the expected CONSUMER_NAME , check the syntax of the enqueue code and verify the recipients are declared correctly. If a recipients list is not used on enqueue, check the subscriber list in the AQ$_<queue_table_name>_S view (note that a single-consumer queue table does not have a subscriber view. This view records all members of the default subscription list which were added using the DBMS_AQADM.ADD_SUBSCRIBER procedure and also those enqueued using a recipient list. SQL> select QUEUE, NAME, ADDRESS from AQ$MULTIPLEQTABLE_S;QUEUE NAME ADDRESS---------- ----------- -------------MULTIPLEQ AQUSER2 AQADM.INQ@M2V102.ESMULTIPLEQ AQUSER1In this example we have 2 subscribers registered with the queue. We have a local subscriber AQUSER1, and a remote subscriber AQUSER2, on the queue INQ, owned by AQADM, at M2V102.ES. Unless overridden with a recipient list during enqueue every message enqueued to this queue will be propagated to INQ at M2V102.ES.For 8.1 style and above multiple consumer queue tables, you can also check the following information at the target:select CONSUMER_NAME, DEQ_TXN_ID, DEQ_TIME, DEQ_USER_ID, PROPAGATED_MSGID from AQ$<queue_table_name> where QUEUE = '<QUEUE_NAME>';For 8.0 style queues, if the queue table supports multiple consumers you can obtain the same information from the history column of the queue table:select h.CONSUMER, h.TRANSACTION_ID, h.DEQ_TIME, h.DEQ_USER, h.PROPAGATED_MSGIDfrom AQ$<queue_table_name> t, table(t.history) h where t.Q_NAME = '<QUEUE_NAME>';A non-NULL TRANSACTION_ID indicates that the message was successfully propagated. Further, the DEQ_TIME indicates the time of propagation, the DEQ_USER indicates the userid used for propagation, and the PROPAGATED_MSGID indicates the message ID of the message that was enqueued at the destination.6. Additional Troubleshooting Steps for Propagation in an Oracle Streams Environment6.1. Is the Propagation Enabled?For a propagation job to propagate messages, the propagation must be enabled. For Streams, a special view called DBA_PROPAGATION exists to convey information about Streams propagations. If messages are not being propagated by a propagation as expected, then the propagation might not be enabled. To query for this:SELECT p.PROPAGATION_NAME, DECODE(s.SCHEDULE_DISABLED, 'Y', 'Disabled','N', 'Enabled') SCHEDULE_DISABLED, s.PROCESS_NAME, s.FAILURES, s.LAST_ERROR_MSGFROM DBA_QUEUE_SCHEDULES s, DBA_PROPAGATION pWHERE p.DESTINATION_DBLINK = NVL(REGEXP_SUBSTR(s.DESTINATION, '[^@]+', 1, 2), s.DESTINATION) AND s.SCHEMA = p.SOURCE_QUEUE_OWNER AND s.QNAME = p.SOURCE_QUEUE_NAME AND MESSAGE_DELIVERY_MODE = 'PERSISTENT' order by PROPAGATION_NAME;At times, the propagation job may become "broken" or fail to start after an error has been encountered or after a database restart. If an error is indicated by the above query, an attempt to disable the propagation and then re-enable it can be made. In the examples below, for the propagation named STRMADMIN_PROPAGATE where the queue name is STREAMS_QUEUE owned by STRMADMIN and the destination database link is ORCL2.WORLD, the commands would be:10.2 and aboveexec dbms_propagation_adm.stop_propagation('STRMADMIN_PROPAGATE'); exec dbms_propagation_adm.start_propagation('STRMADMIN_PROPAGATE'); If the above does not fix the problem, stop the propagation specifying the force parameter (2nd parameter on stop_propagation) as TRUE:exec dbms_propagation_adm.stop_propagation('STRMADMIN_PROPAGATE',true); exec dbms_propagation_adm.start_propagation('STRMADMIN_PROPAGATE');The statistics for the propagation as well as any old error messages are cleared when the force parameter is set to TRUE. Therefore if the propagation schedule is stopped with FORCE set to TRUE, and upon restart there is still an error message in DBA_PROPAGATION, then the error message is current.9.2 or 10.1exec dbms_aqadm.disable_propagation_schedule('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); exec dbms.aqadm.enable_propagation_schedule('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); If the above does not fix the problem, perform an unschedule of propagation and then schedule_propagation:exec dbms_aqadm.unschedule_propagation('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD'); exec dbms_aqadm.schedule_propagation('STRMADMIN.STREAMS_QUEUE','ORCL2.WORLD');Typically if the error from the first query in Section 6.1 recurs after restarting the propagation as shown above, further troubleshooting of the error is needed.6.2. Check Propagation Rule Sets and TransformationsInspect the configuration of the rules in the rule set that is associated with the propagation process to make sure that they evaluate to TRUE as expected. If not, then the object or schema will not be propagated. Remember that when a negative rule evaluates to TRUE, the specified object or schema will not be propagated. Finally inspect any rule-based transformations that are implemented with propagation to make sure they are changing the data in the intended way.The following query shows what rule sets are assigned to a propagation:select PROPAGATION_NAME, RULE_SET_OWNER||'.'||RULE_SET_NAME "Positive Rule Set",NEGATIVE_RULE_SET_OWNER||'.'||NEGATIVE_RULE_SET_NAME "Negative Rule Set"from DBA_PROPAGATION;The next two queries list the propagation rules and their conditions. The first is for the positive rule set, the second is for the negative rule set:set long 4000select rsr.RULE_SET_OWNER||'.'||rsr.RULE_SET_NAME RULE_SET ,rsr.RULE_OWNER||'.'||rsr.RULE_NAME RULE_NAME,r.RULE_CONDITION CONDITION fromDBA_RULE_SET_RULES rsr, DBA_RULES rwhere rsr.RULE_NAME = r.RULE_NAME and rsr.RULE_OWNER = r.RULE_OWNER and RULE_SET_NAME in(select RULE_SET_NAME from DBA_PROPAGATION) order by rsr.RULE_SET_OWNER, rsr.RULE_SET_NAME; set long 4000select c.PROPAGATION_NAME, rsr.RULE_SET_OWNER||'.'||rsr.RULE_SET_NAME RULE_SET ,rsr.RULE_OWNER||'.'||rsr.RULE_NAME RULE_NAME,r.RULE_CONDITION CONDITION fromDBA_RULE_SET_RULES rsr, DBA_RULES r ,DBA_PROPAGATION cwhere rsr.RULE_NAME = r.RULE_NAME and rsr.RULE_OWNER = r.RULE_OWNER andrsr.RULE_SET_OWNER=c.NEGATIVE_RULE_SET_OWNER and rsr.RULE_SET_NAME=c.NEGATIVE_RULE_SET_NAMEand rsr.RULE_SET_NAME in(select NEGATIVE_RULE_SET_NAME from DBA_PROPAGATION) order by rsr.RULE_SET_OWNER, rsr.RULE_SET_NAME;6.3. Determining the Total Number of Messages and Bytes PropagatedAs in Section 3.1, determining if messages are flowing can be instructive to see whether the propagation is entirely hung or just slow. If the propagation is not in flow control (see Section 6.5.2), but the statistics are incrementing slowly, there may be a performance issue. For Streams implementations two views are available that can assist with this that can show the number of messages sent by a propagation, as well as the number of acknowledgements being returned from the target site: the V$PROPAGATION_SENDER view at the Source site and the V$PROPAGATION_RECEIVER view at the destination site. It is helpful to query both to determine if messages are being delivered to the target. Look for the statistics to increase.Source:select QUEUE_SCHEMA, QUEUE_NAME, DBLINK,HIGH_WATER_MARK, ACKNOWLEDGEMENT, TOTAL_MSGS, TOTAL_BYTESfrom V$PROPAGATION_SENDER;Target:select SRC_QUEUE_SCHEMA, SRC_QUEUE_NAME, SRC_DBNAME, DST_QUEUE_SCHEMA, DST_QUEUE_NAME, HIGH_WATER_MARK, ACKNOWLEDGEMENT, TOTAL_MSGS from V$PROPAGATION_RECEIVER;6.4. Check Buffered SubscribersThe V$BUFFERED_SUBSCRIBERS view displays information about subscribers for all buffered queues in the instance. This view can be queried to make sure that the site that the propagation is propagating to is listed as a subscriber address for the site being propagated from:select QUEUE_SCHEMA, QUEUE_NAME, SUBSCRIBER_ADDRESS from V$BUFFERED_SUBSCRIBERS;The SUBSCRIBER_ADDRESS column will not be populated when the propagation is local (between queues on the same database).6.5. Common Streams Propagation Errors6.5.1. ORA-02082: A loopback database link must have a connection qualifier.This error can occur if you use the Streams Setup Wizard in Oracle Enterprise Manager without first configuring the GLOBAL_NAME for your database.6.5.2. ORA-25307: Enqueue rate too high. Enable flow controlDBA_QUEUE_SCHEDULES will display this informational message for propagation when the automatic flow control (10g feature of Streams) has been invoked.Similar to Streams capture processes, a Streams propagation process can also go into a state of 'flow control. This is an informative message that indicates flow control has been automatically enabled to reduce the rate at which messages are being enqueued into at target queue.This typically occurs when the target site is unable to keep up with the rate of messages flowing from the source site. Other than checking that the apply process is running normally on the target site, usually no action is required by the DBA. Propagation and the capture process will be resumed automatically when the target site is able to accept more messages.The following document contains more information:Document 302109.1 Streams Propagation Error: ORA-25307 Enqueue rate too high. Enable flow controlSee the following document for one potential cause of this situation:Document 1097115.1 Oracle Streams Apply Reader is in 'Paused' State6.5.3. ORA-25315 unsupported configuration for propagation of buffered messagesThis error typically occurs when the target database is RAC and usually indicates that an attempt was made to propagate buffered messages with the database link pointing to an instance in the destination database which is not the owner instance of the destination queue. To resolve the problem, use queue-to-queue propagation for buffered messages.6.5.4. ORA-600 [KWQBMCRCPTS101] after dropping / recreating propagationFor cause/fixes refer to:Document 421237.1 ORA-600 [KWQBMCRCPTS101] reported by a Qmon slave process after dropping a Streams Propagation6.5.5. Stopping or Dropping a Streams Propagation HangsSee the following note:Document 1159787.1 Troubleshooting Streams Propagation When It is Not Functioning and Attempts to Stop It Hang6.6. Streams Propagation-Related Notes for Common IssuesDocument 437838.1 Streams Specific PatchesDocument 749181.1 How to Recover Streams After Dropping PropagationDocument 368912.1 Queue to Queue Propagation Schedule encountered ORA-12514 in a RAC environmentDocument 564649.1 ORA-02068/ORA-03114/ORA-03113 Errors From Streams Propagation Process - Remote Database is Available and Unschedule/Reschedule Does Not ResolveDocument 553017.1 Stream Propagation Process Errors Ora-4052 Ora-6554 From 11g To 10201Document 944846.1 Streams Propagation Fails Ora-7445 [kohrsmc]Document 745601.1 ORA-23603 'STREAMS enqueue aborted due to low SGA' Error from Streams Propagation, and V$STREAMS_CAPTURE.STATE Hanging on 'Enqueuing Message'Document 333068.1 ORA-23603: Streams Enqueue Aborted Eue To Low SGADocument 363496.1 Ora-25315 Propagating on RAC StreamsDocument 368237.1 Unable to Unschedule Propagation. Streams Queue is InvalidDocument 436332.1 dbms_propagation_adm.stop_propagation hangsDocument 727389.1 Propagation Fails With ORA-12528Document 730911.1 ORA-4063 Is Reported After Dropping Negative Prop.RulesetDocument 460471.1 Propagation Blocked by Qmon Process - Streams_queue_table / 'library cache lock' waitsDocument 1165583.1 ORA-600 [kwqpuspse0-ack] In Streams EnvironmentDocument 1059029.1 Combined Capture and Apply (CCA) : Capture aborts : ORA-1422 after schedule_propagationDocument 556309.1 Changing Propagation/ queue_to_queue : false -> true does does not work; no LCRs propagatedDocument 839568.1 Propagation failing with error: ORA-01536: space quota exceeded for tablespace ''Document 311021.1 Streams Propagation Process : Ora 12154 After Reboot with Transparent Application Failover TAF configuredDocument 359971.1 STREAMS propagation to Primary of physical Standby configuation errors with Ora-01033, Ora-02068Document 1101616.1 DBMS_PROPAGATION_ADM.DROP_PROPAGATION FAILS WITH ORA-17477. Performance IssuesA propagation may seem to be slow if the queries from Sections 3.1 and 6.3 show that the message statistics are not changing quickly. In Oracle Streams, this more usually is due to a slow apply process at the target rather than a slow propagation. Propagation could be inferred to be slow if the message statistics are changing, and the state of a capture process according to V$STREAMS_CAPTURE.STATE is PAUSED FOR FLOW CONTROL, but an ORA-25307 'Enqueue rate too high. Enable flow control' warning is NOT observed in DBA_QUEUE_SCHEDULES per Section 6.5.2. If this is the case, see the following notes / white papers for suggestions to increase performance:Document 335516.1 Master Note for Streams Performance RecommendationsDocument 730036.1 Overview for Troubleshooting Streams Performance IssuesDocument 780733.1 Streams Propagation Tuning with Network ParametersWhite Paper: http://www.oracle.com/technetwork/database/features/availability/maa-wp-10gr2-streams-performance-130059.pdfWhite Paper: Oracle Streams Configuration Best Practices: Oracle Database 10g Release 10.2, http://www.oracle.com/technetwork/database/features/availability/maa-10gr2-streams-configuration-132039.pdf, See APPENDIX A: USING STREAMS CONFIGURATIONS OVER A NETWORKFor basic AQ propagation, the network tuning in the aforementioned Appendix A of the white paper 'Oracle Streams Configuration Best Practices: Oracle Database 10g Release 10.2' is applicable.ReferencesNOTE:102330.1 - Advanced Queueing MSG_STATE Values and their InterpretationNOTE:102771.1 - Advanced Queueing Propagation using PL/SQLNOTE:1059029.1 - Combined Capture and Apply (CCA) : Capture aborts : ORA-1422 after schedule_propagationNOTE:1079577.1 - Advanced Queuing Propagation Fails With "ORA-22370: incorrect usage of method"NOTE:1083608.1 - 11g Streams and Oracle SchedulerNOTE:1087324.1 - ORA-01405 ORA-01422 reported by Adavanced Queueing Propagation schedules after RAC reconfigurationNOTE:1097115.1 - Oracle Streams Apply Reader is in 'Paused' StateNOTE:1101616.1 - DBMS_PROPAGATION_ADM.DROP_PROPAGATION FAILS WITH ORA-1747NOTE:1159787.1 - Troubleshooting Streams Propagation When It is Not Functioning and Attempts to Stop It HangNOTE:1165583.1 - ORA-600 [kwqpuspse0-ack] In Streams EnvironmentNOTE:118884.1 - How to unschedule a propagation schedule stuck in pending stateNOTE:1203544.1 - AQ PROPAGATION ABORTED WITH ORA-600[OCIKSIN: INVALID STATUS] ON SYS.DBMS_AQADM_SYS.AQ$_PROPAGATION_PROCEDURE AFTER UPGRADENOTE:1204080.1 - AQ Propagation Failing With ORA-25329 After Upgraded From 8i or 9i to 10g or 11g.NOTE:219416.1 - Advanced Queuing Propagation fails with ORA-22922NOTE:222992.1 - DBMS_AQADM.DISABLE_PROPAGATION_SCHEDULE Returns ORA-24082NOTE:253131.1 - Concurrent Writes May Corrupt LOB Segment When Using Auto Segment Space Management (ORA-1555)NOTE:282987.1 - Propagated Messages marked UNDELIVERABLE after Drop and Recreate Of Remote QueueNOTE:298015.1 - Kwqjswproc:Excep After Loop: Assigning To SelfNOTE:302109.1 - Streams Propagation Error: ORA-25307 Enqueue rate too high. Enable flow controlNOTE:311021.1 - Streams Propagation Process : Ora 12154 After Reboot with Transparent Application Failover TAF configuredNOTE:332792.1 - ORA-04061 error relating to SYS.DBMS_PRVTAQIP reported when setting up StatspackNOTE:333068.1 - ORA-23603: Streams Enqueue Aborted Eue To Low SGANOTE:335516.1 - Master Note for Streams Performance RecommendationsNOTE:353325.1 - ORA-24056: Internal inconsistency for QUEUE and destination NOTE:353754.1 - Streams Messaging Propagation Fails between Single and Multi-byte Charactersets when using Chararacter Length Semantics in the ADT.NOTE:359971.1 - STREAMS propagation to Primary of physical Standby configuation errors with Ora-01033, Ora-02068NOTE:363496.1 - Ora-25315 Propagating on RAC StreamsNOTE:365093.1 - ORA-07445 [kwqppay2aqe()+7360] reported on Propagation of a Transformed MessageNOTE:368237.1 - Unable to Unschedule Propagation. Streams Queue is InvalidNOTE:368912.1 - Queue to Queue Propagation Schedule encountered ORA-12514 in a RAC environmentNOTE:421237.1 - ORA-600 [KWQBMCRCPTS101] reported by a Qmon slave process after dropping a Streams PropagationNOTE:436332.1 - dbms_propagation_adm.stop_propagation hangsNOTE:437838.1 - Streams Specific PatchesNOTE:460471.1 - Propagation Blocked by Qmon Process - Streams_queue_table / 'library cache lock' waitsNOTE:463820.1 - Streams Combined Capture and Apply in 11gNOTE:553017.1 - Stream Propagation Process Errors Ora-4052 Ora-6554 From 11g To 10201NOTE:556309.1 - Changing Propagation/ queue_to_queue : false -> true does does not work; no LCRs propagatedNOTE:564649.1 - ORA-02068/ORA-03114/ORA-03113 Errors From Streams Propagation Process - Remote Database is Available and Unschedule/Reschedule Does Not ResolveNOTE:566622.1 - ORA-22275 when propagating >4K AQ$_JMS_TEXT_MESSAGEs from 9.2.0.8 to 10.2.0.1NOTE:727389.1 - Propagation Fails With ORA-12528NOTE:730036.1 - Overview for Troubleshooting Streams Performance IssuesNOTE:730911.1 - ORA-4063 Is Reported After Dropping Negative Prop.RulesetNOTE:731292.1 - ORA-25215 Reported On Local Propagation When Using Transformation with ANYDATA queue tablesNOTE:731539.1 - ORA-29268: HTTP client error 401 Unauthorized Error when the AQ Servlet attempts to Propagate a message via HTTPNOTE:745601.1 - ORA-23603 'STREAMS enqueue aborted due to low SGA' Error from Streams Propagation, and V$STREAMS_CAPTURE.STATE Hanging on 'Enqueuing Message'NOTE:749181.1 - How to Recover Streams After Dropping PropagationNOTE:780733.1 - Streams Propagation Tuning with Network ParametersNOTE:787367.1 - ORA-22275 reported on Propagating Messages with LOB component when propagating between 10.1 and 10.2NOTE:808136.1 - How to clear the old errors from DBA_PROPAGATION view ?NOTE:827184.1 - AQ Propagation with CLOB data types Fails with ORA-22990NOTE:827473.1 - How to alter propagation from queue_to_queue to queue_to_dblinkNOTE:839568.1 - Propagation failing with error: ORA-01536: space quota exceeded for tablespace ''NOTE:846297.1 - AQ Propagation Fails : ORA-00600[kope2upic2954] or Ora-00600[Kghsstream_copyn]NOTE:944846.1 - Streams Propagation Fails Ora-7445 [kohrsmc]

Master Note for Troubleshooting Advanced Queuing and Oracle Streams Propagation Issues (Doc ID 233099.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved.  In this Document  Purpose   Last Re...

Master Notes

Oracle Support Master Note for Streams Downstream Capture - 10g and 11g [Video] (Doc ID 1264598.1)

Master Note for Streams Downstream Capture - 10g and 11g [Video] (Doc ID 1264598.1)Copyright (c) 2010, Oracle Corporation. All Rights Reserved.In this Document  Purpose  Scope and Application  Master Note for Streams Downstream Capture - 10g and 11g [Video]     Downstream Capture     Transport Considerations in Downstream Capture     Instantiation - Implications for Primary Database     Recommended Parameter Settings     TroubleShooting     Performance Issues     Ongoing Streams Related Activities on the Primary Database  ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 11.2.0.2 - Release: 10.1 to 11.2Information in this document applies to any platform.Oracle Server Enterprise Edition - Version: 10.1.0.2 to 11.2.0.2 PurposeThis note provides background and troubleshooting information applicable in a Streams Downstream Capture Environment . Its main purpose as a master note is to bring together reference material in a form which allows a clear understanding of what is required in the configuration as well as to where to look when there are issues. Its focus is setup and understanding relevant parameters settings which are required.Scope and ApplicationThis document is intended for anyone with an interest in understanding Streams downstream capture. Where scripts details are references these are provided for reference to DBAs and Application developers. It is likely these will require modification in view of the fact that the examples presented are for illustration purposes. In order to have a better insight into how the examples work, please ensure you have read and understood the information below.Whilst every care has been taken to ensure that the details are correct and accurate , code should be thoroughly tested to verify it is fit for intended purpose.Please note , the approach adopted in relation to knowledge has been to improve Oracle's product documentation . Thus, relevant content in relation to downstream capture is now found in the primary reference guides for Streams ; if possible, please refer to the latest documentation :- Streams Concepts and Administration;- Streams Replication Administrator's GuideA useful reference in relation to downstream capture specific details is found in : Streams Configuration Best practices ; this was created with respect to 10.2 but is applicable to all later versions.Further questions in relation to this subject may be answered in : FAQ on Downstreams Capture, Note:394575.1 .Master Note for Streams Downstream Capture - 10g and 11g [Video] Downstream CaptureDownstream capture is a configuration in which the capture process runs on a database other than the source database. The source database has to be in archive log mode since the key aspect of downstream Capture of offloading the burden or load of processing the redo generated on the Primary to the downstream database.The following types of downstream capture configurations are possible: real-time downstream capture and archived-log downstream capture. The main consideration of a real-time downstream capture process is that it requires standby logs on the downstream database.Requirements the source database must be running at least Oracle Database 10g and the downstream capture database must be running the same release of Oracle as the source database or later;the downstream database must be running Oracle Database 10g Release 2 to configure real-time downstream capture. In this case, the source database must be running Oracle Database 10g Release 1 or later;the operating system on the source and downstream capture sites must be the same, but the operating system release does not need to be the same. However, the downstream sites can use a different directory structure from the source site;the hardware architecture on the source and downstream capture sites must be the same. For example, a downstream capture configuration with a source database on a 32-bit Sun system must have a downstream database that is configured on a 32-bit Sun system. Other hardware elements, such as the number of CPUs, memory size, and storage configuration, can be different between the source and downstream sites.Advantagescapturing changes does not happen at the source database, this the overhead for supporting Streams on the source database is greatly reduced;capture process administration is greatly simplified, as you can locate capture processes with different source sites on the same database;provide additional protection against database failure and corruption as the redo logs are shipped to a downstream database.  Note : Where multiple capture processes capture changes from a single source database,  the resulting LCRs from these capture processes should never be staged in the same queue if the queue is used by an apply process that applies any of these changes. Instead, a separate queue and apply process should be used to apply changes from each capture process.Real-Time Downstream CaptureIn a real-time downstream capture environment, the primary database is configured so that the transport services use the log writer process (LGWR) at the source database to send redo data to the downstream database either synchronously or asynchronously. At the same time, the LGWR records redo data in the online redo log at the source database.A remote file server process (RFS) at the downstream database receives the redo data over the network and stores the redo data in the standby redo logs.A log switch at the source database causes a log switch at the downstream database and the ARCX process at the downstream database archives the current standby redo log file as necessary.The real-time downstream capture process captures changes from the standby redo log whenever possible and from the archived standby redo log files whenever necessary. A capture process can capture changes in the archived standby redo log files if it falls behind. When it catches up, it resumes capturing changes from the standby redo log.Further detail relating to configuring real time downstream capture , with consideration for RAC source databases is found in : How To Configure Streams Real-Time Downstream Environment, Note:753158.1 .Please note , configuration of Real-Time Downstream differs from Archive-Log Downstream Capture configuration in the following ways :log_archive_* parameters are different ; refer below for example parameter definitions;the downstream capture database must be in archive log mode;standby logs must be created in the downstream capture database ;the capture process should be altered to use real time downstream mineArchived-Log Downstream CaptureThe primary database is configured so that the transport services use the archiver process (ARCX) at the source database. Archived logs from the source database are shipped to the downstream database and the capture process captures the relevant changes.Multiple capture process can be configured in downstream database to capture changes and these capture processes can capture changes from archived redo log files of multiple source databases.Further details on how to set up archive log downstream capture are found in : How To Setup Schema Level Streams Replication with a Downstream Capture Process with Implicit Log Assignment, Note:733691.1. Video - Demonstration of Archive Log Downstream Capture based on : Note:733691.1 (05:00) Transport Considerations in Downstream CaptureA fundamental requirement is that the source can ship redo to the destination database. The sys user on the source database must be able to connect as sys as sysdba at the downstream database. This will require configuration of Secure Sockets Layer (SSL) protocol or more commonly the addition of a remote login password file at the downstream database. If the source database has a remote login password file, then copy it to the appropriate directory on the downstream capture database system.- configure archive log mode on the source ; the downstream database does not need to running in archive log mode if it is running archived-log downstream capture ;- configure network sqlnet tns entries to allow connectivity between the databases ;- create a password file to ensure that user sys on the source can connect as sys on the downstream capture database. The following example creates password files for both databases however it is the downstream database which requires the password file.cd $ORACLE_HOME/dbsorapwd file=orapwstrm1 password=oracle entries=20cp orapwstrm1 orapwstrm2show parameter remote_login_passwordfile-- should show exclusive (non RAC) ; shared (RAC).-- shutdown / startupFollowing the above, you should be able to connect from the source to the destination as follows :connect sys/oracle@strm1.net as sysdbaselect * from global_name;-- Following is the important connection requirementconnect sys/oracle@strm2.net as sysdbaselect * from global_name;For more details refer to the Streams documentation. The main streams activities and their significance in a downstream capture environment are outlined below.Instantiation - Implications for Primary DatabaseRecall that instantiation in a Streams environment involves :preparing the object for instantiation at the source database.ensuring a copy of the object exists at the destination database;setting the instantiation SCN for the database object at the destination (apply) database. The instantiation SCN is an SCN connected with the source database from which the apply process will apply captured changes to the objectAlthough the capture process is running on a different database , there are still implications for the primary database which need to be understood so that the downstream capture environment can be created successfully. These details are discussed as follows.The examples detailed in Note:753158.1 and Note:733691.1 do not separate the build operation from the creation of the capture process. Furthermore , they do not refer to a flashback scn on the primary database corresponding to the first_scn returned from the build - refer to 'create capture' below below.Notes :- any export of data from the primary should always be made in consistent mode export ; - if there is little or no change to the primary database , the referenced notes serve as a reasonable guide . However, this can potentially present a gap in the changes captured relative to when the datapump export was performed which may result in confusing ora-1403 errors after setup.Streams can be configured so that the build activity is separated from the create capture operation. When performed as a separate operation, the build returns a first scn which can be used in subsequent configuration activities :when the capture process is created on the downstream database it should refer to the first SCN returned from the build;if datapump / export is used to move the data to the downstream database, a flashback_scn corresponding to the first scn from the build should be used ;when setting the instantiation scn , this should again refer to the file scn returned from the build.When the above considerations are met, the data on both databases as well as the relevant capture and apply processes are in sync when the processes are started .Create CaptureIn a downstream capture environment , the capture process has to be created separately from add_{tables|schema|database}_rules. The create_capture operation will internally generate a copy of the data dictionary at the primary in the redo stream when a database link to the primary is specified. Additionally, it creates the capture process referring to the first scn returned from the primary.A build operation can be performed on the primary as a separate operation using :set serveroutput onDECLAREscn NUMBER;BEGINDBMS_CAPTURE_ADM.BUILD(first_scn => scn);DBMS_OUTPUT.PUT_LINE('First SCN Value = ' || scn);commit;END;/The first_scn returned should then be used as the first_scn to the create_capture() call.Prepare {tables|schema|database} instantiationThis will result in supplemental logging being generated for the related objects at the primary database when run at the downsteam database .If there is no database link to the primary , perform the prepare at the primary or add the necessary supplemental logging at the primary database for the related objects.Add {tables|schema|database} rulesThis operation also performs a prepare (refer above) operation at the primary database for the related objects and configures relevant rules at the downstream database . This functions also configures relevant rules to allow the related object(s) to be processed by the relevant process.Set {table|schema|database} instantiation scnThis operation sets the {table|schema|database} instantiation scn for the Apply process . This is the SCN at which changes will be applied from.An export / import does not set the instantiation SCN for user objects. Therefore, this is also a required action if you use export / import to move objects from one database to another. Once you have performed this action, verify that instantiation SCNs are set for the schema and tables where appropriate as follows before starting the apply and capture processes.connect strmadmin/strmadminselect * from dba_apply_instantiated_global;select * from dba_apply_instantiated_schemas;select * from dba_apply_instantiated_objects;Note : for individual tables, an instantiation SCN must be present for the object. Similarly, for a schema , a schema instantiation SCN must be exist.Recommended Parameter SettingsThe main issue which may be encountered will relate to log transport or redo not appearing at the downstream database. Before discussing what to look at and where consider the relevant parameters which should be examined. Understanding the related parameters and their values is important in ensuring that set up is free from significant issues. The following outlines which parameters are required. Notes :- it is assumed in the following that log_archive_dest_1 is used as the archive log location and both databases is in archive log mode;- the log_archive_* parameter settings have their meaning defined in the Oracle Server reference manual;- directory specification values are for illustrative purposes only Real-Time Downstream Capture Parameter ValuesThe following serves as a guide of parameter requirements in setting up real-time downstream capture. For further details consult the Streams Concepts and Administration Guide. Primary databasealter system set log_archive_dest_1 = 'location=/bugmnt12/em/celclnx7/SR3.2274614446/user/strm1 mandatory reopen=5' scope=spfile/alter system set log_archive_dest_state_1 = enable scope=spfile/alter system set log_archive_format = 'strm1_%t_%s_%r.dbf' scope=spfile/alter system set log_archive_dest_2 = 'service=strm2.net lgwr async noregister valid_for=(online_logfile,all_roles) db_unique_name=strm2' scope=spfile -- Note 1/alter system set log_archive_dest_state_2 = enable scope=spfile -- Note 2/alter system set log_archive_config='send,dg_config=(strm1,strm2)' scope=both -- Note 3/Note 1 - the public database link for the remote database is strm2.net ; 'lgwr async' the log writer process will write to the remote database asynchronously - if you use 'lgwr sync' you are writing through the network to the remote database which may have an affect in commit time on the source; noregister - logs are not registered in the control file , there is no need to do this ; valid_for should be defined as detailed; db_unique - indicates the unique_name of the remote database (show parameter db_unique_name);Note 2 - this can be set to 'deferred' until the setup is complete; then amended to enable , i.e. initially there is no need to ship the redo information until the setup is complete ; once complete , this parameter can be set to 'enable' to activity activity;Note 3 - this details the unique names involved, again issue show parameter db_unique_name on both databases; also indicates that this is the sending database. Downstream Database alter system set log_archive_dest_1 = 'location=/bugmnt12/em/celclnx7/SR3.2274614446/user/strm2 mandatory reopen=5 valid_for=(online_logfile, primary_role)' scope=spfile/alter system set log_archive_dest_state_1 = enable scope=spfile/alter system set log_archive_format = 'strm2_%t_%s_%r.dbf' scope=spfile/alter system set log_archive_dest_2 = 'location=/bugmnt12/em/celclnx7/SR3.2274614446/user/standby mandatory valid_for=(standby_logfile, primary_role)' scope=spfile -- note 4/alter system set log_archive_dest_state_2 = enable scope=spfile/alter system set log_archive_config='receive,dg_config=(strm1,strm2)' scope=both -- note 5/Note 4 - location where the related archive log files from the primary are stored. logs will initially be written to the standby redo logs and then written out to this location in much the same way as archival of redo;Note 5 - this details the unique names involved, again show parameter db_unique_name on both databases; also indicates that this is the sending database. Archived-Log Downstream Capture Parameter ValuesThe following serves as a guide of parameter requirements in setting up archived log downstream capture. For further details consult the Streams Concepts and Administration Guide.Primary database alter system set log_archive_dest_1 = 'location=/bugmnt12/em/celclnx7/SR3.2274614446/user/strm1 mandatory reopen=5' scope=spfile/alter system set log_archive_dest_state_1 = enable scope=spfile/alter system set log_archive_format = 'strm1_%t_%s_%r.dbf' scope=spfile/alter system set log_archive_dest_2 = 'service=strm2.net arch async noregister valid_for=(online_logfile,all_roles) template=/bugmnt12/em/celclnx7/SR3.2274614446/user/logsfromstrm1/strm1_%t_%s_%r.dbf db_unique_name=strm2' scope=spfile -- Note 6/alter system set log_archive_dest_state_2 = enable scope=spfile -- Note 7/alter system set log_archive_config='send,dg_config=(strm1,strm2)' scope=both -- Note 8/Note 6 - the public database link for the remote database is strm2.net ; 'arch async' indicates the archiver process (ARCX) will write to the remote database asynchronously; noregister - logs are not registered in the control file , there is need to have logs registered in the control file ; valid_for should be defined as detailed; template= - details the location in the remote machine where the logs transferred should be located on transfer; db_unique - indicates the unique_name of the remote database (show parameter db_unique_name);Note 7 - as per Note 2 above;Note 8 - this details the unique names involved, again show parameter db_unique_name on both databases; also indicates that this is the sending database. Downstream Database  alter system set log_archive_dest_1 = 'location=/bugmnt12/em/celclnx7/SR3.2274614446/user/strm2 mandatory reopen=5 valid_for=(online_logfile, primary_role)' scope=spfile/alter system set log_archive_dest_state_1 = enable scope=spfile/alter system set log_archive_format = 'strm2_%t_%s_%r.dbf' scope=spfile/alter system set log_archive_dest_2 = 'location=/bugmnt12/em/celclnx7/SR3.2274614446/user/standby mandatory valid_for=(standby_logfile, primary_role)' scope=spfile -- Note 9/alter system set log_archive_dest_state_2 = enable scope=spfile/alter system set log_archive_config='receive,dg_config=(strm1,strm2)' scope=both -- Note 10/Note 9 - 'valid_for=(standby_logfile' - is in anticipation of moving using real time downstream capture. The primary has specified , by the use of a 'template=', where the logs will be located on the downstream database.Note 10 - this details the unique names involved, again issue show parameter db_unique_name on both databases ; also indicates that this is the receiving database. TroubleShootingIf downstream Capture is not working , it is usually the case that the primary database cannot connect to the downsteam database. i.e typically, the cause may be ora-1017: invalid username/password; logon denied.If the remote_login_passwordfile is not configured or is not the same as that on the primary or the sys passwords are different you will likely encounter ora-1017.Please also refer to the section Troubleshooting Implicit Capture in the 11.2 Streams Concepts and Administration Guide.ErrorsOnce the environment is active, the most common type of error which is likely to be encountered in a streams environment will be data related , i.e this may typically be : ora-1403.For more information, refer to: Master Note for Troubleshooting Streams Apply Errors ORA-1403, ORA-26787 or ORA-26786 ,  Note:265201.1.During setup, it may not be obvious what is happening. Please make sure that you review the source alert.log file as well as the related log writer or archiver trace files - whichever process will be managing redo transport. These will likely point to the root cause of the issue which will likely be configuration / parameter related .Tracing Log Transport IssuesIf there is still no clear indication as to why redo is not being shipped , log transport related trace can be enabled by the setting of the parameter : LOG_ARCHIVE_TRACE. For more information , refer to the Oracle Database Reference Guide for trace levels.Note : most issues will not require any reference to this parameter . Most problems will likely have a visible error which will be connected with an obvious cause and resolution.Performance IssuesIssues relating to performance in a downstream capture environment should be treated in the same way as issues in a local capture environment. If a capture process is waiting on redo , this would suggest that there is a lag in the transport of redo information from the primary ; this will typically depend on whether real-time downstream capture or archived-log downstream capture is used. Other issues should be addressed in the same manner as with a local capture process and streams . A useful starting reference is : Master Note for Streams Performance Recommendations ,  Note:335516.1. Ongoing Streams Related Activities on the Primary DatabaseOnce the downstream capture activity is active, there is one piece of relevant maintenance which should be scheduled at the primary database. Recall in the discussion above, that when the Capture process is created, it will refer to a build in the redo which is linked to its first_scn value. If there is any problem which demands that a Capture process has to be dropped and recreated , the new capture process can refer back to the most recent build scn. This will be the first_scn of the new capture process.NOTE: It is recommended to perform periodic build operations on all streams capture environments. ReferencesNOTE:335516.1 - Master Note for Streams Performance RecommendationsNOTE:394575.1 - FAQ on Downstreams CaptureNOTE:413353.1 - 10.2 Best Practices For Streams in RAC environmentNOTE:418755.1 - Master Note for Streams Recommended ConfigurationNOTE:733691.1 - How To Setup Schema Level Streams Replication with a Downstream Capture Process with Implicit Log AssignmentNOTE:753158.1 - How To Configure Streams Real-Time Downstream EnvironmentNOTE:265201.1 - Master Note for Troubleshooting Streams Apply Errors ORA-1403, ORA-26787 or ORA-26786

Master Note for Streams Downstream Capture - 10g and 11g [Video] (Doc ID 1264598.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved. In this Document  Purpose   Scope and Application   Maste...

Master Notes

Oracle Support Master Note for Streams Recommended Configuration (Doc ID 418755.1)

Master Note for Streams Recommended Configuration (Doc ID 418755.1)Copyright (c) 2010, Oracle Corporation. All Rights Reserved. In this Document  Purpose  Scope and Application  Master Note for Streams Recommended Configuration     Configuration     1.0 Software Version     2.0 Database Parameters     Database Version 9iR2     Database Version 10gR2     3.0 Database Storage     3.1. Tablespace for Streams Administrator queues     3. 2. Separate queues for capture and apply     4.0 Privileges     5.0 Source Site Configuration     5.1. Streams and Flash Recovery Area (FRA)     5.2. Archive Logging must be enabled     5.3. Supplemental logging     5.4. Implement a Heartbeat Table     5.5. Flow Control     5.6. Perform periodic maintenance     Database Version 9iR2 and 10gR1     Database Version 10gR2 and above     5.7. Capture Process Configuration     5.8. Propagation Configuration     5.9. Additional Configuration for RAC Environments for a Source Database     6.0 Target Site Configuration     6.1. Privileges     6.2. Instantiation     6.3. Conflict Resolution     6.4. Apply Process Configuration     6.5. Additional Configuration for RAC Environments for an Apply Database     OPERATION     Global Name     Apply Error Management     Backup Considerations     Batch Processing     Source Queue Growth     Streams Cleanup/Removal     Automatic Optimizer Statistics Collection     MONITORING     Dynamic Streams views      Static Streams Views     Streams Views     Capture Views:     Propagation & Queue Views     Apply Views:     Monitoring Utility STRMMON     Alert Log     Streams Healthcheck Scripts  ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 9.2.0.8 to 11.2.0.1 - Release: 9.2 to 11.2Information in this document applies to any platform.PurposeOracle Streams enables the sharing of data and events in a data stream either within a database or from one database to another. This Note describes best practices for Oracle Streams configurations for both downstream capture and upstream (local) capture in version 9.2 and above. Scope and ApplicationThe information contained in this note targets Replication administrators implementing Streams replication in Oracle 9.2 and higher. This note contains key recommendations for successful implementation of Streams in Oracle database release 9.2 and above. Master Note for Streams Recommended ConfigurationConfigurationTo ensure a successful Streams implementation, use the following recommendations when setting up a Streams environment: Software VersionDatabase Settings: Parameters, Storage, and PrivilegesSource Site ConfigurationTarget Site Configuration1.0 Software VersionOracle recommends to run streams with the latest available patchset, and the list of recommended patches from Document 437838.1 Streams Specific Patches.Please assess if any recommended patch conflicts with existing patches on your system.There is Streams support in both DbControl and GridControl. GridControl should be used to manage multiple databases in a Streams environment.2.0 Database ParametersFor best results in a Streams environment, set the following initialization parameters, as necessary, at each participating instance: global_names, _job_queue_interval, sga_target, streams_pool_size:Database Version 9iR2Parameter Name & RecommendationDescriptionConsiderations_job_queue_interval = 1Scan rate interval (seconds) of job queue. Default is 5This improves the scan rate for propagation jobs to every second, rather than every 5 seconds.COMPATIBLE = 9.2.0.0This parameter specifies the release with which the Oracle server must maintain compatibility. Oracle servers with different compatibility levels can interoperate.GLOBAL_NAMES =trueSpecifies whether a database link is required to have the same name as the database to which it connects. Default is FALSEThis parameter should be set to TRUE at each database that is participating in your Streams environment to eliminate errors resulting from incorrect database connections. This parameter setting is honored by database links.JOB_QUEUE_PROCESSES > 2Specifies the number of Jn job queue processes for the database instanceAQ_TM_PROCESSES >= 1Specifies the number of queue monitor processes for the database instanceLOGMNR_MAX_PERSISTENT_SESSIONS >= Number of capture processesSpecifies the maximum number of persistent LOGMINER mining sessions. Streams Capture Process uses LOGMINER to mine the  redo logs.If there is a need to run multiple Streams capture processes on a single database, then this parameter needs to be set equal to or higher than the number of planned capture processes.LOG_PARALLELISM = 1Specifies the level of concurrency for redo allocation within the database instance.PARALLEL_MAX_SERVERS >= 2Default: Derived from the values of the following parameters:CPU_COUNTPARALLEL_ADAPTIVE_MULTI_USERPARALLEL_AUTOMATIC_TUNINGRange: 0 to 3599Modifiable?: YesSpecifies the maximum number of parallel execution processes and parallel recovery processes for an instance. As demand increases, Oracle will increase the number of processes from the number created at instance startup up to this value.In a Streams environment, each capture process and apply process may use multiple parallel execution servers. Set this initialization parameter to an appropriate value to ensure that there are enough parallel execution servers. For each defined Streams process (capture or apply), increase this parameter by 2+parallelism, where parallelism is the value of the capture or apply parallelism parameter.SHARED_POOL_SIZE Each capture process needs 10MB of shared pool space, by defaultStreams is limited to using a maximum of 10% of the shared pool.The 10% of the shared_pool_size is in reference to the size of the buffer queue before spillover occurs. Shared_pool_size must be significantly larger if Streams captureis implemented, especially if there is a large workload.The typical recommendation is to double the existing shared_pool_size and set the _first_spare_parameter to 50.OPEN_LINKS >= 4Specifies the maximum number of concurrent open connections to remote databases in one session.PROCESSESSpecifies the maximum number of operating system user processes that can simultaneously connect to the database.Make sure the value of this parameter allows for all background processes, such as locks, job queue processes, and parallel execution processes. In Streams, capture processes and apply processes use background processes and parallel execution processes, and propagation jobs use job queue processes.SESSIONSSpecifies the maximum number of sessions that can be created in the system. Because every login requires a session, this parameter effectively determines the maximum number of concurrent users in the system. If you plan to run one or more capture processes or apply processes in a database, then you may need to increase the size of this parameter. Each background process in a database requires a session.SGA_MAX_SIZE Specifies the maximum size of SGA for the lifetime of the instance.If you plan to run multiple capture processes on a single database, then you may need to increase the size of this parameter.Note :_SGA_SIZE should only be increased if a logminer error is returned indicating a need for more memory. Any memory allocated to logminer is used solely by logminer - it is not returned to the shared_poolafter it has been allocated by capture until the capture process is restarted.TIMED_STATISTICSSpecifies whether or not statistics related to time are collected.If you want to collect elapsed time statistics in the data dictionary views related to Streams, then set this parameter to true. The views that include elapsed time statistics include:V$STREAMS_CAPTURE, V$STREAMS_APPLY_COORDINATOR, V$STREAMS_APPLY_READER, V$STREAMS_APPLY_SERVER.Database Version 10gR2Parameter Name & RecommendationDescriptionConsiderations_job_queue_interval = 1Scan rate interval (seconds) of job queue. Default is 5This improves the scan rate for propagation jobs to every second, rather than every 5 seconds.COMPATIBLE = 10.2.0.0This parameter specifies the release with which the Oracle server must maintain compatibility. Oracle servers with different compatibility levels can interoperate.To use the new Streams features introduced in Oracle Database 10g Release 2, this parameter must be set to 10.2.0.0 or higher.To use 10.2 downstream capture on logs from a 10.1 source , this parameter must be set to 10.1.0.0 at the source database and 10.2.0.0 at the downstream capture database.GLOBAL_NAMES =trueSpecifies whether a database link is required to have the same name as the database to which it connects. Default is FALSEThis parameter should be set to TRUE at each database that is participating in your Streams environment to eliminate errors resulting from incorrect database connections. This parameter setting is honored by database links.JOB_QUEUE_PROCESSES > 4 number of propagations definedSpecifies the number of Jn job queue processes for each instance (J000 ... J999). Job queue processes handle requests created by DBMS_JOB.This parameter controls the maximum number of jobs that can run concurrently within the instance and should be set to a value greater than the number of propagations configured for the database. Be sure to increase this parameter if there are any automated jobs configured for the database.LOG_ARCHIVE_DEST_nDefines up to ten log archive destinations, where n is 1, 2, 3, ... 10.A specific archive log destination should be specified if this database is the source for a Streams capture process. Specify a specific destination other than the flash recovery area (FRA) for storing archived logs if a local capture process is enabled. To use downstream capture and copy the redo log files to the downstream database using log transport services, at least one log archive destination must be to the site running the downstream capture process. Avoid copying log files to a remote flash recovery area for downstream capture processing.See Also: Oracle Data Guard Concepts and AdministrationLOG_ARCHIVE_DEST_STATE_nSpecifies the availability state of the corresponding destination. The parameter suffix (1 through 10) specifies one of the ten corresponding LOG_ARCHIVE_DEST_n destination parameters. Enable archive logging to the specified destination for both local and downstream capture. To use downstream capture and copy the redo log files to the downstream database using log transport services, make sure the destination that corresponds to the LOG_ARCHIVE_DEST_n destination for the downstream database is set to enable.PARALLEL_MAX_SERVERSDefault: Derived from the values of the following parameters:CPU_COUNTPARALLEL_ADAPTIVE_MULTI_USERPARALLEL_AUTOMATIC_TUNINGRange: 0 to 3599Modifiable?: YesSpecifies the maximum number of parallel execution processes and parallel recovery processes for an instance. As demand increases, Oracle will increase the number of processes from the number created at instance startup up to this value.In a Streams environment, each capture process and apply process may use multiple parallel execution servers. Set this initialization parameter to an appropriate value to ensure that there are enough parallel execution servers. For each defined Streams process (capture or apply), increase this parameter by 2+parallelism, where parallelism is the value of the capture or apply parallelism parameter.REMOTE_ARCHIVE_ENABLEEnables or disables the sending of redo archival to remote destinations and the receipt of remotely archived redo. To use downstream capture and copy the redo log files to the downstream database using log transport services, this parameter must be set to true at both the source database and the downstream database. This parameter is not required for local capture configuration.SGA_MAX_SIZE Specifies the maximum size of SGA for the lifetime of a database instance. To run multiple Streams processes on a single database, you may need to increase the size of this parameter.SGA_TARGET =0Specifies the total size of all System Global Area (SGA) components.If this parameter is set to a nonzero value, then the size of the Streams pool is managed by Automatic Shared Memory Management.For best results, size the shared_pool and streams_pool explicitly.Tune the STREAMS_POOL_SIZESpecifies (in bytes) the size of the Streams pool. The Streams pool contains buffered queue messages. In addition, the Streams pool is used for internal communications during parallel capture and apply. Refer to V$STREAMS_POOL_ADVICE to determine the correct size to avoid excessive spills.This parameter is modifiable. If this parameter is reduced to zero when an instance is running, then Streams processes and jobs will not run.The size of the Streams pool is affected by each of the following factors:capture process parallelism. Increase the Streams Pool Size by 10 MB for each capture process. In addition, if the capture parameter PARALLELISM is set greater than 1, increase the Streams Pool size by 10Mb * parallelism. For example, if parallelism is set to 3 for a capture process, then increase the Streams pool by 30 MB.Apply process parallelism. Increase the Streams Pool Size by 1 MB for each apply process. In addition, if the apply parameter PARALLELISM is set greater than 1, increase the Streams Pool size by 1Mb * parallelism. For example, if parallelism is set to 5 for an apply process, then increase the Streams pool by 5 MB.Logical Change Records (LCRs) are stored in the buffered queue. Increase the size of the Streams Pool to handle the volume of replicated data managed at both the source and target databases.Minimally set the Streams Pool Size to 256Mb on low activity databases or 500Mb on more active OLTP configurations. Adjust the Streams Pool size to an appropriate value using the V$STREAMS_POOL_ADVICE view to avoid excessive spill from the buffered queue to disk.3.0 Database Storage3.1. Tablespace for Streams Administrator queuesCreate a separate tablespace for the streams administrator schema (STRMADMIN) at each participating Streams database. This tablespace will be used for any objects created in the streams administrator schema, including any spillover of messages from the in-memory queue. For example:CREATE TABLESPACE &streams_tbs_name DATAFILE '&db_file_directory/&db_file_name' SIZE 25 M REUSE AUTOEXTEND ON NEXT 25M MAXSIZE UNLIMITED; ALTER USER strmadmin DEFAULT TABLESPACE &streams_tbs_name QUOTA UNLIMITED ON &streams_tbs_name;3. 2. Separate queues for capture and applyConfigure separate queues for changes that are captured locally and for receiving captured changes from each remote site. This is especially important when configuring bi-directional replication between multiple databases. For example, consider the situation where Database db1.net replicates its changes to databases db2.net, and Database db2.net replicates to db1.net. Each database will maintain 2 queues: one for capturing the changes made locally and other queue receiving changes from the other database.Similarly, for 3 databases (db1.net, db2.net, db3.net) replicating the local changes directly to each other database, there will be 3 queues at each database. For example at db1.net, queue1 for the capture process, and queue2 and queue3 for receiving changes from each of the other databases. The two apply processes on db1.net (apply_from_db2, apply_from_db3) apply the changes, each associated with a specific queue (queue2 or queue3)Queue names should not exceed 24 characters in length. Queue table names should not exceed 24 characters in length. To pre-create a queue for Streams, use the SET_UP_QUEUE procedure in the DBMS_STREAMS_ADM package. If you use the MAINTAIN_TABLES, MAINTAIN_SCHEMAS, or MAINTAIN_GLOBAL procedures to configure Streams and do not identify specific queue names, individual queues will be created automatically.Example: To configure a site (SITEA) that is capturing changes for distribution to another site, as well as receiving changes from that other site (SITEB), configure each queue at SITEA with a separate queue_table as follows:dbms_streams_adm.set_up_queue(queue_table_name='QT_CAP_SITE_A, queue_name=>'CAP_SITEA', )dbms_streams_adm.set_up_queue(queue_table_name='QT_APP_FROM_SITEB', queue_name=>'APP_FROM_SITEB');If desired, the above set_up_queue procedure calls can include a storage_clause parameter to configure separate tablespace and storage specifications for each queue table. Typically, Logical Change Records (LCRs) are queued to an in-memory buffer and processed from memory. However, they can be spilled to disk if they remain in memory too long due to an unavailable destination or on memory pressure (Streams_Pool memory is too low). The storage clause parameter can be used to preallocate space for the queue table or specify an alternative tablespace for the queue table without changing the default tablespace for the Streams Administrator.4.0 PrivilegesThe streams administrator (strmadmin) must be granted the  following on each participating Streams participating database:GRANT EXECUTE ON DBMS_AQADM TO strmadmin;GRANT EXECUTE ON DBMS_APPLY_ADM TO strmadmin;GRANT EXECUTE ON DBMS_CAPTURE_ADM TO strmadmin;GRANT EXECUTE ON DBMS_PROPAGATION_ADM TO strmadmin;GRANT EXECUTE ON DBMS_STREAMS TO strmadmin;GRANT EXECUTE ON DBMS_STREAMS_ADM TO strmadmin;BEGIN DBMS_RULE_ADM.GRANT_SYSTEM_PRIVILEGE(privilege => DBMS_RULE_ADM.CREATE_RULE_SET_OBJ, grantee => 'strmadmin', grant_option => FALSE); END;/BEGIN DBMS_RULE_ADM.GRANT_SYSTEM_PRIVILEGE(privilege => DBMS_RULE_ADM.CREATE_RULE_OBJ, grantee => 'strmadmin', grant_option => FALSE);END;/In order to create capture and apply processes, the Streams Administrator must have DBA privilege. This privilege must be explicitly granted to the Streams Administrator. GRANT DBA to STRMADMIN;In addition, other required privileges must be granted to the Streams Administrator schema (strmadmin) on each participating Streams database with the GRANT_ADMIN_PRIVILEGE procedure:In Oracle 10g and above, all the above (except DBA) can be granted using the procedure:DBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE('STRMADMIN');5.0 Source Site ConfigurationThe following recommendations apply to source databases, ie, databases in which Streams capture is configured.5.1. Streams and Flash Recovery Area (FRA)In Oracle 10g and above, configure a separate log archive destination independent of the Flash Recovery Area for the Streams capture process for the database. Archive logs in the FRA can be removed automatically on space pressure, even if the Streams capture process still requires them. Do not allow the archive logs for Streams capture to reside solely in the FRA.5.2. Archive Logging must be enabledVerify that each source database is running in ARCHIVE LOG mode. For downstream capture sites (ie, databases in which the Streams capture is configured for another database), the database at which the source redo logs are created must have archive logging enabled.5.3. Supplemental loggingConfirm supplemental logging is enabled at each source site. In 9iR2 Streams apply requires unconditional logging of Unique Index and Foreign Key constraints, even if those columns are not modified. This is because of Bug 4198593 Apply incorrectly requires unconditional logging of Unique and FK constraints fixed in 9.2.0.8.If you set the parallelism apply process parameter to a value greater than 1, then you must specify a conditional supplemental log group at the source database for all of the unique and foreign key columns in the tables for which an apply process applies changes. Supplemental logging may be required for other columns in these tables as well, depending on your configuration. Any columns specified in rule-based transformations or used within DML Handlers at target site must be unconditionally logged at the source site.Supplemental logging can be specified at the source either at the database level or for the individual replicated table.In 10gR2, supplemental logging is automatically configured for tables on which primary, unique, or foreign keys are defined when the database object is prepared for Streams capture. The procedures for maintaining streams and adding rules in the DBMS_STREAMS_ADM package automatically prepare objects for a local Streams capture. For downstream capture sites (ie, databases in which the Streams capture is configured for another database), the database at which the source redo logs are created must have supplemental logging for the database objects of interest to the downstream capture process. All target site indexed columns, including the primary key, unique index, and foreign key columns of a replicated table or database must be logged at the source site. Primary Key logging must be unconditionally logged, unique index and foreign keys can be conditionally logged. This supplemental logging is enabled automatically when the source table is prepared for capture with DBMS_CAPTURE_ADM.PREPARE_TABLE_INSTANTIATION.Any columns specified in rule-based transformations or used within DML Handlers at target site must be unconditionally logged at the source site. Supplemental logging for these columns must be configured explicitly by the database administrator, using the table sql syntax: ALTER TABLE... ADD SUPPLEMENTAL LOG... .To Verify that supplemental logging has been specified at the source either at the database level or for the individual replicated table:Database level logging:SELECT supplemental_log_data_pk, supplemental_log_data_ui FROM V$DATABASE;Table level logging:SELECT supplemental_log_data_pk, supplemental_log_data_ui, supplemental_log_data_fk FROM dba_capture_prepared_tables UNIONSELECT supplemental_log_data_pk, supplemental_log_data_ui, supplemental_log_data_fk FROM dba_capture_prepared_schemas UNIONSELECT supplemental_log_data_pk, supplemental_log_data_ui, supplemental_log_data_fk FROM dba_capture_prepared_database;Check supplemental log groupsSelect log_group_name, table_name, decode(always, 'ALWAYS', 'Unconditional', NULL, 'Conditional') ALWAYS from DBA_LOG_GROUPSCheck columns in supplemental log groupsSelect log_group_name, column_name, position from dba_log_group_columns where table_name = 'DEPARTMENTS' and owner='HR'; Refer to Document 782541.1 Streams Replication Supplemental Logging Requirements5.4. Implement a Heartbeat TableTo ensure that the applied_scn of the DBA_CAPTURE view is updated periodically, implement a "heart beat" table. A "heart beat" table is especially useful for databases that have a low activity rate. The streams capture process requests a checkpoint after every 10Mb of generated redo. During the checkpoint, the metadata for streams is maintained if there are active transactions. Implementing a heartbeat table ensures that there are open transactions occurring regularly within the source database enabling additional opportunities for the metadata to be updated frequently. Additionally, the heartbeat table provides quick feedback to the database administrator as to the health of the streams replication.To implement a heartbeat table: Create a table at the source site that includes a date or timestamp column and the global name of the database. Add a rule to capture changes to this table and propagate the changes to each target destination. Make sure that the target destination will apply changes to this table as well. Set up an automated job to update this table at the source site periodically, for example every minute. Refer to Document 461278.1 Example of a Streams Heartbeat Table5.5. Flow ControlIn Oracle 9iR2, when the threshold for memory of the buffer queue is exceeded, Streams will write the messages to disk. This is sometimes referred to as "spillover". When spillover occurs, Streams can no longer take advantage of the in-memory queue optimization. One technique to minimize this spillover is to implement a form of flow control. See the following note for the scripts and pre-requisites:Script to Prevent Excessive Spill of Message From the Streams Buffer Queue To Disk (Doc ID 259609.1)In Oracle 10g and above flow control is automatically handled by the database so there is no need to implement it manually.5.6. Perform periodic maintenanceDatabase Version 9iR2 and 10gR1Periodically force capture to checkpoint. This checkpoint is not the same as a database checkpoint. To force capture to checkpoint, use the capture parameter _CHECKPOINT_FORCE and set the value to YES. Forcing a checkpoint ensure that the DBA_CAPTURE view columns CAPTURED_SCN and APPLIED_SCN are maintained. Database Version 10gR2 and aboveA. Confirm checkpoint retention. In Oracle 10gR2 and above, the mining process checkpoints itself for quicker restart. These checkpoints are maintained in the SYSAUX tablespace by default. The capture parameter, checkpoint_retention_time, controls the amount of checkpoint data retained by moving the FIRST_SCN of the capture process forward. The FIRST_SCN is the lowest possible scn available for capturing changes. When the checkpoint_retention_time is exceeded (default = 60 days), the FIRST_SCN is moved and the Streams metadata tables previous to this scn (FIRST_SCN) can be purged and space in the SYSAUX tablespace reclaimed. To alter the checkpoint_retention_time, use the DBMS_CAPTURE_ADM.ALTER_CAPTURE procedure.B. Dump fresh copy of Dictionary to redo. Issue a DBMS_CAPTURE_ADM.BUILD command to dump a current copy of the data dictionary to the redo logs. Doing this will reduce the amount of logs to be processed in case of additional capture process creation or process rebuild.C. Prepare database objects for instantiation Issue a DBMS_CAPTURE_ADM.PREPARE_*_INSTANTIATION where * indicates the level (TABLE, SCHEMA, GLOBAL) for the database objects captured by Streams. This is used in conjunction with the BUILD in B above for new capture creation or rebuild purposes. 5.7. Capture Process ConfigurationA. Configuring CaptureUse the DBMS_STREAMS_ADM.ADD_*_RULES procedures (ADD_TABLE_RULES,  ADD_SCHEMA_RULES for DML and DDL, ADD_GLOBAL_RULES for DDL only).  These procedures minimize the number of steps required to configure Streams processes. Also, it is possible to create rules for non-existent objects, so be sure to check the spelling of each object specified in a rule carefully. CAPTURE requires a rule set with rules.The ADD_GLOBAL_RULES procedure cannot be used to capture DML changes for entire database. ADD_GLOBAL_RULES can be used to capture all DDL changes for the database.A single Streams capture can process rules for multiple tables or schemas. For best performance, rules should be simple.  Rules that include NOT or LIKE clauses are not simple and will impact the performance of Streams. Minimize the number of rules added into the process rule set.  A good rule of thumb is to keep the number of rules in the rule set to less than 100. If more objects need to be included in the ruleset, consider constructing rules using the IN clause. For example, a rule for the 6 TB_M21* tables in the MYACCT schema would look like the following:(:dml.get_object_owner() = 'MYACCT' and :dml.is_null_tag() = 'Y' and :dml.get_object_name() IN ('TB_M21_1','TB_M21_2','TB_M21_3','TB_M21_40','TB_M21_10','TB_M211B010'))In version 10.2 and above, use the DBMS_STREAMS_ADM. MAINTAIN_* (where *=TABLE,SCHEMA,GLOBAL, TTS) procedures to configure Streams. These procedures automate the entire configuration of the streams processes between databases, following the Streams best practices. For local capture, the default behavior of these procedures is to implement a separate queue for capture and apply. If you are configuring a downstream capture and applying the changes within the same database, override this behavior by specifying the same queue for both the capture_queue_name and apply_queue_name.If the maintain_* procedures are not suitable for your environment, please use the ADD_*_RULES procedures (ADD_TABLE_RULES, ADD_SCHEMA_RULES for DML and DDL, ADD_SUBSET_RULES for DML only, and ADD_GLOBAL_RULES for DDL only). These procedures minimize the number of steps required to configure Streams processes. It is also possible to create rules for non-existent objects, so be sure to check the spelling of each object specified in a rule carefully.The Streams capture process requires a rule set with rules. The ADD_GLOBAL_RULES procedure can be used to capture DML changes for entire database as long as a negative ruleset is created for the capture process that includes rules for objects with unsupported datatypes.. ADD_GLOBAL_RULES can be used to capture all DDL changes for the database.A single Streams capture can process changes for multiple tables or schemas. For best performance, rules for these multiple tables or schemas should be simple. Rules that include LIKE clauses are not simple and will impact the performance of Streams. To eliminate changes for particular tables or objects, specify the include_tagged_lcr clause along with the table or object name in the negative rule set for the Streams process. Setting this clause will eliminate ALL changes, tagged or not, for the table or object.B. Capture ParametersSet the following parameters after a capture process is created:Parameter & RecommendationValuesCommentPARALLELISM=1Default: 1Number of parallel execution servers to configure one or more preparer processes used to prefilter changes for the capture process. Recommended value is 1. _CHEKPOINT_FREQUENCY=500Default: 10 <10.2.0.4Default 1000 in 10.2.0.4 Modify the frequency of logminer checkpoints especially in a database with significant LOB or DDL activity. Larger values decrease the frequency of logminer checkpoints. Smaller numbers increase the frequency of those checkpoints. Logminer checkpoints are not the same as database checkpoints. Availability of logminer checkpoints impacts the time required to recover/restart the capture after database restart. In a low activity database (ie, small amounts of data or the data to be captured is changed infrequently), use a lower value, such as 100.A logminer checkpoint is requested by default every 10Mb of redo mined. If the value is set to 500, a logminer checkpoint is requested after every 500Mb of redo mined. Increasing the value of this parameter is recommended for active databases with significant redo generated per hour.It should not be necessary to configure _CHECKPOINT_FREQUENCY in 10.2.0.4 or higher_SGA_SIZEDefault: 10Amount of memory available from the streams pool for logminer processing. The default amount of streams_pool memory allocated to logminer is 10Mb. Increase this value especially in environments where large LOBs are processed. This parameter should not be increased unless the logminer error ORA-1341 is encountered. Streams pool memory allocated to logminer is unavailble for other usaCapture parameters can be set using the SET_PARAMETER procedure from the DBMS_CAPTURE_ADM package. For example, to set the checkpoint frequency of the streams capture process named CAPTURE_EX, use the following syntax while logged in as the Streams Administrator to request a logminer checkpoint after processing every Gigabyte (1000Mb) of redo:dbms_capture_adm.set_parameter('capture_ex','_checkpoint_frequency','1000');5.8. Propagation ConfigurationA. Configuring PropagationIf the maintain_* procedures are not suitable for your environment(Oracle 9iR2 and 10gR1), please use the ADD_*__PROPAGATION_RULES procedures (ADD_TABLE_PROPAGATION_RULES, ADD_SCHEMA_PROPAGATION_RULES , ADD_GLOBAL_PROPAGATION_RULES for both DML and DDL., ADD_SUBSET_PROPAGATION_RULES for DML only) These procedures minimize the number of steps required to configure Streams processes. Also, it is possible to create rules for non-existent objects, so be sure to check the spelling of each object specified in a rule carefully.The rules in the rule set for propagation can differ from the rules specified for the capture process. For example, to configure that all captured changes be propagated to a target site, a single ADD_GLOBAL_PROPAGATION_RULES procedure can be specified for the propagation even though multiple ADD_TABLE_RULES might have been configured for the capture process.B. Propagation modeFor new propagation processes configured in 10.2 and above. set the queue_to_queue propagation parameter to TRUE. If the database is RAC enabled, an additional service is created typically named in the format: sys$schema.queue_name.global_name when the Streams subscribers are initially created. A streams subscriber is a defined propagation between two Streams queues or an apply process with the apply_captured parameter set to TRUE. This service automatically follows the ownership of the queue on queue ownership switches (ie, instance startup, shutdown, etc). The service name can be found in the network name column of DBA_SERVICES view.If the maintain_* (TABLE,SCHEMA,GLOBAL) procedures are used to configure Streams, queue_to_queue is automatically set to TRUE, if possible. The database link for this queue_to_queue propagation must use a TNS servicename (or connect name) that specifies the GLOBAL_NAME in the CONNECT_DATA clause of the descriptor. See section 6 on Additional Considerations for RAC below.Propagation process configured prior to 10.2 continue to use the dblink mode of propagation. In this situation, if the database link no longer connects to the owning instance of the queue, propagation will not succeed. You can continue to use the 10.1. best practices for this propagation, or during a maintenance window recreate propagation. Make sure that the queue is empty with no unapplied spilled messages before you drop the propagation. Then, recreate the propagation with the queue_to_queue parameter set to TRUE.Queues created prior to 10.2 on RAC instances should be dropped and recreated in order to take advantage of the automatic service generation and queue_to_queue propagation. Be sure to perform this activity when the queue is empty and no new LCRs are being enqueued into the queue.C. Propagation ParametersParameter & RecommendationValuesCommentlatency=5Default: 60 Maximum wait, in seconds, in the propagation window for a message to be propagated after it is enqueued.The default value is 60. Caution: if latency is not specified for this call, then latency will over-write any existing value with this default value (60). For example, if the latency is 60 seconds, then during the propagation window, if there are no messages to be propagated, then messages from that queue for the destination will not be propagated for at least 60 more seconds. It will be at least 60 seconds before the queue will be checked again for messages to be propagated for the specified destination. If the latency is 600, then the queue will not be checked for 10 minutes and if the latency is 0, then a job queue process will be waiting for messages to be enqueued for the destination and as soon as a message is enqueued it will be propagated.Propagation parameters can be set using the ALTER_PROPAGATION_SCHEDULE procedure from the DBMS_AQADM package. For example, to set the latency parameter of the streams propagation from the STREAMS_QUEUE owned by STRMADMIN to the target database whose global_name is DEST_DB for the queue Q1, use the following syntax while logged in as the Streams Administrator:dbms_aqadm.alter_propagation_schedule('strmadmin.streams_queue','DEST_DB',destination_queue=>'Q1',latency=>5);D. Network Connectivity When using Streams propagation across a Wide Area Network (WAN), increase the session data unit (SDU) to improve the propagation performance. The maximum value for SDU is 32K (32767). The SDU value for network transmission is negotiated between the sender and receiver sides of the connection: the minimum SDU value of the two endpoints is used for any individual connection. In order to take advantage of an increased SDU for Streams propagation, the receiving side sqlnet.ora file must include the default_sdu_size parameter. The receiving side listener.ora must indicate the SDU change for the SID. The sending side tnsnames.ora connect string must also include the SDU modification for the particular service. Tuning the tcp/ip networking parameters can significantly improve performance across the WAN. Here are some example tuning parameters for Linux. These parameters can be set in the /etc/sysctl.conf file and running sysctl -p . When using RAC, be sure to configure this at each instance.net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 # increase Linux autotuning TCP buffer limits # min, default, and max # number of bytes to use net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216In addition, the SEND_BUF_SIZE and RECV_BUF_SIZE sqlnet.ora parameters increase the performance of propagation on your system. These parameters increase the size of the buffer used to send or receive the propagated messages. These parameters should only be increased after careful analysis on their overall impact on system performance.For further information, please review the Oracle Net Services Guide5.9. Additional Configuration for RAC Environments for a Source DatabaseArchive Logs The archive log threads from all instances must be available to any instance running a capture process. This is true for both local and downstream capture.Queue OwnershipWhen Streams is configured in a RAC environment, each queue table has an "owning" instance. All queues within an individual queue table are owned by the same instance. The Streams components (capture/propagation/apply) all use that same owning instance to perform their work. This means thata capture process is run at the owning instance of the source queue.a propagation job must run at the owning instance of the queuea propagation job must connect to the owning instance of the target queue.Ownership of the queue can be configured to remain on a specific instance, as long as that instance is available, by setting the PRIMARY _INSTANCE and/or SECONDARY_INSTANCE parameters of DBMS_AQADM.ALTER_QUEUE_TABLE. If the primary_instance is set to a specific instance (ie, not 0), the queue ownership will return to the specified instance whenever the instance is up.Capture will automatically follow the ownership of the queue. If the ownership changes while capture is running, capture will stop on the current instance and restart at the new owner instance. For queues created with Oracle Database 10g Release 2, a service will be created with the service name= schema.queue and the network name SYS$schema.queue.global_name for that queue. If the global_name of the database does not match the db_name.db_domain name of the database, be sure to include the global_name as a service name in the init.ora. For propagations created with the Oracle Database 10g Release 2 code with the queue_to_queue parameter to TRUE, the propagation job will deliver only to the specific queue identified. Also, the source dblink for the target database connect descriptor must specify the correct service (global name of the target database ) to connect to the target database. For example, the tnsnames.ora entry for the target database should include the CONNECT_DATA clause in the connect descriptor for the target database. This clause should specify (CONNECT_DATA=(SERVICE_NAME='global_name of target database')). Do NOT include a specific INSTANCE in the CONNECT_DATA clause.For example, consider the tnsnames.ora file for a database with the global name db.mycompany.com. Assume that the alias name for the first instance is db1 and that the alias for the second instance is db2. The tnsnames.ora file for this database might include the following entries:db.mycompany.com=(description=(load_balance=on)(address=(protocol=tcp)(host=node1-vip)(port=1521))(address=(protocol=tcp)(host=node2-vip)(port=1521))(connect_data=(service_name=db.mycompany.com)))db1.mycompany.com=(description=(address=(protocol=tcp)(host=node1-vip)(port=1521))(connect_data=(service_name=db.mycompany.com)(instance_name=db1)))db2.mycompany.com=(description=(address=(protocol=tcp)(host=node2-vip)(port=1521))(connect_data=(service_name=db.mycompany.com)(instance_name=db2)))Use the italicized tnsnames.ora alias in the target database link USING clause.DBA_SERVICES lists all services for the database. GV$ACTIVE_SERVICES identifies all active services for the database In non_RAC configurations, the service name will typically be the global_name. However, it is possible for users to manually create alternative services and use them in the TNS connect_data specification . For RAC configurations, the service will appear in these views as SYS$schema.queue.global_name.Propagation RestartUse the procedures START_PROPAGATION and STOP_PROPAGATION from DBMS_PROPAGATION_ADM to enable and disable the propagation schedule. These procedures automatically handle queue_to_queue propagation. Example:exec DBMS_PROPAGATION_ADM.STOP_PROPAGATION('name_of_propagation'); or exec DBMS_PROPAGATION_ADM.STOP_PROPAGATION('name_of_propagation',force=>true);exec DBMS_PROPAGATION_ADM.START_PROPAGATION('name_of_propagation');6.0 Target Site ConfigurationThe following recommendations apply to target databases, ie, databases in which Streams apply is configured.6.1. PrivilegesGrant Explicit Privileges to APPLY_USER for the user tablesExamples:Privileges for table level DML: INSERT/UPDATE/DELETE, Privileges for table level DDL: CREATE (ANY) TABLE , CREATE (ANY) INDEX, CREATE (ANY) PROCEDURE6.2. InstantiationSet Instantiation SCNs manually if not using export/import. If manually configuring the instantiation scn for each table within the schema, use the RECURSIVE=>TRUE option on the DBMS_APPLY_ADM.SET_SCHEMA_INSTANTIATION_SCN procedureFor DDL Set Instantiation SCN at next higher level (ie, SCHEMA or GLOBAL level).6.3. Conflict ResolutionIf updates will be performed in multiple databases for the same shared object, be sure to configure conflict resolution. See the Streams Replication Administrator's Guide Chapter 3 Streams Conflict Resolution, for more detail.To simplify conflict resolution on tables with LOB columns, create an error handler to handle errors for the table. When registering the handler using the DBMS_APPLY_ADM.SET_DML_HANDLER procedure, be sure to specify the ASSEMBLE_LOBS parameter as TRUE. Refer to Document  779801.1 Streams Conflict Resolution6.4. Apply Process ConfigurationA. RulesIf the maintain_* procedures are not suitable for your environment, please use the ADD_*_RULES procedures (ADD_TABLE_RULES , ADD_SCHEMA_RULES , ADD_GLOBAL_RULES (for DML and DDL), ADD_SUBSET_RULESAPPLY can be configured with or without a ruleset. The ADD_GLOBAL_RULES can be used to apply all changes in the queue for the database. If no ruleset is specified for the apply process, all changes in the queue are processed by the apply process.A single Streams apply can process rules for multiple tables or schemas located in a single queue that are received from a single source database . For best performance, rules should be simple. Rules that include LIKE clauses are not simple and will impact the performance of Streams. To eliminate changes for particular tables or objects, specify the include_tagged_lcr clause along with the table or object name in the negative rule set for the Streams process. Setting this clause will eliminate all changes, tagged or not, for the table or object.B. ParametersParameterValuesCommentDISABLE_ON_ERROR=N Default: YIf Y, then the apply process is disabled on the first unresolved error, even if the error is not fatal.If N, then the apply process continues regardless of unresolved errors.PARALLELISM= 4 Default: 1 Parallelism configures the number of apply servers available to the apply process for performing user transactions from the source database. Choose a value 4, 8, 12, 16 based on the concurrent replicated workload generated at the source AND the number of CPUs available at the target. TXN_LCR_SPILL_THRESHOLDDefault=10,000New in 10.2. Leave this parameter as default initially. It enables you to specify that an apply process begins to spill messages for a transaction from memory to disk when the number of messages in memory for a particular transaction exceeds the specified number. Setting this parameter to a value that is higher than the default to try to stage everything in memory must be done carefully so that queue spilling is not increased. Setting TXN_LCR_SPILL_THRESHOLD to 'infinite' is not recommended because this will revert Streams to the old pre-10.2 behaviour.The DBA_APPLY_SPILL_TXN and V$STREAMS_APPLY_READER views enable you to monitor the number of transactions and messages spilled by an apply process.Refer to Document 365648.1 Explain TXN_LCR_SPILL_THRESHOLD in Oracle10GR2 StreamsApply parameters can be set using the SET_PARAMETER procedure from the DBMS_APPLY_ADM package. For example, to set the DISABLE_ON_ERROR parameter of the streams apply process named APPLY_EX, use the following syntax while logged in as the Streams Administrator:exec dbms_apply_adm.set_parameter('apply_ex','disable_on_error','n');In some cases, performance can be improved by setting the following hidden parameter. This parameter should be set when the major workload is UPDATEs and the updates are performed on just a few columns of a many-column table. ParameterValuesComment_DYNAMIC_STMTS=YDefault: N If Y, then for UPDATE statements, the apply process will optimize the generation of SQL statements based on required columns._HASH_TABLE_SIZE=1000000Default: 80*parallelismSet the size of the hash table used to calculate transaction dependencies to 1 million.6.5. Additional Configuration for RAC Environments for an Apply DatabaseQueue OwnershipWhen Streams is configured in a RAC environment, each queue table has an "owning" instance. All queues within an individual queue table are owned by the same instance. The Streams components (capture/propagation/apply) all use that same owning instance to perform their work. This means that the database link specified in the propagation must connect to the owning instance of the target queue.the apply process is run at the owning instance of the target queueOwnership of the queue can be configured to remain on a specific instance, as long as that instance is available, by setting the PRIMARY _INSTANCE and SECONDARY_INSTANCE parameters of DBMS_AQADM.ALTER_QUEUE_TABLE. If the primary_instance is set to a specific instance (ie, not 0), the queue ownership will return to the specified instance whenever the instance is up.Apply will automatically follow the ownership of the queue. If the ownership changes while apply is running, apply will stop on the current instance and restart at the new owner instance. Changing the GLOBAL_NAME of the DatabaseSee the OPERATION section on Global_name below. The following are some additional considerations when running in a RAC environment. If the GLOBAL_NAME of the database is changed, ensure that the queue is empty before changing the name and that the apply process is dropped and recreated with the apply_captured parameter = TRUE. In addition, if the GLOBAL_NAME does not match the db_name.db_domain of the database, include the GLOBAL_NAME in the list of services for the database in the database parameter initialization file.OPERATIONA Streams process will automatically restart after a database startup, assuming that the process was in a running state before the database shut down. No special startup or shutdown procedures are required in the normal case.Global NameStreams uses the GLOBAL_NAME of the database to identify changes from or to a particular database. Do not modify the GLOBAL NAME of a Streams database after capture has been created. Changes captured by the Streams capture process automatically include the current global name of the source database. This means that if the global name is modified after a capture process has been configured, the capture process will need to be dropped and recreated following the GLOBAL_NAME modification. In addition, the system-generated rules for capture, propagation, and apply typically specify the global name of the source database. These rule will need to be modified or recreated to adjust the source_database_name. Finally, if the GLOBAL_NAME does not match the db_name.db_domain of the database, include the GLOBAL_NAME in the list of services for the database in the database parameter initialization file.If the global name must be modified on the database, do it at a time when NO user changes are possible on the database and the Streams queues are empty with no outstanding changes to be applied, so that the Streams configuration can be recreated. Keep in mind that all subscribers (propagations to target databses and the target apply processes) must also be recreated if the source database global_name is changed. Follow the directions in the Streams Replication Administrator's Guide for Changing the DBID or GLOBAL NAME of a source database. It is also strongly recommended that the database init.ora parameter global_names be set to TRUE to guarantee that database link names match the global name of the target database.Apply Error ManagementThe view DBA_APPLY_ERROR includes the message_number within the transaction on which the reported error occurred. Use this message number in conjunction with the procedures from the documentation manual Streams Concepts and Administration ( Chapter 22 Monitoring Streams Apply Processes "Displaying Detailed Information About Apply Errors") to print out the column values of each logical change record within the failed transaction. Backup Considerations1. Ensure that any manual backup procedures that include the any of the following statements include a non-null Streams tag:ALTER TABLESPACE ... BEGIN BACKUPALTER TABLESPACE ... END BACKUPThe tag should be chosen such that these DDL commands will be ignored by the capture rule set. To set a streams tag, use the DBMS_STREAMS.SET_TAG procedure. A non-null tag should be specified to avoid capturing these commands.Backups performed using RMAN do not need to set a Streams tag.2. Do not allow any automated backup of the archived logs to remove necessary archive logs. It is especially important in a Streams environment that all necessary archived logs remain available online and in the expected location until the capture process has finished processing them. If a log required by the capture process is unavailable, the capture process will abort. Force a checkpoint (capture/logminer) before beginning the manual backup procedures. To force a checkpoint, explicitly reset the hidden capture parameter _CHECKPOINT_FORCE to 'Y'. The REQUIRED_CHECKPOINT_SCN column of the DBA_CAPTURE view specifies the lowest required SCN to restart capture. A procedure to determine the minimum archive log necessary for successful capture restart is available in the Streams health check script.3. Ensure that all archive logs (from all threads) are available. Database recovery depends on the availability of these logs, and a missing log will result in incomplete recovery.4. Ensure that the APPLY process parameter, COMMIT_SERIALIZATION, is set to the default value, FULL.5. Implement a "heartbeat" table. To ensure that the applied_scn of the DBA_CAPTURE view is updated periodically, implement a "heart beat" table. Implementing a heartbeat table ensures that the metadata is updated frequently. Additionally, the heartbeat table provides quick feedback as to the health of streams replication. Refer to the Source Site Configuration Section: Implement a Hearbeat Table for more details.6. In situations that result in incomplete recovery (Point-in-Time recovery) at the source site, follow the instructions in Chapter 9 of the Streams Replication Administrators Guide Performing Point-in-Time Recovery on the Source in a Single-Source EnvironmentPerforming Point-in-Time Recovery in a Multiple-Source Environment7. In situations that result in incomplete recovery at the destination site, follow the instructions in Chapter 9 of the Streams Replication Administrator's Guide Performing Point-in-Time Recovery on a Destination DatabaseBatch ProcessingFor best performance, the commit point for batch processing should be kept low. It is preferable that excessively large batch processing be run independently at each site. If this technique is utilized, be sure to implement DBMS_STREAMS.SET_TAG to skip the capture of batch processing session. Setting this tag is valid only in the connected session issuing the set_tag command and will not impact the capture of changes from any other database sessions.DDL ReplicationWhen replicating DDL, keep in mind the effect the DDL statement will have on the replicated sites. In particular, do not allow system generated naming for constraints or indexes, as modifications to these will most likely fail at the replicated site. Also, storage clauses may cause some issues if the target sites are not identical.If you decide NOT to replicate DDL in your Streams environment, any table structure change must be performed manually. Refer to Document 313478.1 Performing Manual DDL in a Streams EnvironmentPropagationAt times, the propagation job may become "broken" or fail to start after an error has been encountered or after a database restart. The typical solution is to disable the propagation and then re-enable it. exec dbms_propagation_adm.stop_propagation('propagation_name');exec dbms_propagation_adm.start_propagation('propagation_name');If the above does not fix the problem, perform a stop of propagation with the force parameter and then start propagation again. exec dbms_propagation_adm.stop_propagation('propagation_name',force=>true);exec dbms_propagation_adm.start_propagation('propagation_name');An additional side-effect of stopping the propagation with the force parameter is that the statistics for the propagation are clearedThe above is documented in the Streams Replication Administrator's Guide: Restart Broken PropagationsSource Queue GrowthSource queue may grow if one of the target sites is down for an extended period, or propagation is unable to deliver the messages to a particular target site (subscriber) due to network problems for an extended period. Automatic flow control minimizes the impact of this queue growth. Queued messages (LCRs) for unavailable target sites will spill to disk storage while messages for available sites are processed normally.Propagation is implemented using the DBMS_JOB subsystem. If a job is unable to execute 16 successive times, the job will be marked as "broken" and become disabled. Be sure to periodically check that the job is running successfully to minimize source queue growth due to this problem. Streams Cleanup/RemovalRemoving the Streams administrator schema with DROP USER ..... CASCADE can be used to remove the entire Streams configuration. Automatic Optimizer Statistics CollectionOracle database 10g has the Automatic Optimizer Statistics Collection feature that runs every night and gathers optimizer stats of tables whose stats have become stale. The problem with volatile tables, such as the Streams queue tables, is that it is quite possible that when the stats collection job runs these tables may not have data that is representative of their full load period. For this reason we recommend to customers that for volatile tables, they run the dbms_stats.gather job manually on them when they are at the fullest and then immediately lock the stats of using the PL/SQL API's (dbms_stats.lock ...) provided. This will ensure that when the nightly Automatic Optimizer Statistics Collection job runs, these volatile tables will be skipped and hence not analyzed.These volatile AQ/Streams tables are created through a call to dbms_aqadm.create_queue_table (qtable_name, etc.) or dbms_streams_adm.setup_queue() command with a user defined queue table (qtable_name). In addition to the queue table, the call internally creates the following tables which also tend to be volatile:aq$_{qtable_name}_iaq$_{qtable_name}_haq$_{qtable_name}_taq$_{qtable_name}_paq$_{qtable_name}_daq$_{qtable_name}_cOracle has the ability to restore old stats on tables including data dictionary tables using the dbms_stats.restore... API's. This feature can be used for short term resolution, but the real solution is the first one, where you lock optimizer stats of volatile tables.MONITORINGAll Streams processing is done at the "owning instance" of the queue. To determine the owning instance, use the query below:SELECT q.owner, q.name, t.queue_table, t.owner_instanceFROM DBA_QUEUES q, DBA_QUEUE_TABLES tWHERE t.object_type = 'SYS.ANYDATA' ANDq.queue_table = t.queue_table ANDq.owner = t.owner;To display the monitoring view information, either query the monitoring views from the owning instance or use the GV$ views for dynamic streams views. Dynamic Streams views The views listed below with larger size font are the most commonly monitored runtime views in Streams. The hyperlinks below link to the view descriptions in the Oracle Database 10g Release 2 Database Reference manual.Streams View Name Streams View Name from any RAC instanceV$STREAMS_CAPTUREGV$STREAMS_CAPTUREV$STREAMS_APPLY_COORDINATORGV$STREAMS_APPLY_COORDINATORV$STREAMS_APPLY_READERGV$STREAMS_APPLY_READERV$STREAMS_APPLY_SERVERGV$STREAMS_APPLY_SERVERV$STREAMS_POOL_ADVICEGV$STREAMS_POOL_ADVICEV$STREAMS_TRANSACTIONGV$STREAMS_TRANSACTIONV$BUFFERED_PUBLISHERSGV$BUFFERED_PUBLISHERSV$BUFFERED_QUEUESGV$BUFFERED_QUEUESV$BUFFERED_SUBSCRIBERSGV$BUFFERED_SUBSCRIBERSV$PROPAGATION_RECEIVERGV$PROPAGATION_RECEIVERV$RULEGV$RULEV$RULE_SETGV$RULE_SETV$RULE_SET_AGGREGATE_STATSGV$RULE_SET_AGGREGATE_STATSStatic Streams ViewsThe views listed below with larger size font are the most commonly monitored configuration views in Streams. The hyperlinks below link to the view descriptions in the Oracle Database 10g Release 2 Database Reference manual.Streams ViewsDBA_REGISTERED_ARCHIVED_LOGDBA_RECOVERABLE_SCRIPTDBA_RECOVERABLE_SCRIPT_BLOCKSDBA_RECOVERABLE_SCRIPT_ERRORSDBA_RECOVERABLE_SCRIPT_PARAMSDBA_STREAMS_ADD_COLUMNDBA_STREAMS_ADMINISTRATORDBA_STREAMS_DELETE_COLUMNDBA_STREAMS_GLOBAL_RULESDBA_STREAMS_MESSAGE_CONSUMERSDBA_STREAMS_MESSAGE_RULESDBA_STREAMS_NEWLY_SUPPORTEDDBA_STREAMS_RENAME_COLUMNDBA_STREAMS_RENAME_SCHEMADBA_STREAMS_RENAME_TABLEDBA_STREAMS_RULESDBA_STREAMS_SCHEMA_RULESDBA_STREAMS_TABLE_RULESDBA_STREAMS_TRANSFORM_FUNCTIONDBA_STREAMS_TRANSFORMATIONSDBA_STREAMS_UNSUPPORTEDDBA_RULE_SET_RULESDBA_RULE_SETSDBA_RULESDBA_HIST_BUFFERED_QUEUESDBA_HIST_BUFFERED_SUBSCRIBERSDBA_HIST_RULE_SETDBA_HIST_STREAMS_APPLY_SUMDBA_HIST_STREAMS_CAPTUREDBA_HIST_STREAMS_POOL_ADVICECapture Views:DBA_CAPTUREDBA_CAPTURE_EXTRA_ATTRIBUTESDBA_CAPTURE_PARAMETERSDBA_CAPTURE_PREPARED_DATABASEDBA_CAPTURE_PREPARED_SCHEMASDBA_CAPTURE_PREPARED_TABLESPropagation & Queue ViewsDBA_PROPAGATIONDBA_QUEUE_SCHEDULESDBA_QUEUE_SUBSCRIBERSDBA_QUEUE_TABLESDBA_QUEUESApply Views:DBA_APPLYDBA_APPLY_CONFLICT_COLUMNSDBA_APPLY_DML_HANDLERSDBA_APPLY_ENQUEUEDBA_APPLY_ERRORDBA_APPLY_EXECUTEDBA_APPLY_INSTANTIATED_GLOBALDBA_APPLY_INSTANTIATED_OBJECTSDBA_APPLY_INSTANTIATED_SCHEMASDBA_APPLY_KEY_COLUMNSDBA_APPLY_OBJECT_DEPENDENCIESDBA_APPLY_PARAMETERSDBA_APPLY_PROGRESSDBA_APPLY_SPILL_TXNDBA_APPLY_TABLE_COLUMNSDBA_APPLY_VALUE_DEPENDENCIESMonitoring Utility STRMMONSTRMMON is a monitoring tool focused on Oracle Streams. Using this tool, database administrators get a quick overview of the Streams activity occurring within a database. In a single line display, strmmon reports information The reporting interval and number of iterations to display are configurable. STRMMON is available in the rdbms/demo directory in $ORACLE_HOME. The most recent version of the tool is available from Document 290605.1 Oracle Streams STRMMON Monitoring UtilityAlert LogStreams capture and apply processes report long-running and long transactions in the alert log.Long-running transactions are open transactions with no activity( ie, no new change records , rollback or commit ) for an extended period (20 minutes). Large transactions are open transactions with a large number of change records. The alert log will report the fact that a long-running or large transaction has been seen every 20 minutes. Not all such transactions will be reported - only 1 per 10 minute period. When the commit or rollback is received, this fact will be reported in the alert log as well.Streams Healthcheck ScriptsThe Streams health check script is a collection of queries to determine the configuration of the streams environment. This script should be run at each participating database in a streams configuration. In addition to configuration information, analysis of the rules specified for streams is included to enable quicker diagnosis of problems. A guide to interpreting the output is provided. The healthcheck script is an invaluable tool for problem solving customer issues. The Streams Healthcheck script is available from Document 273674.1 Streams Configuration Report and Health Check ScriptTo browse through the complete list of streams published articles refer to Knowledge > Browse. Then select Oracle Technology -> Database -> Information Integration -> Streams.To learn about Oracle University offerings related to Oracle Streams, refer to Document 762188.1 Oracle University Offerings Related to Oracle Streams.ReferencesNOTE:265201.1 - Master Note for Troubleshooting Streams Apply Errors ORA-1403, ORA-26787 or ORA-26786NOTE:335516.1 - Master Note for Streams Performance RecommendationsNOTE:789445.1 - Master Note for Streams Setup ScriptsNOTE:1264598.1 - Master Note for Streams Downstream Capture - 10g and 11g [Video]NOTE:313279.1 - Master Note for Troubleshooting Streams capture 'WAITING For REDO' or INITIALIZINGNOTE:779801.1 - Streams Conflict ResolutionNOTE:290605.1 - Oracle Streams STRMMON Monitoring UtilityNOTE:730036.1 - Overview for Troubleshooting Streams Performance IssuesNOTE:437838.1 - Streams Specific PatchesNOTE:273674.1 - Streams Configuration Report and Health Check ScriptNOTE:259609.1 - Script to Prevent Excessive Spill of Message From the Streams Buffer Queue To DiskNOTE:365648.1 - Explain TXN_LCR_SPILL_THRESHOLD in Oracle10GR2 StreamsNOTE:782541.1 - Streams Replication Supplemental Logging RequirementsNOTE:313478.1 - Performing Manual DDL in a Streams EnvironmentNOTE:461278.1 - Example of a Streams Heartbeat Table

Master Note for Streams Recommended Configuration (Doc ID 418755.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved.  In this Document   Purpose   Scope and Application   Master Note for...

Master Notes

Master Note: Oracle Lite Message Generating Process (MGP) (Doc ID 1160817.1)

Applies to: Oracle Lite - Version: 10.0.0.0.1 to 10.3.0.3.0 - Release: 10.0 to 10.3Information in this document applies to any platform.What is being announced?The following documents may prove useful to customers concerning the Message Generating Process (MGP):Configuration:Configure MGP to Run with Multiple Mobile Servers in a Farm Using the Same Repository Document ID: 359179.1MGP No Longer Runs After Host System IP Address is Changed - Document ID: 953532.1Performance:Using CONSPERF to Tune the SYNC and MGP Process for an Application - Document ID: 254610.1Mgp Process Experiences Poor Performance And High Cpu Usage With A 10.2 Repository DB - Document ID: 438912.1Performance Issues with the MGP - Document ID: 403963.1Slow MGP Execution when Publication Items Contain Clob/Lob Fields - Document ID: 783838.1Apply Cycle of MGP Stops For a Long Time (minutes) - Document ID: 578123.1MGP Locking Issues With a 3RD Party Application - Document ID: 853433.1Every 100 cycle MGP does some clean up work - Document ID: 799446.1Why Does the MGP Randomly Take Longer to Run Sometimes - Document ID: 1148538.1MGP Process Hangs With Table Lock MOBILEADMIN.C$ALL_CLIENT_ITEMS - Document ID: 433835.1Mgp Compose Cycle Runs, Does Nothing And Exits Without Errors - Document ID: 760798.1Administration:Monitor MGP and SYNC Failures Using SQL Script - Document ID: 415647.1Errors:MGP Compose Postponed Because of Unprocessed Data in the In_queue - Document ID: 402423.1MGP compose timed out for Mobile_UserName: Document ID: 403990.1MGP Compose Fails With ORA-920 On Imported Repository - Document ID: 553952.1Avoid Receiving Error Ora-01000 During MGP Runs - Document ID: 375643.1MGP Fails with ORA-00947: Not Enough Values After Migrating to 10.3.0.2.0 - Document ID: 1072991.1MGP Fails with the Error: MGP Running Outside - Document ID: 309681.1MGP Errors with ORA-942 and ORA-904 During Synchronization - Document ID: 373997.1MGP Fails to Compose/Apply With the Error ORA-907 - Document ID: 551775.1ORA-00904: "L"."CLID$$OWN" DURING MGP COMPOSE - Document ID: 887000.1

Applies to: Oracle Lite - Version: 10.0.0.0.1 to 10.3.0.3.0 - Release: 10.0 to 10.3 Information in this document applies to any platform.What is being announced? The following documents may prove...

Master Notes

Master Note: Best Practices for Oracle Lite 10g (Doc ID 405572.1)

Applies to: Oracle Lite - Version: 10.2.0.1.0 to 10.3.0.1.0 - Release: 10.2 to 10.3Information in this document applies to any platform.Affected Database: OracleWhat is being announced?This article is to reference all useful notes in Oracle Lite and help to make a best approach in Oracle Lite.This is a complement of DocumentationOracle Database Lite Getting Started Guide10g (10.2.0) and 10g (10.3.0)What do you need to do?Notes and Documentations: Understand the synchronization side Inqueue and Outqueue*Reference in documentation Oracle Lite 10.2:Oracle Database Lite Administration and Deployment Guide10g (10.2.0)Part No. B15921-026.1 How Does the Synchronization Process Work *Reference in documentation Oracle Lite 10.3:Oracle Database Lite Administration and Deployment Guide10g (10.3.0)Part Number B28922-015.1 How Does the Synchronization Process WorkSet the trace to help Oracle SupportDocument 307543.1 How To Setup Tracing With Oracle Lite**Reference in documentation Oracle Lite 10.2 :Oracle Database Lite Administration and Deployment Guide10g (10.2.0)Part No. B15921-026 Managing SynchronizationSection 6.3, "Managing the Sync Server"Section 6.7, "Managing Trace Settings and Trace Files" *Reference in documentation Oracle Lite 10.3:Oracle Database Lite Administration and Deployment Guide10g (10.3.0)Part Number B28922-01Section 5 "Managing Synchronization"Section 5.8 "Managing Trace Settings and Trace Files"Document 97616.1 Configuration and Information Regarding the Oracle Lite ODBC Driver**Olite ODBC versions in Oracle Lite 10.2 and 10.3ODBC Driver Name ODBC API Standard VersionOracle Lite 40 ODBC Driver 10.02.00.01 or 10.03.00.01Oracle Lite 40 ODBC Driver (3.51) 10.02.00.02 ( if Oracle Lite patchset Oracle Lite 10.2.0.2.0) or 10.03.00.01Oracle Lite 40 ODBC Driver (Client) 10.02.00.01 or 10.03.00.01 LimitationsDocument 399831.1 Oracle Database Lite LimitationsAdministrationsDocument 429287.1 How To Check If Some Client Are SynchronizingDocument 400725.1 How To Obtain Sync And Sql Trace Files on Oracle Lite clientsDocument 412982.1 How To Install Different JRE On The Client Webtogo Setup.exeDocument 412785.1 How To Delete An Device From Mobile Server by Java API**Warning the java code API in Oracle Lite 10.3 can be deprecated during compilation.Document 234117.1 How to resolve the PoL-2000 error on a ClientDocument 464054.1 CNS-9026 Wrong username/Password - When you specify the correct oneDocument 463100.1 How to install samples demos without use repwizard toolDocument 473461.1 How Create Publication Items On Base Tables With Unsupported Data TypesDocument 463309.1 How To Create an new Administrator User Into and OID / SSO Document 413563.1 Changing Webtogo Port With Mobile Server Running into OAS 10.1.2.0.2Document 368353.1 How to Install Lite10G Client for Pocket PC on Windows Mobile 5 Device

Applies to: Oracle Lite - Version: 10.2.0.1.0 to 10.3.0.1.0 - Release: 10.2 to 10.3 Information in this document applies to any platform. Affected Database: OracleWhat is being announced? This article...

Master Notes

Master Note about The Mobile Device Manager and dmagent.exe (Doc ID 1111283.1)

Applies to: Oracle Lite - Version: 10.0.0.0.1 to 10.3.0.3.0 - Release: 10.0 to 10.3Information in this document applies to any platform.What is being announced?The goal of this article is to give a global picture about The Device Manager and dmagent.exe  in the Oracle Lite Architecture, this note contains the most usefull Note about them. In addition to the Support Notes below, you can get additional information here: My Oracle Support CommunityWhat do you need to do?Bug 5460773 - Device manager displays different device status in a multinode configuration Note 5460773.8 Device Manager Blocks Incoming Sms On Windows Ce Ppc 2003 Devices Note 456761.1 How to use Mobile Manager Device Commands to Initiate Client Upgrade.. Note 836644.1 Autosync Stopped, CNS-9024 and CNS-9032 in OL_SYNC.log Note 859854.1 How Do I Get My Client Device to Run Commands More Frequently Note 302492.1 Oracle Database Lite Statement of Support for Windows XP with Service Pack 2 Note 281097.1 Automatic Sync From Server To Client Does NoT Work Note 436695.1 Mobile client 10G R2 for Windows CE install fails with error code: HTTP/1.1 unauthorized:WIN32:401 Note 330687.1 How to Register the WCESTD500 Platform using the cab file olite.us.stdsdk500.armv4i_sdk.CAB Note 458674.1 Dmagent "Synchronize Database" Command Processing Frozen Note 855714.1 "Resource Not Found" During Sync For Some Clients Using Several Mobile Servers Note 823133.1 One-Off Patch, Not Installed Automatically By The Dmagent.Exe Note 735592.1 CONFIGURE MINOR PATCH UPDATE Note 350993.1 Dmagent not running and / or Device Not Registering With Mobile Server Note 386927.1 How To Set Enable - Disable Devices Via Java API Note 1063823.1 Resolving 'Device Is Not Enabled' Cons-10053 Errors Note 299906.1 ERROR CONS-10053 DeviceManager message: Device is not enabled Note 401503.1 HOW TO DELETE AN DEVICE FROM MOBILE SERVER BY JAVA API Note 412785.1 'Forced Refresh' Does Not Work When Sent As Command Note 302261.1 Client DMAGENT Fails To Register With The Mobile Server - WIN32: 403 Note 400511.1 Adding Parameters to Client Config Files For All New Clients:  Note 436892.1 Resolving 'Device Is Not Enabled' Cons-10053 Errors:  Note 299906.1Who to contact for more information?For more information, please open a thread in the Oracle Lite Community:https://communities.oracle.com/portal/server.pt/community/oracle_lite/274or add a remark/comment in this Note.

Applies to: Oracle Lite - Version: 10.0.0.0.1 to 10.3.0.3.0 - Release: 10.0 to 10.3 Information in this document applies to any platform.What is being announced? The goal of this article is to give a...

Master Notes

TimesTen PLSQL Documentation Library : Master Note Child (Doc Id 1133974.1)

Applies to: TimesTen Data Server - Version: 11.2.1.1.0 and later   [Release: 11.2 and later ]Information in this document applies to any platform.About:This is the Master Note Child for Oracle TimesTen In-Memory Database Documentation.Purpose:This note will list PLSQL Notes in My Oracle Support for Oracle TimesTen In-Memory Database.Usage:To view or download a document please click on the most appropriate note below.Click Note 1088128.1 for the Oracle TimesTen In-Memory Database Master Note Parent.Note 794395.1 HOWTO : Resolve TimesTen Errors TT0970, TT8507 And ORA-6512 Running PL/SQL Cursor Loop In TimesTenNote 885044.1 HOWTO : Resolve TimesTen Error TT8517 - Cannot Attach PL/SQL Shared Memory; PLSQL_MEMORY_ADDRESS Not ValidNote 986922.1 HOWTO : Resolve Error ORA-00904 When Using A TT-specific Expression In PL/SQLNote 958385.1 HOWTO : Resolve Error ORA-4031 Trapped By TimesTen Applications Executing PL/SQLNote 957957.1 HOWTO : Use A TimesTen PL/SQL Procedure To Insert Records Into A TableNote 1080514.1 HOWTO : Call A PLSQL Function In TimesTenNote 1072638.1 HOWTO : Simulate Oracle Commit Inside PL/SQL Cursor With TimesTenNote 1053796.1 HOWTO : Size The TimesTen PLSQL Memory Segment CorrectlyNote 1052910.1 HOWTO : Resolve TimesTen Error 8521 - Cannot Attach PL/SQL Shared Memory

Applies to: TimesTen Data Server - Version: 11.2.1.1.0 and later   [Release: 11.2 and later ] Information in this document applies to any platform.About:This is the Master Note Child for Oracle...

Master Notes

Master Note for OLTP Compression

++++++++++++++++++++++++++++++++++++++++++++++++++++The complete and the most recent version of this article can be viewedfrom My Oracle Support Knowledge Section.Master Note for OLTP Compression (Doc ID 1223705.1)++++++++++++++++++++++++++++++++++++++++++++++++++++Master Note for OLTP Compression [ID 1223705.1]  Modified 18-NOV-2010     Type BULLETIN     Status PUBLISHED In this Document  Purpose  Scope and Application  Master Note for OLTP Compression     Concepts     Compression Effectiveness and Performance     Concurrency / Locking     Administration     Limitations / Restrictions     Testing and Results     Waits     Statistics     Bugs     Notes     Other ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.0. - Release: 11.1 to 11.2Information in this document applies to any platform.PurposeThis document describes the 11g new feature OLTP Compression and provides usage guidelines that help to maximize compression ratio and performance. Scope and ApplicationThe Oracle11g Advanced Compression Option includes OLTP Compression, SecureFiles Deduplication and Compression, Recovery Manager Compression, Data Pump Compression, and DataGuard redo compression. OLTP Compression is one of several types of heap segment compression (HSC) and as it is important to share common terminology for this topic an excerpt from the manual is provided below for clarity.    * When you enable table compression by specifying either COMPRESS or COMPRESS BASIC, you enable basic table compression. Oracle Database attempts to compress data during direct-path INSERT operations when it is productive to do so. The original import utility (imp) does not support direct-path INSERT, and therefore cannot import data in a compressed format.      In earlier releases, this type of compression was called DSS table compression and was enabled using COMPRESS FOR DIRECT_LOAD OPERATIONS. This syntax has been deprecated.      See Also: "Conventional and Direct-Path INSERT" for information on direct-path INSERT operations, including restrictions    * When you enable table compression by specifying COMPRESS FOR OLTP, you enable OLTP table compression. Oracle Database compresses data during all DML operations on the table. This form of compression is recommended for OLTP environments.      In earlier releases, OLTP table compression was enabled using COMPRESS FOR ALL OPERATIONS. This syntax has been deprecated.      Note: Tables with COMPRESS or COMPRESS BASIC use a PCTFREE value of 0 to maximize compression, unless you explicitly set a value for PCTFREE in the physical_attributes_clause. Tables with COMPRESS FOR OLTP or NOCOMPRESS use the PCTFREE default value of 10, to maximize compress while still allowing for some future DML changes to the data, unless you override this default explicitly.    * When you specify COMPRESS FOR QUERY or COMPRESS FOR ARCHIVE, you enable hybrid columnar compression.  With hybrid columnar compression, data can be compressed during bulk load operations. During the load process, data is transformed into a column-oriented format and then compressed. Oracle Database uses a compression algorithm appropriate for the level you specify. In general, the higher the level, the greater the compression ratio. Hybrid columnar compression can result in higher compression ratios, at a greater CPU cost. Therefore, this form of compression is recommended for data that is not frequently updated.COMPRESS FOR QUERY is useful in data warehousing environments. Valid values are LOW and HIGH, with HIGH providing a higher compression ratio. The default is HIGH.COMPRESS FOR ARCHIVE uses higher compression ratios than COMPRESS FOR QUERY, and is useful for compressing data that will be stored for long periods of time. Valid values are LOW and HIGH, with HIGH providing the highest possible compression ratio. The default is LOW.See Also: Oracle Exadata Storage Server Software documentation for more information on hybrid columnar compression, which is a feature of Oracle Exadata.This document focuses on OLTP compression and only briefly discusses basic compression as it applies to tables using OLTP compression.Master Note for OLTP CompressionConceptsOLTP compression differs from basic compression found in prior versions as it allows compression during DML operations and not just during direct path loads. When a table is set to use OLTP compression and rows are inserted into a new block in that table they are inserted uncompressed, just as with a non-compressed table. When enough inserts cause that block to be considered full then it is customarily not available for further inserts. At this time compression is attempted on the block. If the compression gains sufficient free space then the compression is retained and the block is still a candidate for further insertions. If the compression fails to gain sufficient free space then that compression attempt is discarded and the block is considered full. The block itself contains the symbol table that facilitates the one time storage of repeated values. Each block's compression is completely self-contained; no information from outside the block is required to access the information in the block. When a block is accessed it is not decompressed, it remains compressed when it is read into the buffer cache. This extends the benefits of the data compression to allow more rows to be retained in the buffer cache without having to increase the memory allocation. This also means that reading a compressed block has no additional overhead when compared to a non-compressed block.Each time a block is compressed successfully an image of the block prior to compression must be retained in undo for consistent read and transaction rollback. For this reason the amount of undo created when updating a table set for OLTP compression will be greater than the amount of undo created for a non-compressed table. Likewise, whatever is written to undo is also written to the redo logs so there is a corresponding increase in redo when OLTP compression is used.Generally updates involve the same operations as inserts. Like inserts, updates are not immediately compressed. When updating an already compressed block any columns that are not updated are not uncompressed, they continue to use the symbol table and retain their compression. Updated columns are stored initially as with any uncompressed block. When the block approaches full then it compression is attempted with the same implications as with the inserts described above.Deletes from a compressed block are identical to deletes from a non-compressed block, there is no change to consider. If the deletes cause a block that was full to have sufficient free space then the block will be considered again for new inserts.As blocks are retained in compressed form in the buffer cache it is expected that queries and deletes will function at least as quickly and may gain in performance due to fewer physical reads. Inserts and updates will often have no additional overhead, but on occasion will trigger a compression of a block. At that time there is an increase in cpu consumption and if the compression is successful then a corresponding increase in undo and redo generation.Compression Effectiveness and PerformanceThe concepts described above can be used to intuit factors that will improve or degrade the effectiveness and performance of OLTP compression. Some of those factors are listed below and are verified by focused testcases that manipulate those factors and monitor the results.Improving Compression Ratio Use larger blocks.Increase data redundancy within a given block (order of load, partitioning).Offload non-redundant columns (particularly larger columns).Use direct path loads when possible.Improving PerformanceWhen loading many rows use direct path inserts (one compression per block instead of many).Order inserts to take best advantage of redundancy.Leave sufficient free space (PCTFREE) so that updates that lengthen the row fit within the block.Improving the compression ratio will lead to fewer physical reads and improved performance particularly with queries and deletes.Performance is expected to improve for queries and deletes due to the reduction in physical reads and having no additional overhead for those operations. Concurrency may suffer due to increased rows per block and so should be monitored and addressed if needed. Performance for updates can benefit from fewer physical reads but there is occasional overhead when blocks must be compressed. Performance for inserts has the occasional overhead of block compression as well so it is important to use direct path when possible and otherwise to minimize the number of times a given block is compressed.Query optimization via the cost-based optimizer is the same as with non-compressed tables. The statistics for a compressed table however are different from the same table in non-compressed form and so execution plan differences can occur and are to be expected.Concurrency / LockingLocking works the same with compressed and uncompressed blocks except that there are often far more rows per block due to the benefits of compression. Consideration should be given for the number of active transactions that should be expected in a given block at any moment. If ITL waits are observed then increase the ITLs in the block. This can be accomplished by raising PCTFREE so that blocks are not completely filled by updates that increase row length, and in so doing prevent row migration as well. Another option is to raise the INITRANS to force a minimum number of ITLs in each block.If there is contention during inserts with too many sessions attempting to insert into the same blocks, and if MSSM (Manual Segment Space Management) is used, then consider raising the number of freelists. This will spread the inserts over more blocks and reduce contention.AdministrationTable attributes for compression can be changed by altering the table and when this is done new activity will honor that setting. Existing blocks are not immediately modified so if that is needed then the table must be reorganized; alter table move is one common way to do this. The [dba|user|all]_tables views have two columns for compression, 1) "COMPRESSION" which indicates if the table is compressed, and "COMPRESS_FOR" which indicates the type of compression for the table.Limitations / Restrictions[OLTP] Table compression is subject to the following restrictions:    * COMPRESS FOR OLTP and COMPRESS BASIC are not supported for tables with more than 255 columns.    * You cannot drop a column from a table that uses COMPRESS BASIC, although you can set such a column as unused. All of the operations of the ALTER TABLE ... drop_column_clause are valid for tables that use COMPRESS FOR OLTP, COMPRESS FOR QUERY, and COMPRESS FOR ARCHIVE.    * If you specify COMPRESS FOR OLTP, then chained rows are not compressed unless the header for the row remains in the original block and all row columns are moved to another block. If the row chaining results in leaving just the row header in the block and moving all of the row's columns to the next block, and they all fit in the next block, then the columns can be compressed.    * You cannot specify any type of table compression for an index-organized table, any overflow segment or partition of an overflow segment, or any mapping table segment of an index-organized table.    * You cannot specify any type of table compression for external tables or for tables that are part of a cluster.Testing and ResultsOverviewSome simple testing was conducted to support the information and recommended practices in this note. This testing is not intended to establish benchmarks or to be particularly thorough, but it is sufficient to illustrate some of the concepts and points mentioned above.The following practices were observed during the testing in order to improve the value of the statistics:Statistics were collected from another session, not the test session. If a given test modified the test table it was recreated and loaded prior to subsequent tests.The buffer cache was cleared prior to each test to avoid the effect of a warm cache on the results. The log buffer was raised to 64M to minimize the effect of log buffer space activity on the test results.Log switches and checkpoints were managed so that they did not occur during the tests.Test ScenariosScenarioRedundancyLoad MethodCompressionPercent Free#1: Non-Compressed TableYesConventionalNone10%#2: Compressed Without RedundancyNoConventionalOLTP10%#3: Compressed With Redundancy 10%YesConventionalOLTP10%#4: Compressed With Redundancy 20%YesConventionalOLTP20%#5: Compressed With Direct Path LoadYesDirect PathOLTP10%Test SchemaThe testing was performed with the following table. The compression and storage were modified for each test scenario by replacement of the &1 variable. Note that there are no true foreign key relationships here as there are no other tables involved in the testing. The column names were simply meant to convey the nature of the data in the column, i.e., the frequency of redundant values. There were no indexes on this table during testing, and none of the test SQL were reliant on indexes being present for an appropriate execution plan.CREATE TABLE actest( pk NUMBER, fk1 VARCHAR2(15), fk2 DATE, fk3 NUMBER, nr_payload VARCHAR2(80))&1;Test ResultsTEST #1: InsertsThe data for the testing was generated with two different insert statements, one intended to generate data with some redundancy within each block and the other intended to avoid any redundancy within the blocks. These same statements were used for insert append and for insert conventional path and so the &2 was replaced with either /*+ APPEND */ or /* no append */ as called for by the test scenario. INSERT &2 /* nonredundant */ INTO actestSELECT    level-1,    TO_CHAR(MOD(level-1,200),'09999999999999'),    TRUNC(SYSDATE + MOD(level-1,500)),    MOD(level-1,1000),     LPAD(TO_CHAR(level-1,'099999999999999'),80,'X')FROM dualCONNECT BY LEVEL <= 1000000;INSERT &2 /* redundant */ INTO actestSELECT    level-1,    TO_CHAR(TRUNC((level-1)/5000),'09999999999999'),    TRUNC(SYSDATE + TRUNC((level-1)/2000)),    TRUNC((level-1)/1000),    LPAD(TO_CHAR(level-1,'099999999999999'),80,'X')FROM dualCONNECT BY LEVEL <= 1000000;The following statistics show the results of the data loads / inserts from the five scenarios.Storage is reflected in the "Segment blocks used" row.Note that the storage for the non-compressed table and the storage for the compressed table with non-redundant data are comparable. OLTP compression attempts to compress a block and when the data in that block does not occupy less space than before, the block remains uncompressed.When the table was set for free 10% the data occupied about 4% fewer blocks than when the table was set for 20% free. While it is expected that more blocks would be required for the free 20% case, this may be less penalty than would be expected from a change from 10% to 20% free space per block. This is highly dependent on row length and block size; larger blocks and/or shorter rows would make the addition of each row a smaller change in the percentage of the block occupied and so would lead to numbers nearer expectations.The insert direct path (IDL) used the least storage of all the scenarios tested.Performance is reflected primarily in the "CPU used by this session" statistic though some of the other statistics directly impact this evaluation.Compression overhead is related to database configuration, system resources, and the data involved; it is application specific. It is best measured in terms of CPU time as this is the most direct resource involved in compression. There is also an increase in undo and redo when using compression so log related waits can play a vital role in compression performance tuning and analysis.For this specific testcase the inserts into the compressed table required 7.11, 7.23, and 7.96 seconds of CPU depending on the scenario involved whereas the inserts into the non-compressed table required 5.73 seconds of CPU. All that can be done to tune this specific difference is to reduce the number of times that blocks are compressed. Direct path loads assure a single compression per block of data loaded and so are optimal in this regard, causing the least overhead in terms of CPU and reducing undo and redo usage due to the nature of direct path.The increases in undo and redo in the compression cases is due to the pre-images of the blocks prior to compression being saved into undo and subsequently being written to redo.                                      Scenario #1     Scenario #2     Scenario #3     Scenario #4     Scenario #5                                      --------------  --------------  --------------  --------------  --------------                                      Non-Compressed  Compressed      Compressed      Compressed      Compressed                                      Redundant       Non-Redundant   Redundant       Redundant       Redundant                                      Conventional    Conventional    Conventional    Conventional    Direct Path                                      PCTFREE 10      PCTFREE 10      PCTFREE 10      PCTFREE 20      PCTFREE 10                                      --------------  --------------  --------------  --------------  --------------Segment blocks used                            8,195           8,197           7,223           7,498           6,500           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -Elapsed time (seconds)                   00:00:06.06     00:00:07.17     00:00:07.62     00:00:07.96     00:00:06.53           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -CPU used by this session (seconds)              5.73            7.11            7.23            7.96            6.02           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP positive compression                      0               0           7,920          14,444               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP negative compression                      0           7,230               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC IDL Compressed Blocks                          0               0               0               0           6,504           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP Space Saving                              0               0       7,341,774      11,875,780               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -redo size                                 67,254,244     129,084,064     136,293,420     195,226,432         214,684           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -undo change vector size                    2,068,532      61,757,140      68,351,796     122,220,732          48,736TEST #2: Query performing a full table scanSELECT COUNT(1) FROM (SELECT /*+ NO_MERGE */ * FROM actest);Full table scans are generally going to benefit from smaller segments. The actual read of a compressed row is no more involved than the read of a non-compressed row as the column piece for any compressed value is just read from the symbol table in the block instead of within the row piece for the block. For that reason there's no expectation of greater CPU usage attributable to decompression when reading compressed data than when reading non-compressed data. To the contrary, it is expected that CPU and IO consumption will decrease due to retrieving the same data from fewer blocks.                                      Scenario #1     Scenario #2     Scenario #3     Scenario #4     Scenario #5                                      --------------  --------------  --------------  --------------  --------------                                      Non-Compressed  Compressed      Compressed      Compressed      Compressed                                      Redundant       Non-Redundant   Redundant       Redundant       Redundant                                      Conventional    Conventional    Conventional    Conventional    Direct Path                                      PCTFREE 10      PCTFREE 10      PCTFREE 10      PCTFREE 20      PCTFREE 10                                      --------------  --------------  --------------  --------------  --------------Segment blocks used                            8,195           8,197           7,223           7,498           6,500           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -Elapsed time (seconds)                   00:00:01.62     00:00:01.32     00:00:01.38     00:00:01.32     00:00:01.29           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -CPU used by this session (seconds)              1.42            1.32            1.38            1.32            1.30           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP positive compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP negative compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC IDL Compressed Blocks                          0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP Space Saving                              0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -redo size                                          0               0               0               0          11,388           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -undo change vector size                            0               0               0               0               0TEST #3: Query repeated 1,000,000 times reading single rows by rowid in random orderThe random sequence for row retrieval was set prior to testing and maintained throughout all the test scenarios so that the results would be consistent. The following code was used to generate the random values used. This was done only once prior to all testing.CREATE TABLE acrandom( pk NUMBER, rn NUMBER, pkrowid ROWID);INSERT INTO acrandom SELECT LEVEL-1 pk, DBMS_RANDOM.RANDOM(), NULL FROM dual CONNECT BY LEVEL <= 1000000;The following code was used after each load to associate the rowids from the test table with the random ordering in the acrandom table.UPDATE acrandom rSET pkrowid = (        SELECT t.ROWID trowid        FROM actest t        WHERE r.pk = t.pk);The following is the code used to randomly access the rows from the actest table by rowid and in order by the random sequence set above.DECLARE    vRows NUMBER;BEGIN    FOR r IN (SELECT pkrowid FROM acrandom ORDER BY rn, pk) LOOP        SELECT COUNT(1) INTO vRows FROM (SELECT /*+ NO_MERGE */ * FROM actest WHERE rowid = r.pkrowid);    END LOOP;END;This test exercised the same number of random reads in each case and so elapsed time and CPU time are roughly the same across the scenarios. This is another verification that reading a compressed block is no more involved than reading a non-compressed block as discussed in the prior test. With a larger system the compression will have benefits for random reads as long as the physical reads are decreased as a result of a higher buffer cache hit ratio. Clearly the same numbers of rows are accessed from fewer blocks, leading to less physical reads and the same data preserved in the buffer cache using less memory.                                      Scenario #1     Scenario #2     Scenario #3     Scenario #4     Scenario #5                                      --------------  --------------  --------------  --------------  --------------                                      Non-Compressed  Compressed      Compressed      Compressed      Compressed                                      Redundant       Non-Redundant   Redundant       Redundant       Redundant                                      Conventional    Conventional    Conventional    Conventional    Direct Path                                      PCTFREE 10      PCTFREE 10      PCTFREE 10      PCTFREE 20      PCTFREE 10                                      --------------  --------------  --------------  --------------  --------------Segment blocks used                            8,195           8,197           7,223           7,498           6,500           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -Elapsed time (seconds)                   00:02:24.38     00:02:23.73     00:02:23.97     00:02:27.22     00:02:23.96           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -CPU used by this session (seconds)            143.95          143.28          142.80          143.24          143.17           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP positive compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP negative compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC IDL Compressed Blocks                          0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP Space Saving                              0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -redo size                                        188               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -undo change vector size                            0               0               0               0               0TEST #4: Update all rows setting a non-redundant column to the same valueThe following test was run toUPDATE actest SET fk3 = fk3;This test shows that updating a column to the same value does (as expected) generate an update to the row and the ensuing undo and redo that entails. This is not a change of behavior for compression, it is this way with non-compressed tables as well. It is necessary in order to gain expected behavior in other areas, such as triggers, auditing, recovery, and so on. In the third scenario we see that the time increased, this is because when a column is updated it is now stored in the row piece instead of the symbol table. Columns that are not updated still reside in the symbol table unchanged. In particular, the free 10% scenario resulted in more block compressions as the column updates filled the free space triggering the compressions. The free 20% case had more free space to work with and so had fewer block compressions. The direct path table took the longest to update and had the most block compressions.                                      Scenario #1     Scenario #2     Scenario #3     Scenario #4     Scenario #5                                      --------------  --------------  --------------  --------------  --------------                                      Non-Compressed  Compressed      Compressed      Compressed      Compressed                                      Redundant       Non-Redundant   Redundant       Redundant       Redundant                                      Conventional    Conventional    Conventional    Conventional    Direct Path                                      PCTFREE 10      PCTFREE 10      PCTFREE 10      PCTFREE 20      PCTFREE 10                                      --------------  --------------  --------------  --------------  --------------Segment blocks used                            8,195           8,197           7,223           7,498           6,500           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -Elapsed time (seconds)                   00:00:06.64     00:00:04.98     00:00:12.85     00:00:11.76     00:00:21.02           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -CPU used by this session (seconds)              6.24            4.76           11.23           11.54           18.14           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP positive compression                      0               0           1,425             536           2,504           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP negative compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC IDL Compressed Blocks                          0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP Space Saving                              0               0       1,095,027         343,593       2,065,327           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -redo size                                153,024,936     128,612,352     291,122,420     358,759,236               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -undo change vector size                   56,320,708      45,937,872     118,919,436     146,953,284     170,251,100TEST #5: Update all rows setting a redundant column to a new value not present prior to the updateUPDATE actest SET fk2 = TRUNC(SYSDATE)-1;This test showed similar results from the prior test as it did not matter whether the column update value was in the symbol table or not, it would be stored in the row, consume some free space, and lead to a block compression. What is different about this test is that the new value was redundant in all the tests, and more redundant than even the prior redundant data. The timings in the third and fifth scenarios were high for this test and those timings were not due to waits as shown by the CPU time consumed in each. This result is being reviewed internally.                                      Scenario #1     Scenario #2     Scenario #3     Scenario #4     Scenario #5                                      --------------  --------------  --------------  --------------  --------------                                      Non-Compressed  Compressed      Compressed      Compressed      Compressed                                      Redundant       Non-Redundant   Redundant       Redundant       Redundant                                      Conventional    Conventional    Conventional    Conventional    Direct Path                                      PCTFREE 10      PCTFREE 10      PCTFREE 10      PCTFREE 20      PCTFREE 10                                      --------------  --------------  --------------  --------------  --------------Segment blocks used                            8,195           8,197           7,223           7,498           6,500           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -Elapsed time (seconds)                   00:00:03.04     00:00:04.66     00:00:33.15     00:00:11.37     00:01:27.11           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -CPU used by this session (seconds)              3.04            4.58           32.82            9.34           86.50           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP positive compression                      0               0             947           1,498           1,184           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP negative compression                      0               0               0               1               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC IDL Compressed Blocks                          0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP Space Saving                              0               0         876,612         341,639       1,148,648           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -redo size                                 52,710,596     132,373,360     223,392,200     250,096,808     401,957,704           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -undo change vector size                   24,488,364      48,002,288      94,185,364     113,112,500     122,999,392TEST #6: Delete all rowsThis test was included in order to illustrate the impact of compression on deletes. In essence deletes enjoy the same advantages as queries in that they are able to access fewer blocks to accomplish their task. This leads to fewer physical reads and an advantage in the buffer cache as discussed prior. There is also the advantage that the same rows are deleted from fewer blocks leading to fewer writes.DELETE actest;The test showed about 10% improvement in the third and fourth scenarios which is expected to carry through to larger real-life load profiles. There is the implication in the first scenario that some time was lost due to waits as shown by the greater difference between the elapsed time and the CPU time, so the 10% was calculated based on CPU time. The direct path case wasn't as much of an improvement, though it is still better than the non-compressed case, as it caused a greater amount of redo than the other scenarios.                                      Scenario #1     Scenario #2     Scenario #3     Scenario #4     Scenario #5                                      --------------  --------------  --------------  --------------  --------------                                      Non-Compressed  Compressed      Compressed      Compressed      Compressed                                      Redundant       Non-Redundant   Redundant       Redundant       Redundant                                      Conventional    Conventional    Conventional    Conventional    Direct Path                                      PCTFREE 10      PCTFREE 10      PCTFREE 10      PCTFREE 20      PCTFREE 10                                      --------------  --------------  --------------  --------------  --------------Segment blocks used                                0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -Elapsed time (seconds)                   00:00:09.45     00:00:08.86     00:00:06.53     00:00:05.73     00:00:07.05           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -CPU used by this session (seconds)              6.40            6.50            5.83            5.64            6.24           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP positive compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP negative compression                      0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC IDL Compressed Blocks                          0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -HSC OLTP Space Saving                              0               0               0               0               0           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -redo size                                182,547,824     182,540,852     193,953,064     186,564,480     284,471,432           -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -   -undo change vector size                  112,003,216     112,003,456     116,767,100     113,505,684     120,694,960WaitsThere are no added wait events specific to OLTP compression. If a block is being compressed and another session attempts to access it the wait event will be "buffer busy wait" as it is with other block manipulations in the buffer cache.StatisticsStatistic #StatisticDescription407HSC OLTP Space SavingHow many bytes saved in total using OLTP Compression. Take the delta on every compression and adds408HSC OLTP Compressed BlocksHow many blocks are compressed using OLTP Compression. Counts only the first compressions in OLTP code409HSC IDL Compressed BlocksNumber of Blocks compressed using Insert Direct Load (Using DSS Compression)410HSC Compressed Segment Block ChangesTotal number of block changes to Tables/Heaps (Compressed only) 411HSC Heap Segment Block ChangesTotal number of block changes to Tables/Heaps (Compressed or Non-Compressed)412HSC OLTP Non Compressible BlocksBlocks marked as Final (Not to be compressed again)413HSC OLTP positive compressionNumber of times compression released some space414HSC OLTP negative compressionNumber of times compression was negative (consumed more space than before) and reverted to pre-image415HSC OLTP recursive compressionNumber of recursive compressions416HSC OLTP inline compressionNumber of inline compressions417HSC OLTP Drop ColumnNumber of Compressions happening due to Drop Column418HSC OLTP Compression skipped rowsNumber of rows that are skipped for compression (could be deleted or chained)419HSC OLTP compression block checkedNumber of times that OLTP compressed blocks have been validated by block checkingBugsNote 1061366.1 List of Critical Patches Required For Oracle 11g Table CompressionNotesNote 785787.1    11g new features:Advanced Compression overview and advantagesNote 882712.1    11g New feature:All About Advanced Table Compression(Overview,Use,Examples,Restrictions)Note 1101900.1    Performance Issue with Update Statement in Compressed TablespaceNote 987049.1    Performance Issue After Enabling CompressionNote 829068.1    COMPRESS FOR ALL OPERATIONS generates lot of redoNote 1149283.1    The OLTP Compression Saves No Space As Expected Using A Row Too BigNote 805205.1    My Data are not compressedOther ReferencesOracle Technology Network - Advanced Compression

++++++++++++++++++++++++++++++++++++++++++++++++++++ The complete and the most recent version of this article can be viewed from My Oracle Support Knowledge Section.Master Note for OLTP Compression...

Master Notes

Oracle Support Master Note for Troubleshooting Managed Distributed Transactions (Doc ID 100664.1)

Master Note for Troubleshooting Oracle Managed Distributed Transactions (Doc ID 100664.1)Copyright (c) 2010, Oracle Corporation. All Rights Reserved.  In this DocumentWhat is being announced?  What do you need to do?     Oracle Distributed Transactions     Two-Phase Commit (2PC)      In-Doubt Transactions     Recoverer Process (RECO)     Database Parameters     Database Views     DBA_2PC_PENDING     DBA_2PC_NEIGHBORS     V$DBLINK / GV$DBLINK     V$GLOBAL_TRANSACTION     Testing Recovery of Failed Distributed Transactions     Failures in Distributed Transactions     Steps to Troubleshoot an In-Doubt Transaction     Purging the Views     SCN Recovery Steps  Who to contact for more information?  References  Applies to: Oracle Server - Enterprise Edition - Version: 9.2.0.1 to 11.2.0.2 - Release: 9.2 to 11.2Information in this document applies to any platform.What is being announced?This master document discusses Oracle Managed Distributed Transactions; by this, we refer to transactions that span two or more databases and involve Oracle Databases only.The information contained in this document targets DBAs involved in environments that use Oracle Distributed Transactions. Distributed database concepts and troubleshooting steps are included in the note and, where appropriate, references to additional notes and documentation which provide further information on the relevant topic. You can use this information when investigating and troubleshooting in-doubt transactions; a section on SCN based recovery steps is also included.Note that within the scope of this document, for short, the term distributed transactions is used to refer to Oracle Managed Distributed Transactions(also known as homogeneous transactions).The following Oracle Documentation references discuss Oracle Distributed Transactions:Oracle9i Database Administrator's GuideRelease 2 (9.2)Part Number A96521-01Part VI: Distributed Database ManagementOracle10g Database Administrator's Guide10g Release 2 (10.2)Part Number B14231-02Part VII: Distributed Database ManagementOracle11gR1 Database Administrator's Guide11g Release 1 (11.1)Part Number B28310-04Part V: Distributed Database ManagementOracle11gR2 Database Administrator's Guide11g Release 2 (11.2)Part Number E17120-05Part V: Distributed Database ManagementWhat do you need to do?Oracle Distributed TransactionsA distributed transaction modifies data related to two or more databases, it contains DML statements than span many nodes. For a distributed transaction to be succesful all or none of the database nodes involved in the transaction need to commit or rollback the whole transaction.Note the difference between a distributed and a remote transaction; a remote transaction contains one or more DML statements that are executed on the SAME remote node, consider the examples below:- Distributed transaction: insert into table@remotesite;insert into mytable: --local tablecommit;-Remote transaction:insert into table@remotesite;commit;Database links are used to communicate between the databases performing distributed transactions. Location transparency can be achieved via synonyms, database views or PL/SQL procedures.Oracle Distributed Transactions are Operating System and Database version independent; they can expand databases running different Operating Systems and RDBMS releases as long as their combination is certified by Oracle and supported. For more information, refer to: Note 207303.1 Client / Server / Interoperability Support Between Different Oracle VersionsGlobal Database NameThe global name of the database is composed of DB_NAME.DB_DOMAIN, where DB_DOMAIN specifies the network domain.In a distributed environment, it is fundamental that the global name of a database is unique in the network so that each database can be unambiguously identified.Note: The DB_DOMAIN parameter is only relevant at database creation time; once the database is created, changing this parameter in the database initialization file will not take effect in itself, though it is recommended to include it there for documentation purposes.The following sql will allow to check the current name of the database and to change it if necessary:sqlplus> select * from global_name;ORCL.US.ORACLE.COM;sqlplus> ALTER DATABASE RENAME GLOBAL_NAME TO orcl.net;sqlplus> select * from global_name;ORCL.NET;Refer to Managing Global Names in a Distributed System from the Oracle Database Administrator's Guide for further information on the above.Two-Phase Commit (2PC) The two phase-commit mechanism is used to ensure data integrity in a distributed transaction. It is automatically used during all distributed transactions and coordinates either the commit or roll back of all the changes in the transaction as a single, self-contained unit.In short, there are three phases for the 2PC:PREPARE: The initiating node ask each of its referenced nodes to promise to perform a commit or rollback when told to do so. The preparing node will flush the redo log buffer to the online redo log. It converts locks to in-doubt transaction locks on the data blocks and passes its highest SCN value to the initiating node.COMMIT: The initiating node commits and writes to its redo log the committed SCN. The Data Block locks are released.FORGET:  Pending transactions tables are related database views are cleared (dba_2pc_pending/dba_2pc_neighbors)All the above phases take place quickly and transparently to the application where the transaction originated. NOTE:A crash during the PREPARE Phase results in a ROLLBACKA crash during the COMMIT Phase results in either COMMIT or ROLLBACKNode Relations During 2-PC OperationsEach Database involved in a two-phase commit operation performs one or more of the following roles:Client: Database node that requests information from another db.Server: Node which receives a request for information from another database involved in a distributed transaction.Global Coordinator (GC): Node where the distributed transaction originates. This node is responsible for sending messages to the other nodes to prepare and commit (or roll back).Local Coordinator:  Node that needs to access data on other nodes in order to complete its part of the transaction. It is also responsible for coordinating the transaction among the nodes with which it communicates directly.Commit Point Site: Node which commits or rolls back the transaction first, as instructed by the Global Coordinator (GC). This site determines the outcome of the transaction. This node never enters the prepare state. Note: The commit point site is chosen based on the highest COMMIT_POINT_STRENGTH value of all the nodes involved.Session Tree: The communication topology for the Oracle server is fundamentally a tree structured topology. As the statements in a distributed transaction are issued, the nodes and edges of the session tree are defined. There is a certain amount of recursiveness inherent in the tree structure. A session tree is the hierarchical model that describes the relationships between sessions and their roles in the distributed transaction. The nodes which represent the sessions in the session tree have different roles in the 2PC protocol with regard to transaction management.For more detailed information regarding distributed transactions and Oracle implementation of the 2PC mechanism, refer to Note 13229.1 Distributed Database, Transactions and Two Phase Commit In-Doubt TransactionsIf a distributed transaction fails, it may leave in-doubt transactions on one or more databases.An in-doubt transaction is normally automatically resolved when the database or network is restored, this is done by the RECO database background process. An in-doubt distributed transaction occurs when a two-phase commit was interrupted by any type of system or network failure. For example, two databases report to the coordinating database that they were prepared to commit, but the coordinating database instance fails immediately after receiving the messages. The two databases who are prepared to commit are now left awaiting notification out of the outcome. Recoverer Process (RECO)RECO is a mandatory background process that, in a distributed database, automatically resolves failures in distributed transactions. The RECO process of a node automatically connects to other databases involved in an in-doubt distributed transaction; it can use an existing connection or establish a new connection to other nodes involved in the failed transaction.. When RECO reestablishes a connection between the databases, it automatically resolves all in-doubt transactions, removing from each database's pending transaction table any rows that correspond to the resolved transactions.At exponentially growing time intervals, the RECO background process of a node attempts to recover the local portion of an in-doubt distributed transaction. Disabling and Enabling RECOYou can enable and disable RECO using the ALTER SYSTEM statement with the ENABLE/DISABLE DISTRIBUTED RECOVERY options. For example, you can temporarily disable RECO to force the failure of a two-phase commit and manually resolve the in-doubt transaction.-To disable RECO:ALTER SYSTEM DISABLE DISTRIBUTED RECOVERY-To enable RECO and let it automatically resolve indoubt transactionsALTER SYSTEM ENABLE DISTRIBUTED RECOVERY;Database Parameters GLOBAL_NAMES: (default false): It is strongly recommended to set this parameter to TRUE, to enforce the database name uniqueness in the network.Setting it to TRUE enforces that any database link has the same name than the global_name of the database that it connects to.NOTE: changing this parameter from false to true may render already existing database links unusable.OPEN_LINKS: (default 4) Specifies the maximum number of concurrent open connections to remote databases in one session. These connections include database links, as well as external procedures and cartridges, each of which uses a separate process.. It should at least be the same than the maximum number of databases that any transaction can reference to.DISTRIBUTED_LOCK_TIMEOUT: (default 60 seconds): Defines the number of seconds that a distributed transaction waits for a lock. If a session waits longer than this for the lock then ORA-2049 is signalled.note This parameter was hidden in 8i and 9.0 and then made available again in 9.2 onwards.COMMIT_POINT_STRENGTH (default 1). Range of values is any integer from 0 to 255. Its value determines the commit point site in a distributed transaction. The node in the transaction with the highest value for COMMIT_POINT_STRENGTH will be the commit point site. The commit point strength should be set relative to the amount of critical shared data in the database. If no commit point site is set, Oracle will determine which site becomes the commit point site.Database ViewsDBA_2PC_PENDINGDBA_2PC_PENDING Provides Information on distributed transactions awaiting recovery, ie in-doubt transactions. Relevant columns from this view are:LOCAL_TRAN_ID     Local Id of the transaction,in the form of n.n.n; where n is a number GLOBAL_TRAN_ID  Global Transaction ID, unique to all sitesSTATE: One of the following: Collecting/Prepared/Committed/Forced Commit/Forced RollbackMIXED: YES indicates that portions of the transaction have been committed and portions rolled back (forcibly)ADVICEC indicates CommitR indicates RollbackThis field is populated only if the application has issued one of the following statements prior to beginning the distributed transaction: ALTER SESSION ADVICE COMMIT or  ALTER SESSION ADVICE ROLLBACKTRAN_COMMENTCommit comment text. This column is populated only if the application has issued a COMMIT with a comment, for example: COMMIT COMMENT  'comment detail'FAIL_TIME Time that the row has been inserted in the viewFORCE_TIMEIf the transaction has been forced the time will be displayed; NULL otherwiseRETRY_TIMETime that the RECO process last attempted to resolve the transactionOS_USEROS USerID of the local user that created the transactionOS_TERMINALTerminal from where the local portion of the transaction originatedHOSTName of the machine where the local transaction originatedDB_USERDatabase username who originated the distributed transactionCOMMIT#If the transaction is committed this column represents the global commit numberDBA_2PC_NEIGHBORSDBA_2PC_NEIGHBORS  Displays information on incoming and outgoing connections for pending transactions within an Oracle distributed transaction. Relevant columns:LOCAL_TRAN_IDLocal identifier of the transactionIN_OUTConnection Type:IN for incomingOUT for outgoingDATABASEFor incoming connections, this is the client db global_nameFor outgoing connections, this value represents the database linkDBUSER_OWNERFor incoming connections, the Oracle UsernameFor outgoing connections, the owner of the database linkINTERFACE Used to locate the Global Commit Point site:For incoming links, C indicates that this site or one of the descendants on an outgoing link is the commit point site.For outgoing links, C indicates that the destination database DBID is the commit point site.If we are in-doubt, INTERFACE is N and then the top-level database either is the commit point site or can locate the commit point site.DBIDThe global name of the remote databaseSESS#Local session number for the connection at this database. Sessions are numbered consequently, starting by 1BRANCHTransaction branch ID of the connection at this database. Branch IDs for incoming connections are two byte hexadecimal numbers; the first byte is the remote parent's session ID, and the second byte is its branch ID.V$DBLINK / GV$DBLINKV$DBLINK Lists all open database links in your session, that is, all database links with the IN_TRANSACTION column set to YES.GV$DBLINK Lists all open database links in your session along with their corresponding instances. This view is useful in an Oracle Real Application Clusters configuration.It might be useful to determine which database link connections are currently open in your own session. Note that if you connect as SYSDBA, you cannot query a view to determine all the links open for all sessions; you can only access the link information in the session within which you are working.V$GLOBAL_TRANSACTIONV$GLOBAL_TRANSACTION provides information on the currently active distributed transactions.Testing Recovery of Failed Distributed TransactionsOracle provides a means to force failure of distributed transactions so that you can test and validate the distributed transaction recovery procedures according to the point of failure.Issuing a COMMIT with a comment such as ORA-2PC-CRASH-TEST-n, -where n is a number from 1 to 10-.You can test a variety of scenarios, according to the value of n, as shown below, 1   Crash commit point after collect2   Crash non-commit-point site after collect3   Crash before prepare (non-commit-point site)4   Crash after prepare (non-commit-point site)5   Crash commit point site before commit6   Crash commit point site after commit7   Crash non-commit-point site before commit8   Crash non-commit-point site after commit9   Crash commit point site before forget10 Crash non-commit-point site before forgetFor example, the following statement returns the following messages if the local commit point strength is greater than the remote commit point strength and both nodes are updated: COMMIT COMMENT 'ORA-2PC-CRASH-TEST-7'; ORA-02054: transaction 1.93.29 in-doubt ORA-02059: ORA_CRASH_TEST_7 in commit commentAt this point, the in-doubt distributed transaction appears in the DBA_2PC_PENDING view. If enabled, RECO automatically resolves the transaction.Refer to Note 126069.1 Manually Resolving In-Doubt Transactions: Different Scenarios, for detailed test results regarding each of the situations above.Failures in Distributed TransactionsError messages related to distributed transactions fail in the range ORA-02040 - ORA-02099, refer to Oracle11g Database Error Messages for details on those.Often, abnormal conditions that occur during 2PC are caused by either a network or a server failure whilst the transaction is in the prepare and commit phases.Some errors that you might see in the alert.log when working in environments that use distributed transactions are shown below:ORA-02053: transaction <txnId> committed, some remote DBs may be in-doubtThe transaction has been locally committed, however we have lost communication with one or more local coordinators.ORA-02054: transaction <txnId> in-doubtThe transaction is neither committed or rolled back locally, and we have lost communication with the global coordinator.ORA-02050: transaction <txnId> rolled back, some remote DBs may be in-doubtIndicates that a communication error ocurred during the two-phase commitORA-01591: lock held by in-doubt distributed transaction <txnId> Encountering the above error and users/applications unable to proceed with their work. In this case, Oracle automatically rolls back the user attempted transaction and the DBA has now to manually commit or rollback the in-doubt transaction. NOTE: Reads are blocked because, until the transaction is resolved, Oracle does not assume which version of the data to display for a query user.- Distributed transactions time out waiting to acquire locks or hold locks themselves for an excessive amount of time. If a distributed transaction cannot obtain a required lock after DISTRIBUTED_LOCK_TIMEOUT seconds then the following error is returned:ORA-02049 timeout: distributed transaction waiting for lockIncreasing the value of the distributed_lock_timeout/retry the issued SQL, are typical approaches when this error is encountered. If it becomes a recurrent problem and certain transactions appear to be hanging for ever or causing contention in the database, then further investigation is required, and there is a need determine what other transaction is holding the lock and what type of problem exists.To further analyze the above, refer to Note 789517.1 ORA-02049: timeout: distributed transaction waiting for lock' Error: How to Obtain a System State Trace BEFORE the Error Occurs, While Still Experiencing the Contention - Additionally , there are some other rare circumstances in which further action needs to be taken by the DBA in order to resolve the in-doubt transaction as there is underlying metadata corruption. Please refer to the following note for a discussion on how to investigate and overcome those: Note 401302.1 How To Resolve Stranded DBA_2PC_PENDING EntriesSteps to Troubleshoot an In-Doubt TransactionSee the example below of how to troubleshoot a sample in-doubt transaction: 1-  Problem reported: Users querying a local table in database DB102C encounter the following errors and notify the DBA :ORA-01591: lock held by in-doubt distributed transaction 10.24.340In the alert.log :DISTRIB TRAN DB102D.UK.ORACLE.COM.ea27958e.9.46.338 is local tran 10.24.340 (hex=0a.18.154) insert pending prepared tran, scn=5017243 (hex=0.004c8e9b)You note down the Local Transaction ID,10.24.340, and query DBA_2PC_PENDING:2-  Query DBA_2PC_PENDINGselect * from dba_2pc_pending where local_tran_id='10.24.340';LOCAL_TRAN_ID : 10.24.340GLOBAL_TRAN_ID : DB102D.UK.ORACLE.COM.ea27958e.9.46.338STATE : preparedMIXED : noADVICE :TRAN_COMMENT : FAIL_TIME : 02-nov-2010 02:06:02FORCE_TIME :RETRY_TIME : 02-nov-2010 02:06:02OS_USER : NT AUTHORITY\SYSTEMOS_TERMINAL : USER33-lapHOST : USER33-lapDB_USER : TESTUSERCOMMIT# : 5017243From the above output the following can be determined:This node is not the Global Coordinator as the local_tran_id is different than the last portion of the Global_Tran_ID; ie the distributed transaction did not originate at this node.GLOBAL_TRANSACTION_ID represents the common transaction ID that will be the same on the every node for a distributed transaction. It is of the form global_database_name.hhhhhhhh.local_transaction_id where global_database_name is the database name of the global coordinator and hhhhhhhh represents in hexadecimal the internal database identifier of the global coordinator.The transaction in this node is in PREPARED StateLook at the ADVICE and TRAN_COMMENT columns for information about this transaction, if any of those columns are populated, they could help you decide whether the local portion of the transaction should be rolled back or committed. (COMMIT COMMENT... or SET TRANSACTION... NAME populates tran_comment column)3 - Query DBA_2PC_NEIGHBORSWe climb the session tree so that we find coordinators, until we eventually reach the global coordinator. Along the way, you might find a coordinator that has resolved the transaction. If not, you can eventually work your way to the commit point site, which will have always have resolved the in-doubt transaction. To trace the session tree, query DBA_2PC_NEIGHBORS on each node- On the local node:select * from dba_2pc_neighbors;LOCAL_TRAN_ID : 10.24.340IN_OUT : inDATABASE : DB102D.UK.ORACLE.COMDBUSER_OWNER : TESTUSERINTERFACE : NDBID : ea27958eSESS# : 1BRANCH : 09002E00520100000104As discussed, DBA_2PC_NEIGHBORS provides information about connections associated with an in-doubt transaction. Information for each connection is different, based on whether the connection is inbound (IN_OUT = in) or outbound (IN_OUT = out).In our example we see that IN_OUT = in, indicating that our db (db102c) is a server of the DB102D client as specified in the database column.DBUSER_OWNER shows the connecting user to our db102c databaseINTERFACE = N indicates that neither DB102C nor any of its dependent sites is the commit point site.- On the remote node referenced in the transaction:select * from dba_2pc_neighbors;LOCAL_TRAN_ID : 9.46.338IN_OUT : inDATABASE :DBUSER_OWNER : TESTUSERINTERFACE : NDBID :SESS# : 1BRANCH : 0000-----------------LOCAL_TRAN_ID : 9.46.338IN_OUT : outDATABASE : DB102C.UK.ORACLE.COMDBUSER_OWNER : TESTUSERINTERFACE : NDBID : bae6b90fSESS# : 1BRANCH : 4For the OUT connection :column DBUSER_OWNER shows the owner of the dblink column DATABASE shows the database link name that has accessed the remote server (db102c).The local txn ID matches the end portion of the global_transaction_id indicating that this site is the global coordinator, where the transaction originated.This site is the commit point site and has already committed the local portion of the transaction as seen by the state column:Conclusion: This node already committed the local transaction and afterwards it crashed leaving the GC still waiting for the 'commit' response from  the commit point site.4 - Resolve the in-doubt transactionGiven that the commit point site has already committed this transaction, then manually commit it at the DB102C site; to do so the following syntax can be used:COMMIT FORCE 'DB102D.UK.ORACLE.COM.ea27958e.9.46.338','5017245'; -OR-COMMIT FORCE '10.24.340','5017245';The COMMIT FORCE clause lets you manually commit an in-doubt distributed transaction. The transaction is identified by the 'string' containing its local or global transaction ID. You can use integer to specifically assign the transaction a system change number (SCN). If you omit integer, then the transaction is committed using the current SCN.In order to ensure global integrity, we use the commit# of the commit point site (highest global commit#)By specifying the SCN for the transaction when forcing a transaction to commit, the in-doubt transaction is committed with the SCN assigned when it was committed at other nodes.In this way, we can maintain the synchronized commit time of the distributed transaction even if there is a failure. We only specify an SCN only when we can determine the SCN of the same transaction already committed at another node.In cases where the in-doubt transaction is to be ROLLBACK you would use syntax ROLLBACK FORCE <local_txn_ID> to set the state to forced rollback. Purging the ViewsDBMS_TRANSACTION.PURGE_LOST_DB_ENTRY is used to manually purge the details from the views once the in-doubt transaction has been resolved:execute DBMS_TRANSACTION.PURGE_LOST_DB_ENTRY ('10.24.340');In the case of a mixed transaction, where portions of the database have already committed and other portions have been rolled back forced, use execute DBMS_TRANSACTION.PURGE_MIXED ('txnID');Refer to DBMS_TRANSACTION.PURGE_LOST_DB_ENTRY  for further details  regarding purging Pending Rows from the Data DictionaryNOTE: If using UNDO_MANAGEMENT=auto and DBMS_TRANSACTION.PURGE_LOST_DB_ENTRY fails withORA-30019: Illegal rollback Segment operation in Automatic Undo mode, use thefollowing workaround:-- check the current value of _smu_debug_mode (default 0):SQL> show parameter debug   -- if default 0, it will show no entry-- set it temporarily to 4:SQL> alter system set "_smu_debug_mode" = 4; -- in 9.2x alter session can be used instead.SQL> commit;SQL>execute DBMS_TRANSACTION.PURGE_LOST_DB_ENTRY('local_tran_id');SQL> commit;SQL> alter system set "_smu_debug_mode" = <original value>;SQL> commit; SCN Recovery Steps Complete Recovery:1. At the down site recover completely if possible (treat as regular recovery).2. SCN will appear in the alert.log after recover is done on the crashed Node. Incomplete Recovery:1. If time-based or cancel-based recovery was used on the crashed node, other sites must be placed back to the same point in time for global consistency. Get last SCN from the alert.log of the crashed node.2. At each node perform a shutdown normal or immediate.3. Take a cold backup.4. Restore the control file if necessary.5. Restore the last backup of all datafiles along with archived redo logs.6. Choose which tool to use to perform SCN recovery - either SQLPLUS or RMAN.7. Connect as sysdba, startup mount, check status from v$datafile to make sure all datafiles are online. Issue 'Alter database datafile '?/?/?' online;', for each datafile with status offline, to bring it online.8. Issue the following command using the latest SCN from alert.log on the Node that had to be recovered: RECOVER DATABASE UNTIL CHANGE '1.13.5197'; NOTE: If for some reason (e.g.when issuing commit force command) automatic recovery (RECO process) needs to be disabled, we can disable it and reenable via alter system as mentioned before: ALTER SYSTEM DISABLE DISTRIBUTED RECOVERY; Use this command to wake RECO up after that: ALTER SYSTEM ENABLE DISTRIBUTED RECOVERY;Who to contact for more information?For further diagnosis and help troubleshooting distributed transactions, contact Oracle Support or login to Streams and Distributed Database Community to share your experiences with fellow peers. ReferencesNOTE:1012842.102 - ORA-2019 ORA-2058 ORA-2068 ORA-2050: Failed Distributed TransactionsNOTE:126069.1 - Manually Resolving In-Doubt Transactions: Different ScenariosNOTE:13229.1 - Distributed Database, Transactions and Two Phase CommitNOTE:159377.1 - How to Purge a Distributed Transaction from a DatabaseNOTE:207303.1 - Client / Server / Interoperability Support Between Different Oracle VersionsNOTE:401302.1 - How To Resolve Stranded DBA_2PC_PENDING EntriesOracle9i Database Administrator's Guide - Part VI: Distributed Database ManagementOracle10g Database Administrator's Guide - Part VII: Distributed Database ManagementOracle11g Database Administrator's GuidePart V: Distributed Database Management

Master Note for Troubleshooting Oracle Managed Distributed Transactions (Doc ID 100664.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved.   In this Document What is being announced?  What do...

Master Notes

Oracle Support Master Note for Troubleshooting Streams Apply Errors ORA-1403, ORA-26787 or ORA-26786 (Doc ID 265201.1)

Master Note for Troubleshooting Streams Apply Errors ORA-1403, ORA-26787 or ORA-26786 (Doc ID 265201.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved.In this Document  Purpose  Last Review Date  Instructions for the Reader  Troubleshooting Details     Retrieve information about the error     Identify root cause     Fixing the error     Avoid new errors  ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 9.2.0.1 to 11.2.0.2 - Release: 9.2 to 11.2Information in this document applies to any platform. PurposeThis article covers how to diagnose Apply Streams errors when the Apply process gets a data error when applying a transaction, like errors ORA-1403, ORA-26787 or ORA-26786This do not cover critical errors suffered by the apply process that are not related to data errors. Last Review DateOctober 15, 2010  Instructions for the ReaderA Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting. Troubleshooting DetailsOn a streams environment it is required that data information retrieved by the capture process is in sync with the information that the apply process is going to apply. When this data is not not completely in sync any DML captured may raise an error on the apply side.A good is example are the errors ORA-1403, ORA-26787 and ORA-26786 which typically occurs  when UPDATE or DELETE statements are attempted on an existing row and the OLD_VALUES of the LCR do not match the current values at the target database. Please see a sample video on how to troubleshoot these errors: Video - Sample on Troubleshooting an error on Streams Apply error queue (4:35) There are 2 potential actions to run when you get a data error on the apply error queue. Delete the transaction/s from the error queue or re-execute the transaction/s in the error queue after fixing the data in the apply side. To do this, it is necessary to get the information from the LCR getting the error and compare with the information existing in database if any. Retrieve information about the errorWhen an apply process gets a data error applying an LCR, it rollbacks the whole transaction and insert the whole transaction in the error queue that could be checked through DBA_APPLY_ERROR view.The most useful information, when analyzing a data error, in this view are the columns:APPLY_NAME, Name of the apply process that get the error when applying the transaction.LOCAL_TRANSACTION_ID, which contains the id of the transaction that has been rolled back and that we will use to solve the transaction.MESSAGE_NUMBER, Number of the message that has hit the errorERROR_NUMBER, Error number hitERROR_MESSAGE, Error message hitMESSAGE_COUNT, Number of LCRs in the transactionYou can query this view yourself or you can see the errors through other mechanisms like db console, OEM console, RDA, Streams Health Check or print_errors procedure that is described below.It is recommended to generate a Streams Health Check report from source site and target site when troubleshooting these errors. View failed LCRTo see what information the apply process has tried to apply, you have to print that LCR. Depending on the size (MESSAGE_COUNT) of the transaction that has failed, it could be interesting to print the whole transaction or a single LCR.To do this print you can make use of procedures print_transaction, print_errors, print_lcr and print_any documented on :Oracle Streams Concepts and Administration  Chapter - Monitoring Streams Apply Processes     Section - Displaying Detailed Information About Apply ErrorsThese procedures are also available through Note 405541.1 - Procedure to Print LCRsTo print the whole transaction, you can use print_transaction procedure, to print the error on the error queue you can use procedure print_errors and to print a single_transaction you can do it as follows:SET SERVEROUTPUT ON;DECLARE   lcr SYS.AnyData;BEGIN    lcr := DBMS_APPLY_ADM.GET_ERROR_MESSAGE                (<MESSAGE_NUMBER>, <LOCAL_TRANSACTION_ID>);    print_lcr(lcr);END;/Check data on the target side Based on the information retrieved on previous step, it will be necessary to retrieve information of the data indeed stored in database, if any. Do a select in the problematic table using the information in the previous step and referring to the values on old value columns.NOTE: Streams will not trigger an ORA-1403 for columns not updated at the source site and not supplementally logged either. The OLD column value willnot be included in the LCR for comparison at the destination.You may need to dump the problematic row from both the source and destination site using the Dump Function. Check for any data mismatch. Be sure to dump ALL primary key columns and date columns.For Example:SELECT dump(deptno), dump(dname), dump(loc) FROM dept where deptno=40;See Note 108377.1 - How Do You Read the Output from the Dump Function? Enable TracingOn certain situations, it could be necessary to enable tracing and re-execute failed transaction to observe exactly when and where the error is being raised.To do this, you'll have to do as strmadmin:alter session set max_dump_file_size=unlimited;alter session set events '10046 trace name context forever, level 4';exec DBMS_APPLY_ADM.EXECUTE_ERROR ('<local_transaction_id>');   -- or DBMS_APPLY_ADM.EXECUTE_ALL_ERRORS -- Error will be reportedalter session set events '10046 trace name context off';These operations create a trace file on udump/diag directory. To interpret this output use Note 39817.1 - Interpreting Raw SQL_TRACE and DBMS_SUPPORT.START_TRACE output. Identify root cause Here is a list of potential root causes that may lead to the error; there could be more than one reason to get an error when applying an LCR. When a LCR is being applied and we hit on an error, we are stopping on the first error it may find.Supplemental Logging Ensure that all key columns that uniquely identify a row (usually primary key) at source site are included in the LCR that has failed; if they are not this could be a problem of inaccurate supplemental logging.Verify that supplemental logging has been specified at the source either at the database level or for the individual replicated table.Database level loggingSELECT supplemental_log_data_pk, supplemental_log_data_ui FROM V$DATABASE;Table level loggingCheck supplemental log groups. For Example:ALTER TABLE scott.deptADD SUPPLEMENTAL LOG GROUP dept_log_group (deptno, dname, loc) ALWAYS;Select owner,log_group_name, table_name, always from dba_log_groups;Check columns in supplemental log groups:Select log_group_name, column_name, positionfrom dba_log_group_columnswhere table_name = 'DEPT' and owner='SCOTT';  Key columns identificationEnsure that LCR failing have value assigned for key columns on target site, this means to verify if the target table has primary key and LCR contains values for all the columns. If target table do not have primary key, you can either add the primary key constraint to the table, or use the DBMS_APPLY_ADM.SET_KEY_COLUMNS procedure to specify the identification key for apply. When using SET_KEY_COLUMNS be sure to list ALL of the key columns, in order, in the column list. Do not specify each column in a separate call to SET_KEY_COLUMNS.his has even more importance if the tables that are being managed contain LOB columns, otherwise you will get errors ORA-1403 or ORA-26572 while applying LOB LCR as described on Note 567623.1In the case that the columns of the primary key on source site do not match the columns of the primary key  at target, it will be necessary that target site key columns are supplementally logged at the source database, so the LCR will not contain the values needed to identify the row at the target site. This applies to using SET_KEY_COLUMNS as well.  Triggers and ConstraintsThe LCR identified as cause of the error indicates in which row we are hitting the error, but note that the operation may involve additional operations fired by a trigger or by a constraint in the case of a DELETE operation.Therefore ensure that triggers associated to target table has firing property set to fire once. This verification is done using DBMS_DDL.IS_TRIGGER_FIRE_ONCE procedure. By default, DML and DDL triggers are set to fire once. If a trigger is not set to fire once it will fire when a relevant change is made by an apply process and/or when a relevant change results from the execution of one or more apply errors using the EXECUTE_ERROR or EXECUTE_ALL_ERRORS procedure in the DBMS_APPLY_ADM package.set serveroutput onDECLARE   res boolean;BEGIN   res := DBMS_DDL.IS_TRIGGER_FIRE_ONCE('triggerowner'.'triggername');   IF res THEN     DBMS_OUTPUT.PUT_LINE('Fire Once? YES');   ELSE     DBMS_OUTPUT.PUT_LINE('Fire Once? NO');   END IF;END;/If necessary set the property using DBMS_DDL.SET_TRIGGER_FIRING_PROPERTY.Verify if table being affected has a foreign key with includes a ON DELETE CASCADE clause, as  the error may come because cascade operation. Please disable the constraint in this case or create an error handler to manage the ORA-1403 error for DELETE statements. Transformations and HandlersDeclarative Transformations, Custom-rule based transformations and DML or DDL handlers do transform the contents and the format of an LCR, if this modification is not correct it may lead to errors by the apply process. It is important to review if the transformation that is ocurring is the desired one and if not correct it accordingly.Transformations may occur on any phase of a Streams flow (Capture, Propagation and Apply), so it is also important to check if there has been any transformation on source site. Transformations occur at rule evaluation time, so if the evaluation of the LCR is positive then the transformation is going to occur. Transformations are registered on DBA_STREAMS_TRANSFORMATIONS view and you can easyly identify them on a Streams HealthCheck output on sections:Capture Rule Transformations By Rule SetPropagation Rule Transformations By Rule SetApply Rule Transformations By Rule SetRule Transformations StatusIn the other side we have apply handlers that only occurs on apply site, handler are executed directly by the apply server and are going to occur after a transformation (if defined) has taken place. It is important to distinguish 3 group of handlers:DML and DDL handlers, these handlers are executed by the apply slave before applying the LCR.Precommit handlers, these handlers are executed before running COMMIT of the transaction; remember that DDL have an implicit commit.Error handlers, these handlers is executed by apply slave after applying the LCR but only in the case there has been an error while applying the LCRHandlers are registered on DBA_APPLY_DML_HANDLERS and DBA_APPLY views or you can find further information for them on a Streams Health Check output on sections:Apply Process HandlersApply HandlersApply Dml Handlers DDL operationsUnder certain circumstances a DDL operation may fire DML operations, for example :  alter table add column default (x)).As per the design of Streams, logminer session associated to Capture process is ignoring these DMLs and are not being captured.In the case that Streams set up in place do not replicate DDLs, when replicating DMLs we may find a data mismatch.  Unrecoverable operationsCapture process trust on redo/archive log information to replicate information; but is it possible not to store information of operations in redo files, this is done through unrecoverable operations or nologging settings.It is possible to set table and tablespace settings to nologging, that means that operations on that table/tablespace will not write redo information into redo log files. So, please review if the table or tablespace where the table has been stored has this setting enabled at any moment. You can check DBA_TABLES and DBA_TABLESPACES views.In addition to this settings and although table and tablespace settings are correct, it is possible to force operations not to write on redo log files, this is done by specifying the operation as NOLOGGING or UNRECOVERABLE; this usually occurs on massive data loads. Please check if you have done a massive data load in source database recently as UNRECOVERABLE, if so, maybe does worth to repeat massive data load on target site.Other possible causesBug 2849670This bug occurs when no primary key columns are being used on target site, so DBMS_APPLY_ADM.SET_KEY_COLUMNS procedure has been used and there is a DELETE LCR being processed. It will occur if schema_name has been misspelled when using DBMS_APPLY_ADM.SET_KEY_COLUMNS.The solution is to define properly schema_name on  DBMS_APPLY_ADM.SET_KEY_COLUMNS.Data modification on target siteIf cause is not any of previous one, this probably is the cause.This occurs when target data has been manipulated manually or automatically, causing a data mismatch.Compare the column values for the existing row in the target table with the values of the LCR. LOB columns should not be compared.Be sure to expand the DATE columns to show full year value and time. ALTER SESSION SET NLS_DATE_FORMAT='HH24:MI:SS DD/MM/YYYY';Starting on release 11g the package DBMS_COMPARISON is included. This package allows to do a massive data comparison between source data and target data, allowing to identify quickly and simple the diverge data between the tables, it is also very useful to solve massive ORA-1403, ORA-26787 and ORA-26786 errors reported. In addition to this it allows to repair the divergence data. Please refer to Oracle Streams Replication  Administrator Guide for further information on how to use DBMS_COMPARISON with Streams. Fixing the errorOnce the root cause of the error has been identified, then there are two options:Re-execute the transaction after fixing root cause.Ignore failed transaction and delete the error from error queue.Fixing root cause, could be different types of actions like disabling constraints, altering a table,...but what it is not possible is to modify the LCR that is stored in the error queue.If the only solution is altering the LCR, then it will be necessary to define a handler that will be executed by re-execution of the error. Data CorrectionData correction is a typical solution to data mismatch; in the case that data there is uni-directional replication just correct the data on the apply siteBut if bi-directional replication is configured, it will be necessary that data manipulation to be done is not replicated back to source site causing new error there; to achieve this target you can set a streams tag before doing data correction on target site.exec DBMS_STREAMS.SET_TAG('ff');  >> fix the data on the apply siteexec DBMS_STREAMS.SET_TAG(null);Starting on release 11g you can converge the data on  source and remote table through package DBMS_COMPARISON, if it has been used to identify the differences, on bi-directional streams environments you should be aware of the tags too. In this way if parameter converge_options is set to DBMS_COMPARISON.CMP_CONVERGE_LOCAL_WINS, then parameter remote_converge_tag should be set and; if parameter converge_options is set to DBMS_COMPARISON.CMP_CONVERGE_REMOTE_WINS, then parameter local_converge_tag should be set.Please refer to Oracle Streams Replication Administrator Guide for further information on how to use DBMS_COMPARISON with Streams. Reexecuting the errorTo re-execute transaction you can two procedures. DBMS_APPLY_ADM.EXECUTE_ERROR to reexecute only one transaction or, DBMS_APPLY_ADM.EXECUTE_ALL_ERRORS to re-execute the all failed transactions in the error queue. This should be done as Streams Administrator.exec DBMS_APPLY_ADM.EXECUTE_ERROR ('<local_transaction_id>'); or exec DBMS_APPLY_ADM.EXECUTE_ALL_ERRORS ();If the root cause the error has been properly corrected, the transaction will apply properly and the error will disappear from error queue.In the case the execution fails again the error will remain in the error queue. Remember that there could be multiple reason for a transaction to fail and although the error returned is the same, the root cause for the problem can be different and therefore identification of the root cause have to be done again. Deleting error from error queueIf your conclusion is that the transaction that has failed is not relevant anymore for your system and therefore it can be removed, then you can delete the failed transaction.To delete transaction you can use procedures DBMS_APPLY_ADM.DELETE_ERROR, to delete 1 failed transaction or DBMS_APPLY_ADM.DELETE_ALL_ERRORS to delete the whole error queue.exec DBMS_APPLY_ADM.DELETE_ERROR ('<local_transaction_id>');orexec DBMS_APPLY_ADM.DELETE_ALL_ERRORS ();Please be aware that deleting a transaction from error queue, it is not possible to recover that information unless a new Capture process is defined.Avoid new errorsTo avoid data errors in future you have to identify root cause of the errors, depending on the root cause the solution can be to define a handler or maybe implement conflict resolution methodsFor further information about conflict resolution see Streams Conflict Resolution chapter on Streams Replication Administrator's Guide, or See Note 230049.1 - Streams Conflict Resolution.Setting conflict resolution will have effect since the moment that you set it up, but you still need to re-executed failed transactions.Also note that even though conflict resolution is in place you still may receive these errors.ReferencesNOTE:108377.1 - How Do You Read the Output from the Dump Function?NOTE:215372.1 - Streams Apply Process Fails With ORA-1031: Insufficient PrivilegesNOTE:230049.1 - Streams Conflict ResolutionNOTE:273674.1 - Streams Configuration Report and Health Check ScriptNOTE:345119.1 - ORA-26688: missing key in lcrNOTE:39817.1 - Interpreting Raw SQL_TRACE and DBMS_SUPPORT.START_TRACE outputNOTE:405541.1 - Procedure to Print LCRsNOTE:463295.1 - Compare and Converge in an Oracle Streams Replication EnvironmentNOTE:466882.1 - Streams Apply Process Aborts On Decimal Values Using Tg4sybase - Error ORA-28500NOTE:567623.1 - Streams Apply fails with ORA-26572 when the table has a BLOB column and no primary or substitute key definedNOTE:760421.1 - Streams Apply Reports ORA-1422NOTE:858554.1 - SQL to Compare Streams Replicated Table Objects between Source and Target Sites

Master Note for Troubleshooting Streams Apply Errors ORA-1403, ORA-26787 or ORA-26786 (Doc ID 265201.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved. In this Document   Purpose  Last...

Master Notes

Oracle Support Master Note for Troubleshooting Streams Capture 'WAITING For REDO' or INITIALIZING (Doc ID 313279.1)

Master Note for Troubleshooting Streams Capture 'WAITING For REDO' or INITIALIZING (Doc ID 313279.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved. In this DocumentPurpose  Last Review Date  Instructions for the Reader  Troubleshooting Details  CAUSE  1.Incorrect use of RMAN backup command or RMAN Issue   2.Logs Stored in FRA - Flash Recovery Area   3.Logs not registered in dba_registered_archived_log   4.Logs marked as corrupted in system.logmnr_log$   5.required_checkpoint_scn issues   6.Capture Prevalidation of Logs in dba_registered_archived_log  7.Logs deleted in error or damaged due to OS/Storage issues   SOLUTION   1.SCN reported in the Capture state related message - determine logs the SCN relates to   2.Check Capture tracefile to understand if mising logfile detailed  3.Determine if there are logfiles unregistered  4.Determine if there are logs marked as corrupt  5.RMAN - Recovery Manager can be used to identify archivelog files which may be missing   References Applies to: Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 10.2.0.4 - Release: 10.2 to 10.2Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.1.0.7   [Release: 11.1 to 11.1]Oracle Server - Enterprise Edition - Version: 10.1.0.3 to 10.1.0.5   [Release: 10.1 to 10.1]Information in this document applies to any platform.Oracle Server Enterprise Edition - Version: 10.1.0.3 to 10.1.0.5Oracle Server Enterprise Edition - Version: 10.2.0.1 to 10.2.0.4Oracle Server Enterprise Edition - Version: 11.1.0.6 to 11.1.0.7 PurposeStreams capture process can exist in different states. The different states of the capture process is explained in the following note. Note 471713.1 Different States of Capture & Apply Process This article provides troubleshooting steps when capture is stuck in INITIALIZING/DICTIONARY INITIALIZATION, WAITING FOR DICTIONARY REDO: FIRST SCN <SCN> ; WAITING FOR REDO: LAST SCN MINED <SCN> states.Last Review DateOctober 25, 2010  Instructions for the ReaderA Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting. Troubleshooting DetailsCapture process does not advance and appears stuck in one of the following states : select * from v$streams_capture shows : 1. INITALIZING / DICTIONARY INITIALIZATION (the state alternates between these states), or 2. WAITING FOR DICTIONARY REDO: FIRST SCN , or 3. WAITING FOR REDO: LAST SCN MINED , or 4. WAITING FOR DICTIONARY REDO: FILE The nature of the problem could be : - a missing logfile; or - a logfile is not registered; or - a logfile is corrupted; or - Capture is verifying/prechecking logfiles The process will remain in this state until the log is located, it is registered, the corruption is resolved in which case it will also be necessary to reregister the log or the Capture process has checked the logfiles on disk  CAUSECommon issues which can cause Capture to stop in this state are : 1.Incorrect use of RMAN backup command or RMAN Issue Only certain RMAN commands are Streams aware. There are also a number of RMAN related issues where logfiles can be deleted when Streams may still need to access these. Refer to Note 421176.1 for more details. 2.Logs Stored in FRA - Flash Recovery Area It is recommended that the FRA should be avoided with Streams. Archive log files can be deleted and ORA-1291 reported when archive logs are located in a Flash Recovery Area (FRA). More details can be found with respect to FRA in Note 305648.1. This is a recognised behaviour and for this reason it is recommended that archive logs not be located in a FRA where a Capture process will mine from this location. This is documented in the Oracle Streams Concepts and Administration 10g Release 2 (10.2) guide under the section Troublshooting Capture Problems subsection : Are Required Redo Log Files Missing. It should also be pointed out that archive logs may be deleted from the FRA when the instance is shutdown and restarted. Those archive logfiles deleted logs should be reported in the alert.log file. 3.Logs not registered in dba_registered_archived_log This issue is more likely at a downstream database. This could be due to transport issues from upstream to downstream database. 4.Logs marked as corrupted in system.logmnr_log$ Manual intervention will likely be necessary to correct this. 5.required_checkpoint_scn issues Logs may need to be manually deleted which can then affect Capture restart if the required_checkpoint_scn (dba_capture view) is not advancing. This is a key value for Capture as discussed below. 6.Capture Prevalidation of Logs in dba_registered_archived_logThe Capture process has to access all the logs in dba_registered_archived_log before it can start mining changes. Thus, in the case of message :WAITING FOR DICTIONARY REDO: FILE, it may be that logs are being accessed but the state of the capture process has not been updated to indicate that it is prevalidating logs.7.Logs deleted in error or damaged due to OS/Storage issues The following is useful background which should be used to help understand what logs are relevant. In a non RAC environment, all logs from the logfile containing the required_checkpoint_scn have to be present on disk. In a RAC environment, all logs from all threads have to be present with respect to the required_checkpoint_scn; this is also applicable for downstream Capture environments where the primary / upstream database is RAC related but the downstream database may be single instance. These logfiles are also added to the dba_registered_archived_log view. Before Capture starts mining changes , all the logs in the dba_registered_archived_log have to be located. On downstream Capture environments, this can present confusion as the Capture process may appear to be stuck / not moving (due to large numbers of logs in the view) whilst in fact it is actually accessing log files. This is more of an issue for Downstream Capture environments where a Capture process may be down for some time but logs are still being registered by the RFS processes associated with the primary database. The logfiles that a Capture process requires on restart as specified by the required_checkpoint_scn can be located from the Streams Healthcheck report output - Note 273674.1. Refer to the section headed : '++ Minimum Archive Log Necessary to Restart Capture ++' Note: the same information can also be obtain from Note 275323.1 or Note 290143.1. SOLUTION In order to determine the problem logfile(s) and understand what is going on please provide to Oracle Suport Services the Streams Healthcheck output (Note 273674.1note /)) and the alert.log file(s) from Upstream (if relevant) and downstream databases. Note: please provide details from all nodes. Additionally, please collect the following:1.SCN reported in the Capture state related message - determine logs the SCN relates to connect / as sysdbaset pagesize 1000 col first_scn format 999999999999999999 col next_scn format 999999999999999999 alter session set nls_date_format='dd-mon-yyyy hh24:mi:ss'; select source_database,thread#,sequence#,name,modified_time,first_scn,next_scn,dictionary_begin,dictionary_end from dba_registered_archived_log where between first_scn and next_scn; Note : If the reported is the corresponds to the next_scn SCN value for a thread then it is likely the problem is with the next log in that thread; it may be missing or not registered. 2.Check Capture tracefile to understand if mising logfile detailedIf no trace file is present, identify the Capture and Reader processes from the following : connect / as sysdba set pagesize 1000 -- Capture define capture_name='' select p.spid Spid, 'C00'||c.capture#||' '||upper(lp.role) "Capture Process ", c.capture_name "Capture Name", p.pga_used_mem "PGA Memory Used", p.pga_alloc_mem "PGA Memory Allocated", p.pga_max_mem "PGA Maximum Memory" from v$streams_capture c, v$logmnr_process lp, v$session s, v$process p where c.logminer_id = lp.session_id and lp.role in ('reader','preparer','builder') and lp.sid = s.sid and lp.serial# = s.serial# and s.paddr = p.addr and c.capture_name = '&capture_name' union select p.spid, 'C00'||c.capture#||' Coordinator', c.capture_name, p.pga_used_mem, p.pga_alloc_mem, p.pga_max_mem from v$streams_capture c, v$session s, v$process p where c.sid = s.sid and c.serial# = s.serial# and s.paddr = p.addr and c.capture_name = '&capture_name' order by 6,5; Use the SPID (OS Process or thread id) for both the Capture process and Reader process as follows : Note: it may be that a reader process does not exist at the point a particular state is being reported.  connect / as sysdba -- Capture BOTH Process and Reader Process :oradebug setospid -- event 1349 / level 32768 decimal oradebug event 1349 trace name context forever, level 32768 oradebug setospid   -- event 1349 / level 32768 decimal  oradebug event 1349 trace name context forever, level 32768 -- wait 5 minutes, disable trace : oradebug event 1349 trace name context off oradebug setospid   oradebug event 1349 trace name context off  Check the background_dump_dest location for trace files containing the process ids (spids) from the above.  The trace files should identify the thread / sequence number and the problematic logfile which Capture is looking for. Note: ensure that you have the latest patchset installed otherwise relevant trace information may not be present. If it is still not obvious what the file being accessed is specify more complete event trace : oradebug event 1349 trace name context forever, level 26148863 -- wait 5 minutes, disable trace : oradebug event 1349 trace name context off For example, in a Capture trace file you may see something like the following : krvxalfs: Error 308 raised while opening log /bugmnt21/em/celclnx8/tar20093554.6/app/oracle/product/10.2.0.3/dbs/arch1_35_677241985.dbf. i.e ora-308 indicates : missing logfile 3.Determine if there are logfiles unregisteredIf there is a gap in the registered logfiles, the Capture process will stop. Specify the name of your capture process below as well as thread number, i.e repeat for each thread used. Note : The plsql block will need be be run more than once for RAC configurations. connect / as sysdbaset serveroutput on declare -- amend the following as required. user_thread number := 1; -- change as necessaryuser_capture varchar2(100) := ''; --change as necessary rcscn number; minseq number; maxseq number; rseq number; cnt number; cursor mlsns is select user_thread thread, sequence# from ( select (lvl + minseq) sequence# from (select * from (select level lvl from dual connect by level <= maxseq - minseq)) minus select sequence# from dba_registered_archived_log where thread#=user_thread) order by 1; begin select required_checkpoint_scn into rcscn from dba_capture where capture_name = user_capture; select min(sequence#) into rseq from dba_registered_archived_log where thread# = user_thread and consumer_name = user_capture and rcscn between first_scn and next_scn; select min(sequence#) into minseq from dba_registered_archived_log where thread# = user_thread and consumer_name = user_capture and sequence# >= rseq; select max(sequence#) into maxseq from dba_registered_archived_log where thread# = user_thread and consumer_name = user_capture; select count(*) into cnt from ( select (lvl + minseq) sequence# from (select * from (select level lvl from dual connect by level <= maxseq - minseq)) minus select sequence# from dba_registered_archived_log where thread#=user_thread) ; if (cnt is not null) then for rec in mlsns loop dbms_output.put_line('Log not registered ' ||rec.sequence#); end loop; else dbms_output.put_line('There are no gaps in registered logs.'); end if; end; / For any logfile which are not registered, ensure that these are on disk and register these using : alter database register or replace logical logfile '&fqfn' for '&capture_name'; 4.Determine if there are logs marked as corruptA corrupt archive log file is indicated by the contents field of system.logmnr_log$ having value 16. connect / as sysdba set pagesize 1000 select * from system.logmnr_log$ where contents = 16 order by sequence#; If any logs as marked as corrupted , proceed as follows : - dump out the logfile; the dump pf the logfile will appear in the user_dump_dest location.connect / as sysdba oradebug setmypidoradebug unlimitalter session set events '1354 trace name context forever, level 32768'; alter session set events '1348 trace name context forever, level 1032'; alter system dump logfile '&fqfn'; It may be that logfile will need to be restored or copied once again from the primary. One the above has been done the logfile can be reregistered as follows : - take a backup/copy of system.logmnr_log$ : connect / as sysdba create table system.logmnr_log$_bak as select * from system.logmnr_log$; - delete the entry in system.logmnr_log$ which is marked as corrupted :  delete from system.logmnr_log$ where contents = 16 and thread# = and sequence# = ; commit; - reregister the logfile once again as detailed above.5.RMAN - Recovery Manager can be used to identify archivelog files which may be missing Note: this can only be used on upstream environments - not downstream databases. This is due to the fact that no controlfile entry is created for a registered logfile in the downstream database.For Oracle 10.2, logfiles being mined by a Capture process are also logged to the alert.log. This information together with the Capture restart information from the Streams Healthcheck as well as logfile validation information reported by RMAN can be useful in identifying those logfile(s) that are missing. The RMAN command 'crosscheck archivelog' command can check the physical presence of an archive log file. Note: you do not have to be familiar with RMAN for backup and restore or have any experience of RMAN to use the commands detailed below.The approach when using RMAN is as follows : - First of all determine where a Capture process will start from it were restarted. This would be a convenient point to specify where to check the existence of logs from. - Record the restart SCN , the Thread number and Log Sequence number.  Note: The Thread number and the Log Sequence number should typically be visible in the logfile name. This may not be the case however and will depends on the setting of the parameter : log_archive_format.Use either the restart Capture SCN (YYYYYYYYY) or the Log Sequence (ZZZZ) number from which to start the crosscheck validation. The thread number can also be specified on some commands (X); in order to understand if RAC or multiple threads are present issue : select * from gv$instance; Having performed the above, start RMAN :  Note : 1. If you specify a logfile you will receive no RMAN command output to the screen. Therefore, whilst getting to grips with RMAN you would be advised not to specify a logfile. Please also use set echo on so that reevant commands can be seen in log file. 2. for problems relating to the running of RMAN , refer to the Backup and Recovery Advanced User's Guide.  -- start rman without specifying a logfile. rman target / -- start rman and specify a logfile to record command output rman target / log /tmp/crosscheck.log Useful commands to allow logs on disk to be crosschecked on disk are as follows : -- ensure command used is echoed to log file. set echo on - crosscheck archivelog all; (crosscheck all logs on disk) - crosscheck archivelog from sequence ZZZZ thread X; (* crosscheck all the logs on disk from the sequence number) - crosscheck archivelog from scn = YYYYYYYYYY; (* crosscheck all the logs on disk from the scn) - crosscheck archivelog like '%%'; (crosscheck all the logs on disk from a particular location) - crosscheck archivelog like '%%'; (crosscheck that a specific logfile exists) * above allow the starting point of a the crosscheck command to be specified. Therefore , these should be used together with restart information detailed in the Streams healthcheck. Consider the following example where the Streams Healthcheck reports that the capture process will restart from :Capture will restart from SCN 2381906 in the following file: /la2/1_228_657120968.dbf (date/time) /la1/1_227_657120968.dbf (date/time) The following commands can be used in order to determine whether logs are available on disk or otherwise. rman target / set echo on - crosscheck archivelog all; - crosscheck archivelog from sequence 227 thread 1; - crosscheck archivelog from scn = 2381906; - crosscheck archivelog like '%la1%'; - crosscheck archivelog like '%1_227_657120968.dbf%'; Checking RMAN output If a logfile is not found, RMAN will report : 'validation failed for archived log' error. Note: This is really only significant if a log is missing after the required_checkpoint_scn where a Capture process may restart from. If a logfile is found, RMAN will report : 'validation succeeded for archived log'.... archive log filename= /arch1_127_657120968.dbf recid=127 stamp=673799334 validation failed for archived log archive log filename= /arch1_128_657120968.dbf recid=128 stamp=673806193 validation succeeded for archived log ... To exit RMAN, type : exit Search through the logfile for 'failed' keyword. This will hopefully identify the problematic logfile. i.e (on Unix) grep -i failed /tmp/crosscheck.log If all logfiles are validated and RMAN performed the crosscheck validation from the location that Capture would restart from , then the logs missing may likely come after the last logs reported by RMAN. Further inspection may be required.  ReferencesBUG:5770059 - REQUIRED_CHECKPOINT_SCN NOT MOVING FORWARDBUG:6154377 - NO LOGMNR CHECKPOINTS ARE BEING EXECUTED

Master Note for Troubleshooting Streams Capture 'WAITING For REDO' or INITIALIZING (Doc ID 313279.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved.  In this Document Purpose  Last Review...

Master Notes

Oracle Support Master Note for Streams Setup Scripts (Doc ID 789445.1)

Master Note for Streams Setup Scripts (Doc ID 789445.1)Copyright (c) 2010, Oracle Corporation. All Rights Reserved. What is being announced?  What do you need to do?     Initial Setup     Advanced Setup     Administration     FAQ     Troubleshooting     Monitoring  Who to contact for more information?  ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 10.1.0.2 to 11.2.0.2 - Release: 10.1 to 11.2Information in this document applies to any platform.Oracle Server Enterprise Edition - Version: 10.1.0.2 to 11.2.0.2 What is being announced?This note is a compilation of various types of streams replication setup scripts . The master index note provides links to scripts that are to be used as sample scripts . User needs to make all the required changes according to their requirements.These scripts are to be used by DBAs and application developers to setup streams replication.This sample code is provided for educational purposes only and not supported by Oracle Support Services. It has been tested internally, however, and works as documented. We do not guarantee that it will work for you, so be sure to test it in your environment before relying on it.Proofread this sample code before using it! Due to the differences in the way text editors, e-mail packages and operating systems handle text formatting (spaces, tabs and carriage returns), this sample code may not be in an executable state when you first receive it. Check over the sample code to ensure that errors of this type are corrected. var LEO_HIGHLIGHTS_INFINITE_LOOP_COUNT = 300; var LEO_HIGHLIGHTS_MAX_HIGHLIGHTS = 50; var LEO_HIGHLIGHTS_IFRAME_TOP_ID = "leoHighlights_top_iframe"; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID = "leoHighlights_bottom_iframe"; var LEO_HIGHLIGHTS_IFRAME_DIV_ID = "leoHighlights_iframe_modal_div_container"; var LEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_WIDTH = 520; var LEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_HEIGHT = 391; var LEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_WIDTH = 520; var LEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_HEIGHT = 665; var LEO_HIGHLIGHTS_IFRAME_TOP_POS_X = 0; var LEO_HIGHLIGHTS_IFRAME_TOP_POS_Y = 0; var LEO_HIGHLIGHTS_IFRAME_TOP_WIDTH = 520; var LEO_HIGHLIGHTS_IFRAME_TOP_HEIGHT = 294; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_POS_X = 96; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_POS_Y = 294; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_WIDTH = 425; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_HEIGHT = 97; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_WIDTH = 425; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_HEIGHT = 371; var LEO_HIGHLIGHTS_SHOW_DELAY_MS = 300; var LEO_HIGHLIGHTS_HIDE_DELAY_MS = 750; var LEO_HIGHLIGHTS_SHOW_DELAY_NO_UNDER_MS = 850; var LEO_HIGHLIGHTS_BACKGROUND_STYLE_DEFAULT = "transparent none repeat scroll 0% 0%"; var LEO_HIGHLIGHTS_BACKGROUND_STYLE_HOVER = "rgb(245, 245, 0) none repeat scroll 0% 0%"; var LEO_HIGHLIGHTS_ROVER_TAG = "711-36858-13496-14"; createInlineScriptElement("var%20LEO_HIGHLIGHTS_DEBUG%20%3D%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20false%3B%0Avar%20LEO_HIGHLIGHTS_DEBUG_POS%20%3D%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20false%3B%0A%20%20%20%0Avar%20_leoHighlightsPrevElem%20%3D%20null%3B%0Avar%20_leoHighlightsSnoozed%20%3D%20false%3B%0A%0A/**%0A%20*%20Checks%20if%20the%20passed%20in%20class%20exists%0A%20*%20@param%20c%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsClassExists%28c%29%20%7B%0A%20%20%20return%20typeof%28c%29%20%3D%3D%20%22function%22%20%26%26%20typeof%28c.prototype%29%20%3D%3D%20%22object%22%20?%20true%20%3A%20false%3B%0A%7D%0A%0A/**%0A%20*%20Checks%20if%20the%20firebug%20console%20is%20available%0A%20*%20@param%20c%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsFirebugConsoleAvailable%28c%29%20%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsClassExists%28_FirebugConsole%29%20%26%26%20%0A%20%20%20%20%20%20%20%20%20window.console%20%26%26%20console.log%20%26%26%20%0A%20%20%20%20%20%20%20%20%20%28console%20instanceof%20_FirebugConsole%29%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20return%20true%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%7D%0A%20%20%20catch%28e%29%7B%7D%0A%20%20%20%0A%20%20%20return%20false%3B%0A%7D%20%0A%0A%0A/**%0A%20*%20General%20method%20used%20to%20debug%20exceptions%0A%20*%20%0A%20*%20@param%20location%0A%20*%20@param%20e%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsReportExeception%28location%2Ce%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsFirebugConsoleAvailable%28%29%20||LEO_HIGHLIGHTS_DEBUG%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20var%20logString%3Dlocation%2B%22%3A%20%22%2Be%2B%22%5Cn%5Ct%22%2Be.name%2B%22%5Cn%5Ct%22%2B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%28e.number%260xFFFF%29%2B%22%5Cn%5Ct%22%2Be.description%3B%0A%20%20%20%20%20%20%20%20%20if%28_leoHighlightsFirebugConsoleAvailable%28%29%29%0A%20%20%20%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20console.error%28logString%29%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20console.trace%28%29%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28LEO_HIGHLIGHTS_DEBUG%29%0A%20%20%20%20%20%20%20%20%20alert%28logString%29%3B%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%7B%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20log%20a%20string%20to%20the%20firebug%20console%0A%20*%20%0A%20*%20@param%20str%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsDebugLog%28str%29%0A%7B%20%20%20%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsFirebugConsoleAvailable%28%29%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20console.log%28typeof%28_FirebugConsole%29%2B%22%20%22%2Bstr%29%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22_leoHighlightsDebugLog%28%29%20%22%2Bstr%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20get%20an%20attribute%20and%20decode%20it.%0A%20*%20%0A%20*%20@param%20elem%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetAttrib%28elem%2Cid%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20val%3Delem.getAttribute%28id%29%3B%0A%20%20%20%20%20%20return%20decodeURI%28val%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22_leoHighlightsGetAttrib%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%20%20%20return%20null%3B%0A%7D%0A%0A/**%0A%20*%20Checks%20if%20this%20is%20within%20a%20frame%20by%20checking%20for%20a%20parent.%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsIsFrame%28%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%20%20%0A%20%20%20%20%20%20return%20%28window%21%3Dtop%29%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22_leoHighlightsIsFrame%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%0A%20%20%20return%20false%3B%0A%7D%0A%0A/**%0A%20*%20This%20is%20a%20dimensions%20object%0A%20*%20%0A%20*%20@param%20width%0A%20*%20@param%20height%0A%20*%20@return%0A%20*/%0Afunction%20LeoHighlightsDimension%28width%2Cheight%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09this.width%3Dwidth%3B%0A%20%20%20%09this.height%3Dheight%3B%0A%20%20%20%09this.toString%3Dfunction%28%29%20%7B%20return%20%28%22%28%22%2Bthis.width%2B%22%2C%22%2Bthis.height%2B%22%29%22%29%3B%7D%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22new%20LeoHighlightsDimension%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20is%20a%20Position%20object%0A%20*%20%0A%20*%20@param%20x%0A%20*%20@param%20y%0A%20*%20@return%0A%20*/%0Afunction%20LeoHighlightsPosition%28x%2Cy%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09this.x%3Dx%3B%0A%20%20%20%09this.y%3Dy%3B%0A%20%20%20%09this.toString%3Dfunction%28%29%20%7B%20return%20%28%22%28%22%2Bthis.x%2B%22%2C%22%2Bthis.y%2B%22%29%22%29%3B%7D%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22new%20LeoHighlightsPosition%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0Avar%20LEO_HIGHLIGHTS_ADJUSTMENT%20%3D%20new%20LeoHighlightsPosition%283%2C3%29%3B%0Avar%20LEO_HIGHLIGHTS_IFRAME_TOP_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_TOP_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_TOP_HEIGHT%29%3B%0Avar%20LEO_HIGHLIGHTS_IFRAME_BOTTOM_HOVER_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_HEIGHT%29%3B%0Avar%20LEO_HIGHLIGHTS_IFRAME_BOTTOM_CLICK_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_HEIGHT%29%3B%0A%0Avar%20LEO_HIGHLIGHTS_DIV_HOVER_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_HEIGHT%29%3B%0Avar%20LEO_HIGHLIGHTS_DIV_CLICK_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_HEIGHT%29%3B%0A%0A%0A/**%0A%20*%20Sets%20the%20size%20of%20the%20passed%20in%20element%0A%20*%20%0A%20*%20@param%20elem%0A%20*%20@param%20dim%20%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsSetSize%28elem%2Cdim%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09//%20Set%20the%20popup%20location%0A%20%20%20%09elem.style.width%20%3D%20dim.width%20%2B%20%22px%22%3B%0A%20%20%20%09if%28elem.width%29%0A%20%20%20%09%09elem.width%3Ddim.width%3B%0A%20%20%20%09elem.style.height%20%20%3D%20dim.height%20%2B%20%22px%22%3B%0A%20%20%20%09if%28elem.height%29%0A%20%20%20%09%09elem.height%3Ddim.height%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsSetSize%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20can%20be%20used%20for%20a%20simple%20one%20argument%20callback%0A%20*%0A%20*%20@param%20callName%0A%20*%20@param%20argName%0A%20*%20@param%20argVal%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsSimpleGwCallBack%28callName%2CargName%2C%20argVal%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20if%28argName%29%0A%20%20%20%20%20%20%09gwObj.addParam%28argName%2CargVal%29%3B%0A%20%20%20%20%20%20gwObj.callName%28callName%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsSimpleGwCallBack%28%29%20%22%2BcallName%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20gets%20a%20url%20argument%20from%20the%20current%20document.%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetUrlArg%28url%2C%20name%20%29%0A%7B%0A%09%20%20name%20%3D%20name.replace%28/[%5C[]/%2C%22%5C%5C%5C[%22%29.replace%28/[%5C]]/%2C%22%5C%5C%5C]%22%29%3B%0A%09%20%20var%20regexS%20%3D%20%22[%5C%5C?%26]%22%2Bname%2B%22%3D%28[^%26%23]*%29%22%3B%0A%09%20%20var%20regex%20%3D%20new%20RegExp%28%20regexS%20%29%3B%0A%09%20%20var%20results%20%3D%20regex.exec%28url%29%3B%0A%09%20%20if%28%20results%20%3D%3D%20null%20%29%0A%09%20%20%20%20return%20%22%22%3B%0A%09%20%20else%0A%09%20%20%20%20return%20results[1]%3B%0A%7D%0A%0A%0A/**%0A%20*%20This%20allows%20to%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsRedirectTop%28url%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%20%20%09%0A%20%20%20%09top.location%3Durl%3B%09%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsRedirectTop%28%29%22%2Ce%29%3B%0A%20%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20find%20an%20element%20by%20Id%0A%20*%20%0A%20*%20@param%20elemId%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsFindElementById%28elemId%2Cdoc%29%0A%7B%0A%09try%0A%09%7B%0A%09%20%20%20if%28doc%3D%3Dnull%29%0A%09%20%20%20%20%20%20doc%3Ddocument%3B%0A%09%20%20%20%0A%09%09var%20elem%3Ddoc.getElementById%28elemId%29%3B%0A%09%09if%28elem%29%0A%09%09%09return%20elem%3B%0A%09%09%0A%09%09/*%20This%20is%20the%20handling%20for%20IE%20*/%0A%09%09if%28doc.all%29%0A%09%09%7B%0A%09%09%09elem%3Ddoc.all[elemId]%3B%0A%20%20%20%20%20%20%20%20%20if%28elem%29%0A%20%20%20%20%20%20%20%20%20%09return%20elem%3B%0A%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%20for%20%28%20var%20i%20%3D%20%28document.all.length-1%29%3B%20i%20%3E%3D%200%3B%20i--%29%20%7B%0A%20%20%20%20%20%20%20%20%20%09elem%3Ddoc.all[i]%3B%0A%20%20%20%20%20%20%20%20%20%09if%28elem.id%3D%3DelemId%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20return%20elem%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%09%09%7D%0A%09%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsFindElementById%28%29%22%2Ce%29%3B%0A%20%20%20%7D%0A%09return%20null%3B%0A%7D%0A%0A/**%0A%20*%20Get%20the%20location%20of%20one%20element%20relative%20to%20a%20parent%20reference%0A%20*%0A%20*%20@param%20ref%0A%20*%20%20%20%20%20%20%20%20%20%20%20the%20reference%20element%2C%20this%20must%20be%20a%20parent%20of%20the%20passed%20in%0A%20*%20%20%20%20%20%20%20%20%20%20%20element%0A%20*%20@param%20elem%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetLocation%28ref%2C%20elem%29%20%7B%0A%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsGetLocation%20%22%2Belem.id%29%3B%0A%20%20%20%0A%20%20%20var%20count%20%3D%200%3B%0A%20%20%20var%20location%20%3D%20new%20LeoHighlightsPosition%280%2C0%29%3B%0A%20%20%20var%20walk%20%3D%20elem%3B%0A%20%20%20while%20%28walk%20%21%3D%20null%20%26%26%20walk%20%21%3D%20ref%20%26%26%20count%20%3C%20LEO_HIGHLIGHTS_INFINITE_LOOP_COUNT%29%20%7B%0A%20%20%20%20%20%20location.x%20%2B%3D%20walk.offsetLeft%3B%0A%20%20%20%20%20%20location.y%20%2B%3D%20walk.offsetTop%3B%0A%20%20%20%20%20%20walk%20%3D%20walk.offsetParent%3B%0A%20%20%20%20%20%20count%2B%2B%3B%0A%20%20%20%7D%0A%20%20%20%0A%20%20%20_leoHighlightsDebugLog%28%22Location%20is%3A%20%22%2Belem.id%2B%22%20-%20%22%2Blocation%29%3B%0A%0A%20%20%20return%20location%3B%0A%7D%0A%0A/**%0A%20*%20This%20is%20used%20to%20update%20the%20position%20of%20an%20element%20as%20a%20popup%0A%20*%20%0A%20*%20@param%20IFrame%0A%20*%20@param%20anchor%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsUpdatePopupPos%28iFrame%2Canchor%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20//%20Gets%20the%20scrolled%20location%20for%20x%20and%20y%0A%20%20%20%20%20%20var%20scrolledPos%3Dnew%20LeoHighlightsPosition%280%2C0%29%3B%0A%20%20%20%20%20%20if%28%20self.pageYOffset%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20scrolledPos.x%20%3D%20self.pageXOffset%3B%0A%20%20%20%20%20%20%20%20%20scrolledPos.y%20%3D%20self.pageYOffset%3B%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.documentElement%20%26%26%20document.documentElement.scrollTop%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20scrolledPos.x%20%3D%20document.documentElement.scrollLeft%3B%0A%20%20%20%20%20%20%20%20%20scrolledPos.y%20%3D%20document.documentElement.scrollTop%3B%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.body%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20scrolledPos.x%20%3D%20document.body.scrollLeft%3B%0A%20%20%20%20%20%20%20%20%20scrolledPos.y%20%3D%20document.body.scrollTop%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20total%20dimensions%20to%20see%20what%20scroll%20bars%20might%20be%20active%20*/%0A%20%20%20%20%20%20var%20totalDim%3Dnew%20LeoHighlightsDimension%280%2C0%29%0A%20%20%20%20%20%20if%20%28document.all%20%26%26%20document.documentElement%20%26%26%20%0A%20%20%20%20%20%20%09document.documentElement.clientHeight%26%26document.documentElement.clientWidth%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%09totalDim.width%20%3D%20document.documentElement.scrollWidth%3B%0A%20%20%20%20%20%20%09totalDim.height%20%3D%20document.documentElement.scrollHeight%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20else%20if%20%28document.all%29%0A%20%20%20%20%20%20%7B%20/*%20This%20is%20in%20IE%20*/%0A%20%20%20%20%20%09%20%09totalDim.width%20%3D%20document.body.scrollWidth%3B%0A%20%20%20%20%20%20%09totalDim.height%20%3D%20document.body.scrollHeight%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20else%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%09%20totalDim.width%20%3D%20document.width%3B%0A%20%20%20%20%20%20%09%20totalDim.height%20%3D%20document.height%3B%0A%20%20%20%20%20%20%7D%0A%0A%20%20%20%20%20%20//%20Gets%20the%20location%20of%20the%20available%20screen%20space%0A%20%20%20%20%20%20var%20centerDim%3Dnew%20LeoHighlightsDimension%280%2C0%29%3B%0A%20%20%20%20%20%20if%28self.innerWidth%20%26%26%20self.innerHeight%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20centerDim.width%20%3D%20self.innerWidth-%28totalDim.height%3Eself.innerHeight?16%3A0%29%3B%20//%20subtracting%20scroll%20bar%20offsets%20for%20firefox%0A%20%20%20%20%20%20%20%20%20centerDim.height%20%3D%20self.innerHeight-%28totalDim.width%3Eself.innerWidth?16%3A0%29%3B%20%20//%20subtracting%20scroll%20bar%20offsets%20for%20firefox%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.documentElement%20%26%26%20document.documentElement.clientHeight%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20centerDim.width%20%3D%20document.documentElement.clientWidth%3B%0A%20%20%20%20%20%20%20%20%20centerDim.height%20%3D%20document.documentElement.clientHeight%3B%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.body%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20centerDim.width%20%3D%20document.body.clientWidth%3B%0A%20%20%20%20%20%20%20%20%20centerDim.height%20%3D%20document.body.clientHeight%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20//%20Get%20the%20current%20dimension%20of%20the%20popup%20element%0A%20%20%20%20%20%20var%20iFrameDim%3Dnew%20LeoHighlightsDimension%28iFrame.offsetWidth%2CiFrame.offsetHeight%29%0A%20%20%20%20%20%20if%20%28iFrameDim.width%20%3C%3D%200%29%0A%20%20%20%20%20%20%09iFrameDim.width%20%3D%20iFrame.style.width.substring%280%2C%20iFrame.style.width.indexOf%28%27px%27%29%29%3B%0A%20%20%20%20%20%20if%20%28iFrameDim.height%20%3C%3D%200%29%0A%20%20%20%20%20%20%09iFrameDim.height%20%3D%20iFrame.style.height.substring%280%2C%20iFrame.style.height.indexOf%28%27px%27%29%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Calculate%20the%20position%2C%20lower%20right%20hand%20corner%20by%20default%20*/%0A%20%20%20%20%20%20var%20position%3Dnew%20LeoHighlightsPosition%280%2C0%29%3B%0A%20%20%20%20%20%20position.x%3DscrolledPos.x%2BcenterDim.width-iFrameDim.width-LEO_HIGHLIGHTS_ADJUSTMENT.x%3B%0A%20%20%20%20%20%20position.y%3DscrolledPos.y%2BcenterDim.height-iFrameDim.height-LEO_HIGHLIGHTS_ADJUSTMENT.y%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28anchor%21%3Dnull%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20//centerDim%20in%20relation%20to%20the%20anchor%20element%20if%20available%0A%20%20%20%20%20%20%20%20%20var%20topOrBottom%20%3D%20false%3B%0A%20%20%20%20%20%20%20%20%20var%20anchorPos%3D_leoHighlightsGetLocation%28document.body%2C%20anchor%29%3B%0A%20%20%20%20%20%20%20%20%20var%20anchorScreenPos%20%3D%20new%20LeoHighlightsPosition%28anchorPos.x-scrolledPos.x%2CanchorPos.y-scrolledPos.y%29%3B%0A%20%20%20%20%20%20%20%20%20var%20anchorDim%3Dnew%20LeoHighlightsDimension%28anchor.offsetWidth%2Canchor.offsetHeight%29%0A%20%20%20%20%20%20%20%20%20if%20%28anchorDim.width%20%3C%3D%200%29%0A%20%20%20%20%20%20%20%20%20%09anchorDim.width%20%3D%20anchor.style.width.substring%280%2C%20anchor.style.width.indexOf%28%27px%27%29%29%3B%0A%20%20%20%20%20%20%20%20%20if%20%28anchorDim.height%20%3C%3D%200%29%0A%20%20%20%20%20%20%20%20%20%09anchorDim.height%20%3D%20anchor.style.height.substring%280%2C%20anchor.style.height.indexOf%28%27px%27%29%29%3B%0A%0A%20%20%20%20%20%20%20%20%20//%20Check%20if%20the%20popup%20can%20be%20shown%20above%20or%20below%20the%20element%0A%20%20%20%20%20%20%20%20%20if%20%28centerDim.height%20-%20anchorDim.height%20-%20iFrameDim.height%20-%20anchorScreenPos.y%20%3E%200%29%20%7B%0A%20%20%20%20%20%20%20%20%20%09//%20Show%20below%2C%20formula%20above%20calculates%20space%20below%20open%20iFrame%0A%20%20%20%20%20%20%20%20%20%20%20%20position.y%20%3D%20anchorPos.y%20%2B%20anchorDim.height%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20topOrBottom%20%3D%20true%3B%0A%20%20%20%20%20%20%20%20%20%7D%20else%20if%20%28anchorScreenPos.y%20-%20anchorDim.height%20-%20iFrameDim.height%20%3E%200%29%20%7B%0A%20%20%20%20%20%20%20%20%20%09//%20Show%20above%2C%20formula%20above%20calculates%20space%20above%20open%20iFrame%0A%20%20%20%20%20%20%20%20%20%09position.y%20%3D%20anchorPos.y%20-%20iFrameDim.height%20-%20anchorDim.height%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20topOrBottom%20%3D%20true%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsUpdatePopupPos%28%29%20-%20topOrBottom%3A%20%22%2BtopOrBottom%29%3B%0A%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%20if%20%28topOrBottom%29%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20//%20We%20attempt%20top%20attach%20the%20window%20to%20the%20element%0A%20%20%20%20%20%20%20%20%20%09position.x%20%3D%20anchorPos.x%20-%20iFrameDim.width%20/%202%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20if%20%28position.x%20%3C%200%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%09position.x%20%3D%200%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20else%20if%20%28position.x%20%2B%20iFrameDim.width%20%3E%20scrolledPos.x%20%2B%20centerDim.width%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%09position.x%20%3D%20scrolledPos.x%20%2B%20centerDim.width%20-%20iFrameDim.width%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsUpdatePopupPos%28%29%20-%20topOrBottom%3A%20%22%2Bposition%29%3B%0A%20%20%20%20%20%20%20%20%20%7D%20else%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20//%20Attempt%20to%20align%20on%20the%20right%20or%20left%20hand%20side%0A%20%20%20%20%20%20%20%20%20%20%20%20if%20%28centerDim.width%20-%20anchorDim.width%20-%20iFrameDim.width%20-%20anchorScreenPos.x%20%3E%200%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20position.x%20%3D%20anchorPos.x%20%2B%20anchorDim.width%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20else%20if%20%28anchorScreenPos.x%20-%20anchorDim.width%20-%20iFrameDim.width%20%3E%200%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%09position.x%20%3D%20anchorPos.x%20-%20anchorDim.width%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20else%20%20//%20default%20to%20below%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20position.y%20%3D%20anchorPos.y%20%2B%20anchorDim.height%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsUpdatePopupPos%28%29%20-%20sideBottom%3A%20%22%2Bposition%29%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Make%20sure%20that%20we%20don%27t%20go%20passed%20the%20right%20hand%20border%20*/%0A%20%20%20%20%20%20if%28position.x%2BiFrameDim.width%3EcenterDim.width-20%29%0A%20%20%20%20%20%20%09position.x%3DcenterDim.width-%28iFrameDim.width%2B20%29%3B%0A%20%20%20%20%20%20%09%09%0A%20%20%20%20%20%20//%20Make%20sure%20that%20we%20didn%27t%20go%20passed%20the%20start%0A%20%20%20%20%20%20if%28position.x%3C0%29%0A%20%20%20%20%20%20%20%20%20position.x%3D0%3B%0A%20%20%20%20%20%20if%28position.y%3C0%29%0A%20%20%20%20%20%20%09position.y%3D0%3B%0A%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22Popup%20info%20id%3A%20%20%20%20%20%20%20%22%20%2BiFrame.id%2B%22%20-%20%22%2Banchor.id%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cnscrolled%20%20%20%20%20%20%20%20%20%20%22%20%2B%20scrolledPos%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cncenter/visible%20%20%20%20%22%20%2B%20centerDim%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cnanchor%20%28absolute%29%20%22%20%2B%20anchorPos%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cnanchor%20%28screen%29%20%20%20%22%20%2B%20anchorScreenPos%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5CnSize%20%28anchor%29%20%20%20%20%20%22%20%2B%20anchorDim%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5CnSize%20%28popup%29%20%20%20%20%20%20%22%20%2B%20iFrameDim%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5CnResult%20pos%20%20%20%20%20%20%20%20%22%20%2B%20position%29%3B%0A%0A%20%20%20%20%20%20//%20Set%20the%20popup%20location%0A%20%20%20%20%20%20iFrame.style.left%20%3D%20position.x%20%2B%20%22px%22%3B%0A%20%20%20%20%20%20iFrame.style.top%20%20%3D%20position.y%20%2B%20%22px%22%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsUpdatePopupPos%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20show%20the%20passed%20in%20element%20as%20a%20popup%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@param%20size%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsShowPopup%28anchorId%2Csize%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%20%09var%20popup%3Dnew%20LeoHighlightsPopup%28anchorId%2Csize%29%3B%0A%20%20%20%09popup.show%28%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsShowPopup%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20transform%20the%20passed%20in%20url%20to%20a%20rover%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetRoverUrl%28url%29%0A%7B%0A%09var%20rover%3DLEO_HIGHLIGHTS_ROVER_TAG%3B%0A%09var%20roverUrl%3D%22http%3A//rover.ebay.com/rover/1/%22%2Brover%2B%22/4?%26mpre%3D%22%2BencodeURI%28url%29%3B%0A%09%0A%09return%20roverUrl%3B%0A%7D%0A%0A/**%0A%20*%20Sets%20the%20size%20of%20the%20bottom%20windown%20part%0A%20*%20%0A%20*%20@param%20size%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsSetBottomSize%28size%2CclickId%29%0A%7B%0A%20%20%20/*%20Get%20the%20elements%20*/%0A%20%20%20var%20iFrameBottom%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20var%20iFrameDiv%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_DIV_ID%29%3B%0A%0A%20%20%20/*%20Figure%20out%20the%20correct%20sizes%20*/%0A%20%20%20var%20iFrameBottomSize%3D%28size%3D%3D1%29?LEO_HIGHLIGHTS_IFRAME_BOTTOM_CLICK_SIZE%3ALEO_HIGHLIGHTS_IFRAME_BOTTOM_HOVER_SIZE%3B%0A%20%20%20var%20divSize%3D%28size%3D%3D1%29?LEO_HIGHLIGHTS_DIV_CLICK_SIZE%3ALEO_HIGHLIGHTS_DIV_HOVER_SIZE%3B%0A%0A%20%20%20/*%20Refresh%20the%20iFrame%27s%20url%2C%20by%20removing%20the%20size%20arg%20and%20adding%20it%20again%20*/%0A%20%20%20leoHighlightsUpdateUrl%28iFrameBottom%2Csize%2CclickId%29%3B%0A%0A%20%20%20/*%20Clear%20the%20hover%20flag%2C%20if%20the%20user%20shows%20this%20at%20full%20size%20*/%0A%20%20%20_leoHighlightsPrevElem.hover%3Dsize%3D%3D1?false%3Atrue%3B%0A%0A%20%20%20_leoHighlightsSetSize%28iFrameBottom%2CiFrameBottomSize%29%3B%0A%20%20%20_leoHighlightsSetSize%28iFrameDiv%2CdivSize%29%3B%0A%7D%0A%0A/**%0A%20*%20Class%20for%20a%20Popup%20%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@param%20size%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20LeoHighlightsPopup%28anchorId%2Csize%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22LeoHighlightsPopup%28%29%20%22%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09this.anchorId%3DanchorId%3B%0A%20%20%20%09this.anchor%3D_leoHighlightsFindElementById%28this.anchorId%29%3B%0A%20%20%20%09this.topIframe%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%20%20%20%20%20%20this.bottomIframe%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20%09this.iFrameDiv%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_DIV_ID%29%3B%0A%20%20%20%09%0A%20%20%20%09this.topIframe.src%3Dunescape%28this.anchor.getAttribute%28%27leoHighlights_url_top%27%29%29%3B%3B%0A%20%20%20%20%20%20this.bottomIframe.src%3Dunescape%28this.anchor.getAttribute%28%27leoHighlights_url_bottom%27%29%29%3B%3B%0A%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%221%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.topIframe.style.top%2B%22%2C%20%22%2Bthis.topIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%222%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.bottomIframe.style.top%2B%22%2C%20%22%2Bthis.bottomIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09leoHighlightsSetSize%28size%29%3B%0A%20%20%20%09%0A%20%20%20%09this.updatePos%3Dfunction%28%29%20%7B%20_leoHighlightsUpdatePopupPos%28this.iFrameDiv%2Cthis.anchor%29%3B%7D%3B%0A%20%20%20%20%20%20this.show%3Dfunction%28%29%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20this.updatePos%28%29%3B%20%0A%20%20%20%20%20%20%20%20%20%20%20%20this.iFrameDiv.style.visibility%20%3D%20%22visible%22%3B%20%0A%20%20%20%20%20%20%20%20%20%20%20%20this.iFrameDiv.style.display%20%3D%20%22block%22%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20this.updatePos%28%29%3B%0A%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%223%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.topIframe.style.top%2B%22%2C%20%22%2Bthis.topIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%224%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.bottomIframe.style.top%2B%22%2C%20%22%2Bthis.bottomIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20%7D%3B%20%20%20%20%20%20%20%0A%20%20%20%09this.scroll%3Dfunction%28%29%20%7B%20this.updatePos%28%29%3B%7D%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22new%20LeoHighlightsPopup%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20updates%20the%20url%20for%20the%20iFrame%0A%20*%20%0A%20*%20@param%20iFrame%0A%20*%20@param%20size%0A%20*%20@param%20clickId%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsUpdateUrl%28iFrame%2Csize%2CclickId%2CdestUrl%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsUpdateUrl%28%29%20%22%2BdestUrl%29%3B%0A%20%20%20%20%20%20var%20url%3DiFrame.src%3B%0A%20%20%20%20%20%20var%20idx%3Durl.indexOf%28%22%26size%3D%22%29%3B%0A%20%20%20%20%20%20if%28idx%3E%3D0%29%0A%20%20%20%20%20%20%20%20%20url%3Durl.substring%280%2Cidx%29%3B%0A%20%20%20%20%20%20%0A//%20%20%20%20%20%20size%3D1%3B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsUpdateUrl%28%29%20size%3D%22%2Bsize%2B%22%20%20%22%2Burl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28size%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20url%2B%3D%28%22%26size%3D%22%2Bsize%29%3B%0A%20%20%20%20%20%20if%28clickId%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20url%2B%3D%28%22%26clickId%3D%22%2BclickId%29%3B%0A%20%20%20%20%20%20if%28destUrl%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20url%2B%3D%28%22%26url%3D%22%2BdestUrl%29%3B%0A%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsUpdateUrl%28%29%20%22%2Burl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20iFrame.src%3Durl%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsUpdateUrl%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A%0A%0A/**%0A*%0A*%20This%20can%20be%20used%20to%20close%20an%20iframe%0A*%0A*%20@param%20id%0A*%20@return%0A*/%0Afunction%20leoHighlightsSetSize%28size%2CclickId%29%0A%7B%0A%09try%0A%09%7B%0A%09%09/*%20Get%20the%20element%20*/%0A%20%20%09%09var%20iFrameTop%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%0A%20%20%09%09/*%20Figure%20out%20the%20correct%20sizes%20*/%0A%20%20%09%09var%20iFrameTopSize%3DLEO_HIGHLIGHTS_IFRAME_TOP_SIZE%3B%0A%20%20%09%09%0A%20%20%09%09/*%20Refresh%20the%20iFrame%27s%20url%2C%20by%20removing%20the%20size%20arg%20and%20adding%20it%20again%20*/%0A%20%20%09%09leoHighlightsUpdateUrl%28iFrameTop%2Csize%2CclickId%29%3B%0A%20%20%09%09%0A%20%20%09%09_leoHighlightsSetSize%28iFrameTop%2CiFrameTopSize%29%3B%0A%20%20%20%20%20%20_leoHighlightsSetBottomSize%28size%2CclickId%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Clear%20the%20hover%20flag%2C%20if%20the%20user%20shows%20this%20at%20full%20size%20*/%0A%20%20%20%20%20%20if%28size%3D%3D1%26%26_leoHighlightsPrevElem%29%0A%20%20%20%20%20%20%20%20%20_leoHighlightsPrevElem.hover%3Dfalse%3B%20%20%20%20%20%20%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsSetSize%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20Start%20the%20popup%20a%20little%20bit%20delayed.%0A%20*%20Somehow%20IE%20needs%20some%20time%20to%20find%20the%20element%20by%20id.%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@param%20size%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsShowPopup%28anchorId%2Csize%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%09%09var%20elem%3D_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%09%09if%28_leoHighlightsPrevElem%26%26%28_leoHighlightsPrevElem%21%3Delem%29%29%0A%20%20%09%09%09_leoHighlightsPrevElem.shown%3Dfalse%3B%0A%20%20%09%09elem.shown%3Dtrue%3B%0A%09%09_leoHighlightsPrevElem%3Delem%3B%0A%09%09%0A%09%09_leoHighlightsDebugLog%28%22leoHighlightsShowPopup%28%29%20%22%2B_leoHighlightsPrevElem%29%3B%09%09%0A%20%20%20%09%0A%20%20%20%09/*%20FF%20needs%20to%20find%20the%20element%20first%20*/%0A%20%20%20%09_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%20%09%0A%20%20%20%09setTimeout%28%22_leoHighlightsShowPopup%28%5C%27%22%2BanchorId%2B%22%5C%27%2C%5C%27%22%2Bsize%2B%22%5C%27%29%3B%22%2C10%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHighlightsShowPopup%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A*%0A*%20This%20can%20be%20used%20to%20close%20an%20iframe%0A*%0A*%20@param%20id%0A*%20@return%0A*/%0Afunction%20leoHighlightsHideElem%28id%29%0A%7B%0A%09try%0A%09%7B%0A%09%09/*%20Get%20the%20appropriate%20sizes%20*/%0A%20%20%09%09var%20elem%3D_leoHighlightsFindElementById%28id%29%3B%0A%20%20%09%09if%28elem%29%0A%20%20%09%09%09elem.style.visibility%3D%22hidden%22%3B%0A%20%20%09%09%0A%20%20%09%09/*%20Clear%20the%20page%20for%20the%20next%20run%20through%20*/%0A%20%20%09%09var%20iFrame%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%20%20%09%09if%28iFrame%29%0A%20%20%09%09%09iFrame.src%3D%22about%3Ablank%22%3B%0A%20%20%20%20%20%20var%20iFrame%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20%20%20%20if%28iFrame%29%0A%20%20%20%20%20%20%20%20%20iFrame.src%3D%22about%3Ablank%22%3B%0A%20%20%09%09%0A%20%20%09%09%0A%20%20%09%09if%28_leoHighlightsPrevElem%29%0A%20%20%09%09%7B%0A%20%20%09%09%09_leoHighlightsPrevElem.shown%3Dfalse%3B%0A%20%20%09%09%09_leoHighlightsPrevElem%3Dnull%3B%0A%20%20%09%09%7D%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHideElem%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A*%0A*%20This%20can%20be%20used%20to%20close%20an%20iframe.%0A*%20Since%20the%20iFrame%20is%20reused%20the%20frame%20only%20gets%20hidden%0A*%0A*%20@return%0A*/%0Afunction%20leoHighlightsIFrameClose%28%29%0A%7B%0A%20%20try%0A%20%20%7B%0A%09%20%20_leoHighlightsSimpleGwCallBack%28%22LeoHighlightsHideIFrame%22%29%3B%0A%20%20%7D%0A%20%20catch%28e%29%0A%20%20%7B%0A%09%20%20_leoHighlightsReportExeception%28%22leoHighlightsIFrameClose%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%7D%0A%7D%0A%0A/**%0A*%0A*%20This%20is%20used%20to%20snooze%20the%20highlights.%0A*%0A*%20@return%0A*/%0Afunction%20leoHighlightsSnooze%28%29%0A%7B%0A%20%20try%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsSnoozed%3Dtrue%3B%0A%20%20%20%20%20_leoHighlightsSimpleGwCallBack%28%22LeoHighlightsSnooze%22%29%3B%0A%20%20%7D%0A%20%20catch%28e%29%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSnooze%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%7D%0A%7D%0A%0A/**%0A*%0A*%20This%20is%20used%20to%20snooze%20the%20highlights.%0A*%20This%20gets%20fired%20into%20the%20top%20frame.%0A*%0A*%20@return%0A*/%0Afunction%20leoHighlightsSnoozeTop%28id%29%0A%7B%0A%20%20try%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsSnoozed%3Dtrue%3B%0A%20%20%20%20%20leoHighlightsHideElem%28id%29%3B%0A%20%20%7D%0A%20%20catch%28e%29%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSnoozeTop%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20should%20handle%20the%20click%20events%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleClick%28anchorId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28_leoHighlightsIsFrame%28%29%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%09%09var%20anchor%3D_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%09%09anchor.hover%3Dfalse%3B%0A%20%20%09%09if%28anchor.startTimer%29%0A%20%20%09%09%09clearTimeout%28anchor.startTimer%29%3B%0A%20%20%20%09%0A%20%20%20%20%20%20/*%20Report%20the%20click%20event%20*/%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22clicked%22%2C%20window.document.domain%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_keywords%27%29%2Cnull%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_accept%27%29%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_reject%27%29%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09leoHighlightsShowPopup%28anchorId%2C1%29%3B%0A%20%20%20%09return%20false%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHighlightsHandleClick%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20should%20handle%20the%20hover%20events%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleHover%28anchorId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28_leoHighlightsIsFrame%28%29%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%09%09var%20anchor%3D_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%09%09anchor.hover%3Dtrue%3B%0A%20%20%09%09%0A%20%20%20%20%20%20/*%20Report%20the%20hover%20event%20*/%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22hovered%22%2C%20window.document.domain%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_keywords%27%29%2Cnull%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_accept%27%29%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_reject%27%29%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09leoHighlightsShowPopup%28anchorId%2C0%29%3B%0A%20%20%20%09return%20false%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHighlightsHandleHover%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20will%20handle%20the%20mouse%20over%20setup%20timers%20for%20the%20appropriate%20timers%0A%20*%20%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleMouseOver%28id%29%0A%7B%0A%09try%0A%09%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%3B%0A%09%20%20%20%0A%09%20%20%20if%28_leoHighlightsIsFrame%28%29%29%0A%09%20%20%20%20%20%20return%3B%0A%09%20%20%20%0A%09%09var%20anchor%3D_leoHighlightsFindElementById%28id%29%3B%09%09%0A%0A%09%09/*%20Clear%20the%20end%20timer%20if%20required%20*/%0A%09%09if%28anchor.endTimer%29%0A%09%09%09clearTimeout%28anchor.endTimer%29%3B%0A%09%09anchor.endTimer%3Dnull%3B%0A%09%09%0A%09%09anchor.style.background%3DLEO_HIGHLIGHTS_BACKGROUND_STYLE_HOVER%3B%0A%09%09%0A%09%09var%20underline%3D_leoHighlightsGetAttrib%28anchor%2C%22leohighlights_underline%22%29%3D%3D%27true%27%3B%0A%09%09%0A%09%09/*%20The%20element%20is%20already%20showing%20we%20are%20done%20*/%0A%09%09if%28anchor.shown%29%0A%09%09%09return%3B%0A%09%09%0A%09%09/*%20Setup%20the%20start%20timer%20if%20required%20*/%0A%09%09anchor.startTimer%3DsetTimeout%28function%28%29%7B%0A%09%09%09leoHighlightsHandleHover%28anchor.id%29%3B%0A%09%09%09anchor.hover%3Dtrue%3B%0A%09%09%09%7D%2Cunderline?LEO_HIGHLIGHTS_SHOW_DELAY_MS%3ALEO_HIGHLIGHTS_SHOW_DELAY_NO_UNDER_MS%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleMouseOver%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20handle%20the%20mouse%20over%20setup%20timers%20for%20the%20appropriate%20timers%0A%20*%20%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleMouseOut%28id%29%0A%7B%0A%09try%0A%09%7B%09%0A%09%09var%20anchor%3D_leoHighlightsFindElementById%28id%29%3B%0A%09%09%0A%09%09/*%20Clear%20the%20start%20timer%20if%20required%20*/%0A%09%09if%28anchor.startTimer%29%0A%09%09%09clearTimeout%28anchor.startTimer%29%3B%0A%09%09anchor.startTimer%3Dnull%3B%0A%09%09%0A%09%09anchor.style.background%3DLEO_HIGHLIGHTS_BACKGROUND_STYLE_DEFAULT%3B%0A%09%09if%28%21anchor.shown||%21anchor.hover%29%0A%09%09%09return%3B%0A%09%09%0A%09%09/*%20Setup%20the%20start%20timer%20if%20required%20*/%0A%09%09anchor.endTimer%3DsetTimeout%28function%28%29%7B%0A%09%09%09leoHighlightsHideElem%28LEO_HIGHLIGHTS_IFRAME_DIV_ID%29%3B%0A%09%09%09anchor.shown%3Dfalse%3B%0A%09%09%09_leoHighlightsPrevElem%3Dnull%3B%0A%09%09%09%7D%2CLEO_HIGHLIGHTS_HIDE_DELAY_MS%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleMouseOut%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20This%20handles%20the%20mouse%20movement%20into%20the%20currently%20opened%20window.%0A%20*%20Just%20clear%20the%20close%20timer%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleIFrameMouseOver%28%29%0A%7B%0A%09try%0A%09%7B%0A%09%09if%28_leoHighlightsPrevElem%26%26_leoHighlightsPrevElem.endTimer%29%0A%09%09%09clearTimeout%28_leoHighlightsPrevElem.endTimer%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleIFrameMouseOver%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20This%20handles%20the%20mouse%20movement%20into%20the%20currently%20opened%20window.%0A%20*%20Just%20clear%20the%20close%20timer%0A%20*%20%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleIFrameMouseOut%28%29%0A%7B%0A%09try%0A%09%7B%0A%09%09if%28_leoHighlightsPrevElem%29%0A%09%09%09leoHighlightsHandleMouseOut%28_leoHighlightsPrevElem.id%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleIFrameMouseOut%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A/**%0A%20*%20This%20is%20a%20method%20is%20used%20to%20make%20the%20javascript%20within%20IE%20runnable%0A%20*/%0Avar%20leoHighlightsRanUpdateDivs%3Dfalse%3B%0Afunction%20leoHighlightsUpdateDivs%28%29%0A%7B%0A%09try%0A%09%7B%0A%09%09/*%20Check%20if%20this%20is%20an%20IE%20browser%20and%20if%20divs%20have%20been%20updated%20already%20*/%0A%09%09if%28document.all%26%26%21leoHighlightsRanUpdateDivs%26%26%21_leoHighlightsIsFrame%28%29%29%0A%09%09%7B%0A%09%09%09leoHighlightsRanUpdateDivs%3Dtrue%3B%20//%20Set%20early%20to%20prevent%20running%20twice%0A%09%09%09for%28var%20i%3D0%3Bi%3CLEO_HIGHLIGHTS_MAX_HIGHLIGHTS%3Bi%2B%2B%29%0A%09%09%09%7B%0A%09%09%09%09var%20id%3D%22leoHighlights_Underline_%22%2Bi%3B%0A%09%09%09%09var%20elem%3D_leoHighlightsFindElementById%28id%29%3B%0A%09%09%09%09if%28elem%3D%3Dnull%29%0A%09%09%09%09%09break%3B%0A%09%09%09%09%0A%09%09%09%09if%28%21elem.leoChanged%29%0A%09%09%09%09%7B%0A%09%09%09%09%09elem.leoChanged%3Dtrue%3B%0A%09%09%09%09%0A%09%09%09%09%09/*%20This%20will%20make%20javaScript%20runnable%20*/%09%09%09%09%0A%09%09%09%09%09elem.outerHTML%3Delem.outerHTML%3B%0A%09%09%09%09%7D%0A%09%09%09%7D%0A%09%09%7D%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsUpdateDivs%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0Aif%28document.all%29%0A%09setTimeout%28leoHighlightsUpdateDivs%2C200%29%3B%0A%0A%0A/**%0A%20*%20This%20is%20used%20to%20report%20events%20to%20the%20plugin%0A%20*%20@param%20key%0A%20*%20@param%20domain%0A%20*%20@param%20keywords%0A%20*%20@param%20vendorId%0A%20*%20@param%20accept%0A%20*%20@param%20reject%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsReportEvent%28key%2C%20domain%2Ckeywords%2CvendorId%2Caccept%2Creject%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22key%22%2Ckey%29%3B%0A%20%20%20%20%20%20if%28domain%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22domain%22%2Cdomain%29%3B%0A%20%20%20%20%20%20if%28keywords%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22keywords%22%2Ckeywords%29%3B%0A%20%20%20%20%20%20if%28vendorId%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22vendorId%22%2CvendorId%29%3B%0A%20%20%20%20%20%20if%28accept%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22accept%22%2Caccept%29%3B%0A%20%20%20%20%20%20if%28reject%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22reject%22%2Creject%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsEvent%22%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlights%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20expand%20or%20collapse%20the%20window%20base%20on%20it%20prior%20state%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsToggleSize%28clickId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsToggleSize%28%29%20%22%2B_leoHighlightsPrevElem%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20hover%20flag%20and%20change%20the%20status%20*/%0A%20%20%20%20%20%20var%20size%3D_leoHighlightsPrevElem.hover?1%3A0%3B%0A%20%20%20%20%20%20_leoHighlightsSetBottomSize%28size%2CclickId%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsToggleSize%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20Call%20into%20the%20kvm%20that%20will%20then%20do%20a%20callback%20into%20the%20top%20window%0A%20*%20The%20top%20window%20will%20then%20call%20leoH%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsSetSecondaryWindowUrl%28url%2C%20customerId%2C%20phraseId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrl%28%29%20%22%2Burl%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22url%22%2C%20url%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22phraseId%22%2C%20phraseId%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22customerId%22%2C%20customerId%29%3B%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsSetSecondaryWindowUrl%22%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSetSecondaryWindowUrl%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20Call%20into%20the%20kvm%20that%20will%20then%20do%20a%20callback%20into%20the%20top%20window%0A%20*%20The%20top%20window%20will%20then%20call%20leoH%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsSetSecondaryWindowUrlCallback%28url%2C%20customerId%2C%20phraseId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%20%22%2Burl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Clear%20the%20hover%20flag%2C%20if%20the%20user%20shows%20this%20at%20full%20size%20*/%0A%20%20%20%20%20%20var%20size%3D_leoHighlightsPrevElem.hover?0%3A1%3B%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%20%22%2B_leoHighlightsPrevElem%2B%22%20--%20%22%2B_leoHighlightsPrevElem.hover%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20elements%20*/%0A%20%20%20%20%20%20var%20iFrameBottom%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20%20%20%20leoHighlightsUpdateUrl%28iFrameBottom%2Csize%2Cnull%2Curl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%20%22%2Burl%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20set%20the%20text%20to%20the%20Top%20%0A%20*%20%0A%20*%20@param%20txt%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsSetExpandTxt%28txt%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20var%20topIFrame%20%3D%20_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%20%20%20%20%20%20if%28topIFrame%3D%3Dnull%29%0A%20%20%20%20%20%20%20%20%20return%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20current%20url%20*/%0A%20%20%20%20%20%20var%20url%3DtopIFrame.src%3B%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28url%3D%3Dnull%29%0A%20%20%20%20%20%20%20%20%20return%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Extract%20the%20previous%20hash%20if%20present%20*/%0A%20%20%20%20%20%20var%20idx%3D-1%3B%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28%28idx%3Durl.indexOf%28%27%23%27%29%29%3E0%29%0A%20%20%20%20%20%20%20%20%20url%3Durl.substring%280%2Cidx%29%3B%0A%0A%20%20%20%20%20%20/*%20Append%20the%20text%20to%20the%20end%20*/%0A%20%20%20%20%20%20url%2B%3D%22%23%22%2BencodeURI%28txt%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Set%20the%20iframe%20with%20the%20new%20url%20that%20contains%20the%20hash%20tag%20*/%0A%20%20%20%20%20%20topIFrame.src%3Durl%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSetExpandTxt%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/*----------------------------------------------------------------------*/%0A/*%20Methods%20provided%20to%20the%20highlight%20providers...%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20*/%0A/*----------------------------------------------------------------------*/%0A%0A/**%0A%20*%20This%20will%20set%20the%20expand%20text%20for%20the%20Top%20window%0A%20*/%0Afunction%20leoHL_SetExpandTxt%28txt%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHL_SetExpandTxt%28%29%20%22%2Btxt%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsSimpleGwCallBack%28%22LeoHighlightsSetExpandTxt%22%2C%22expandTxt%22%2Ctxt%29%3B%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_SetExpandTxt%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20@return%0A%20*/%0Afunction%20leoHL_RedirectTop%28url%2CparentId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20try%7B%0A%20%20%20%20%20%20%20%20%20var%20domain%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22domain%22%29%0A%20%20%20%20%20%20%20%20%20var%20keywords%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22keywords%22%29%0A%20%20%20%20%20%20%20%20%20var%20vendorId%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22vendorId%22%29%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22clickthrough%22%2C%20domain%2Ckeywords%2C%20vendorId%29%3B%0A%20%20%20%20%20%20%7Dcatch%28e%29%7B%0A%20%20%20%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_RedirectTop%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%20%20%20%7D%0A%20%20%09%09%0A%20%20%20%09_leoHighlightsRedirectTop%28url%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHL_RedirectTop%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20@return%0A%20*/%0Afunction%20LeoHL_RedirectTop%28url%2CparentId%29%0A%7B%0A%20%20%20leoHL_RedirectTop%28url%2CparentId%29%3B%0A%7D%0A%0A/**%0A%20*%20This%20will%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20@return%0A%20*/%0Afunction%20leoHL_RedirectTopAd%28url%2CparentId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20try%7B%0A%20%20%20%20%20%20%20%20%20var%20domain%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22domain%22%29%0A%20%20%20%20%20%20%20%20%20var%20keywords%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22keywords%22%29%0A%20%20%20%20%20%20%20%20%20var%20vendorId%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22vendorId%22%29%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22advertisement.click%22%2C%20domain%2Ckeywords%2C%20vendorId%29%3B%0A%20%20%20%20%20%20%7Dcatch%28e%29%7B%0A%20%20%20%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_RedirectTopAd%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsRedirectTop%28url%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_RedirectTopAd%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20set%20the%20size%20of%20the%20iframe%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHl_setSize%28size%2Curl%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09/*%20Get%20the%20clickId%20*/%0A%20%20%20%09var%20clickId%3D_leoHighlightsGetUrlArg%28%20url%2C%22clickId%22%29%0A%20%20%20%09%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22size%22%2Csize%29%3B%0A%20%20%20%20%20%20if%28clickId%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22clickId%22%2CclickId%2B%22_blah%22%29%3B%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsSetSize%22%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHl_setSize%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20toggle%20the%20size%20of%20the%20window%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHl_ToggleSize%28%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsToggleSize%22%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHl_ToggleSize%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A");Initial SetupNote: 786528.1 How to Create STRMADMIN User and Grant PrivilegesNote: 259688.1 Grant Streams Administrator User Privileges using DBMS_STREAMS_AUTH PackageNote: 418755.1 10gR2 Streams Recommended ConfigurationNote: 413353.1 10.2 Best Practices For Streams in RAC environment Note: 335516.1 Streams Performance RecommendationsNote: 782541.1 Streams Replication Supplemental Logging RequirementsNote: 437838.1 Streams Specific PatchesNote: 789500.1 Streams Table Level Replication Setup ScriptNote: 301431.1 How To Setup One-Way Schema Level Streams ReplicationNote: 459922.1 How To Setup Database Level Streams ReplicationNote: 471845.1 Streams Bi-Directional SetupNote: 462102.1 Hub and Spoke Streams Replication Setup between 3 DatabasesNote: 753158.1 How To Configure Streams Real-Time Downstream EnvironmentNote: 733691.1 How To Setup Schema Level Streams Replication with an Archived-log Downstream Capture Process with Implicit Log AssignmentNote: 316893.1 DML and DDL Auditing Using StreamsNote: 382745.1 Setup Tablespace Streams ReplicationNote: 864973.1 How to setup Streams replication using DBMS_STREAMS_ADM.MAINTAIN_* set of proceduresNote: 728911.1 Setup 11g Synchronous Streams CaptureNote: 463820.1 Streams Combined Capture and Apply in 11gAdvanced SetupNote: 784899.1 Setup Streams Replication Between Different Source and Target Schemas with Different Table StructuresNote: 783203.1 How to Setup Custom Rule Based TransformationNote: 780642.1 How To Setup Table Level Streams Replication with Propagation Subset RulesNote: 781625.1 How to Add/Remove a column from a Streams Replicated Table using Declarative Rule TransformationsNote: 309575.1 How To Transform a Schema Name In Streams for DML and Simple DDL OperationsNote: 784126.1 How to Transform a Schema Name in Streams using Declarative Rule TransformationsNote: 814602.1 How to Exclude Operations in a Streams EnvironmentNote: 239571.1 How To Exclude Specific DML Operations From The Capture Process When Using StreamsAdministrationNote: 335502.1 How To Reinstantiate a Single Table in a Streams EnvironmentNote: 550955.1 Instantiating Objects in a 10g Streams Using Original Export/Import and Data Pump Export/Import - ExampleNote: 551106.1 Instantiating Objects in Streams Using Transportable Tablespace or RMANNote: 749036.1 How to Re-Synchronize Streams Replicated Objects OnlineNote: 779801.1 Streams Conflict ResolutionNote: 336265.1 Best Practices For Managing Backups In A Streams EnvironmentNote: 779407.1 Steps To Add An Active Site To An Existing Bi-directional Streams EnvironmentNote: 753233.1 Steps To Add An Active Table To Existing Streams EnvironmentNote: 394127.1 An example of how to use DBMS_STREAMS_TABLESPACE_ADM.CLONE_TABLESPACENote: 259609.1 Script to Prevent Excessive Spill of Message From the Streams Buffer Queue To Disk FAQ Note: 394575.1 FAQ on Downstreams CaptureTroubleshootingNote: 230898.1 How To Troubleshoot the Streams Apply ProcessNote: 265201.1 Troubleshooting Streams Apply Error ORA-1403, ORA-26787 or ORA-26786Note: 746247.1 Troubleshoot and resolve Streams Capture when it is Paused for flow controlNote: 313279.1 Master Note for Troubleshooting Streams capture 'WAITING For REDO' or INITIALIZING Note: 855964.1 How to do SQL Trace for the Streams ProcessesNote: 729860.1 Troubleshooting Queries in StreamsNote: 422252.1 How to skip a transaction at APPLY site Note: 275323.1 Minimum Archive Log Necessary to Restart 9iR2 Streams Capture ProcessNote: 290143.1 Minimum Achived Log Necessary to Restart 10g and 11g Streams Capture Process Note: 471695.1 Required Steps to Recreate a Capture Process @ Note: 421183.1 How To Enable Capture, Propagation, Apply Tracing@ Note: 472440.1 How to Purge Apply Spill in 10.2 Monitoring10g Grid Control offers an excellent set of manageability & monitoring features for Streams, and it should be used as the admin interface for Streams. Available externally on OTN.http://http//www.oracle.com/technology/obe/demos/11gr2/streams11_2/streams_and_em/streams_and_em_viewlet_swf.htmlNote: 784021.1 Managing Streams from Oracle Enterprise Manager 10g Release 5 Grid ControlNote: 273674.1 Streams Configuration Report and Health Check ScriptNote: 735976.1 All Replication Configuration Views For Streams, AQ, CDCand Advanced ReplicationNote: 290605.1 Oracle Streams STRMMON Monitoring UtilityNote: 761815.1 Determining Latency for Streams Processes at Different States11gR2 Important notes can also be found fromhttp://apex.oracle.com/pls/apex/f?p=9830:28:0::NO:RIR:IR_PRODUCT,IR_PRODUCT_SUITE,IR_PRODUCT_COMPONENT,IR_RELEASE,IR_TYPE,IRC_ROWFILTER,IR_FUNCTIONAL_CATEGORY:,,,,,,Information%20IntegrationWho to contact for more information?For diagnosis and troubleshooting while setting up Streams contact Oracle Support and also upload the Stream Healthcheck Report. var LEO_HIGHLIGHTS_INFINITE_LOOP_COUNT = 300; var LEO_HIGHLIGHTS_MAX_HIGHLIGHTS = 50; var LEO_HIGHLIGHTS_IFRAME_TOP_ID = "leoHighlights_top_iframe"; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID = "leoHighlights_bottom_iframe"; var LEO_HIGHLIGHTS_IFRAME_DIV_ID = "leoHighlights_iframe_modal_div_container"; var LEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_WIDTH = 520; var LEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_HEIGHT = 391; var LEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_WIDTH = 520; var LEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_HEIGHT = 665; var LEO_HIGHLIGHTS_IFRAME_TOP_POS_X = 0; var LEO_HIGHLIGHTS_IFRAME_TOP_POS_Y = 0; var LEO_HIGHLIGHTS_IFRAME_TOP_WIDTH = 520; var LEO_HIGHLIGHTS_IFRAME_TOP_HEIGHT = 294; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_POS_X = 96; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_POS_Y = 294; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_WIDTH = 425; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_HEIGHT = 97; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_WIDTH = 425; var LEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_HEIGHT = 371; var LEO_HIGHLIGHTS_SHOW_DELAY_MS = 300; var LEO_HIGHLIGHTS_HIDE_DELAY_MS = 750; var LEO_HIGHLIGHTS_SHOW_DELAY_NO_UNDER_MS = 850; var LEO_HIGHLIGHTS_BACKGROUND_STYLE_DEFAULT = "transparent none repeat scroll 0% 0%"; var LEO_HIGHLIGHTS_BACKGROUND_STYLE_HOVER = "rgb(245, 245, 0) none repeat scroll 0% 0%"; var LEO_HIGHLIGHTS_ROVER_TAG = "711-36858-13496-14"; createInlineScriptElement("var%20LEO_HIGHLIGHTS_DEBUG%20%3D%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20false%3B%0Avar%20LEO_HIGHLIGHTS_DEBUG_POS%20%3D%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20false%3B%0A%20%20%20%0Avar%20_leoHighlightsPrevElem%20%3D%20null%3B%0Avar%20_leoHighlightsSnoozed%20%3D%20false%3B%0A%0A/**%0A%20*%20Checks%20if%20the%20passed%20in%20class%20exists%0A%20*%20@param%20c%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsClassExists%28c%29%20%7B%0A%20%20%20return%20typeof%28c%29%20%3D%3D%20%22function%22%20%26%26%20typeof%28c.prototype%29%20%3D%3D%20%22object%22%20?%20true%20%3A%20false%3B%0A%7D%0A%0A/**%0A%20*%20Checks%20if%20the%20firebug%20console%20is%20available%0A%20*%20@param%20c%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsFirebugConsoleAvailable%28c%29%20%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsClassExists%28_FirebugConsole%29%20%26%26%20%0A%20%20%20%20%20%20%20%20%20window.console%20%26%26%20console.log%20%26%26%20%0A%20%20%20%20%20%20%20%20%20%28console%20instanceof%20_FirebugConsole%29%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20return%20true%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%7D%0A%20%20%20catch%28e%29%7B%7D%0A%20%20%20%0A%20%20%20return%20false%3B%0A%7D%20%0A%0A%0A/**%0A%20*%20General%20method%20used%20to%20debug%20exceptions%0A%20*%20%0A%20*%20@param%20location%0A%20*%20@param%20e%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsReportExeception%28location%2Ce%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsFirebugConsoleAvailable%28%29%20||LEO_HIGHLIGHTS_DEBUG%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20var%20logString%3Dlocation%2B%22%3A%20%22%2Be%2B%22%5Cn%5Ct%22%2Be.name%2B%22%5Cn%5Ct%22%2B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%28e.number%260xFFFF%29%2B%22%5Cn%5Ct%22%2Be.description%3B%0A%20%20%20%20%20%20%20%20%20if%28_leoHighlightsFirebugConsoleAvailable%28%29%29%0A%20%20%20%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20console.error%28logString%29%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20console.trace%28%29%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28LEO_HIGHLIGHTS_DEBUG%29%0A%20%20%20%20%20%20%20%20%20alert%28logString%29%3B%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%7B%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20log%20a%20string%20to%20the%20firebug%20console%0A%20*%20%0A%20*%20@param%20str%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsDebugLog%28str%29%0A%7B%20%20%20%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsFirebugConsoleAvailable%28%29%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20console.log%28typeof%28_FirebugConsole%29%2B%22%20%22%2Bstr%29%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22_leoHighlightsDebugLog%28%29%20%22%2Bstr%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20get%20an%20attribute%20and%20decode%20it.%0A%20*%20%0A%20*%20@param%20elem%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetAttrib%28elem%2Cid%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20val%3Delem.getAttribute%28id%29%3B%0A%20%20%20%20%20%20return%20decodeURI%28val%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22_leoHighlightsGetAttrib%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%20%20%20return%20null%3B%0A%7D%0A%0A/**%0A%20*%20Checks%20if%20this%20is%20within%20a%20frame%20by%20checking%20for%20a%20parent.%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsIsFrame%28%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%20%20%0A%20%20%20%20%20%20return%20%28window%21%3Dtop%29%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22_leoHighlightsIsFrame%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%0A%20%20%20return%20false%3B%0A%7D%0A%0A/**%0A%20*%20This%20is%20a%20dimensions%20object%0A%20*%20%0A%20*%20@param%20width%0A%20*%20@param%20height%0A%20*%20@return%0A%20*/%0Afunction%20LeoHighlightsDimension%28width%2Cheight%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09this.width%3Dwidth%3B%0A%20%20%20%09this.height%3Dheight%3B%0A%20%20%20%09this.toString%3Dfunction%28%29%20%7B%20return%20%28%22%28%22%2Bthis.width%2B%22%2C%22%2Bthis.height%2B%22%29%22%29%3B%7D%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22new%20LeoHighlightsDimension%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20is%20a%20Position%20object%0A%20*%20%0A%20*%20@param%20x%0A%20*%20@param%20y%0A%20*%20@return%0A%20*/%0Afunction%20LeoHighlightsPosition%28x%2Cy%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09this.x%3Dx%3B%0A%20%20%20%09this.y%3Dy%3B%0A%20%20%20%09this.toString%3Dfunction%28%29%20%7B%20return%20%28%22%28%22%2Bthis.x%2B%22%2C%22%2Bthis.y%2B%22%29%22%29%3B%7D%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22new%20LeoHighlightsPosition%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0Avar%20LEO_HIGHLIGHTS_ADJUSTMENT%20%3D%20new%20LeoHighlightsPosition%283%2C3%29%3B%0Avar%20LEO_HIGHLIGHTS_IFRAME_TOP_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_TOP_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_TOP_HEIGHT%29%3B%0Avar%20LEO_HIGHLIGHTS_IFRAME_BOTTOM_HOVER_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_BOTTOM_COLLAPSED_HEIGHT%29%3B%0Avar%20LEO_HIGHLIGHTS_IFRAME_BOTTOM_CLICK_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_BOTTOM_EXPANDED_HEIGHT%29%3B%0A%0Avar%20LEO_HIGHLIGHTS_DIV_HOVER_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_TOTAL_COLLAPSED_HEIGHT%29%3B%0Avar%20LEO_HIGHLIGHTS_DIV_CLICK_SIZE%20%3D%20new%20LeoHighlightsDimension%28LEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_WIDTH%2CLEO_HIGHLIGHTS_IFRAME_TOTAL_EXPANDED_HEIGHT%29%3B%0A%0A%0A/**%0A%20*%20Sets%20the%20size%20of%20the%20passed%20in%20element%0A%20*%20%0A%20*%20@param%20elem%0A%20*%20@param%20dim%20%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsSetSize%28elem%2Cdim%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09//%20Set%20the%20popup%20location%0A%20%20%20%09elem.style.width%20%3D%20dim.width%20%2B%20%22px%22%3B%0A%20%20%20%09if%28elem.width%29%0A%20%20%20%09%09elem.width%3Ddim.width%3B%0A%20%20%20%09elem.style.height%20%20%3D%20dim.height%20%2B%20%22px%22%3B%0A%20%20%20%09if%28elem.height%29%0A%20%20%20%09%09elem.height%3Ddim.height%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsSetSize%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20can%20be%20used%20for%20a%20simple%20one%20argument%20callback%0A%20*%0A%20*%20@param%20callName%0A%20*%20@param%20argName%0A%20*%20@param%20argVal%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsSimpleGwCallBack%28callName%2CargName%2C%20argVal%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20if%28argName%29%0A%20%20%20%20%20%20%09gwObj.addParam%28argName%2CargVal%29%3B%0A%20%20%20%20%20%20gwObj.callName%28callName%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsSimpleGwCallBack%28%29%20%22%2BcallName%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20gets%20a%20url%20argument%20from%20the%20current%20document.%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetUrlArg%28url%2C%20name%20%29%0A%7B%0A%09%20%20name%20%3D%20name.replace%28/[%5C[]/%2C%22%5C%5C%5C[%22%29.replace%28/[%5C]]/%2C%22%5C%5C%5C]%22%29%3B%0A%09%20%20var%20regexS%20%3D%20%22[%5C%5C?%26]%22%2Bname%2B%22%3D%28[^%26%23]*%29%22%3B%0A%09%20%20var%20regex%20%3D%20new%20RegExp%28%20regexS%20%29%3B%0A%09%20%20var%20results%20%3D%20regex.exec%28url%29%3B%0A%09%20%20if%28%20results%20%3D%3D%20null%20%29%0A%09%20%20%20%20return%20%22%22%3B%0A%09%20%20else%0A%09%20%20%20%20return%20results[1]%3B%0A%7D%0A%0A%0A/**%0A%20*%20This%20allows%20to%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsRedirectTop%28url%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%20%20%09%0A%20%20%20%09top.location%3Durl%3B%09%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsRedirectTop%28%29%22%2Ce%29%3B%0A%20%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20find%20an%20element%20by%20Id%0A%20*%20%0A%20*%20@param%20elemId%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsFindElementById%28elemId%2Cdoc%29%0A%7B%0A%09try%0A%09%7B%0A%09%20%20%20if%28doc%3D%3Dnull%29%0A%09%20%20%20%20%20%20doc%3Ddocument%3B%0A%09%20%20%20%0A%09%09var%20elem%3Ddoc.getElementById%28elemId%29%3B%0A%09%09if%28elem%29%0A%09%09%09return%20elem%3B%0A%09%09%0A%09%09/*%20This%20is%20the%20handling%20for%20IE%20*/%0A%09%09if%28doc.all%29%0A%09%09%7B%0A%09%09%09elem%3Ddoc.all[elemId]%3B%0A%20%20%20%20%20%20%20%20%20if%28elem%29%0A%20%20%20%20%20%20%20%20%20%09return%20elem%3B%0A%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%20for%20%28%20var%20i%20%3D%20%28document.all.length-1%29%3B%20i%20%3E%3D%200%3B%20i--%29%20%7B%0A%20%20%20%20%20%20%20%20%20%09elem%3Ddoc.all[i]%3B%0A%20%20%20%20%20%20%20%20%20%09if%28elem.id%3D%3DelemId%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20return%20elem%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%09%09%7D%0A%09%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsFindElementById%28%29%22%2Ce%29%3B%0A%20%20%20%7D%0A%09return%20null%3B%0A%7D%0A%0A/**%0A%20*%20Get%20the%20location%20of%20one%20element%20relative%20to%20a%20parent%20reference%0A%20*%0A%20*%20@param%20ref%0A%20*%20%20%20%20%20%20%20%20%20%20%20the%20reference%20element%2C%20this%20must%20be%20a%20parent%20of%20the%20passed%20in%0A%20*%20%20%20%20%20%20%20%20%20%20%20element%0A%20*%20@param%20elem%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetLocation%28ref%2C%20elem%29%20%7B%0A%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsGetLocation%20%22%2Belem.id%29%3B%0A%20%20%20%0A%20%20%20var%20count%20%3D%200%3B%0A%20%20%20var%20location%20%3D%20new%20LeoHighlightsPosition%280%2C0%29%3B%0A%20%20%20var%20walk%20%3D%20elem%3B%0A%20%20%20while%20%28walk%20%21%3D%20null%20%26%26%20walk%20%21%3D%20ref%20%26%26%20count%20%3C%20LEO_HIGHLIGHTS_INFINITE_LOOP_COUNT%29%20%7B%0A%20%20%20%20%20%20location.x%20%2B%3D%20walk.offsetLeft%3B%0A%20%20%20%20%20%20location.y%20%2B%3D%20walk.offsetTop%3B%0A%20%20%20%20%20%20walk%20%3D%20walk.offsetParent%3B%0A%20%20%20%20%20%20count%2B%2B%3B%0A%20%20%20%7D%0A%20%20%20%0A%20%20%20_leoHighlightsDebugLog%28%22Location%20is%3A%20%22%2Belem.id%2B%22%20-%20%22%2Blocation%29%3B%0A%0A%20%20%20return%20location%3B%0A%7D%0A%0A/**%0A%20*%20This%20is%20used%20to%20update%20the%20position%20of%20an%20element%20as%20a%20popup%0A%20*%20%0A%20*%20@param%20IFrame%0A%20*%20@param%20anchor%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsUpdatePopupPos%28iFrame%2Canchor%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20//%20Gets%20the%20scrolled%20location%20for%20x%20and%20y%0A%20%20%20%20%20%20var%20scrolledPos%3Dnew%20LeoHighlightsPosition%280%2C0%29%3B%0A%20%20%20%20%20%20if%28%20self.pageYOffset%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20scrolledPos.x%20%3D%20self.pageXOffset%3B%0A%20%20%20%20%20%20%20%20%20scrolledPos.y%20%3D%20self.pageYOffset%3B%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.documentElement%20%26%26%20document.documentElement.scrollTop%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20scrolledPos.x%20%3D%20document.documentElement.scrollLeft%3B%0A%20%20%20%20%20%20%20%20%20scrolledPos.y%20%3D%20document.documentElement.scrollTop%3B%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.body%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20scrolledPos.x%20%3D%20document.body.scrollLeft%3B%0A%20%20%20%20%20%20%20%20%20scrolledPos.y%20%3D%20document.body.scrollTop%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20total%20dimensions%20to%20see%20what%20scroll%20bars%20might%20be%20active%20*/%0A%20%20%20%20%20%20var%20totalDim%3Dnew%20LeoHighlightsDimension%280%2C0%29%0A%20%20%20%20%20%20if%20%28document.all%20%26%26%20document.documentElement%20%26%26%20%0A%20%20%20%20%20%20%09document.documentElement.clientHeight%26%26document.documentElement.clientWidth%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%09totalDim.width%20%3D%20document.documentElement.scrollWidth%3B%0A%20%20%20%20%20%20%09totalDim.height%20%3D%20document.documentElement.scrollHeight%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20else%20if%20%28document.all%29%0A%20%20%20%20%20%20%7B%20/*%20This%20is%20in%20IE%20*/%0A%20%20%20%20%20%09%20%09totalDim.width%20%3D%20document.body.scrollWidth%3B%0A%20%20%20%20%20%20%09totalDim.height%20%3D%20document.body.scrollHeight%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20else%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%09%20totalDim.width%20%3D%20document.width%3B%0A%20%20%20%20%20%20%09%20totalDim.height%20%3D%20document.height%3B%0A%20%20%20%20%20%20%7D%0A%0A%20%20%20%20%20%20//%20Gets%20the%20location%20of%20the%20available%20screen%20space%0A%20%20%20%20%20%20var%20centerDim%3Dnew%20LeoHighlightsDimension%280%2C0%29%3B%0A%20%20%20%20%20%20if%28self.innerWidth%20%26%26%20self.innerHeight%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20centerDim.width%20%3D%20self.innerWidth-%28totalDim.height%3Eself.innerHeight?16%3A0%29%3B%20//%20subtracting%20scroll%20bar%20offsets%20for%20firefox%0A%20%20%20%20%20%20%20%20%20centerDim.height%20%3D%20self.innerHeight-%28totalDim.width%3Eself.innerWidth?16%3A0%29%3B%20%20//%20subtracting%20scroll%20bar%20offsets%20for%20firefox%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.documentElement%20%26%26%20document.documentElement.clientHeight%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20centerDim.width%20%3D%20document.documentElement.clientWidth%3B%0A%20%20%20%20%20%20%20%20%20centerDim.height%20%3D%20document.documentElement.clientHeight%3B%0A%20%20%20%20%20%20%7D%20else%20if%28%20document.body%20%29%20%7B%0A%20%20%20%20%20%20%20%20%20centerDim.width%20%3D%20document.body.clientWidth%3B%0A%20%20%20%20%20%20%20%20%20centerDim.height%20%3D%20document.body.clientHeight%3B%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20//%20Get%20the%20current%20dimension%20of%20the%20popup%20element%0A%20%20%20%20%20%20var%20iFrameDim%3Dnew%20LeoHighlightsDimension%28iFrame.offsetWidth%2CiFrame.offsetHeight%29%0A%20%20%20%20%20%20if%20%28iFrameDim.width%20%3C%3D%200%29%0A%20%20%20%20%20%20%09iFrameDim.width%20%3D%20iFrame.style.width.substring%280%2C%20iFrame.style.width.indexOf%28%27px%27%29%29%3B%0A%20%20%20%20%20%20if%20%28iFrameDim.height%20%3C%3D%200%29%0A%20%20%20%20%20%20%09iFrameDim.height%20%3D%20iFrame.style.height.substring%280%2C%20iFrame.style.height.indexOf%28%27px%27%29%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Calculate%20the%20position%2C%20lower%20right%20hand%20corner%20by%20default%20*/%0A%20%20%20%20%20%20var%20position%3Dnew%20LeoHighlightsPosition%280%2C0%29%3B%0A%20%20%20%20%20%20position.x%3DscrolledPos.x%2BcenterDim.width-iFrameDim.width-LEO_HIGHLIGHTS_ADJUSTMENT.x%3B%0A%20%20%20%20%20%20position.y%3DscrolledPos.y%2BcenterDim.height-iFrameDim.height-LEO_HIGHLIGHTS_ADJUSTMENT.y%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28anchor%21%3Dnull%29%0A%20%20%20%20%20%20%7B%0A%20%20%20%20%20%20%20%20%20//centerDim%20in%20relation%20to%20the%20anchor%20element%20if%20available%0A%20%20%20%20%20%20%20%20%20var%20topOrBottom%20%3D%20false%3B%0A%20%20%20%20%20%20%20%20%20var%20anchorPos%3D_leoHighlightsGetLocation%28document.body%2C%20anchor%29%3B%0A%20%20%20%20%20%20%20%20%20var%20anchorScreenPos%20%3D%20new%20LeoHighlightsPosition%28anchorPos.x-scrolledPos.x%2CanchorPos.y-scrolledPos.y%29%3B%0A%20%20%20%20%20%20%20%20%20var%20anchorDim%3Dnew%20LeoHighlightsDimension%28anchor.offsetWidth%2Canchor.offsetHeight%29%0A%20%20%20%20%20%20%20%20%20if%20%28anchorDim.width%20%3C%3D%200%29%0A%20%20%20%20%20%20%20%20%20%09anchorDim.width%20%3D%20anchor.style.width.substring%280%2C%20anchor.style.width.indexOf%28%27px%27%29%29%3B%0A%20%20%20%20%20%20%20%20%20if%20%28anchorDim.height%20%3C%3D%200%29%0A%20%20%20%20%20%20%20%20%20%09anchorDim.height%20%3D%20anchor.style.height.substring%280%2C%20anchor.style.height.indexOf%28%27px%27%29%29%3B%0A%0A%20%20%20%20%20%20%20%20%20//%20Check%20if%20the%20popup%20can%20be%20shown%20above%20or%20below%20the%20element%0A%20%20%20%20%20%20%20%20%20if%20%28centerDim.height%20-%20anchorDim.height%20-%20iFrameDim.height%20-%20anchorScreenPos.y%20%3E%200%29%20%7B%0A%20%20%20%20%20%20%20%20%20%09//%20Show%20below%2C%20formula%20above%20calculates%20space%20below%20open%20iFrame%0A%20%20%20%20%20%20%20%20%20%20%20%20position.y%20%3D%20anchorPos.y%20%2B%20anchorDim.height%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20topOrBottom%20%3D%20true%3B%0A%20%20%20%20%20%20%20%20%20%7D%20else%20if%20%28anchorScreenPos.y%20-%20anchorDim.height%20-%20iFrameDim.height%20%3E%200%29%20%7B%0A%20%20%20%20%20%20%20%20%20%09//%20Show%20above%2C%20formula%20above%20calculates%20space%20above%20open%20iFrame%0A%20%20%20%20%20%20%20%20%20%09position.y%20%3D%20anchorPos.y%20-%20iFrameDim.height%20-%20anchorDim.height%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20topOrBottom%20%3D%20true%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsUpdatePopupPos%28%29%20-%20topOrBottom%3A%20%22%2BtopOrBottom%29%3B%0A%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20%20%20%20if%20%28topOrBottom%29%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20//%20We%20attempt%20top%20attach%20the%20window%20to%20the%20element%0A%20%20%20%20%20%20%20%20%20%09position.x%20%3D%20anchorPos.x%20-%20iFrameDim.width%20/%202%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20if%20%28position.x%20%3C%200%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%09position.x%20%3D%200%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20else%20if%20%28position.x%20%2B%20iFrameDim.width%20%3E%20scrolledPos.x%20%2B%20centerDim.width%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%09position.x%20%3D%20scrolledPos.x%20%2B%20centerDim.width%20-%20iFrameDim.width%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsUpdatePopupPos%28%29%20-%20topOrBottom%3A%20%22%2Bposition%29%3B%0A%20%20%20%20%20%20%20%20%20%7D%20else%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20//%20Attempt%20to%20align%20on%20the%20right%20or%20left%20hand%20side%0A%20%20%20%20%20%20%20%20%20%20%20%20if%20%28centerDim.width%20-%20anchorDim.width%20-%20iFrameDim.width%20-%20anchorScreenPos.x%20%3E%200%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20position.x%20%3D%20anchorPos.x%20%2B%20anchorDim.width%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20else%20if%20%28anchorScreenPos.x%20-%20anchorDim.width%20-%20iFrameDim.width%20%3E%200%29%0A%20%20%20%20%20%20%20%20%20%20%20%20%09position.x%20%3D%20anchorPos.x%20-%20anchorDim.width%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20else%20%20//%20default%20to%20below%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20position.y%20%3D%20anchorPos.y%20%2B%20anchorDim.height%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%22_leoHighlightsUpdatePopupPos%28%29%20-%20sideBottom%3A%20%22%2Bposition%29%3B%0A%20%20%20%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Make%20sure%20that%20we%20don%27t%20go%20passed%20the%20right%20hand%20border%20*/%0A%20%20%20%20%20%20if%28position.x%2BiFrameDim.width%3EcenterDim.width-20%29%0A%20%20%20%20%20%20%09position.x%3DcenterDim.width-%28iFrameDim.width%2B20%29%3B%0A%20%20%20%20%20%20%09%09%0A%20%20%20%20%20%20//%20Make%20sure%20that%20we%20didn%27t%20go%20passed%20the%20start%0A%20%20%20%20%20%20if%28position.x%3C0%29%0A%20%20%20%20%20%20%20%20%20position.x%3D0%3B%0A%20%20%20%20%20%20if%28position.y%3C0%29%0A%20%20%20%20%20%20%09position.y%3D0%3B%0A%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22Popup%20info%20id%3A%20%20%20%20%20%20%20%22%20%2BiFrame.id%2B%22%20-%20%22%2Banchor.id%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cnscrolled%20%20%20%20%20%20%20%20%20%20%22%20%2B%20scrolledPos%20%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cncenter/visible%20%20%20%20%22%20%2B%20centerDim%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cnanchor%20%28absolute%29%20%22%20%2B%20anchorPos%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5Cnanchor%20%28screen%29%20%20%20%22%20%2B%20anchorScreenPos%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5CnSize%20%28anchor%29%20%20%20%20%20%22%20%2B%20anchorDim%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5CnSize%20%28popup%29%20%20%20%20%20%20%22%20%2B%20iFrameDim%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2B%20%22%5CnResult%20pos%20%20%20%20%20%20%20%20%22%20%2B%20position%29%3B%0A%0A%20%20%20%20%20%20//%20Set%20the%20popup%20location%0A%20%20%20%20%20%20iFrame.style.left%20%3D%20position.x%20%2B%20%22px%22%3B%0A%20%20%20%20%20%20iFrame.style.top%20%20%3D%20position.y%20%2B%20%22px%22%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsUpdatePopupPos%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20show%20the%20passed%20in%20element%20as%20a%20popup%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@param%20size%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsShowPopup%28anchorId%2Csize%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%20%09var%20popup%3Dnew%20LeoHighlightsPopup%28anchorId%2Csize%29%3B%0A%20%20%20%09popup.show%28%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22_leoHighlightsShowPopup%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20transform%20the%20passed%20in%20url%20to%20a%20rover%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsGetRoverUrl%28url%29%0A%7B%0A%09var%20rover%3DLEO_HIGHLIGHTS_ROVER_TAG%3B%0A%09var%20roverUrl%3D%22http%3A//rover.ebay.com/rover/1/%22%2Brover%2B%22/4?%26mpre%3D%22%2BencodeURI%28url%29%3B%0A%09%0A%09return%20roverUrl%3B%0A%7D%0A%0A/**%0A%20*%20Sets%20the%20size%20of%20the%20bottom%20windown%20part%0A%20*%20%0A%20*%20@param%20size%0A%20*%20@return%0A%20*/%0Afunction%20_leoHighlightsSetBottomSize%28size%2CclickId%29%0A%7B%0A%20%20%20/*%20Get%20the%20elements%20*/%0A%20%20%20var%20iFrameBottom%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20var%20iFrameDiv%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_DIV_ID%29%3B%0A%0A%20%20%20/*%20Figure%20out%20the%20correct%20sizes%20*/%0A%20%20%20var%20iFrameBottomSize%3D%28size%3D%3D1%29?LEO_HIGHLIGHTS_IFRAME_BOTTOM_CLICK_SIZE%3ALEO_HIGHLIGHTS_IFRAME_BOTTOM_HOVER_SIZE%3B%0A%20%20%20var%20divSize%3D%28size%3D%3D1%29?LEO_HIGHLIGHTS_DIV_CLICK_SIZE%3ALEO_HIGHLIGHTS_DIV_HOVER_SIZE%3B%0A%0A%20%20%20/*%20Refresh%20the%20iFrame%27s%20url%2C%20by%20removing%20the%20size%20arg%20and%20adding%20it%20again%20*/%0A%20%20%20leoHighlightsUpdateUrl%28iFrameBottom%2Csize%2CclickId%29%3B%0A%0A%20%20%20/*%20Clear%20the%20hover%20flag%2C%20if%20the%20user%20shows%20this%20at%20full%20size%20*/%0A%20%20%20_leoHighlightsPrevElem.hover%3Dsize%3D%3D1?false%3Atrue%3B%0A%0A%20%20%20_leoHighlightsSetSize%28iFrameBottom%2CiFrameBottomSize%29%3B%0A%20%20%20_leoHighlightsSetSize%28iFrameDiv%2CdivSize%29%3B%0A%7D%0A%0A/**%0A%20*%20Class%20for%20a%20Popup%20%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@param%20size%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20LeoHighlightsPopup%28anchorId%2Csize%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22LeoHighlightsPopup%28%29%20%22%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09this.anchorId%3DanchorId%3B%0A%20%20%20%09this.anchor%3D_leoHighlightsFindElementById%28this.anchorId%29%3B%0A%20%20%20%09this.topIframe%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%20%20%20%20%20%20this.bottomIframe%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20%09this.iFrameDiv%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_DIV_ID%29%3B%0A%20%20%20%09%0A%20%20%20%09this.topIframe.src%3Dunescape%28this.anchor.getAttribute%28%27leoHighlights_url_top%27%29%29%3B%3B%0A%20%20%20%20%20%20this.bottomIframe.src%3Dunescape%28this.anchor.getAttribute%28%27leoHighlights_url_bottom%27%29%29%3B%3B%0A%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%221%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.topIframe.style.top%2B%22%2C%20%22%2Bthis.topIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%222%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.bottomIframe.style.top%2B%22%2C%20%22%2Bthis.bottomIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09leoHighlightsSetSize%28size%29%3B%0A%20%20%20%09%0A%20%20%20%09this.updatePos%3Dfunction%28%29%20%7B%20_leoHighlightsUpdatePopupPos%28this.iFrameDiv%2Cthis.anchor%29%3B%7D%3B%0A%20%20%20%20%20%20this.show%3Dfunction%28%29%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20this.updatePos%28%29%3B%20%0A%20%20%20%20%20%20%20%20%20%20%20%20this.iFrameDiv.style.visibility%20%3D%20%22visible%22%3B%20%0A%20%20%20%20%20%20%20%20%20%20%20%20this.iFrameDiv.style.display%20%3D%20%22block%22%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20this.updatePos%28%29%3B%0A%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%223%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.topIframe.style.top%2B%22%2C%20%22%2Bthis.topIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsDebugLog%28%224%29%20LeoHighlightsPopup%28%29%20%28%22%2Bthis.bottomIframe.style.top%2B%22%2C%20%22%2Bthis.bottomIframe.style.left%2B%22%29%22%29%3B%0A%20%20%20%20%20%20%7D%3B%20%20%20%20%20%20%20%0A%20%20%20%09this.scroll%3Dfunction%28%29%20%7B%20this.updatePos%28%29%3B%7D%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22new%20LeoHighlightsPopup%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20updates%20the%20url%20for%20the%20iFrame%0A%20*%20%0A%20*%20@param%20iFrame%0A%20*%20@param%20size%0A%20*%20@param%20clickId%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsUpdateUrl%28iFrame%2Csize%2CclickId%2CdestUrl%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsUpdateUrl%28%29%20%22%2BdestUrl%29%3B%0A%20%20%20%20%20%20var%20url%3DiFrame.src%3B%0A%20%20%20%20%20%20var%20idx%3Durl.indexOf%28%22%26size%3D%22%29%3B%0A%20%20%20%20%20%20if%28idx%3E%3D0%29%0A%20%20%20%20%20%20%20%20%20url%3Durl.substring%280%2Cidx%29%3B%0A%20%20%20%20%20%20%0A//%20%20%20%20%20%20size%3D1%3B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsUpdateUrl%28%29%20size%3D%22%2Bsize%2B%22%20%20%22%2Burl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28size%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20url%2B%3D%28%22%26size%3D%22%2Bsize%29%3B%0A%20%20%20%20%20%20if%28clickId%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20url%2B%3D%28%22%26clickId%3D%22%2BclickId%29%3B%0A%20%20%20%20%20%20if%28destUrl%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20url%2B%3D%28%22%26url%3D%22%2BdestUrl%29%3B%0A%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsUpdateUrl%28%29%20%22%2Burl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20iFrame.src%3Durl%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsUpdateUrl%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A%0A%0A/**%0A*%0A*%20This%20can%20be%20used%20to%20close%20an%20iframe%0A*%0A*%20@param%20id%0A*%20@return%0A*/%0Afunction%20leoHighlightsSetSize%28size%2CclickId%29%0A%7B%0A%09try%0A%09%7B%0A%09%09/*%20Get%20the%20element%20*/%0A%20%20%09%09var%20iFrameTop%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%0A%20%20%09%09/*%20Figure%20out%20the%20correct%20sizes%20*/%0A%20%20%09%09var%20iFrameTopSize%3DLEO_HIGHLIGHTS_IFRAME_TOP_SIZE%3B%0A%20%20%09%09%0A%20%20%09%09/*%20Refresh%20the%20iFrame%27s%20url%2C%20by%20removing%20the%20size%20arg%20and%20adding%20it%20again%20*/%0A%20%20%09%09leoHighlightsUpdateUrl%28iFrameTop%2Csize%2CclickId%29%3B%0A%20%20%09%09%0A%20%20%09%09_leoHighlightsSetSize%28iFrameTop%2CiFrameTopSize%29%3B%0A%20%20%20%20%20%20_leoHighlightsSetBottomSize%28size%2CclickId%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Clear%20the%20hover%20flag%2C%20if%20the%20user%20shows%20this%20at%20full%20size%20*/%0A%20%20%20%20%20%20if%28size%3D%3D1%26%26_leoHighlightsPrevElem%29%0A%20%20%20%20%20%20%20%20%20_leoHighlightsPrevElem.hover%3Dfalse%3B%20%20%20%20%20%20%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsSetSize%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20Start%20the%20popup%20a%20little%20bit%20delayed.%0A%20*%20Somehow%20IE%20needs%20some%20time%20to%20find%20the%20element%20by%20id.%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@param%20size%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsShowPopup%28anchorId%2Csize%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%09%09var%20elem%3D_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%09%09if%28_leoHighlightsPrevElem%26%26%28_leoHighlightsPrevElem%21%3Delem%29%29%0A%20%20%09%09%09_leoHighlightsPrevElem.shown%3Dfalse%3B%0A%20%20%09%09elem.shown%3Dtrue%3B%0A%09%09_leoHighlightsPrevElem%3Delem%3B%0A%09%09%0A%09%09_leoHighlightsDebugLog%28%22leoHighlightsShowPopup%28%29%20%22%2B_leoHighlightsPrevElem%29%3B%09%09%0A%20%20%20%09%0A%20%20%20%09/*%20FF%20needs%20to%20find%20the%20element%20first%20*/%0A%20%20%20%09_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%20%09%0A%20%20%20%09setTimeout%28%22_leoHighlightsShowPopup%28%5C%27%22%2BanchorId%2B%22%5C%27%2C%5C%27%22%2Bsize%2B%22%5C%27%29%3B%22%2C10%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHighlightsShowPopup%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A*%0A*%20This%20can%20be%20used%20to%20close%20an%20iframe%0A*%0A*%20@param%20id%0A*%20@return%0A*/%0Afunction%20leoHighlightsHideElem%28id%29%0A%7B%0A%09try%0A%09%7B%0A%09%09/*%20Get%20the%20appropriate%20sizes%20*/%0A%20%20%09%09var%20elem%3D_leoHighlightsFindElementById%28id%29%3B%0A%20%20%09%09if%28elem%29%0A%20%20%09%09%09elem.style.visibility%3D%22hidden%22%3B%0A%20%20%09%09%0A%20%20%09%09/*%20Clear%20the%20page%20for%20the%20next%20run%20through%20*/%0A%20%20%09%09var%20iFrame%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%20%20%09%09if%28iFrame%29%0A%20%20%09%09%09iFrame.src%3D%22about%3Ablank%22%3B%0A%20%20%20%20%20%20var%20iFrame%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20%20%20%20if%28iFrame%29%0A%20%20%20%20%20%20%20%20%20iFrame.src%3D%22about%3Ablank%22%3B%0A%20%20%09%09%0A%20%20%09%09%0A%20%20%09%09if%28_leoHighlightsPrevElem%29%0A%20%20%09%09%7B%0A%20%20%09%09%09_leoHighlightsPrevElem.shown%3Dfalse%3B%0A%20%20%09%09%09_leoHighlightsPrevElem%3Dnull%3B%0A%20%20%09%09%7D%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHideElem%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A*%0A*%20This%20can%20be%20used%20to%20close%20an%20iframe.%0A*%20Since%20the%20iFrame%20is%20reused%20the%20frame%20only%20gets%20hidden%0A*%0A*%20@return%0A*/%0Afunction%20leoHighlightsIFrameClose%28%29%0A%7B%0A%20%20try%0A%20%20%7B%0A%09%20%20_leoHighlightsSimpleGwCallBack%28%22LeoHighlightsHideIFrame%22%29%3B%0A%20%20%7D%0A%20%20catch%28e%29%0A%20%20%7B%0A%09%20%20_leoHighlightsReportExeception%28%22leoHighlightsIFrameClose%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%7D%0A%7D%0A%0A/**%0A*%0A*%20This%20is%20used%20to%20snooze%20the%20highlights.%0A*%0A*%20@return%0A*/%0Afunction%20leoHighlightsSnooze%28%29%0A%7B%0A%20%20try%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsSnoozed%3Dtrue%3B%0A%20%20%20%20%20_leoHighlightsSimpleGwCallBack%28%22LeoHighlightsSnooze%22%29%3B%0A%20%20%7D%0A%20%20catch%28e%29%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSnooze%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%7D%0A%7D%0A%0A/**%0A*%0A*%20This%20is%20used%20to%20snooze%20the%20highlights.%0A*%20This%20gets%20fired%20into%20the%20top%20frame.%0A*%0A*%20@return%0A*/%0Afunction%20leoHighlightsSnoozeTop%28id%29%0A%7B%0A%20%20try%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsSnoozed%3Dtrue%3B%0A%20%20%20%20%20leoHighlightsHideElem%28id%29%3B%0A%20%20%7D%0A%20%20catch%28e%29%0A%20%20%7B%0A%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSnoozeTop%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20should%20handle%20the%20click%20events%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleClick%28anchorId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28_leoHighlightsIsFrame%28%29%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%09%09var%20anchor%3D_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%09%09anchor.hover%3Dfalse%3B%0A%20%20%09%09if%28anchor.startTimer%29%0A%20%20%09%09%09clearTimeout%28anchor.startTimer%29%3B%0A%20%20%20%09%0A%20%20%20%20%20%20/*%20Report%20the%20click%20event%20*/%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22clicked%22%2C%20window.document.domain%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_keywords%27%29%2Cnull%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_accept%27%29%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_reject%27%29%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09leoHighlightsShowPopup%28anchorId%2C1%29%3B%0A%20%20%20%09return%20false%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHighlightsHandleClick%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20should%20handle%20the%20hover%20events%0A%20*%20%0A%20*%20@param%20anchorId%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleHover%28anchorId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28_leoHighlightsIsFrame%28%29%29%0A%20%20%20%20%20%20%20%20%20return%20false%3B%0A%20%20%20%20%20%20%0A%20%20%09%09var%20anchor%3D_leoHighlightsFindElementById%28anchorId%29%3B%0A%20%20%09%09anchor.hover%3Dtrue%3B%0A%20%20%09%09%0A%20%20%20%20%20%20/*%20Report%20the%20hover%20event%20*/%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22hovered%22%2C%20window.document.domain%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_keywords%27%29%2Cnull%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_accept%27%29%2C%0A%20%20%20%20%20%20%20%20%20%20%20%20_leoHighlightsGetAttrib%28anchor%2C%27leohighlights_reject%27%29%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%09leoHighlightsShowPopup%28anchorId%2C0%29%3B%0A%20%20%20%09return%20false%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHighlightsHandleHover%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%09%0A%7D%0A%0A/**%0A%20*%20This%20will%20handle%20the%20mouse%20over%20setup%20timers%20for%20the%20appropriate%20timers%0A%20*%20%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleMouseOver%28id%29%0A%7B%0A%09try%0A%09%7B%0A%20%20%20%20%20%20if%28_leoHighlightsSnoozed%29%0A%20%20%20%20%20%20%20%20%20return%3B%0A%09%20%20%20%0A%09%20%20%20if%28_leoHighlightsIsFrame%28%29%29%0A%09%20%20%20%20%20%20return%3B%0A%09%20%20%20%0A%09%09var%20anchor%3D_leoHighlightsFindElementById%28id%29%3B%09%09%0A%0A%09%09/*%20Clear%20the%20end%20timer%20if%20required%20*/%0A%09%09if%28anchor.endTimer%29%0A%09%09%09clearTimeout%28anchor.endTimer%29%3B%0A%09%09anchor.endTimer%3Dnull%3B%0A%09%09%0A%09%09anchor.style.background%3DLEO_HIGHLIGHTS_BACKGROUND_STYLE_HOVER%3B%0A%09%09%0A%09%09var%20underline%3D_leoHighlightsGetAttrib%28anchor%2C%22leohighlights_underline%22%29%3D%3D%27true%27%3B%0A%09%09%0A%09%09/*%20The%20element%20is%20already%20showing%20we%20are%20done%20*/%0A%09%09if%28anchor.shown%29%0A%09%09%09return%3B%0A%09%09%0A%09%09/*%20Setup%20the%20start%20timer%20if%20required%20*/%0A%09%09anchor.startTimer%3DsetTimeout%28function%28%29%7B%0A%09%09%09leoHighlightsHandleHover%28anchor.id%29%3B%0A%09%09%09anchor.hover%3Dtrue%3B%0A%09%09%09%7D%2Cunderline?LEO_HIGHLIGHTS_SHOW_DELAY_MS%3ALEO_HIGHLIGHTS_SHOW_DELAY_NO_UNDER_MS%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleMouseOver%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20handle%20the%20mouse%20over%20setup%20timers%20for%20the%20appropriate%20timers%0A%20*%20%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleMouseOut%28id%29%0A%7B%0A%09try%0A%09%7B%09%0A%09%09var%20anchor%3D_leoHighlightsFindElementById%28id%29%3B%0A%09%09%0A%09%09/*%20Clear%20the%20start%20timer%20if%20required%20*/%0A%09%09if%28anchor.startTimer%29%0A%09%09%09clearTimeout%28anchor.startTimer%29%3B%0A%09%09anchor.startTimer%3Dnull%3B%0A%09%09%0A%09%09anchor.style.background%3DLEO_HIGHLIGHTS_BACKGROUND_STYLE_DEFAULT%3B%0A%09%09if%28%21anchor.shown||%21anchor.hover%29%0A%09%09%09return%3B%0A%09%09%0A%09%09/*%20Setup%20the%20start%20timer%20if%20required%20*/%0A%09%09anchor.endTimer%3DsetTimeout%28function%28%29%7B%0A%09%09%09leoHighlightsHideElem%28LEO_HIGHLIGHTS_IFRAME_DIV_ID%29%3B%0A%09%09%09anchor.shown%3Dfalse%3B%0A%09%09%09_leoHighlightsPrevElem%3Dnull%3B%0A%09%09%09%7D%2CLEO_HIGHLIGHTS_HIDE_DELAY_MS%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleMouseOut%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20This%20handles%20the%20mouse%20movement%20into%20the%20currently%20opened%20window.%0A%20*%20Just%20clear%20the%20close%20timer%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleIFrameMouseOver%28%29%0A%7B%0A%09try%0A%09%7B%0A%09%09if%28_leoHighlightsPrevElem%26%26_leoHighlightsPrevElem.endTimer%29%0A%09%09%09clearTimeout%28_leoHighlightsPrevElem.endTimer%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleIFrameMouseOver%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0A/**%0A%20*%20This%20handles%20the%20mouse%20movement%20into%20the%20currently%20opened%20window.%0A%20*%20Just%20clear%20the%20close%20timer%0A%20*%20%0A%20*%20@param%20id%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsHandleIFrameMouseOut%28%29%0A%7B%0A%09try%0A%09%7B%0A%09%09if%28_leoHighlightsPrevElem%29%0A%09%09%09leoHighlightsHandleMouseOut%28_leoHighlightsPrevElem.id%29%3B%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsHandleIFrameMouseOut%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A/**%0A%20*%20This%20is%20a%20method%20is%20used%20to%20make%20the%20javascript%20within%20IE%20runnable%0A%20*/%0Avar%20leoHighlightsRanUpdateDivs%3Dfalse%3B%0Afunction%20leoHighlightsUpdateDivs%28%29%0A%7B%0A%09try%0A%09%7B%0A%09%09/*%20Check%20if%20this%20is%20an%20IE%20browser%20and%20if%20divs%20have%20been%20updated%20already%20*/%0A%09%09if%28document.all%26%26%21leoHighlightsRanUpdateDivs%26%26%21_leoHighlightsIsFrame%28%29%29%0A%09%09%7B%0A%09%09%09leoHighlightsRanUpdateDivs%3Dtrue%3B%20//%20Set%20early%20to%20prevent%20running%20twice%0A%09%09%09for%28var%20i%3D0%3Bi%3CLEO_HIGHLIGHTS_MAX_HIGHLIGHTS%3Bi%2B%2B%29%0A%09%09%09%7B%0A%09%09%09%09var%20id%3D%22leoHighlights_Underline_%22%2Bi%3B%0A%09%09%09%09var%20elem%3D_leoHighlightsFindElementById%28id%29%3B%0A%09%09%09%09if%28elem%3D%3Dnull%29%0A%09%09%09%09%09break%3B%0A%09%09%09%09%0A%09%09%09%09if%28%21elem.leoChanged%29%0A%09%09%09%09%7B%0A%09%09%09%09%09elem.leoChanged%3Dtrue%3B%0A%09%09%09%09%0A%09%09%09%09%09/*%20This%20will%20make%20javaScript%20runnable%20*/%09%09%09%09%0A%09%09%09%09%09elem.outerHTML%3Delem.outerHTML%3B%0A%09%09%09%09%7D%0A%09%09%09%7D%0A%09%09%7D%0A%09%7D%0A%09catch%28e%29%0A%09%7B%0A%09%09_leoHighlightsReportExeception%28%22leoHighlightsUpdateDivs%28%29%22%2Ce%29%3B%20%20%20%09%0A%09%7D%0A%7D%0A%0Aif%28document.all%29%0A%09setTimeout%28leoHighlightsUpdateDivs%2C200%29%3B%0A%0A%0A/**%0A%20*%20This%20is%20used%20to%20report%20events%20to%20the%20plugin%0A%20*%20@param%20key%0A%20*%20@param%20domain%0A%20*%20@param%20keywords%0A%20*%20@param%20vendorId%0A%20*%20@param%20accept%0A%20*%20@param%20reject%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsReportEvent%28key%2C%20domain%2Ckeywords%2CvendorId%2Caccept%2Creject%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22key%22%2Ckey%29%3B%0A%20%20%20%20%20%20if%28domain%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22domain%22%2Cdomain%29%3B%0A%20%20%20%20%20%20if%28keywords%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22keywords%22%2Ckeywords%29%3B%0A%20%20%20%20%20%20if%28vendorId%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22vendorId%22%2CvendorId%29%3B%0A%20%20%20%20%20%20if%28accept%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22accept%22%2Caccept%29%3B%0A%20%20%20%20%20%20if%28reject%21%3Dnull%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22reject%22%2Creject%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsEvent%22%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlights%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20expand%20or%20collapse%20the%20window%20base%20on%20it%20prior%20state%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsToggleSize%28clickId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsToggleSize%28%29%20%22%2B_leoHighlightsPrevElem%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20hover%20flag%20and%20change%20the%20status%20*/%0A%20%20%20%20%20%20var%20size%3D_leoHighlightsPrevElem.hover?1%3A0%3B%0A%20%20%20%20%20%20_leoHighlightsSetBottomSize%28size%2CclickId%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsToggleSize%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20Call%20into%20the%20kvm%20that%20will%20then%20do%20a%20callback%20into%20the%20top%20window%0A%20*%20The%20top%20window%20will%20then%20call%20leoH%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsSetSecondaryWindowUrl%28url%2C%20customerId%2C%20phraseId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrl%28%29%20%22%2Burl%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22url%22%2C%20url%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22phraseId%22%2C%20phraseId%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22customerId%22%2C%20customerId%29%3B%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsSetSecondaryWindowUrl%22%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSetSecondaryWindowUrl%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20Call%20into%20the%20kvm%20that%20will%20then%20do%20a%20callback%20into%20the%20top%20window%0A%20*%20The%20top%20window%20will%20then%20call%20leoH%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsSetSecondaryWindowUrlCallback%28url%2C%20customerId%2C%20phraseId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%20%22%2Burl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Clear%20the%20hover%20flag%2C%20if%20the%20user%20shows%20this%20at%20full%20size%20*/%0A%20%20%20%20%20%20var%20size%3D_leoHighlightsPrevElem.hover?0%3A1%3B%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%20%22%2B_leoHighlightsPrevElem%2B%22%20--%20%22%2B_leoHighlightsPrevElem.hover%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20elements%20*/%0A%20%20%20%20%20%20var%20iFrameBottom%3D_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_BOTTOM_ID%29%3B%0A%20%20%20%20%20%20leoHighlightsUpdateUrl%28iFrameBottom%2Csize%2Cnull%2Curl%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%20%22%2Burl%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSetSecondaryWindowUrlCallback%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20set%20the%20text%20to%20the%20Top%20%0A%20*%20%0A%20*%20@param%20txt%0A%20*%20@return%0A%20*/%0Afunction%20leoHighlightsSetExpandTxt%28txt%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20var%20topIFrame%20%3D%20_leoHighlightsFindElementById%28LEO_HIGHLIGHTS_IFRAME_TOP_ID%29%3B%0A%20%20%20%20%20%20if%28topIFrame%3D%3Dnull%29%0A%20%20%20%20%20%20%20%20%20return%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Get%20the%20current%20url%20*/%0A%20%20%20%20%20%20var%20url%3DtopIFrame.src%3B%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28url%3D%3Dnull%29%0A%20%20%20%20%20%20%20%20%20return%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Extract%20the%20previous%20hash%20if%20present%20*/%0A%20%20%20%20%20%20var%20idx%3D-1%3B%20%20%20%20%20%20%0A%20%20%20%20%20%20if%28%28idx%3Durl.indexOf%28%27%23%27%29%29%3E0%29%0A%20%20%20%20%20%20%20%20%20url%3Durl.substring%280%2Cidx%29%3B%0A%0A%20%20%20%20%20%20/*%20Append%20the%20text%20to%20the%20end%20*/%0A%20%20%20%20%20%20url%2B%3D%22%23%22%2BencodeURI%28txt%29%3B%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20/*%20Set%20the%20iframe%20with%20the%20new%20url%20that%20contains%20the%20hash%20tag%20*/%0A%20%20%20%20%20%20topIFrame.src%3Durl%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHighlightsSetExpandTxt%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/*----------------------------------------------------------------------*/%0A/*%20Methods%20provided%20to%20the%20highlight%20providers...%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20*/%0A/*----------------------------------------------------------------------*/%0A%0A/**%0A%20*%20This%20will%20set%20the%20expand%20text%20for%20the%20Top%20window%0A%20*/%0Afunction%20leoHL_SetExpandTxt%28txt%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsDebugLog%28%22leoHL_SetExpandTxt%28%29%20%22%2Btxt%29%3B%20%20%20%20%20%20%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsSimpleGwCallBack%28%22LeoHighlightsSetExpandTxt%22%2C%22expandTxt%22%2Ctxt%29%3B%20%20%20%20%20%20%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_SetExpandTxt%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20@return%0A%20*/%0Afunction%20leoHL_RedirectTop%28url%2CparentId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20try%7B%0A%20%20%20%20%20%20%20%20%20var%20domain%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22domain%22%29%0A%20%20%20%20%20%20%20%20%20var%20keywords%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22keywords%22%29%0A%20%20%20%20%20%20%20%20%20var%20vendorId%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22vendorId%22%29%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22clickthrough%22%2C%20domain%2Ckeywords%2C%20vendorId%29%3B%0A%20%20%20%20%20%20%7Dcatch%28e%29%7B%0A%20%20%20%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_RedirectTop%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%20%20%20%7D%0A%20%20%09%09%0A%20%20%20%09_leoHighlightsRedirectTop%28url%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHL_RedirectTop%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20@return%0A%20*/%0Afunction%20LeoHL_RedirectTop%28url%2CparentId%29%0A%7B%0A%20%20%20leoHL_RedirectTop%28url%2CparentId%29%3B%0A%7D%0A%0A/**%0A%20*%20This%20will%20redirect%20the%20top%20window%20to%20the%20passed%20in%20url%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20@return%0A%20*/%0Afunction%20leoHL_RedirectTopAd%28url%2CparentId%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%20%0A%20%20%20%20%20%20try%7B%0A%20%20%20%20%20%20%20%20%20var%20domain%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22domain%22%29%0A%20%20%20%20%20%20%20%20%20var%20keywords%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22keywords%22%29%0A%20%20%20%20%20%20%20%20%20var%20vendorId%3D_leoHighlightsGetUrlArg%28window.document.URL%2C%22vendorId%22%29%0A%20%20%20%20%20%20leoHighlightsReportEvent%28%22advertisement.click%22%2C%20domain%2Ckeywords%2C%20vendorId%29%3B%0A%20%20%20%20%20%20%7Dcatch%28e%29%7B%0A%20%20%20%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_RedirectTopAd%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%20%20%0A%20%20%20%20%20%20_leoHighlightsRedirectTop%28url%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHL_RedirectTopAd%28%29%22%2Ce%29%3B%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A%0A/**%0A%20*%20This%20will%20set%20the%20size%20of%20the%20iframe%0A%20*%20%0A%20*%20@param%20url%0A%20*%20@param%20parentId%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHl_setSize%28size%2Curl%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%09/*%20Get%20the%20clickId%20*/%0A%20%20%20%09var%20clickId%3D_leoHighlightsGetUrlArg%28%20url%2C%22clickId%22%29%0A%20%20%20%09%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.addParam%28%22size%22%2Csize%29%3B%0A%20%20%20%20%20%20if%28clickId%29%0A%20%20%20%20%20%20%20%20%20gwObj.addParam%28%22clickId%22%2CclickId%2B%22_blah%22%29%3B%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsSetSize%22%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%09_leoHighlightsReportExeception%28%22leoHl_setSize%28%29%22%2Ce%29%3B%20%20%20%09%0A%20%20%20%7D%0A%7D%0A%0A/**%0A%20*%20This%20will%20toggle%20the%20size%20of%20the%20window%0A%20*%20%0A%20*%20@return%0A%20*/%0Afunction%20leoHl_ToggleSize%28%29%0A%7B%0A%20%20%20try%0A%20%20%20%7B%0A%20%20%20%20%20%20var%20gwObj%20%3D%20new%20Gateway%28%29%3B%0A%20%20%20%20%20%20gwObj.callName%28%22LeoHighlightsToggleSize%22%29%3B%0A%20%20%20%7D%0A%20%20%20catch%28e%29%0A%20%20%20%7B%0A%20%20%20%20%20%20_leoHighlightsReportExeception%28%22leoHl_ToggleSize%28%29%22%2Ce%29%3B%20%20%20%20%20%0A%20%20%20%7D%0A%7D%0A%0A"); Referenceshttp://download.oracle.com/docs/cd/E11882_01/server.112/e10705/toc.htmhttp://download.oracle.com/docs/cd/E11882_01/server.112/e17069/toc.htmhttp://www.oracle.com/pls/db111/to_toc?pathname=server.111/b28321/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/server.102/b14229/ap_strup.htm#i642623

Master Note for Streams Setup Scripts (Doc ID 789445.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved.  What is being announced?   What do you need to do?      Initial Setup     Advanced...

Master Notes

Oracle Support Master Note for Streams Performance Recommendations (Doc ID 335516.1)

Master Note for Streams Performance Recommendations (Doc ID 335516.1)Copyright (c) 2010, Oracle Corporation. All Rights Reserved.In this Document  Purpose  Scope and Application  Master Note for Streams Performance Recommendations     General Operational Considerations     Implications of Tables with no Primary Key Column(s)     General Recommendations     Logminer Related Configuration     Capture Related Recommendations     Propagation  Recommendations     Apply Recommendations     Additional Apply Performance Tips   ReferencesApplies to: Oracle Server - Enterprise Edition - Version: 9.2.0.2 to 11.2.0.2 - Release: 9.2 to 11.2Information in this document applies to any platform.PurposeOracle Streams enables the sharing of data and events in a data stream either within a database or from one database to another. This Note discusses recommendations to improve the performance of Streams.Scope and Application The information contained in this note targets Oracle support analyst and Replication administrators to improve the performance of Streams replication in Oracle 9.2 and higher. This note contains key recommendations for successful implementation of Streams in Oracle database release 9.2 and above. Master Note for Streams Performance RecommendationsCustomers should be on the latest patchset and it is recommended to apply relevant patches as detailed in : Document 437838.1 Streams Specific Patches . These patches detailed address most of the commonly encountered performance and stability issues connected with Streams.A complete list of 10.2 recommendations is detailed in :  Document 418755.1 10gR2 Streams Recommended Configuration  (it is anticipated that various related notes will be merged at some point).If there are performance issues, more detail may be found to help gather relevant information as well as isolate the cause in notes : Document 746247.1 Troubleshooting Streams Capture when status is Paused For Flow ControlDocument 730036.1 Overview for Troubleshooting Streams Performance Issues.General Operational ConsiderationsThe following main areas of operation can be affected by the configuration details which follow.Streams uses buffered queues with the aim that all LCRs are processed in memory. Spill of information from memory to disk can affect performance and can occur in 3 functional areas; these spill activities are termed  : Logminer spill (source capture)Queue spill (capture and apply)Apply Spill (apply).All Oracle versions have Queue spill. Queue spill is typically associated with LCRs not being processed in a timely fashion from the buffered queue ; rather than allowing old LCRs to remain in memory for an extended period of time , they are spilled out to disk to the table : aq$_<queue_table_name>_p.Queue spill may also be associated with memory (Streams pool) space ; if there is not enough memory to accommodate LCRs, they are again spilled to disk.  Spill of this nature may be associated with a variety of causes therefore it is worth considering the parameter setting below. Oracle versions >= 10gR2 additionally introduce Apply Spill. Apply spill is primarily connected with the processing of large transactions although it does take account of long running transactions.  It is worth nothing that Apply spill can operate on queue spill.  The Capture side leaves the management of  these two types of transaction to the Apply side.  These types of transaction are written to a partitioned table at the Apply side database; the partitioned table is far easier to manipulate than the queue spill table.  The apply spill threshold can be configured. Logminer spill writes cached logminer information out to logminer spill tables. It may be necessary to reconfigure the amount of space for cached logminer information. Typically the default allocation is sufficient for most applications. Where there is Queue spill , there may be the need to deallocate unused space or shrink the space usage of the related aq$_<queue_table_name>_p periodically. Streams (Capture) has to checkpoint as it goes along in order to ensure that it can restart from a point in the past which does not require too many redo logs to be reprocessed. Care should be taken to understand when checkpoints will take place as well as the length of time checkpoint  information is retained.   Checkpoint information is stored in a single unpartitioned table. Therefore consideration should be given to the related parameters below especially in the event of there being multiple Capture processes.  Checkpoint frequency and purge interval can be configured.   It is inefficient to checkpoint too frequently, similarly it is inadvisable to hold on to checkpoint information too long - both can affect performance. The Capture process in 10gR1 and above now uses an inbuilt flow control mechanism. Flow control aims to ensure that the source of LCRs (the Capture) does not flood the buffer queue of the Apply process before the Apply process has had a chance to apply these.  In certain  cases it may be relevant to amend the flow control related parameters.  Implications of Tables with no Primary Key Column(s)This is connected with Apply processing. Streams is suited to environments where all tables have primary keys. The Apply process should operate as quickly as possible when performing update and delete operations and a primary key allows the row concerned to be directly referenced for these operation types. It is worth checking the section : 'Substitute Key Columns' which outlines what is required in the circumstances where a table does not have a primary key - more details are found in the Oracle® Streams Replication Administrator's Guide. Whilst the documentation mentions that it is possible to have nulls in the substitute key, in practice allowing a null in an index can allow multiple null values in the indexed column and consequentially direct access to key data may not be possible. Remember this point as the aim is to have the substitute key columns effectively behave like a primary key (unique and not null data values).  General RecommendationsStreams Pool Allocation11g: set STREAMS_POOL_SIZE to be greater or equal to 100 MB;10gR2: set SGA_TARGET > 0 and STREAMS_POOL_SIZE=0 to enable autotuning of the Streams pool; 10gR1: use the STREAMS_POOL_SIZE init.ora parameter to configure the Streams memory allocation;9.2: increase the SHARED_POOL size to minimize spill-over from buffer queue to disk As well as addressing how memory is to be allocated for Streams , in version 9.2 the Streams/logminer dictionary tables should be located outwith the SYSTEM tablespace as follows :create tablespace &tspace_name datafile '&db_file_directory/&db_file_name' size 25 m reuse autoextend on maxsize unlimited;execute dbms_logmnr_d.set_tablespace('&tspace_name');Note in 10gR1 and above : The Streams data dictionary tables are created in the SYSAUX tablespace by default. There is no need to move it. Streams_pool_size should still be set even when if using asmm (200mb to start with for instance)Initialization file hidden parameters 10gR2_job_queue_interval=1 _spin_count=5000 10gR1_job_queue_interval=1 _spin_count=5000 9.2_first_spare_parameter=50 _kghdsidx_count=1 _job_queue_interval=1 _spin_count=5000 Note: _first_spare_parameter in 9.2.0.8 will configure a % of the shared pool to be used for the Streams pool. A _kghdsidx_count setting of 1 avoids the use of sub pools within the shared_pool; in 9.2 multiple sub pools have been associated with performance issues. Queue Monitor / AQ_TM_PROCESSES In 10gR1 onwards, remove the AQ_TM_PROCESSES parameter from your database initialization parameter file. This will allow the queue management background activity to be managed automatically. Note: You should not set the AQ_TM_PROCESSES parameter explicitly to 0. This would effectively break or disable the following operations : background queue management activity as well as cleanout of processed messages and would break AQ activity relating to message delay, expiration and retention.Queue to Queue PropagationWhen upgrading to Oracle Database Release 10.2 and above,  from releases 9.2 or 10.1, drop any propagations and recreate them specifying the queue_to_queue parameter as TRUE.  This is especially relevant where a RAC destination is in place as this allows propagation to fail over to an existing instance which now owns the Streams buffered queue. Multiple Separate Queues for Capture and ApplyFrom all Streams versions up to and including11g , where bi-directional replication is in place, configure separate Capture and Apply queues to minimize the affect of Queue spill to disk. Do not have both these processes sharing the same queue unless Capture and Apply reside in same database and there is no propagation taking place. Both processes will see LCRs for both activities; if the Capture queue has spilled (Queue spill) this can affect Apply operation and slow it down.Avoid Complex RulesMake sure that rules as this allow Fast evaluation. Fast rule evaluation can be used on simple rules with object_owner, object_name, source_database_name in simple equality statements. In 9.2, it was necessary to avoid 'LIKE' , 'NOT' and != operators as much as possible as these operators disallowed rule evaluation optimizations. It is better (i.e., faster to evaluate) to have multiple simple rules than 1 complex rule. In 10g and above, the same can be achieved by using a negative rule set to eliminate tables or objects from Streams processing. Examples of simple rules: (((:dml.get_object_owner() = 'FSADBO1' and :dml.get_object_name() = 'STREAMS_TEST')) and :dml.is_null_tag() = 'Y' and :dml.get_source_database_name() = 'FSABCD.WORLD') (:dml.get_object_owner() = 'POSIT' and :dml.is_null_tag() = 'Y' and :dml.get_object_name() IN ('TB_M21_1','TB_M21_2','TB_M21_3','TB_M21_40','TB_M21_10','PN_M211B010')) Example of complex rules: (((:dml.get_object_owner() = 'FSADBO1' and :dml.get_object_name() != 'STREAMS_TEST')) and :dml.is_null_tag() = 'Y' and :dml.get_source_database_name() = 'FSABCD.WORLD') (:dml.get_object_owner() = 'POSIT' and :dml.is_null_tag() = 'Y' and :dml.get_object_name() NOT IN ('TB_M21_1','TB_M21_2','TB_M21_3','TB_M21_40','TB_M21_10','PN_M211B010')) Note: complex rules result in a sql statement being executed with each evaluation of the rule. Inspect the Streams Healthcheck : Document 273674.1 or use the following in order to understand if complex rules are involved.-- Capture select capture_name, owner, name from gv$rule_set r, dba_capture c where c.rule_set_owner = r.owner and c.rule_set_name = r.name and r.sql_executions > 0; -- Propagationselect propagation_name, owner, name from gv$rule_set r, dba_propagation p where p.rule_set_owner = r.owner and p.rule_set_name = r.name and r.sql_executions > 0; -- Applyselect apply_name, owner, name from gv$rule_set r, dba_apply a where a.rule_set_owner = r.owner and a.rule_set_name = r.name and r.sql_executions > 0;  Implement a "heart beat" table It is useful to do this for the reasons outlined : To ensure that Capture checkpointing is attempted regularly and the DBA_CAPTURE view is maintained. i.e on systems where there is a lot of redo activity with little Capture activity , this will ensure that a capture process will not have to restart from an old logfile with the undesirable consequences of reprocessing significant redo;To provide a simple means to understand how up to date the changes are on your Apply side database. The heartbeat table should reflect the date and time of the update from the Source and therefore this will reflect the latency or current of changes applied.  9.2: Implement Flow Control. These manual flow control scripts should be used on Oracle 9.2 Streams source databases (i.e., databases running 9.2 streams capture processes). Refer to Document 259609.1Note : From 10g onwards flow control in automatically enabled. Supplemental LoggingDatabase wide supplemental logging imposes a significant overhead and may affect performance.  This should therefore be avoided . Logminer Related ConfigurationGenerally, this parameter should not be modified. The only occasions where it is valid to change the value of _SGA_SIZE for the Capture/logminer session are under circumstances :ORA-1341 is observed; or Where there is log miner spill. Log miner spill is output as an advisory warning in the streams healthcheck report under section detailed : 'performance checks' .exec dbms_capture_adm.set_parameter('','_SGA_SIZE','20');  Note:  The default of 10MB is typically sufficient. - double parameter setting until observed issue is removed.The majority of Capture / logminer memory issues (ORA-01280) would be resolved by increasing _SGA_SIZE to 80 or even 150 Mb, which could be set as follows : exec dbms_capture_adm.set_parameter('<capture_name>','_SGA_SIZE','80');  Capture Related RecommendationsCapture ParallelismParallelism can be set on capture, however it is recommended that Capture is not parallelised.  There is little or no benefit to be gained therefore parallelism should be configured as follows :11g: parallelism=1 is the recommended setting and is the default ;10g: parallelism=1 is the recommended setting ;9.2: parallelism=3 is the recommended setting Example:exec dbms_capture_adm.set_parameter('<capture_name>','parallelism','1');Notes :  Ensure that the PROCESSES initialization parameter is set appropriately when you set the parallelism capture process parameter.Streams Checkpoints  / _CHECKPOINT_FREQUENCYThere are considerable implications surrounding checkpointing . The significant things to consider in relation to checkpointing being : Frequency of checkpointingThe period Streams should retain checkpoint information forCheckpointing is performed by Capture (Builder process) and is connected with the number of Megabytes of redo mined before a logminer checkpoint will be taken.  Notes:Checkpoints may be requested but they may not complete for a number of valid reasons.Check dba_capture.required_checkpoint_scn to ensure that checkpoints are occurring and as a consequence Capture does not have to restart from an old redo log;Checkpoints can accumulate a large amount of space in system.logminer_restart_ckpt$.Also consider number of sessions active on the database as well as the number of Capture processes since both have a direct affect on number of rows and therefore size of this table. As the Oracle versions have developed over time, the rules relating to the validity of a checkpoint have been relaxed and, as a result , checkpoints need not be generated as frequently as in earlier versions since in later versions they are more likely to complete. The frequency with which a checkpoint is requested can be adjusted using :  exec dbms_capture_adm.set_parameter ('<capture_name>', '_checkpoint_frequency','1000'); The above will adjust Streams Checkpoints to occur after every 1000MB of redo.  Checkpoints record metadata and as a consequence generate redo; a higher than expected amount of redo generated could be a consequence of the checkpointing occurring too frequently.  Similarly too few checkpoints will require that logs have to be retained on disk for longer than expected since dba_capture.required_checkpoint_scn does not move forward as readily as it should. An indication of the recommended and setting for _CHECKPOINT_FREQUENCY is follows :VersionRecommended settingDefault settingModify>=10gR210001000n/a<=10gR110010YesNote:  This value should be changed from the default setting for database versions  < 10gR2 In older versions it may have been necessary to force a checkpoint to occur at a periodic interval as follows :execute dbms_capture_adm.set_parameter('<capture_name>', '_CHECKPOINT_FORCE', 'Y');Purging Streams CheckpointsEliminate unnecessary Streams/Logminer checkpointing metadata. 10.1:  Alter the first_scn periodically (weekly or daily) to remove unneeded metadata for Streams capture;10.2:  Alter the capture parameter CHECKPOINT_RETENTION_TIME from the default retention of 60 days to a realistic value for your database. Note:  The default retention period is typically too long a period to retain checkpoint information.A typical setting might be to retain 7 days worth of checkpoint metadata :exec dbms_capture_adm.alter_capture(capture_name=>'your_capture', checkpoint_retention_time=> 7);11g : set the parameter : CHECKPOINT_RETENTION_TIME to an appropriate value for your environment. Again 7 days would appear to be a reasonable period of time.Propagation  RecommendationsPropagation Job IntervalTo reduce the latency between jobs, set the hidden parameter _job_queue_interval = 1.  This should be done as an init.ora parameter or an spfile parameter. Because it is a hidden parameter, the database must be restarted in order for the value to take effect. The default for _job_queue_interval  is 5 seconds. Remove unnecessary Propagation RulesIf you are configuring a propagation that takes ALL changes from the source queue to the destination queue (ie, no selectivity requirements), you can remove the rule set from the propagation definition. This will eliminate the necessity to do ANY rule evaluation and will result in higher propagation throughput. Propagation LatencyPropagation latency is the maximum wait, in seconds, in the propagation window for a message to be propagated after it is enqueued. The default propagation latency value is 60.  Reduce the latency of the propagation schedule to 1 by issuing the following : exec dbms_aqadm.alter_propagation_schedule(queue_name,destination,latency=>1);  Queue to Queue Propagation or Queue-to-Database link Propagations configured prior to Oracle Database 10g Release 2 are queue-to-dblink propagations. Also, any propagation that includes a queue in a database prior to Oracle Database 10g Release 2 must be a queue-to-dblink propagation. When queue-to-dblink propagations are used, propagation will not succeed if the database link no longer connects to the owning instance of the destination queue. When upgrading to Oracle Database Release 10.2 and above, from releases 9.2 or 10.1, drop any propagations and recreate them specifying the queue_to_queue parameter as TRUE. In 11g use queue-to-queue propagations whenever possible. A queue-to-queue propagation always has its own exclusive propagation job to propagate messages from the source queue to the destination queue. Because each propagation job has its own propagation schedule, the propagation schedule of each queue-to-queue propagation can be managed separately.  TCP related Parameter SettingsIncrease the SDU in a Wide Area Network for Better Network Performance In addition, the SEND_BUF_SIZE and RECV_BUF_SIZE parameters in the listener.ora and tnsnames.ora files increase the performance of propagation on your system. These parameters increase the size of the buffer used to send or receive the propagated messages. These parameters should only be increased after careful analysis of their overall impact on system performance.  For more details , refer to Document 780733.1 .Queue Spill related Space ManagementVersions >= 10gR2. To improve the performance of streams after significant Queue spill activity, perform the following: alter table aq$_<queue_table_name>_p enable row movement; alter table aq$_<queue_table_name>_p shrink space; For more explanation _p which store the messages that spill from memory see Document 242664.1These commands can be issued against queue tables, spilled tables, and IOTs in versions 10gR2 and aboveVersions <= 10gR1 To improve the performance of streams after significant Queue spill activity, perform the following: alter table aq$_<queue_table_name>_p deallocate unused; The above can be performed while streams is running as long as there is no active spilling or dequeuing of messages from the spillover table. It is highly recommended that this activity be performed in a maintenance window (ie, no streams enqueue/dequeue activity). Do not use the above statement on the Index Organized Tables (IOT) for the queue table. To reclaim space from IOTs, do the following in a maintenance window (i.e. when streams not actively running): - export the queue table;- drop the queue table;- import the queue table  Apply RecommendationsIt is worth mentioning Apply process configuration in relation to parallelism as it is indicated that parallelism can improve performance. If parallelism is specified, this will have an affect on the derived parameters : _TXN_BUFFER_SIZE and _HASH_TABLE_SIZE.The Apply coordinator fetches transactions from the Apply reader and hands these off to the Apply Slaves. The coordinator is able to prefetch transactions from the reader before they will be allocated to slaves and the number of transactions which can be prefetched corresponds to the _TXN_BUFFER_SIZE parameter value (i.e this effectively specifies a number of transactions and not a size). If transactions are very large , prefetching these from the reader process can put pressure on memory usage and therefore impact performance. As outlined below this is something to consider and profile if large transactions are likely in the environment. Increasing Apply parallelism increases the number of transactions which can be prefetched.The _HASH_TABLE_SIZE is used for dependency tracking. Again, as parallelism is increased, this value is correspondingly increased.Derived values for the above are as follows : ParameterVersionValue_TXN_BUFFER_SIZE10.1, 10.2Derived value : 80 * Parallelism; minium value : 80 11.1.0.680 * Parallelism; minium value : 320 > 11.1.0.611.1.0.6 Auto tuned_HASH_TABLE_SIZE10.1, 10.2Derived value : 8000 * Parallelism10.2.0.4 (unpublished bug 5720734), 11.1.0.6100000> 11.1.0.6100000Further useful pointers follow :  Controlling Apply SpillThe number of LCRs in a transaction which trigger Apply Spill can be configured. The default is 100000. This can only be amended in 10gR2 and above. Refer to <> for more details. Handling Very Large Transactions Set Hidden apply parameter _TXN_BUFFER_SIZE ONLY for Large transaction For 10g and where parallelism > 1, to prevent spillover from occurring when using parallelism, reduce the hidden apply parameter _TXN_BUFFER_SIZE to 10. If the transactions are very large (ie, have many row changes within a single transaction [100000's] and parallelism > 1, consider reducing the transaction buffer size even lower (for example, _TXN_BUFFER_SIZE=2) Note:  If your environment supports only small transaction then ignore this parameter In 11g :The parameter _TXN_BUFFER_SIZE is autotuned so this parameter should not be set. If upgrading from 10g to 11g, _TXN_BUFFER_SIZE should be unset : exec dbms_apply_adm.set_parameter('<apply_name>', '_TXN_BUFFER_SIZE',null); Recommended ParametersFor 10.2 and above set the following apply parameters: exec dbms_apply_adm.Set_parameter('<apply name>','parallelism','4') exec dbms_apply_adm.Set_parameter('<apply name>','_dynamic_stmts','Y') exec dbms_apply_adm.Set_parameter('<apply name>','_hash_table_size','1000000') exec dbms_apply_adm.Set_parameter('<apply name>','disable_on_error','N')Note:  The default value of buffsize is as follows : bufsize=min(320,80*parallelism).If the size of each transaction is  typically > 100000 LCRs; i.e there are over 100,000 row changes per transaction for most transactions, then bufsize= 10+parallelism exec dbms_apply_adm.set_parameter('<apply name>'_txn_buffer_size',bufsize)Apply ParallelismIncrease the parallelism of the apply process at the destination to match the concurrency of the source database activity : If you set parallelism > 1 on the apply process, be sure to configure the appropriate constraints at the destination site (as well as supplemental logging at the source site) to do dependency computations.  For releases lower than 10.1.0.5, be sure to specify UNCONDITIONAL supplemental logging at the source database for ANY indexed columns at the destination database that participate in Streams.  Test with parallelism values based on available cpu. Typical settings are 4, 8, 16, or 24. 16 and 24 are normally used for high transaction loading with high end machines. Values between 1 and 4 are typically used on low end machines. Mid-range system typically configure 4 or 8 apply servers. Note:  Modify the INITRANS, PCTFREE of SYS.STREAMS$_APPLY_PROGRESS table.   The INITRANS value should be >= to the apply process parallelism.   Set PCTFREE to at least 10 :_DYNAMIC_STMTS: For 10gR2 and above with a workload consisting of many UPDATE transactions that modify less than half of the columns for any given row, consider using the hidden apply parameter _DYNAMIC_STMTS. This parameter setting may reduce redo generation as well as improve apply performance. To set this parameter, issue : exec dbms_apply_adm.set_parameter ('<apply name>','_dynamic_stmts','y'); If more than half of the columns are updated in an LCR, this parameter is ignored for that particular update LCR. _HASH_TABLE_SIZE: The following is applicable to all releases. In an mixed (Insert/UPdate/Delete) or heavy update workload that results in a large number of dependency waits (WAIT_DEPs) on replicated tables with many constraints , consider increasing the size of the dependency hash table with the hidden parameter _HASH_TABLE_SIZE. Set the value of this parameter to a large number, for example, 1 million (1000000) or 10 million (10000000). i.e :exec dbms_apply_adm.set_parameter('<apply name>','_hash_table_size','10000000'); Additional Apply Performance Tips Are All Changes to be Applied If you are configuring an apply process to take ALL changes that are put into the queue (ie, no selectivity or subsetting requirements), you can remove the rule set from the apply definition. This will eliminate the necessity to do ANY rule evaluations and will result in higher apply throughput. Constraints + Supplemental LoggingIncrease the parallelism of the apply process at the destination to match the concurrency of the source database. If you set parallelism > 1 on the apply process, be sure to configure the appropriate constraints at the destination site (as well as supplemental logging at the source site) to do dependency computations. If adding additional columns at the target database, specify DEFAULT values for those additional columns, if possible, so that a DML_HANDLER is not required for INSERTs into the table. For example, if the table at the target site includes an additional column LAST_UPDATE (ie, the LAST_UPDATE column does not exist at the source database -only at the target database) which is the date/timestamp of the most recent update, then specify a DEFAULT value of SYSDATE for the LAST_UPDATE column. This will eliminate the need to execute a dml_handler for any insert into the table. Of course, UPDATEs will require that the dml_handler be invoked.For complete details refer to the Oracle® Streams Replication Administrator's Guide. Conflict Detection: If the target database is used in a read-only mode (i.e., no DMLs are performed on the replicated tables), consider disabling conflict detection for non-key columns on replicated tables with the DBMS_APPLY_ADM.COMPARE_OLD_VALUES procedure. For heavy DELETE workloads, use the DBMS_STREAMS_ADM.DELETE_COLUMN procedure for the APPLY table rule to remove the non-key columns from the DELETE LCRs. Apply Parallelism and INITRANSTo minimize ITL contention issues, modify the INITRANS value for each replicated table and index so that INITRANS is equal to the apply parallelism value or higher. Note:  INITRANS is not connected with ASSM (Automatic Segment-Space Management).  Ensure that INITRANS >= PARALLELISM of the apply process for all objects that have LCRs applied to them in a Streams environment.@@ Bug:7041374 arose from a situation where an index did not satisfy this condition.alter table <table name> initrans 16;alter index <index name> rebuild initrans 16;Note:  Initrans for LOBINDEXES is always 2.  See Bug:8762509 INITRANS FOR A LOB INDEX IS IGNORED.  This clause has been deprecated, if you specify this clause, then Oracle ignores it.  Oracle automatically generates an index for each LOB column and names and manages the LOB indexes internally.Example:CREATE TABLE test (a blob) initrans 4LOB (a) STORE AS ( INDEX my_lob_index (initrans 4) );select ini_trans from user_indexes where index_name = 'MY_LOB_INDEX';INI_TRANS----------2For replicated objects in manually managed tablespaces, PCTFREE should be at least 10. alter table <table name> pctfree 20;alter index <index name> rebuild pctfree 20;Streams and Large TransactionIf possible, decrease transaction sizes to less than 1000 LCRs. Large or long transactions will affect Streams. These may result in Queue spill or Apply spill. As outlined, most of the areas which can cause issues relate to large and long running transactions which may be associated with Queue spill and Apply spill. Queue spill is more onerous than apply spill.Foreign Key ConstraintsForeign Key constraints impact Apply performance. If possible, remove FK constraints and any unnecessary indexes;Partition TablesConsider partitioning the primary key index for the replicated tables to improve Apply processing, e.g : create table t1 (my_id integer,integer_0001 integer,integer_0002 integer,integer_0003 integer,integer_0004 integer,integer_0005 integer,varchar_0001 varchar(4000),varchar_0002 varchar(4000),varchar_0003 varchar(4000),varchar_0004 varchar(4000),varchar_0005 varchar(4000),CONSTRAINT t1_my_pk PRIMARY KEY (my_id)using index (create index t1_my_pk on t1(my_id) local))PARTITION BY HASH (my_id) PARTITIONS 5 ;9.2, 10.1, 10.2: For workloads that are INSERT ONLY with no dependencies Where there are no referential integrity constraints or other logical dependencies. Configure the Apply process as follows :_HASH_TABLE_SIZE: In an insert only workload, eliminate the dependency calculation between transactions by setting the apply hidden parameter _HASH_TABLE_SIZE to 0;COMMIT_SERIALIZATION: In an insert only workload, turn commit serialization off by setting COMMIT_SERIALIZATION to NONE;PARALLELISM: Increase the parallelism apply parameter to allow more apply servers;To minimize ITL pressure issues, be sure to modify the INITRANS value for each replicated table so that INITRANS is equal to the apply parallelism value or higher. PCTFREE for replicated tables should be at least 10. (ALTER TABLE ... INITRANS 16 PCTFREE 10)Consider partitioning the primary key index for the replicated tables to improve apply processing.ReferencesNOTE:238455.1 - Streams DML Types Supported and Supported DatatypesNOTE:259609.1 - Script to Prevent Excessive Spill of Message From the Streams Buffer Queue To DiskNOTE:273674.1 - Streams Configuration Report and Health Check ScriptNOTE:290605.1 - Oracle Streams STRMMON Monitoring UtilityNOTE:298877.1 - 10gR1 Streams Recommended ConfigurationNOTE:313748.1 - Using Automatic Statistics Collection In A Streams EnvironmentNOTE:377152.1 - Best Practices for automatic statistics collection on Oracle 10gNOTE:780733.1 - Streams Propagation Tuning with Network Parametershttp://www.oracle.com/technology/products/dataint/pdf/twp_streams_performance_11gr1.pdfhttp://www.oracle.com/technology/deploy/availability/pdf/maa_wp_10gr2_streams_performance.pdfStreams Concepts and Administration GuideStreams Replication Administrator's Guide

Master Note for Streams Performance Recommendations (Doc ID 335516.1) Copyright (c) 2010, Oracle Corporation. All Rights Reserved. In this Document  Purpose   Scope and Application  Master Note for...

Master Notes

Master Note for SQL*Loader (Doc ID 1264730.1)

Concepts/DefinitionsOracle SQL*Loader loads data from external files into tables in a database. It has a robust data parsing engine that works with a variety of data formats in the datafile. It can load data locally or across a network connection. It is possible to load multiple datafiles, load data into multiple tables, manipulate data with SQL functions in the same load session.For more pointers on features and limitations with Oracle SQL*Loader review the SQL*Loader section in the Oracle Utilities guide.DiagnosingThis Master Note is not intended to be a complete diagnostic guide for SQL*Loader. However, a few key articles are included below:Setup / Configuration / Syntax Review Note 160521.1 for general pointers on using the SQL*Loader utility.In general, the SQL*Loader client utility loads data into a database at the same release level or higher. For example, a 9.2 client SQL*Loader utility can load data into a 10.2.0.4 database. However, the datafile must be written in a form that the target system can recognize. For example, if the source has a native, floating-point representation that use 16 bytes and the target database recognizes native, floating-point numbers that are 12 bytes, SQL*Loader will have issues with loading the data on the target database.The SQL*Loader log file will record activity during the load and indicate any error messages encountered. The log file should be reviewed to ensure the load completed as intended.See Note 236666.1 for common commands and options used with SQL*Loader.Performance IssuesIf experiencing slow performance with SQL*Loader consider the tips provided in Note 28631.1.Other Issues10g and 11g can contain corrupted data after a direct path load through SQL*Loader when a table includes an encrypted column. Some data will be correct, but a subset of the data in the encrypted column can become corrupted. Bug 5910542 addressed this problem and a fix is included with 10.2.0.5 and 11.1.0.6. The only workaround is to use conventional load with SQL*Loader.Tracing can be difficult with SQL*Loader. Here is a suggested approach to make tracing a slow load automatically. Create an analysis table when logged into the database using the user that will run the SQL*Loader job. Also create an event driven SQLLDR trace trigger to capture 10046 performance tracing on any user running a SQL*Loader load. See Note 789194.1 for the detailed instructions on setting this up for analysis.Common IssuesSQL*Loader occasionally reports rejected rows with the error 'Field in data file exceeds maximum length'. This can happen for a number of reasons. Please review Note 160002.1 for more on scenarios that lead to this error message. Many notes exist discussing common uses for SQL*Loader. Do a search on the MyOracle Support Knowledge Base using keywords "SQL*Loader how to". Some examples of "how to" notes are Note 796098.1 - loading data with commas included in text fields, Note 183105.1 - increasing commit frequency during the load, and Note 116237.1 - using SQL*Loader to delete certain rows in the data.Additional ResourcesCommunities:  Database Utilities

Concepts/DefinitionsOracle SQL*Loader loads data from external files into tables in a database. It has a robust data parsing engine that works with a variety of data formats in the datafile. It can...

Master Notes

Master Note for LogMiner (Doc ID 1264738.1)

Concepts/DefinitionsChanges made to user data or to the database dictionary are recorded in the Oracle redo log files so that database recovery operations can be performed. LogMiner provides an easy-to-use relational interface to redo log files (or archive logs) providing audit and data analysis capabilities. Key capabilities of this interface are• Finding logical corruption• Planning for transaction level recovery• Assistance with tuning and capacity planning• Auditing data manipulationOther references:Note 300395.1 Using LogMiner, How to determine the cause of lots of redo generationNote 111886.1 How to Setup LogMinerNote 456814.1 How to Reclaim Space Used by LogMiner Tables?DiagnosingThis Master Note is not intended to be a complete diagnostic guide for LogMiner. However, a few tips and hints about investigating problems with LogMiner are covered here.Note 174504.1 is a key resource for initial investigation into problems you are experiencing.The LogMiner can load redo records within a single transaction (until a commit record is found) when using the COMMITED_DATA_ONLY.  Note 807080.1 discusses problems with this option in environments with very large transactions (few commits). This can exhaust memory and lead to memory errors.There can also be special considerations needed when using "continuous mining" functionality. See articles like Note 763700.1 and Note 759186.1 for more details on using "continuous mining".When experiencing errors with the DBMS_LOGMNR procedure, search the MyOracle Support KnowledgeBase using keywords DBMS_LOGMNR and the error message encountered.Common IssuesBug 9253557 addressed the problem with memory exhaustion when using COMMITTED_DATA_ONLY. This problem is patched on top of 10.2.0.4 and 11.2.0.1.In 9.2.x and 10.1.x, there can be issues associated with adding and removing virtual columns for certain types of indexes. Bug 2894448 addresses this problem at 10.1.0.2 and on top of 9.2.0.8.There can be character conversion issues with LogMiner when mining the redo/undo information on a different database with a different characterset defined than on the source database. Bug 8753157 addresses this problem and a patch is available at 10.2.0.5.Additional ResourcesCommunities: Database Utilities

Concepts/DefinitionsChanges made to user data or to the database dictionary are recorded in the Oracle redo log files so that database recovery operations can be performed. LogMiner provides an...

Master Notes

Master Note for Data Pump (Doc ID 1264715.1)

Concepts/DefinitionsOracle Data Pump technology enables very high-speed movement of data and metadata from one database to another. Oracle Data Pump is made up of three distinct parts:• The command-line clients, expdp and impdp• The DBMS_DATAPUMP PL/SQL package (also known as the Data Pump API)• The DBMS_METADATA PL/SQL package (also known as the Metadata API)The Data Pump is a server based utility vs. the traditional Export / Import which are client based utilities. The Oracle Data Pump is not compatible with the Export / Import functionality.The Oracle Data Pump can also use NETWORK_LINK functionality to move data and metadata from a remote database without a dumpfile using the network technology between the two databases.DiagnosingThis Master Note is not intended to be a complete diagnostic guide with Data Pump. However, a few key articles are included below:Setup / Configuration / Syntax Users need certain privileges to run Data Pump. See Note 351598.1 for details on minimum requirements for this utility. All messages regarding work in progress, completed, and errors encountered can be written to a log file using the Data Pump LOGFILE parameter. This log file should be your first place to check for problems with the Data Pump. You can also include the undocumented parameter METRICS=y to include additional logging information about number of objects and the time it took to process them in the log file.Error messages that are report in the log file do not automatically indicate failures in Data Pump job. Some are reported as warning messages and informational. For example, messages about objects already existing and being skipped. In that case, you may need to adjust the IMPDP command parameters to recreate those objects or append data to those existing objects.Other resources:Note 266875.1 Export/Import DataPump Parameter DIRECTORY - How to Specify a DirectoryNote 430221.1 How To Reload Datapump Utility EXPDP/IMPDPSearch the Oracle Knowledge Base using keywords "how to datapump" for a variety of How To articles to help with parameter syntax for commonly required Data Pump jobs.In some cases, the performance of the EXPDP or IMPDP utilities may be slower than the traditional EXP / IMP utilities. Refer to Note 286496.1 for pointers on tracing long running Data Pump jobs to investigate where bottlenecks or performance issues are occurring. The Data Pump can parallelize certain operations. However, this does not always mean the job will be faster than the traditional EXP / IMP tools. Refer to Note 365459.1 for more details on performance improvements and restrictions on parallelism with Data Pump.Characterset and Compatibility IssuesUnlike previous tools, the Data Pump uses the characterset of the source database to ensure proper conversion to the target database. There can still be issues with character loss in the conversion process.Note 227332.1 NLS considerations in Import/Export - Frequently Asked QuestionsNote 457526.1 Possible data corruption with Impdp if importing NLS_CHARACTERSET is different from exporting NLS_CHARACTERSETNote 436240.1 ORA-2375 ORA-12899 ORA-2372 Errors While Datapump Import Done From Single Byte Characterset to Multi Byte Characterset DatabaseNote 553337.1 Export/Import DataPump Parameter VERSION - Compatibility of Data Pump Between Different Oracle Versions [Video]Note 864582.1 Examples using Data pump VERSION parameter and its relationship to database compatible parameterPerformance IssuesThe Data Pump may be performing slower than expected. When investigating performance issues like this, it is important to eliminate factors outside the Data Pump utility first. Is overall performance on the database slow? If so this could be the root issue for the slow Data Pump job. Is overall performance impacted currently on the server? Again the slow Data Pump job could be "victim" of issues with CPU or memory on the server.Determine if there are bottlenecks with CPU utilization and/or memory usage on the Operating System. In some cases, stress for these resources can lead to paging issue and that can be the root issue of the performance issues with Data Pump.Investigating this area is different from platform to platform. Check with your OS Administrator for assistance on analysis at the server level.At the database level, investigate performance information using tools like AWR or ADDM. Refer to Note 748642.1 for pointers on generating and use the AWR tool. Note 842884.1 provides high level tips and hints about interpreting data in the AWR report. Focus on "top waits", and I/O information.  Note 22908.1 discusses latch contention. If the performance issues are narrowed down to the Data Pump utility, refer to articles below:Note 552424.1 Export/Import DataPump Parameter ACCESS_METHOD - How to Enforce a Method of Loading and Unloading Data ?Note 457526.1 Possible data corruption with Impdp if importing NLS_CHARACTERSET is different from exporting NLS_CHARACTERSETNote 227332.1 NLS considerations in Import/Export - Frequently Asked QuestionsNote 436240.1 ORA-2375 ORA-12899 ORA-2372 Errors While Datapump Import Done From Single Byte Characterset to Multi Byte Characterset DatabaseNote 553337.1 Export/Import DataPump Parameter VERSION - Compatibility of Data Pump Between Different Oracle Versions [Video]Note 864582.1 Examples using Data pump VERSION parameter and its relationship to database compatible parameterCommon IssuesBug 7362589 indicates with 11g performance can be very slow when exporting a small proportion of objects relative to the number in the database. Bug 8845859 indicates issues with 11g performance on partition exports. See Note 1050907.1Note 812864.1 discusses problems prior to 10.2.0.4 with REMAP_SCHEMAData Pump is slower when using TABLE_EXISTS_ACTION=TRUNCATE on 10g with Index Organized Table (IOT) objects exist. This is expected behavior. See Note 780784.1 for more details.Prior to 10.2.0.4, Data Pump export loses global indexes for local domain indexes. Bug 5152232 is documented in Note 781759.1.There is some confusion about parallel operations with indexes and Data Pump. Note 402511.1 documents how this works more completely.Exporting Flashback archive tables will throw ORA-1426 errors. See Note 742739.1.Additional ResourcesCommunity: Database Utilities

Concepts/Definitions Oracle Data Pump technology enables very high-speed movement of data and metadata from one database to another. Oracle Data Pump is made up of three distinct parts:• The...

Master Notes

Master Note for Export and Import (Doc ID 1264691.1)

Concepts/DefinitionsOracle Export and Import utilities provide a simple way for you to transfer data objectsbetween Oracle databases, even if they reside on platforms with different hardware and software configurations. When you run Export against an Oracle database, objects (such as tables) are extracted, followed by their related objects (such as indexes, comments, and grants), if any. The extracted data is written to an export dump file. The Import utility reads the object definitions and table data from the dump file. The new Data Pump utility is not able to read the dumpfile from the Export utility.DiagnosingThis Master Note is not intended to be a complete diagnostic guide for the Export or Import utilities. However, a few key articles are listed below.The Export and Import utilities have not changed significantly since 9.2.x. All messages regarding work in progress, completed, and errors encountered can be written to a log file using the LOG parameter. This log file should be the first place to check for problems with the Export or Import. The Export and Import FAQ Note 175624.1 documents usage and common issues and defects.Export and Import are client tools which work under the characterset defined by NLS_LANG. If the characterset defined by NLS_LANG is not a superset of the characterset used by the export tool, it will be necessary for character conversion during the import. There can be issues with character loss in this conversion process.Note 48644.1 Identifying the Export Character SetNote 15095.1 Export/Import and NLS ConsiderationsNote 227332.1 NLS considerations in Import/Export - Frequently Asked QuestionsNote 227337.1 Changing the Database Character Set - Frequently Asked QuestionsIf export or import appears to be hanging or running slower than expected, refer to Note 258418.1 for pointers on tracing the process doing the export or import using the 10046 event. Also check hints and tips in Note 155902.1.If experiencing errors when running export or import, search for known issue matching your scenario using the error number(s) starting with the last error and working backwards through the list.Common IssuesNote 62427.1 2Gb or Not 2Gb - File limits in OracleNote 30428.1 Exporting to Tape on Unix SystemNote 230627.1 9i Export/Import Process for Oracle Applications Release 11iNote 305211.1 Very Slow Performance While Import for Particular TableNote 739241.1 Import Of A Large Table With A TIMESTAMP Column Is SlowNote 727411.1 Database Export slow while exporting triggers - XDB InstalledNote 729248.1 Conventional & Direct Path Export/Import Are Extremely Slow Note 783835.1 Export Slow After Applying 10.2.0.4 Patchset on top of 10.2.0.3Additional Resources Community: Database Utilities

Concepts/Definitions Oracle Export and Import utilities provide a simple way for you to transfer data objectsbetween Oracle databases, even if they reside on platforms with different hardware and...

Master Notes

Oracle Support Master Note for Issues Related to the Performance of SQL Queries.

The Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes for issues related to the performance of SQL queries. It can be found in My Oracle Support Note:199083.1Master Note: Query Tuning Overview (Doc ID 199083.1) SQL Query Handling in Oracle Every statement submitted to the Oracle engine passes through a number of phases that determine, for example, the statements validity or if it has run before etc. In terms of SQL Tuning the most important area is Query optimization which is nominally part of the PARSE phase.  Frequently Asked Questions Note:398838.1 * FAQ: Query Tuning Frequently Asked Questions Frequently Asked Query Tuning questions. Contains links to articles on numerous topics. Troubleshooting Query Tuning IssuesThe following sections can assist with troubleshooting any SQL Tuning issues that may be encountered providing guidance of actions to take and suggested workarounds. Support Action plans Support have generated a number of articles containing recommended actions to assist with the troubleshooting of various problems: Note:742112.1 * Query Performance Degradation - Recommended Actions Recommended actions for troubleshooting issues where the performance of an individual query has worsened Note:745216.1 * Query Performance Degradation - Upgrade Related - Recommended Actions Recommended actions for troubleshooting issues where the performance of an individual query has worsened following a software upgrade Note:150895.1 * Wrong Results Issues - Recommended Actions Recommended actions for troubleshooting issues where the rows returned from a query are unexpected Guidance on Specific Issues: Note:67522.1 * Diagnosing Why a Query is Not Using an Index Explains why queries may not be able to fully utilise indexes Note:69992.1 Why is my hint ignored? Explains why supplied hints may not appear to be used Note:604256.1 * Why is a Particular Query Slower on One Machine than Another? Suggests reasons why performance on one machine may not be as good as on another Note:223806.1 * Query with Unchanged Execution Plan is Slower than Previously Suggests reasons why performance of a query may differ even though the access methods it uses are the same Note:160089.1 Why are my queries slow after upgrading my database? Suggests reasons why performance may degrade after making system changes Recommended diagnostics Note:68735.1 * Diagnostics for Query Tuning Problems Suggested Information to collect for resolving Tuning Issues Note:235530.1 * Recommended Method for Obtaining a Formatted Explain Plan How to collect access path information for a query Note:376442.1 * Recommended Method for Obtaining 10046 trace for Tuning How to collect 10046 trace Note:199081.1 Overview Reference for SQL_TRACE, TKProf and Explain Plan Details of Trace, TKProf and Explain Note:46234.1 Interpreting Explain plan This article details, with explain plan examples, all possible access methods, join methods, view usage, sorting, filtering, parallel execution and handling of bind variables and remote queries. It provides a short (but expert level) introduction to Oracle's SQL processing and gives deep insight in to each step that can be seen in explain plan output.  Stored Outlines Stored outlines are a method of fixing the access path for a query without modifying the query itself. Details can be found in the following: Note:67536.1 Stored Outline Quick Reference Reference article for Stored Outlines Note:445126.1 HOW TO: Create a Stored Outline Based Upon an Existing Cursor Outline how to create an outline for a query that has been executed Note:728647.1 How to Transfer Stored Outlines from One Database to another (9i and above) How to move an outline from a 'good' instance to a target instance Note:730062.1 How to Edit a Stored Outline to Use the Plan from Another Stored Outline How to edit an outline to use a target outline Query Tuning Advice For each SQL statement, there is a multitude of different approaches that could be used to retrieve the required data. Optimization is the process of choosing the most efficient way to retrieve this data based upon the evaluation of a number of different criteria. Oracle currently (as of Oracle11g Release 2) utilises a Cost based approach using a Cost Based Optimizer (CBO). The CBO bases optimization choices on pre-gathered table and index statistics as well as the capability to dynamically sample statistical data to accurately reflect real data volumes. For more information on the Oracle optimizer see: Note:10626.1 Cost Based Optimizer (CBO) Overview This article is a comprehensive description about the CBO. Most suitable for beginners in the tuning area. It was written for version 7 so does not consider the new methods.  Note:754931.1 Cost Based Optimizer - Common Misconceptions and Issues - 10g and Above This reference article covers many areas of query optimization and predominately focuses on solving common issues. For further information on the optimizer see: Oracle Database Performance Tuning Guide11g Release 2 (11.2)Part Number E10821-05Chapter 11 The Query OptimizerSection 11.4 Understanding the Query Optimizer Statistics Optimizer decisions are based on statistics collected against the objects in the database. For advice and recommendations on the collection of statistics, see the following: Note:1226841.1 How To: Gather Statistics for the Cost Based Optimizer Recommendations for object statistics gathering Note:470316.1 Using Actual System Statistics (Collected CPU and IO information Recommendations for gathering system statistics Note:44961.1 Gathering statistics frequency and strategy guidelines General advice and overall strategy for gathering statistics Query Tuning The vast majority of queries submitted to the Oracle optimizer(s) perform adequately and consistently return correct results. Sometimes, however, the optimizer is presented with information that, when it is compared to the real information, is incorrect or insufficient for determination of the optimal access method for the query. In these rare cases, manual intervention may be necessary to obtain the desired performance. Support has created a large number of articles to help guide analysts tune queries. A number of these are listed below together with brief descriptions of their contents. Additional detail can be found in the following: Note:179668.1 * TROUBLESHOOTING: Tuning Slow Running Queries Suggests methods that can be used to modify the performance of slow queries Note:33089.1 * TROUBLESHOOTING GUIDE: SQL Tuning Presents a number of common Solutions to SQL Tuning Issues Note:372431.1 * TROUBLESHOOTING: Tuning a New Query Suggests methods that can be used to tune a new query Note:122812.1 * TROUBLESHOOTING: Tuning Queries That Cannot be Modified Suggests methods that can be used to modify the performance of queries that cannot be changed Note:163563.1 * TROUBLESHOOTING: Advanced Query Tuning Provides step by step guidelines for dealing with Query Tuning problems Hints Hints are a mechanism that can be used to force the optimizer to choose a specific access path. For details see: Note:29236.1 QREF: SQL Statement HINTS Reference article explaining hint syntax. Hints provide a mechanism to direct the optimizer to choose a certain query execution plan.  Note:50607.1 How to specify an INDEX Hint Explains how to specify working index hints in queries with worked SQL and explain plan examples. For more background information see: Oracle Database Performance Tuning Guide11g Release 2 (11.2)Part Number E10821-05Chapter 19 Using Optimizer Hints Documentation Note:1195363.1 Database Performance and SQL Tuning Documentation on OTN Links to the main Tuning and Performance documentation Tuning Support See the following article for details of what Query Tuning Support offers: Note:166650.1 Working Effectively With Global Customer Support Refer to the "Performance Tuning" section for details

The Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes for issues related to the performance of SQL queries.It can be found in My Oracle...

Master Notes

Oracle Support Master Note for Target Maintenance Through 10g Enterprise Manager Grid Control

For most current information refer Master Note for Target Maintenance Through 10g Enterprise Manager Grid Control (Doc ID 1202453.1) Master Note for Target Maintenance Through 10g Enterprise Manager Grid Control (Doc ID 1202453.1) In this Document  Purpose  Scope and Application  Master Note for Target Maintenance Through 10g Enterprise Manager Grid Control     Concepts Related to Grid Control Targets     Diagnostic Tools Available for Troubleshooting Target Maintenance Issues     Troubleshooting Target Maintenance Issues     Best Practices (Certification, Maintenance Activities, OCM, Healthcheck, CPU & PSU)  ReferencesApplies to: Enterprise Manager Grid Control - Version: 10.1.0.2 to 10.2.0.5 - Release: 10.1 to 10.2Information in this document applies to any platform.PurposeThis Master Note helps understand 10g Grid Control Target Maintenance and provides assistance in using diagnostics effectively to debug/troubleshoot and resolve issues encountered.Scope and ApplicationThis document is intended to assist Enterprise Manager Grid Control Administrators effectively troubleshoot Target Maintenance Issues in the 10g Grid Control. This document covers the following topics:1. Concepts related to Grid Control Targets2. Diagnostic Tools Available for Troubleshooting Target Maintenance Issues3. Troubleshooting Target Maintenance Issues4. Best Practices (Maintenance Activities, OCM, Healthcheck, CPU & PSU and Certification)Master Note for Target Maintenance Through 10g Enterprise Manager Grid ControlConcepts Related to Grid Control TargetsA Grid Control Target can be broadly defined as a single entity / component that can be monitored, managed and configured via the Enterprise Manager Grid Control. For this purpose an Management Agent must be installed on the host where the target resides. Examples of a target include:- A Linux host computer- An Oracle9i Application Server or an instance of Oracle HTTP Server - A 10.2.0.5 Database - A Web Application, etc For a list of certified targets, refer to Note 412431.1: Oracle Enterprise Manager 10g Grid Control Certification Checker1. Key Characteristics of a TargetTarget Type: Represents a class of managed entities / Targets, which can be monitored by the Grid Control. Each of the supported target types have an internal name assigned to uniquely identify them. Some examples: host, oracle_database, oracle_ias, oracle_listener, etc.Target Name: Name of the target as displayed in the Grid Console. In most cases, this is same as the name of the target identified by the Agent during the discovery but it is possible to provide a different target name during manual discovery of a target as described in Note 755630.1: Grid Control Target Maintenance: How to Change the Display Name of Targets in Grid Console?TARGET_GUID: Based on the Target Name, the Repository assigns a unique identifier to the Target. This TARGET_GUID is used for identifying the target when performing any target-related operation from the Grid Console. For this purpose, no two targets in the Grid Control can have the same target name. The second target being added with the same name will be marked as a duplicate target.Monitoring Configuration and Target Properties: Properties using which the Target can be identified and accessed from the Grid Console.Monitoring Configuration includes details such as Target Name, Hostname, Connection properties using which the Agent can connect to the Target, ORACLE_HOME, etc. These details are saved by the Agent in the targets.xml file. For targets such as Databases, Application Server, Listener etc these details can also be viewed in the Grid Console by clicking on the 'Monitoring Configuration' link in the Target Homepage.A Host target is configured automatically, hence no properties can be modified.User-defined Target Properties are optional descriptive attributes that can be associated with the target, which can be used for searching, filtering, notifications or classification purposes. For more details, refer to Note 750070.1: Grid Control Target Maintenance: How to Add and Populate Target Properties in the Grid Console Targets PagesMetadata and Default Collections: Metadata is "Data that describes data". With respect to a Target, Metadata defines the target type and the methods used to monitor it. This includes:Target-Type Metadata: is read-only definition of a Target Type. Each target type has its own metadata, which is defined in <target_type>.xml file under the <AGENT_HOME>/sysman/admin/metadata directory in each Agent installation. Each file defines properties of the target type, metrics for this target type and the method used to collect the metrics.Target Specific Metadata (dynamic properties): The properties of the target itself, for example: DBName, VersionCategory, OpenMode etc for a Database Target. The method to collect these properties is also defined in the <target_type>.xml files under the <AGENT_HOME>/sysman/admin/metadata directory. The Target Manager component of the Agent computes these properties at the time of the Agent startup.Note 1101615.1: Description of Important Components / Threads in a 10g Enterprise Manager Grid Control Agent, Topic: Target ManagerDefault Collections: Associated with the metadata of a target type and defines the collection schedule for the pre-defined list of metrics. These are defined in a <target_type>.xml file in the <AGENT_HOME>/sysman/admin/default_collection directory in each Agent installation.Availability Rule: Defines how to calculate whether the target is Up / Down. The logic is mostly straight forward for a single target but will involve some calculations based on the status of the child-targets in case of a composite / aggregate target. This is not applicable to some composite targets such as Groups, System. The status of a Target could be one of the following: Up, Down, Under Blackout, Unknown (due to Metric Collection Error), Unknown due to Agent Unreachable (Agent Down or Host unreachable), Status Pending.2. File: targets.xmlThe list of targets discovered by an Agent is stored in the targets.xml file. This is located at <AGENT_HOME>/sysman/emd in case of a standalone Agent installation and in<AGENT_HOME>/<nodename>/sysman/emd in case of a RAC Agent installation.For details, refer to Note 234872.1: Understanding the Enterprise Manager 10g Grid Control Management Agent, Directory Structure and Key Configuration Files.For a machine to be seen in the Grid Console, this file should at the minimum have entries for the Host and Agent (oracle_emd) targets. If the file has been lost and a valid backup does not exist, the Host and Agent entries can be added using the steps inNote 303105.1: Grid Control Target Maintenance: How To Recreate the Targets.xml File in Grid Control 10.1.xNote 365252.1: Grid Control Target Maintenance: How To Recreate the Targets.xml File in Grid Control 10.2.x Using 'agentca' CommandsNote:- A valid backup of the targets.xml file should be maintained always.- The targets.xml file should not be manually modified for any reason unless advised by EM Support / Development. Incorrect edition of this file can corrupt the file and cause the Agent to crash.3. Target DiscoveryA Target can be monitored from the Grid Console only if it has been discovered in the Grid Console. Hence, the Target discovery is necessary for important management operations such as fault detection, configuration and inventory tracking, real-time monitoring, and historical reporting. The discovery mechanisms are described in Note 239224.1: Grid Control Target Maintenance: Overview of Target Discovery in 10g Grid ControlIf there are multiple Oracle Central Inventories in the machine, the Agent should be made aware of the required inventories for discovering the Oracle Homes. Steps are described inNote 292084.1: How to Enable and Troubleshoot Multiple-Inventory Support for the Agent/ECM4. Target Deletion / RemovalTo understand the steps for deleting a Target from the Grid Console, refer to Note 271691.1: Grid Control Target Maintenance: Understanding the Target Deletion / Removal Process in the Enterprise Manager Grid Control 5. Other  Target ConceptsFor more details related to a Grid Control Target, refer to Note 1214933.1: Grid Control Target Maintenance: Understanding Concepts Related to Enterprise Manager Grid Control TargetsBack to Top********************************************************************************Diagnostic Tools Available for Troubleshooting Target Maintenance IssuesEMDiagkitThe EMDiagkit is a diagnostic tool developed to assist in diagnosis and correction of Enterprise Manager 10g Framework issues. At present, the tool allows us to extract necessary troubleshooting data from the EM Repository Schema using the repvfy utility.The details for installation, usage of EMDiagkit are available in           Note 421053.1: EMDiagkit Download and Master IndexEMDiagkit (repvfy) also has options for obtaining the target details and its availability from the Repository. These options are described in:Note 399899.1: Grid Control Target Maintenance: Troubleshooting Script for Target Availability in Enterprise Manager Grid ControlNote 1228803.1: Grid Control Target Maintenance: Script for Target Analysis (dump) in Enterprise Manager Grid ControlRDAThe Remote Diagnostic Agent (RDA) can be executed specifically with the Grid Control / OMS profile name: GridControl, Agent profile name: AGT and the Database profile name: DB10g / DB11g in order to reduce the number of questions that need to be answered and also to collect all details of the OMS / Agent / Database Homes correctly.The steps to execute the RDA with GridControl and profiles are explained in:           Note 1057051.1: How to Run the RDA against a Grid Control InstallationIt is highly recommended that the latest EMDiagkit is installed and executed in the OMS home, before running the RDA. Other tools and options are described in Note 1098262.1: Master Note for Diagnostic Tools for 10g Enterprise Manager Grid Control ComponentsBack to Top********************************************************************************Troubleshooting Target Maintenance IssuesProblems During Target DiscoveryDiscovery problems can be classified as:Permissions / Formatting issues of the targets.xml file: The <AGENT_HOME>/sysman/log/emagent.trc shows errors such as ...oracle.sysman.emSDK.emd.comm.OperationException: Could not save target to targets.xml - Could not save target to targets.xmlFor resolving this error, refer to Note 745448.1:Grid Control Target Maintenance: Trying to Perform Target Discovery Fails with "Could not save target to targets.xml"The targets.xml file should not be modified manually unless advised by Support / Development. Incorrect modification of this file can corrupt the file. For example:Note 945002.1: Grid Control Target Maintenance: Trying to Manually Discover a Database Results in "LPX-00245: extra data after end of document"Incorrect Format for the Hostname: The <hostname> of the machine is referenced for the entries of many other targets in the targets.xml file. If the hostname in various target entries do not match with each other, then trying to access the target details from the Grid Console can return errors such as:Error finding target <target name> from the repository. The target does not exist or you may not have access to the target.For details, refer to Note 752693.1: Grid Control Target Maintenance: Acessing the Target Reports "Error finding target from the repository. The target does not exist or you may not have the access to the target"Multiple Agents on a Machine: If multiple Agents are installed on a machine, then the host target will be discovered by all the Agents and will be treated as a duplicate one in the Grid Console. For details, refer to Note 363062.1: Grid Control Target Maintenance: Host Target Does Not Appear in Grid Control After Agent Is Removed From Grid ControlDiscovery of Targets in Oracle Homes, installed as different OS users (in Unix/Linux): If the Agent is installed as a OS User different from the OS user who has installed the other Oracle Home's in a machine, then the Agent should have sufficient permissions to read the required files in the other Oracle homes for the target discovery.Some of the issues face for certain target discoveries are described in the below documents:Note 437078.1: Problem iAS Discovery/Monitoring Fails If Agent Is Installed Under Another OS User Than iAS Note 787571.1: E-PSEM How to Setup PeopleSoft Plug-in With Installations Done by Different Users ? Note 371539.1: Grid Control Target Maintenance: Agent Does Not Discover Targets in ORACLE_HOME's Installed by Other OS UsersRe-discovery Failing due to Incomplete Deletion: If you remove a target and try to add it again with the same name before the deletion is complete, you will get error such as: java.sql.SQLException: ORA-20600: The specified target is in the process of being deleted. You must wait till the deletion is complete to re-use the same target name or can manually discover the target using the steps in Note 417690.1: Grid Control Target Maintenance: How To Manually Discover a Target in the Grid Console?Adding Targets to Groups During Manual Discovery:A target can get added to a Group automatically at the time of manual discovery. This feature however adds the target to all available groups. This issue has been described in Note 846470.1: Grid Control Target Maintenance: Manual Discovery Automatically Adds Targets to all Groups, NO Option to Choose a Particular Group.Back to TopProblems During Target DeletionFor diagnosing targets pending in deletion, refer to Note 271714.1: Grid Control Target Maintenance: How to Diagnose Target Removal issues in Enterprise Manager Grid ControlTroubleshooting Target Availability'Agent Unreachable' Status: A target will be shown in the 'Agent Unreachable' status if the Agent monitoring this target is not reachable. There are several reasons why the Management Agent will show a status of "Agent Unreachable":    1. The Agent is not running.     2. The Agent cannot resolve the OMS hostname after the initial successful heartbeat.    3. The Agent is running and has files to upload but cannot upload files to the OMS.     4. The OMS has been locked down to receive only HTTPS connections from the Agents         but this particular Agent is not configured for HTTPS communications, etc.For troubleshooting such issues, refer to Note 271126.1: Grid Control Target Maintenance: Steps to Diagnose Issues Related to "Agent Unreachable" StatusIncorrect Target Status: If a target is running but is not shown as Up in the Grid Console, this will affect monitoring of this target from the console as none of the metrics will be collected by the Agent. To debug such issues, refer to Note 730757.1: Grid Control Target Maintenance: Troubleshooting Incorrect Target Status Issues in Grid Control Host Preferred Credentials To troubleshoot errors when setting or using a feature that makes use of the Host Preferred credentials, refer to Note.757425.1: Troubleshooting Host Preferred Credentials Searching My Oracle Support Documents for Target Maintenance IssuesAs the search is specific to Enterprise Manager Grid Control issues, we recommend that the search be performed only under the Grid Control section, using the following navigation:Login to My Oracle Support then Click Knowledge -> Enterprise Management -> Enterprise Manager Consoles - Packs - and Plugins -> Enterprise Manager Grid Control ->All of Enterprise Manager Grid Control.To find documents related to Target Maintenance in Grid Control, login to My Oracle Support portal and query the 'Knowledge' with the following keywords:Grid Control Target Maintenance: <actual issue or error message seen>Some examples:Grid Control Target Maintenance: Steps to Remove an Orphaned Target Using the EMDiag KitGrid Control Target Maintenance: How to Manage Duplicate Targets in the Grid Console?Using RDA and EMDiagkit for troubleshooting Target Maintenance IssuesThe RDA output generated with the GridControl profile is very useful in obtaining all the configuration files and log/trace files together. These can assist in identifying if a problem is specific to the Console UI or other components like the OMS / Repository. The EMDiagkit (repvfy) output is very useful in diagnosing problems / mis-configurations with Grid Control Repository objects, which can result in errors when maintaining the Targets. In addition, repvfy provides options for dumping the target's configuration and availability details.Note: It is highly recommended that the latest EMDiagkit (repvfy) is installed and executed in the OMS home, before running the RDA. This will ensure that the RDA picks up the latest data collected by the EMDiagkit.Back to Top********************************************************************************Best Practices (Certification, Maintenance Activities, OCM, Healthcheck, CPU & PSU)This section lists some of the best practices which will help prevent problems with Target Maintenance.EM Certification CheckerIt is strongly recommended that you always use a certified combination of OMS, Agent and Repository Database for managing Targets which are certified with this combination.The Enterprise Manager certification details are available in:Note 412431.1: Oracle Enterprise Manager 10g Grid Control Certification CheckerMaintenance ActivitiesExecute EMDiagKit at regular intervals (once per week or more frequently, depending on your setup) and check for any new problems that are reported for the OMS / Repository operations.Always maintain a valid backup of the targets.xml file in the Agent machine. The targets.xml file should not be manually modified for any reason unless advised by EM Support / Development. Incorrect edition of this file can corrupt the file and cause the Agent to crash.The Discovery attempted by an Agent is always a "Best-effort approach" i.e the Agent tries all possible methods defined in the perl scripts to discover the targets in the machine. If a particular target is still not discovered, then it can be manually added to the Grid Console. Refer to Note 417690.1: Grid Control Target Maintenance: How To Manually Discover a Target in the Grid Console? 'OMS and Repository' is a special target monitored by the Agent on the OMS machine. When using this Agent in cloning operations to install Agents on other machines, follow the steps inNote 579156.1: How to Clone a Management Agent And Known IssuesTopic: Steps to clone Chained Management Agent (The Agent which is installed as part of the OMS installation is also known as the chained Agent).If the options mentioned in the above note are not used correctly, this will result in multiple 'OMS and Repository' targets in the Grid Console, which will affect the details shown for the Management Services and repository pages. Take valid backups of the Agent, OMS and Repository Database Homes at regular intervals, to restore back any configuration files that are deleted by accident.For a 10.2.0.5 OMS, the 'emctl exportconfig oms' command can be used to backup the necessary OMS configuration details. Refer to the details in Oracle Enterprise Manager Administration 10g Release 5 (10.2.0.5), Chapter - 9 Backup, Recovery, and Disaster Recovery. Topic : OMS Backup and Recovery.Back to TopOCM Oracle Configuration Manager (OCM) works with My Oracle Support to enable proactive support capability that helps you organize, collect and manage your Oracle configurations by providing Proactive configuration-specific notification of Security and General Alerts, HealthCheck recommendations based on Support Best practices when using configuration auto-collection, Simplified Service Request logging, tracking and reporting and Project cataloging of key milestones and contacts associated with your configurations.The lists of all available OCM collections for the Oracle Products are available here:Oracle Configuration Manager CollectionsAmong these the following topics are related to the Enterprise Manager Components: 2.52 Oracle Enterprise Manager 10g Grid Control Management Agent: 2.54 Oracle Enterprise Manager 10g Grid Control Management Service 2.53 Oracle Enterprise Manager 10g Grid Control Management Repository 2.72 Oracle Grid Control Repository (for oracle_emrep target) 2.38 Oracle Agent Deployment Configuration (oracle_emd target) 2.73 Oracle Home 2.23 HostNote: The above list is expected to be expanded as and when new collections are introduced in future.It is also advisable to review the collections available for the Database instance, so that the Database hosting the repository can be monitored as well: 2.10 Database Instance 2.78 Oracle ListenerThe installation steps and benefits from OCM collections are explained in: Oracle Configuration Manager Quick Start Guide Note 548815.1: My Oracle Support Configuration Management FAQ Note 728982.1: Oracle Configuration Manager Security Overview and Collections Overview Note 728473.1: Oracle Configuration Manager PrerequisitesNote 728989.1: Oracle Configuration Manager Installation and Administration Guide HealthcheckHealthchecks are executed dynamically against the Oracle Configuration Manager uploaded configurations in My Oracle Support. These checks, based on Oracle Best practices, will proactively notify you of potential problems in your environment, and provide recommendations that help you improve system performance and avoid problems in your Oracle environment. If you are receiving any Healthcheck alerts in My Oracle support, then refer to the following document for the alert details and its corresponding document for resolving the same: Note 868955.1: My Oracle Support Health Checks CatalogFor Healthchecks specific to the Enterprise Manager and Repository Database, refer to the sections titled: Enterprise Manager (for the OMS) Oracle Database (for the Database hosting the Repository) Back to TopCPU and PSUCPUCritical Patch Updates (CPU) is the primary means of releasing security fixes for Oracle products. They are released on the Tuesday closest to the 15th day of January, April, July and October. This page lists all the currently available Critical Patch Updates (CPUs) in chronological order and is updated whenever new Critical Patch is released. You can also subscribe to the CPU Email Alerts using the steps listed here.To obtain the latest CPU patch details for the Enterprise Manager Grid Control and its dependent products - Oracle Application Server and Oracle Database:- In the page, click on the link shown for the latest CPU in the table under the 'Critical Patch Updates'.- The next page, lists all the products which have security fixes in the chosen CPU release. Scroll down to 'Patch Availability Table ..' topic and find the table with details for the Product Group and Patch Availability and Installation Information. - In the table, find the row related to Product Group: 'Oracle Enterprise Manager' and pick up the document number given in the Patch Availability and Installation Information column. In the document, navigate to:              "Critical Patch Update Availability for Oracle Products" and then to             "Oracle Enterprise Manager Grid Control"PSUPatch Set Updates (PSU) are proactive cumulative patches containing recommended bug fixes that are released on a regular and predictable schedule. PSUs are on the same quarterly schedule as the Critical Patch Updates (CPU), specifically the Tuesday closest to the 15th of January, April, July, and October. The PSUs serve as a new baseline version for reporting issues to Oracle, hence it is always recommended to be on the latest PSU release.For more details on PSU, refer Note 854428.1: Patch Set Updates for Oracle Products For Enterprise Manager specific PSU, refer Note 822485.1: Oracle Recommended Patches -- Oracle Enterprise ManagerChoosing between CPU / PSU patches The PSU and CPU released each quarter contain the same security content. However, the patches employ different patching mechanisms, so customers need to choose wisely which patch satisfies their needs better:A PSU can be applied on the CPU released at the same time or on an any earlier CPU for the base release version. A PSU can be applied on any earlier PSU or the base release version. CPUs are only created on the base release version.  Once a PSU has been installed, the recommended way to get future security content is to apply subsequent PSUs. Reverting from PSU back to CPU, while possible, would require significant effort, and so is not advised. Getting CPU / PSU patch recommendations via OCM OCM also collects and recommends the latest CPU and PSU patch that can be applied to a particular Oracle Home. These details can be seen in the My Oracle Support ->Patches and Updates -> Patch Recommendations section - 'Security' patch recommendations include the CPU patches.- 'Other Recommendations' include the PSU patches.Back to TopReferencesNOTE:1081865.1 - Master Note for 10g Grid Control OMS Process Control (Start, Stop and Status) & ConfigurationNOTE:1082009.1 - Master Note for 10g Grid Control Agent Process Control (Start, Stop & Status) & ConfigurationNOTE:1086343.1 - Master Note for 10g Grid Control Enterprise Manager Communication and Upload issuesNOTE:1087997.1 - Master Note for 10g Enterprise Manager Grid Control Agent Performance & Core Dump issuesNOTE:1092513.1 - Master Note for 10g Enterprise Manager Grid Control Security FrameworkNOTE:1098262.1 - Master Note for Diagnostic Tools for 10g Enterprise Manager Grid Control ComponentsNOTE:1161003.1 - Master Note for 10g Grid Control OMS Performance IssuesNOTE:1190323.1 - Master Note for 10g Grid Console Browser / User-interface Issues

For most current information refer Master Note for Target Maintenance Through 10g Enterprise Manager Grid Control (Doc ID 1202453.1)   Master Note for Target Maintenance Through 10g Enterprise Manager...

Master Notes

Oracle Support Master Note for 10g Grid Console Browser / User-interface Issues

For most current information refer Master Note for 10g Grid Console Browser / User-interface Issues (Doc ID 1190323.1)  Master Note for 10g Grid Console Browser / User-interface Issues (Doc ID 1190323.1) In this Document  Purpose  Scope and Application  Master Note for 10g Grid Console Browser / User-interface Issues     Grid Console User-interface Concepts     Diagnostic Tools Available for Troubleshooting Grid Console User-interface Issues     Troubleshooting Grid Console Browser / User-Interface Issues     Best Practices (Certification, Maintenance Activities, OCM, Healthcheck, CPU & PSU)  ReferencesApplies to: Enterprise Manager Grid Control - Version: 10.1.0.2 to 10.2.0.5 - Release: 10.1 to 10.2Information in this document applies to any platform.PurposeThis Master Note helps understand Grid Console Browser / User-Interface Issues and provides assistance in using diagnostics effectively to debug/troubleshoot and resolve issues encountered.Scope and ApplicationThis document is intended to assist Enterprise Manager Grid Control Administrators effectively troubleshoot Grid Console Browser / User-interface Issues. This document covers the following topics:1. Grid Console User-interface Concepts.2. Diagnostic Tools Available for Troubleshooting Grid Console User-interface Issues.3. Troubleshooting Console Browser / User-Interface Issues.4. Best Practices (Maintenance Activities, OCM, Healthcheck, CPU & PSU and Certification).Master Note for 10g Grid Console Browser / User-interface IssuesGrid Console User-interface ConceptsThe Grid Console, also known as Grid Control Console / Enterprise Manager (EM) Console is the user / web-based interface to the Grid Control setup. From the Grid Console, you can monitor and administer your entire computing environment from one location on the network. All the services within your enterprise, including hosts, databases, listeners, application servers, and so on, are easily managed from one central location. Thus, you can access the Grid Control setup from any client machine which has a web-browser installed, provided that the communication channels to the OMS machine is open. The 10g Grid Console uses the default OracleAS Web Cache port (for example, port 7777 on UNIX systems and port 80 on Windows systems) to connect to the Oracle HTTP Server component of the OMS. The connection can be via http / https. For example:http://omsmachine.domain:7777/emhttps://omsmachine.domain:4444/emIf the communication is successful, then the above URL will show a login page where details of a valid Enterprise Manager Administrator and password can be entered.For additional details, refer to:List of Supported browsersNote 412431.1: Oracle Enterprise Manager Grid Control Certification CheckerSection: Browser Certifications Identifying the Console URLNote 392979.1: How to Identify the URL to login to the 10g Enterprise Manager Grid ConsoleNote 763844.1: How to Verify the Hostname/IP Address Resolution Between the 10g Enterprise Manager Grid Control Components? Language and Timezone in the UI pagesNote 413864.1: Grid Control UI: How to Choose / Modify the Enterprise Manager Grid Console Language?Note 338481.1: Grid Control UI: How are the Date / Time for the 'Page Refreshed' Value in the Enterprise Manager Pages Computed?Securing and Authenticating access to Grid ConsoleNote 452290.1: How to secure/restrict access to the Enterprise Manager Grid Control console Note 736103.1: How to Configure a Third Party Certificate for HTTPS Access to the Grid Console?Note 1092513.1: Master Note for 10g Enterprise Manager Grid Control Security Framework, Section: Grid Control Components Security Objectives,Category area: Grid Control Console AuthenticationGrid Console UsersDuring the OMS installation and configuration, the SYSMAN user (who also owns the repository schema) is created as a super-administrator in the Grid Console. By, default, the SYS / SYSTEM users in the repository database are also configured as Super-Administrators for the Grid Console. New administrators can be created as needed from the setup -> Administrators page, after logging into the Console as one of the above users. For more details, refer to:Note 1092513.1: Master Note for 10g Enterprise Manager Grid Control Security FrameworkSection: Grid Control Components Security ObjectivesCategory area: Preferred Credentials and Target AccessTo be able to successfully login to the Grid Console, one must have a valid Enterprise Manager Administrator user name and its password.Back to TopModifying the default Session TimeoutNote 234875.1: How to Change the Default Session Timeout for the 10g Enterprise Manager Grid Console Connections?Adding a custom Logo / Image in the ConsoleNote 395029.1: Grid Console UI: How to add a Custom Logo / Banner in the EM Grid Console?Note: The steps in the above note should be used to modify the banner in the Grid Console only. Do not modify any other images or gif's similarly, this will affect the normal manner in which the details are displayed in the Grid Console and will not be supported.Enabling Accessibility Mode for users of Assistive technologyNote 459856.1: Grid Console UI: What Is "Accessibility Mode" and How to Enable / Disable it?Management Packs Oracle offers a number of management options, or "Management Packs," that bring together a set of technologies related to a specific area of Enterprise Manager.Each pack has several premium features bundled as part of that pack. Having a pack licensed or unlicensed on a target has a very significant impact on the user experience because the corresponding links, which need the pack, are enabled or disabled accordingly.For detailed description of the Enterprise Manager Management Packs and their Licensing, refer to Oracle Enterprise Manager Licensing Information 10g Release 5 (10.2.0.5.0) For enabling and disabling the Packs from Grid Console UI, refer toNote 556922.1: Grid Console UI: How to Enable / Disable Management Pack Access from the Grid Console UI?Abode SVG Viewer / Adobe Flash Player requirementNote 562922.1: Grid Console UI: Why is Adobe SVG Viewer / Adobe Flash Player Needed for Grid Console?Additional Details:1. In any of the Console pages, clicking on the 'Help' button at the top right-hand corner of the page will launch a help window with context-specific (for the page where the button is clicked) details. You can also click on the 'Search' tab and search for details on a specific topic.2. Clicking on the 'About Oracle Enterprise Manager' link at the bottom left-hand corner of the page will display the version of the OMS.3. In 10.2.0.x Grid Control version, clicking on the '+' sign next to 'About Oracle Enterprise Manager' will display the abbreviations for the different Management Packs required for the functionality will be displayed in the UI. Refer to Note 414671.1: Grid Console UI: How to Display the Management Pack Details for a Particular Functionality in the Grid Control 10gR2 UI?Back to Top********************************************************************************Diagnostic Tools Available for Troubleshooting Grid Console User-interface IssuesExtended UI Tracing for Console Operations against Repository DatabaseThe SQL queries that are being run against the Repository Database, for a console-level activity can be traced using the Extended / SQL Console UI Trace URL:         http://<omshost>:<port>/em/console/admin/rep/extendedSQLTraceThe UI tracing is similar to enabling 10046 trace in the repository database for the session from the console. For more details, refer to Note 436592.1: Steps to Enable Extended SQL / UI Trace for Enterprise Manager Grid Console Sessions Against Repository Schema'EM SQL History' for Administrative / Real-time Operations against Target DatabaseFor many of the Database-related pages in the Grid Console, the data is obtained by making a direct SQL*Net connection from the OMS machine to the target Database. This connection bypasses the Grid Agent on the Database machine; hence it is possible to view certain pages of the database even if the Agent is not running. Such database pages can be classified as:Administrative: Pages which are related to DBA activities against the database, for example: checking the Tablespace information from the Database Administration (till 10.2.0.4 GC) or the Server (from 10.2.0.5 GC) page, adding a datafile, performing a DDL operation against a Table, enabling backup, modifying init.ora parameters etc. Real-time Monitoring: Pages where the real-time information for the Database is shown, for example: the Database Performance page.For more details on tracing the queries executed by such pages, refer to Note 357318.1: How to Identify the SQL Used by OMS For Administrative / Real-time Monitoring Pages of a Database Target?Monitoring Console page Performance via BeaconsRefer to the details in Oracle Enterprise Manager Administration 10g Release 5 (10.2.0.5),Chapter 11 - Sizing Your Enterprise Manager Deployment,Topic: The Oracle Enterprise Manager Performance PageEMDiagkitThe EMDiagkit is a diagnostic tool developed to assist in diagnosis and correction of Enterprise Manager 10g Framework issues. At present, the tool allows us to extract necessary troubleshooting data from the EM Repository Schema using the repvfy utility.The details for installation, usage of EMDiagkit are available in Note 421053.1: EMDiagkit Download and Master IndexRDAThe Remote Diagnostic Agent (RDA) can be executed specifically with the Grid Control / OMS profile name: GridControl and the Database profile name: DB10g / DB11g in order to reduce the number of questions that need to be answered and also to collect all details of the OMS / Database Homes correctly.The steps to execute the RDA with GridControl and profiles are explained in:           Note 1057051.1: How to Run the RDA against a Grid Control InstallationIt is highly recommended that the latest EMDiagkit is installed and executed in the OMS home, before running the RDA. This will ensure that the RDA picks up the latest data collected by the EMDiagkit. Any operation performed in the Console will also be logged in the emoms.log / emoms.trc of the OMS. Enabling DEBUG level trace and reproducing the issue can provide more details about the Console thread, performing the operations.Back to Top********************************************************************************Troubleshooting Grid Console Browser / User-Interface IssuesAccessibility and performance of the Grid Console UI pages, to a very large extent, depend on the performance of the OMS and Repository components. Hence, before troubleshooting any problems related to the Grid Console UI, the following should be verified:1. Is the OMS up and running?       cd <OMS_HOME>/bin       emctl status oms2. Is it possible to resolve the hostname / IP address of the OMS machine, from the machine where the Grid Console is being accessed? Refer to Note 763844.1: How to Verify the Hostname/IP Address Resolution Between the 10g Enterprise Manager Grid Control Components?3. If the OMS is behind a firewall, is the OMS HTTP Server port open in the firewall for communication to succeed? Refer to Note 1088393.1: How to Verify the Communication Between the 10g Enterprise Manager Grid Control Components via Firewall/Proxy?4. If the security settings in the browser is restrictive, then trying to access the Grid console can result in a blank page. Verify the security settings, if necessary with the help of your System Administrator, and modify them so that there is unrestricted communication to the OMS. Example: Note 365102.1: Accessing the 10g Enterprise Manager Grid Console URL Returns a Blank Page Without Login Screen / Errors 5. To verify if the problem is specific to a particular machine / browser, try to access the Grid Console URL from another machine / browser.6. If you get an "Internal Error has occurred. Check the log file for details" on any Grid Console page, it actually indicates a problem with the OMS or the code used in that console page for a particular operation. Checking the <OMS_HOME>/sysman/log/emoms.trc will provide more errors regarding the Console.For understanding the entries in the OMS trace files, refer to the details in Note 1098262.1: Master Note for Diagnostic Tools for 10g Enterprise Manager Grid Control Components, Section: Diagnostic Tools for the 10g Oracle Management Service (OMS),Topic : OMS Log/Trace Files7. Depending on the privileges / roles assigned to the Administrator who is logging into the Grid Console, certain Console pages / targets / features may not be accessible. For details on the privileges / roles, refer toOracle Enterprise Manager Administration 10g Release 5 (10.2.0.5),Chapter 3 Enterprise Manager Security,Topic : Privileges and Roles Some more problems and troubleshooting steps are listed below:Communication Failure between the Grid Console and OMSIncorrect communication setup between the Grid Console (User Interface) and the OMS can result in the following problems:Trying to access the Grid Console URL from a certified web-browser results in the error: "page cannot be displayed"OR502 Bad GatewayThe following error occurred:[code=DNS_HOST_NOT_FOUND] The host name was not found during DNS lookup. Contact your system administrator if the problem is not found by retrying the URL. Trying to access the Grid console using unsecured URL (using http) fails with:ForbiddenYou don't have permission to access /em/console/home on this server.-----------------------------------------------Oracle-Application-Server-10g/10.1.2.2.0 Oracle-HTTP-Server Server at omsmachine.domain: 7778But access to the secured URL (using https) works fine.For troubleshooting steps, refer to Note 1089443.1: How to Troubleshoot Communication Between the Grid Console (UI) and Oracle Management Service (OMS) Components in 10g Enterprise Manager Grid Control To find documents related to communication problems between the Grid Console (UI) and OMS, login to My Oracle Support portal and query the 'Knowledge' with the following keywords:         Accessing the 10g Grid Console URL Results <actual error message seen>Some examples:        Accessing the 10g Grid Console URL Results in 502 Bad Gateway       Accessing the 10g Grid Console URL Results in 'page cannot be displayed' Slow Console UI PagesIf a particular page in the Grid Console is slow, to gather more details refer to the steps in Note 1098262.1:Master Note for Diagnostic Tools for 10g Enterprise Manager Grid Control ComponentsSection: Diagnostic Tools for the Enterprise Manager Console OperationsTo improve the Login performance of the Console Home page, refer to the steps under Oracle Enterprise Manager Administration 10g Release 5 (10.2.0.5),Chapter 12 - Maintaining and Troubleshooting the Management Repository,Improving the Login Performance of the Console Home PageSome examples:Note 836290.1: Grid Control Performance: OMS Crashes When Running Customized Report with 'CHART FROM SQL' Element in ConsoleNote 781347.1: Grid Control Performance: OMS Crashes With 'OutOfMemory' Errors When Editing / Configuring a GroupNote 436690.1: Grid Console Performance: Pages in Console are Slow and Sometimes Return TimeoutLogin FailuresLogin to the Grid Console may fail errors such as:Login operation failed.ORYour username and/or password are invalid.ORThis user is not a valid Enterprise Manager AdministratorORAuthentication failed. Verify username/password that you have provided. If you believe you entered correct credentials, your account may have been locked, contact system administrator to unlock your account. (From 10.2.0.5 OMS version only)The Login failures can occur it theUsername entered is not a valid Enterprise Manager Administrator Password for the Administrator is entered wrongly.Administrator account has been locked at the Repository Database level. Administrator account has been disabled from logging into the Grid Console, etc.To find documents related to communication problems between the Grid Console (UI) and OMS, login to My Oracle Support portal and query the 'Knowledge' with the following keywords:          Logging into the 10g Grid Console URL Results <actual error message seen>Some known issues:Note 784532.1: Logging into the 10g Grid Console URL Results in "Login Operation Failed"Note 455165.1: Logging into the 10g Grid Console URL Results in "This user is not a valid Enterprise Manager Administrator"Certificate Errors when Accessing HTTPS Grid Console URLWhen the OMS is secured, the SSL certificates are self-generated (signed by Oracle Certificate Authority), by default, and not signed by any third party Certificate Authority. If this certificate is not imported in the Trusted Root Certification Authorities store of the browser, then 'Certificate Errors' will be seen when accessing the HTTPS Grid Console URL from a browser. Steps for importing the certificate in an Internet Explorer browser are described in Note 437660.1: Enterprise Manager Console UI: Accessing the Grid Console / DBConsole / Metric Browser shows "Certificate Errors".Note: Third-party Certificates can be used from 10.2.0.5 Grid Control version onwards. Refer to Note 736103.1: How to Configure a Third Party Certificate for HTTPS Access to the Grid Console?Language or Timezone issuesThe Language, timestamp / timezone details and format shown in the Grid Console pages depend on multiple formats:Language settings chosen in the browser in which the Grid Console is being accessed. Language that has been chosen when installing the Grid Control software.Font files that have been installed in the OMS home, for the language chosen at the time of installation.The Timezone value using which the OMS has been started up. Timezone value using which the Agents on the Target machines have been started up. Incorrect calculation between the OMS machine timestamp / timezone and the target machine timestamp / timezone, for computing the timestamp in the Target related pages (these would be code defects in the product).Some known issues:Note 340952.1 Enterprise Manager Console UI: Cannot view Chinese / Japanese (Asian) Characters on the Buttons of Enterprise Manager Console Note 968704.1: Grid Console UI: Grid Console UI: 'Page Refreshed' / 'Latest Data Collected' Timestamp in the Console Pages Does not Include AM / PMTo find documents related to timezone issues in the Grid Console, login to My Oracle Support portal and query the 'Knowledge' with the following keywords:    Grid Console UI: 'Page Refreshed'     OR Grid Console UI:  'Latest Data Collected From Target'Some examples:Note 959775.1: Grid Console UI: 'Latest Data Collected From Target' on Host Performance Page shows Time of Target Machine but Timezone of RepositoryNote 829751.1: Grid Console UI: Timezone in the 'Page Refreshed' Value is Shown as GMT Rather than Local Timezone of the OMSSearching My Oracle Support Documents for Grid Console User-Interface IssuesAs the search is specific to Enterprise Manager Grid Control issues, we recommend that the search be performed only under the Grid Control section, using the following navigation:Login to My Oracle Support then Click Knowledge -> Enterprise Management -> Enterprise Manager Consoles - Packs - and Plugins -> Enterprise Manager Grid Control ->All of Enterprise Manager Grid Control.Using RDA and EMDiagkit for troubleshooting User-Interface IssuesThe RDA output generated with the GridControl profile is very useful in obtaining all the configuration files and log/trace files together. These can assist in identifying if a problem is specific to the Console UI or other components like the OMS / Repository. The EMDiagkit output is very useful in diagnosing problems / mis-configurations with Grid Control Repository objects, which can result in errors when accessing the Grid Console URL.Note: It is highly recommended that the latest EMDiagkit is installed and executed in the OMS home, before running the RDA. This will ensure that the RDA picks up the latest data collected by the EMDiagkit.Back to Top********************************************************************************Best Practices (Certification, Maintenance Activities, OCM, Healthcheck, CPU & PSU)This section lists some of the best practices which will help prevent problems with Grid Console Browser / User-interface Issues.EM Certification CheckerIt is strongly recommended that you always use a certified combination of OMS, Agent and Repository Database for managing Targets which are certified with this combination.The Enterprise Manager certification details are available in:Note 412431.1: Oracle Enterprise Manager 10g Grid Control Certification CheckerMaintenance ActivitiesEnable Log Rotation for the access_log and error_log files created by the httpd_em.conf file:Note 436690.1: Grid Console Performance: Pages in Console are Slow and Sometimes Return TimeoutDo not use the browser 'Back' button to navigate to the earlier seen pages in the Grid Console. Most pages do not support this action and may result in errors or inconsistent data.Execute EMDiagKit at regular intervals (once per week or more frequently, depending on your setup) and check for any new problems that are reported for the OMS / Repository operations.Take valid backups of the Agent, OMS and Repository Database Homes at regular intervals, to restore back any configuration files that are deleted by accident.For a 10.2.0.5 OMS, the 'emctl exportconfig oms' command can be used to backup the necessary OMS configuration details. Refer to the details in Oracle Enterprise Manager Administration 10g Release 5 (10.2.0.5), Chapter - 9 Backup, Recovery, and Disaster Recovery. Topic : OMS Backup and Recovery. Back to TopOCM Oracle Configuration Manager (OCM) works with My Oracle Support to enable proactive support capability that helps you organize, collect and manage your Oracle configurations by providing Proactive configuration-specific notification of Security and General Alerts, HealthCheck recommendations based on Support Best practices when using configuration auto-collection, Simplified Service Request logging, tracking and reporting and Project cataloging of key milestones and contacts associated with your configurations.The lists of all available OCM collections for the Oracle Products are available here:Oracle Configuration Manager CollectionsAmong these the following topics are related to the Enterprise Manager Components: 2.52 Oracle Enterprise Manager 10g Grid Control Management Agent: 2.54 Oracle Enterprise Manager 10g Grid Control Management Service 2.53 Oracle Enterprise Manager 10g Grid Control Management Repository 2.72 Oracle Grid Control Repository (for oracle_emrep target) 2.38 Oracle Agent Deployment Configuration (oracle_emd target) 2.73 Oracle Home 2.23 HostNote: The above list is expected to be expanded as and when new collections are introduced in future.It is also advisable to review the collections available for the Database instance, so that the Database hosting the repository can be monitored as well: 2.10 Database Instance 2.78 Oracle ListenerThe installation steps and benefits from OCM collections are explained in: Oracle Configuration Manager Quick Start Guide Note 548815.1: My Oracle Support Configuration Management FAQ Note 728982.1: Oracle Configuration Manager Security Overview and Collections Overview Note 728473.1: Oracle Configuration Manager PrerequisitesNote 728989.1: Oracle Configuration Manager Installation and Administration Guide HealthcheckHealthchecks are executed dynamically against the Oracle Configuration Manager uploaded configurations in My Oracle Support. These checks, based on Oracle Best practices, will proactively notify you of potential problems in your environment, and provide recommendations that help you improve system performance and avoid problems in your Oracle environment. If you are receiving any Healthcheck alerts in My Oracle support, then refer to the following document for the alert details and its corresponding document for resolving the same: Note 868955.1: My Oracle Support Health Checks CatalogFor Healthchecks specific to the Enterprise Manager and Repository Database, refer to the sections titled: Enterprise Manager (for the OMS) Oracle Database (for the Database hosting the Repository) Back to TopCPU and PSUCPUCritical Patch Updates (CPU) is the primary means of releasing security fixes for Oracle products. They are released on the Tuesday closest to the 15th day of January, April, July and October. This page lists all the currently available Critical Patch Updates (CPUs) in chronological order and is updated whenever new Critical Patch is released. You can also subscribe to the CPU Email Alerts using the steps listed here.To obtain the latest CPU patch details for the Enterprise Manager Grid Control and its dependent products - Oracle Application Server and Oracle Database:- In the page, click on the link shown for the latest CPU in the table under the 'Critical Patch Updates'.- The next page, lists all the products which have security fixes in the chosen CPU release. Scroll down to 'Patch Availability Table ..' topic and find the table with details for the Product Group and Patch Availability and Installation Information. - In the table, find the row related to Product Group: 'Oracle Enterprise Manager' and pick up the document number given in the Patch Availability and Installation Information column. In the document, navigate to:              "Critical Patch Update Availability for Oracle Products" and then to             "Oracle Enterprise Manager Grid Control"PSUPatch Set Updates (PSU) are proactive cumulative patches containing recommended bug fixes that are released on a regular and predictable schedule. PSUs are on the same quarterly schedule as the Critical Patch Updates (CPU), specifically the Tuesday closest to the 15th of January, April, July, and October. The PSUs serve as a new baseline version for reporting issues to Oracle, hence it is always recommended to be on the latest PSU release.For more details on PSU, refer Note 854428.1: Patch Set Updates for Oracle Products For Enterprise Manager specific PSU, refer Note 822485.1: Oracle Recommended Patches -- Oracle Enterprise ManagerChoosing between CPU / PSU patches The PSU and CPU released each quarter contain the same security content. However, the patches employ different patching mechanisms, so customers need to choose wisely which patch satisfies their needs better:A PSU can be applied on the CPU released at the same time or on an any earlier CPU for the base release version. A PSU can be applied on any earlier PSU or the base release version. CPUs are only created on the base release version.  Once a PSU has been installed, the recommended way to get future security content is to apply subsequent PSUs. Reverting from PSU back to CPU, while possible, would require significant effort, and so is not advised. Getting CPU / PSU patch recommendations via OCM OCM also collects and recommends the latest CPU and PSU patch that can be applied to a particular Oracle Home. These details can be seen in the My Oracle Support ->Patches and Updates -> Patch Recommendations section - 'Security' patch recommendations include the CPU patches.- 'Other Recommendations' include the PSU patches.Back to TopReferencesNOTE:1081865.1 - Master Note for 10g Grid Control OMS Process Control (Start, Stop and Status) & ConfigurationNOTE:1082009.1 - Master Note for 10g Grid Control Agent Process Control (Start, Stop & Status) & ConfigurationNOTE:1086343.1 - Master Note for 10g Grid Control Enterprise Manager Communication and Upload issuesNOTE:1087997.1 - Master Note for 10g Enterprise Manager Grid Control Agent Performance & Core Dump issuesNOTE:1092513.1 - Master Note for 10g Enterprise Manager Grid Control Security FrameworkNOTE:1098262.1 - Master Note for Diagnostic Tools for 10g Enterprise Manager Grid Control ComponentsNOTE:1161003.1 - Master Note for 10g Grid Control OMS Performance Issues

For most current information refer Master Note for 10g Grid Console Browser / User-interface Issues (Doc ID 1190323.1)     Master Note for 10g Grid Console Browser / User-interface Issues (Doc...

Master Notes

Oracle Support Master Note for Real Application Clusters (RAC), Oracle Clusterware and Oracle Grid Infrastructure (Doc ID 1096952.1)

QUICKLINKS:RAC Assurance Support Team: RAC Starter Kit and Best Practices (Doc ID 810394.1)RAC Frequently Asked Questions (Doc ID 220970.1)Are you implementing Oracle RAC or Oracle Grid Infrastructure for the first time?   Are you an experienced RAC or Oracle Clusterware user, but want feedback on the latest knowledge and Best Practices?   This blog is intended to give you insights into the quickest way to find information about RAC, Oracle Clusterware and Oracle Grid Infrastructure.  The contents of this blog replicate the data found in the My Oracle Support Master Note#1096952.1 for Oracle RAC, Oracle Clusterware and Oracle Grid Infrastructure.This Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes with respect to Oracle Real Application Clusters, Oracle Clusterware and Oracle Grid Infrastructure implementations. This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest, within the RAC, Clusterware and Grid Infrastructure spaces. This includes the following categories: RAC Assurance RAC Starter Kits - Installation and Best Practices Information Additional Notes on RAC Installation and Frequently asked Questions: RAC and Clusterware Monitoring, Troubleshooting & Data Collection: Tools Troubleshooting and Data Collection My Oracle Support references RAC and Clusterware Testing and Performance Tuning RAC and Clusterware Maintenance and Patching References General Maintenance References Patching Best Practices and References Upgrade and Platform Migration for RAC RAC and Oracle Applications RAC ASM Documentation Maximum Availability Architecture References Scope and ApplicationOracle Real Application Clusters (RAC) is a cluster database with a shared cache architecture that overcomes the limitations of traditional shared-nothing and shared-disk approaches to provide a highly scalable and available database solution for all your business applications. Oracle RAC provides the foundation for enterprise grid computing. Oracle Clusterware is a portable cluster software that allows clustering of single servers so that they cooperate as a single system. Oracle Clusterware provides the required infrastructure for Oracle Real Application Clusters while also enabling high availability protection of any Oracle or third party application.Oracle Automatic Storage Management (ASM) provides a virtualization layer between the database and storage. It treats multiple disks as a single disk group, and lets you dynamically add or remove disks while keeping databases online. ASM greatly simplifies the storage complexities often associated with an Oracle database while increasing availability and performance of the database.With Oracle grid infrastructure 11g release 2 (11.2), Oracle Automatic Storage Management (Oracle ASM) and Oracle Clusterware are installed into a single home directory, which is referred to as the Grid Infrastructure home. The installation of the combined products is called Oracle Grid Infrastructure. However, Oracle Clusterware and Oracle Automatic Storage Management remain separate products. This note applies to the following versions of these products: Oracle Clusterware - Version 10.1.0.2 to 11.1.0.7 Oracle Grid Infrastructure - Version 11.2.0.x Oracle Real Application Clusters - Version 10.1.0.2 to 11.2.0.xMaster Note for Real Application Clusters (RAC) Oracle Clusterware and Oracle Grid InfrastructureRAC Assurance RAC Starter Kits - Installation and Best Practices The goal of the Oracle Real Application Clusters (RAC) Starter Kits are to provide the latest information on generic and platform specific best practices for implementing an Oracle cluster using Oracle Clusterware or Oracle Grid Infrastructure (11gR2) and Oracle Real Application Clusters. These documents are compiled and maintained based on Oracle Support's ongoing experience with its global RAC customer base. The process used to install, configure, and create an Oracle Cluster and an Oracle Real Application Clusters (RAC) database shares much in common across all supported operating system platforms. Despite these commonalities, many aspects of the deployment process are O/S specific. As such, there is a Generic Starter Kit, as well as a separate Starter Kit for each supported platform. These Starter Kits provide the latest information in terms of installation help, as well as best practices for ongoing/existing implementations. <Document ID 810394.1> RAC Assurance Support Team: RAC and Oracle Clusterware Starter Kit and Best Practices (Generic)<Document ID 811306.1> RAC Assurance Support Team RAC Starter Kit (Linux) <Document ID 811280.1> RAC Assurance Support Team RAC Starter Kit (Solaris) <Document ID 811271.1> RAC Assurance Support Team RAC Starter Kit (Windows) <Document ID 811293.1> RAC Assurance Support Team RAC Starter Kit (AIX) <Document ID 811303.1> RAC Assurance Support Team RAC Starter Kit (HP-UX) The Generic Starter Kit has attached documents with sample test plan outlines for system testing and basic load-testing to validate the infrastructure is setup correctly, as well as a sample project plan outline for your RAC implementation. In addition, best practices for Storage and networking and other areas are included. The Platform-specific Starter Kit notes complement this with Step-by-Step Installation Guides, for each particular platform, as well as best practices for the specific platform in question. Additional Notes on RAC and Clusterware Installation and Frequently asked QuestionsWhile the Starter Kits referenced above will contain much of what you need to get started, additional notes and resources that are important to take heed of, particularly for a new implementation, can be found here in the following FAQ's and other references: <Document ID 220970.1> RAC: Frequently Asked Questions<Document ID 316817.1> CLUSTER VERIFICATION UTILITY FAQ <Document ID 332257.1> Using Oracle Clusterware with Vendor Clusterware FAQ <Document ID 397460.1> Oracle's Policy for Supporting RAC 10g with Symantec SFRAC <Document ID 391771.1> OCFS2 - Frequently Asked Questions <Document ID 787420.1> Cluster Interconnect in Oracle 10g and 11gKey notes related to OS Installation requirements for a RAC or Oracle Clusterware environment<Document ID 359515.1> Mount Options for Oracle files when used with NAS devices<Document ID 401132.1> How to install Oracle Clusterware with shared storage on block devices <Document ID 357472.1> Configuring device-mapper for CRS/ASM <Document ID 169706.1> Oracle Database on AIX,HP-UX,Linux,MacOSX,Solaris,Tru64 Operating Systems Installation and Configuration Requirements Quick Reference (8.0.5 to 11.1) Key notes introducing differences in Oracle 11gR2<Document ID 1053147.1> 11gR2 Clusterware and Grid Home - What You Need to Know<Document ID 948456.1> Pre 11.2 Database Issues in 11gR2 Grid Infrastructure Environment <Document ID 1050908.1> How to Troubleshoot Grid Infrastructure Startup Issues<Document ID 887522.1> 11gR2 Grid Infrastructure Single Client Access Name (SCAN)ExplainedRAC and Clusterware Monitoring, Troubleshooting and Data Collection NotesTo effectively manage your RAC cluster, it is important to know how to monitor, troubleshoot and collect data, both for your own diagnosis, and also to provide supporting information when logging a service request. The following Notes will give you information on how to do both: Tools for Monitoring RAC and Clusterware Environments <Document ID 301137.1> OS Watcher User Guide<Document ID 459694.1> Procwatcher: Script to Monitor and Examine Oracle and CRS <Document ID 736752.1> Introducing Cluster Health Monitor <Document ID 811151.1> How to install Oracle Cluster Health Monitor (former IPD/OS) on Windows <Document ID 314422.1> Remote Diagnostics Agent (RDA) <Document ID 422893.1> 11g Understanding Automatic Diagnostic Repository <Document ID 559365.1> Using Diagwait as a Diagnostic.  References needed for Troubleshooting and Data Collection <Document ID 556679.1> Data Gathering for Troubleshooting RAC Issues<Document ID 289690.1> Data Gathering for Troubleshooting CRS issues <Document ID 330358.1> CRS 10gR2/ 11gR1/ 11gR2 Diagnostic Collection Guide <Document ID 557204.1> Crsd Ocssd Evmd Log Rotation Policy <Document ID 563566.1> gc lost blocks diagnostics <Document ID 219361.1> Troubleshoot ORA-29740 errors in a RAC Environment. <Document ID 745960.1> 11g How to Unpack a Package in to ADR <Document ID 443529.1> 11g Quick Steps to Package and Send Critical Error Diagnostic Information to Support <Document ID 726446.1> What Is The Difference Between RDA and Configuration Manager <Document ID 77483.1> External Support FTP site: Information Sheet  RAC and Clusterware Testing and Performance Tuning Much of RAC performance tuning is no different from "single instance" tuning. However, there are some additional areas to check in a RAC environment when tuning or troubleshooting performance related issues. The following notes will make you aware of some of the areas where performance tuning and diagnosis has a specific RAC flavor: <Document ID 390374.1> Oracle Performance Diagnostic Guide (OPDG)<Document ID 181489.1> Tuning Inter-Instance Performance in RAC and OPS <Document ID 561414.1> Transactional Sequences in Applications in a RAC environment <Document ID 395314.1> RAC Hangs due to small cache size on SYS.AUDSES$ <Document ID 563566.1> gc lost blocks diagnostics <Document ID 361323.1> HugePages on Linux: What It Is... and What It Is Not... <Document ID 280939.1> Checklist for Performance Problems with Parallel Execution <Document ID 759565.1> Turning NUMA on can cause database hangs  RAC and Clusterware Certification, Maintenance, and Patching References Maintenance of a RAC environment is similar in many respects to maintaining a single-instance environment. However, with the Clusterware and Grid Infrastructure stacks, there are some additional tasks to be aware of regarding ongoing maintenance, as well as maintenance that may be brought on by changes to your environment. Below are some references speaking to the areas of general maintenance that may be needed or affect RAC environments in a particular way, as well as references to patching, which is a key part of an ongoing maintenance strategy. Certification References for RAC and Clusterware Environments<Document ID 184875.1> How To Check The Certification Matrix for Real Application Clusters<Document ID 1065024.1> Database 11g Release 2 Certification Highlights<Document ID 337737.1> Oracle Clusterware - ASM - Database Version CompatibilityGeneral Maintenance References for RAC and Clusterware Environments <Document ID 428681.1> How to ADD/REMOVE/REPLACE/MOVE Oracle Cluster Registry (OCR) and Voting Disk<Document ID 276434.1> Modifying the VIP or VIP Hostname of a 10g or 11g Oracle Clusterware Node. <Document ID 283684.1> How to Change Interconnect/Public Interface IP or Subnet in Oracle Clusterware <Document ID 754594.1> Potential disk corruption when adding a node to a cluster<Document ID 403743.1> VIP Failover Take Long Time After Network Cable Pulled <Document ID 563905.1> Implementing LIBUMEM for CRS on Solaris 64 with 3rd Party Clusterware <Document ID 92602.1> How to Password Protect the Listener Patching Best Practices and References for RAC and Oracle Clusterware As part of an overall maintenance strategy, it is critical that customers have a formal strategy to stay in front of known issues and bugs. To make it easier for customers to obtain and deploy fixes for known critical issues please refer to the following list of references. <Document ID 209768.1> Database, FMW, Em Grid Control and OCS Software Error Correction Support Policy.<Document ID 316900.1> Oracle 10g Release 2 (10.2) Support Status and Alerts <Document ID 454507.1> Oracle 11g Release 1 (11.1) Support Status and Alerts <Document ID 742060.1> Release Schedule of Current Database Releases. <Document ID 756671.1> Oracle Recommended Patches -- Oracle Database. <Document ID 854428.1> Introduction to Patch Set Updates (PSUs) <Document ID 850471.1> Oracle Announces First Patch Set Update For Oracle Database Release 10.2 <Document ID 405820.1> 10.2.0.X CRS Bundle Patch Information. <Document ID 810663.1> 11.1.0.X CRS Bundle Patch Information. <Document ID 1082394.1> 11.2.0.X Grid Infrastructure Bundle Patch Information. <Document ID 363254.1> Applying one-off Oracle Clusterware patches in a mixed version home environment <as yet unpublished> Note 844983.1 Apply CRS Bundle Patch Using opatch auto Option. <Document ID 761111.1> Online Patches (Hot Patching). <Document ID 438314.1> Critical Patch Update - Introduction to Database n-Apply CPUs. <Document ID 1082394.1> 11.2.0.X Grid Infrastructure PSU Known Issues Upgrade and Platform Migration for RAC and Oracle Clusterware When migrating a RAC cluster to a new platform or upgrading a RAC cluster from an older version of Oracle to the current version of Oracle, the following references will be helpful: <Document ID 466181.1> Oracle 10gR2 Upgrade Companion.<Document ID 601807.1> Oracle 11gR1 Upgrade Companion. <Document ID 785351.1> Oracle 11gR2 Upgrade Companion. <Document ID 429825.1> Complete Checklist for Manual Upgrades to 11gR1. <Document ID 747457.1> How to Convert 10g Single-Instance database to 10g RAC using Manual Conversion procedure. <Document ID 338706.1> Cluster Ready Services (CRS) rolling upgrade References for running RAC with Applications <Document ID 790189.1> Oracle Clusterware and Application Failover Management RAC and Oracle E-Business <Document ID 362135.1> Configuring Oracle Applications Release 11i with Oracle10g Release 2 Real Application Clusters and Automatic Storage Management.<Document ID 380489.1> Using Load-Balancers with Oracle E-Business Suite Release 12 <Document ID 727171.1> Implementing Load Balancing On Oracle E-Business Suite - Documentation For Specific Load Balancer Hardware <Document ID 217368.1> Advanced Configurations and Topologies for Enterprise Deployments of E-Business Suite 11i <Document ID 403347.1> MAA Roadmap for the E-Business Suite <Document ID 388577.1> Using Oracle 10gR2 RAC and Automatic Storage Management with Oracle E-Business Suite Release 12 RAC ASM Documentation Automatic Storage Management (ASM) is an evolution in file system and volume management functionality for Oracle database files. ASM further enhances automation and simplicity in storage management that is critical to the success of the Oracle grid architecture. ASM also improves file system scalability and performance, manageability and database availability for RAC environments. Use of ASM with RAC is an Oracle Best Practice. <Document ID 1187723.1> Master Note for Automatic Storage Management (ASM)<Document ID 1187674.1> Master Note for Oracle Database Machine and Exadata Storage Server MAA/Standby DocumentationData Guard provides the management, monitoring, and automation software infrastructure to create and maintain one or more standby databases to protect Oracle data from failures, disasters, errors, and data corruptions. As users commit transactions at a primary database, Oracle generates redo records and writes them to a local online log file. <Document ID 1101938.1> Master Note for Data Guard Using My Oracle Support Effectively<Document ID 374370.1> New Customers Start Here<Document ID 736737.1> My Oracle Support - The Next Generation Support Platform<Document ID 747242.5> My Oracle Support Configuration Management FAQ<Document ID 868955.1> My Oracle Support Health Checks Catalog<Document ID 166650.1> Working Effectively With Global Customer Support<Document ID 199389.1> Escalating Service Requests with Oracle Support Services Keywords and search terms: RAC, ASM, CRS, Cluster, Clusterware, DB Machine, Exadata, GI, Grid, Infrastructure, OCR, Oracle, RAC, Votedsk, votedisk, 'Voting Disk', ohasd, ocssd, crsd, evmd, srvctl, crsctl, 'Grid Infrastructure', 'Real Application Clusters', HA, Scalability, ACFS, ADVM*** - For reference to the content in this blog, and for the latest and most up to date information, refer to Note:1096952.1  Master Note for Real Application Clusters (RAC) Oracle Clusterware and Oracle Grid Infrastructure

QUICKLINKS: RAC Assurance Support Team: RAC Starter Kit and Best Practices (Doc ID 810394.1) RAC Frequently Asked Questions (Doc ID 220970.1) Are you implementing Oracle RAC or Oracle Grid...

Master Notes

Master Note for Oracle Rdb Product Family on OpenVMS (Doc ID 1162187.1)

In this Document  What is being announced?     Current Releases     Oracle Rdb     Oracle CODASYL DBMS     Oracle SQL/Services and OCI Services for Rdb     Oracle JDBC for Rdb     Oracle ODBC Driver for Rdb     Oracle Rdb Developer Tools for Visual Studio .NET (ORDP & DDEX)     Oracle Rdb Extension for SQL Developer     Oracle CDD/Repository     Oracle Trace / Expert for Rdb     Oracle Rdb Extension for OEM 10g     Replication Option for RdbApplies to: Oracle ODBC Driver for Rdb - Version: 3.0 and later   [Release: and later ]Oracle CDD/Repository - Version: 7.0 and later    [Release: 7.0 and later]Oracle SQL/Services for Rdb on OpenVMS - Version: 7.0 and later    [Release: 7.0 and later]Oracle Rdb Server on OpenVMS - Version: 7.0 and later    [Release: 7.0 and later]Oracle CODASYL DBMS - Version: 7.0 and later    [Release: 7.0 and later]Information in this document applies to any platform.What is being announced?This Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes for the Oracle Rdb Product Family.Current ReleasesOracle Rdb                                                                               7.2.4.1Oracle CODASYL DBMS                                                        7.2.4.1Oracle SQL/Services / OCI Services for Rdb                     7.3.0.3Oracle JDBC for Rdb                                                              7.3Oracle ODBC Driver for Rdb                                                 3.3.2.0Oracle Rdb Developer Tools for Visual Studio                  7.3.2 (formerly Oracle Rdb Data Provider for .NET)Oracle Rdb Extension for SQL Developer                          7.3 Oracle CDD/Repository                                                         7.2.0.2Oracle Trace for Rdb                                                              7.2.0.3/2.4.1.1Oracle Rdb Extension for OEM 10g                                     7.2.1.0Oracle Replication Option for Rdb                                       7.2Note: Oracle Rdb 7.2.4.1 and Oracle CODASYL DBMS 7.2.4.1 are certified to install and run on OpenVMS 8.4Oracle Rdb  Documentation     RDBPROD: Comprehensive Rdb Versions List     RDBPROD: Frequently Asked AIJ Backup and Recovery Questions     Oracle Rdb Documentation Index     Oracle Rdb Release Notes Read Me Index        RDBPROD: New Features Rdb 7.2     RDBPROD: New Features Rdb 7.1     Oracle Rdb Server 7.0 Desupport Notice     Oracle Rdb Server 6.1 Desupport Notice     Oracle Rdb Server 6.0 Desupport Notice  Troubleshooting     Remote Diagnostic Agent (RDA) for the Rdb Product Family on OpenVMS     RDBPROD: Procedure to summarize a bugcheck dump     RDBPROD: Diagnosing and Understanding Common Locking Problems  White Papers / Frequently Used Notes     RDBPROD: Oracle Rdb Row Cache Hints & Tricks     RDBPROD: Oracle Rdb Virtual Memory Options on OpenVMS     RDBPROD: Overview of Rdb & CODASYL DBMS Database Replication With Hot Standby     Guide to Performance and Tuning: Managing the AIP Data Structure     Guide to Performance and Tuning: Space Management and Thresholds     Guide to Using SQL: Computed and Automatic Columns     Guide to Using SQL: Identity Columns     RDBPROD: Oracle Rdb Upgrade Paths and V7.x RMU/CONVERT Restrictions     RDBPROD: Improving Query Performance Using Sampled Selectivity     RDBPROD: Using Query Outlines     RDBPROD: Rdb: Poor Insert Performance---Excessive Pages Checked     RDBPROD: Planning for Success: Database Backup & Recovery     RDBPROD: Understanding Oracle Rdb's TSNBLK and SEQBLK     RDBPROD: How to identify fragmented rows in a table     RDBPROD: How to Determine if Indexes are FragmentedOracle CODASYL DBMS  Documentation     RDBPROD: DBMS Supported Version     Oracle CODASYL DBMS Documentation Index     Oracle CODASYL DBMS Release Notes & Readme Index     Oracle CODASYL DBMS 7.0 Desupport Notice     Oracle CODASYL DBMS 6.1 Desupport Notice     Oracle CODASYL DBMS 6.0 Desupport Notice  Troubleshooting     Remote Diagnostic Agent (RDA) for the Rdb Product Family on OpenVMS     RDBPROD: Procedure to summarize a bugcheck dump  White Papers / Frequently Used Notes     RDBPROD: Overview of Rdb & CODASYL DBMS Database Replication With Hot StandbyOracle SQL/Services and OCI Services for Rdb  Documentation     SQL/Services, OCI Services for Rdb (SQLNet), ODBC, and OEM Versions List     RDBPROD: Oracle SQL/Services - OCI Services for Rdb F.A.Q.        SQL/Services, OCI Services for Rdb (SQLNet), ORDT & JDBC for Rdb        Documentation Index     OCI Services for Rdb (SQLNet), SQL/Services, ODBC, ORDT, JDBC for Rdb        Release Notes & Readme Index  Troubleshooting     Remote Diagnostic Agent (RDA) for the Rdb Product Family on OpenVMS     RDBPROD: Procedure to summarize a bugcheck dump     RDBPROD: Guide to Solving OCI Services Connection Problems     RDBPROD: Finding Out Why an OCI Services Dispatcher Will Not Start     RDBPROD: How to Enable SQL*Net Client Tracing to Debug OCI Services Network Issues     RDBPROD: ORA-01017 on Connection to Rdb via OCI Services      RDBPROD: How to Troubleshoot Performance Problems Involving Access to an Rdb Database                         via a DBLINK  White Papers / Frequently Used Notes     RDBPROD: How to Configure OCI Services for Rdb     RDBPROD: 10G Client Gives ORA-12705: Cannot Access NLS Data Files or                          Invalid Environment Specified     RDBPROD: ERROR ORA-12504: TNS:listener was not given the SID in CONNECT_DATA     RDBPROD: ORA-3134: Connections to This Server Version are No Longer Supported                          When Accessing Rdb from Oracle 10g Client     RDBPROD: A Query via DB Link from Oracle to Rdb Fails Intermittently with ORA-02068                         and ORA-03113Oracle JDBC for Rdb  Documentation     RDBPROD: Oracle Rdb Data Provider for .Net and JDBC for Rdb Compatibility Matrix     Oracle� Rdb JDBC Driver F.A.Q.     RDBPROD: Oracle JDBC for Rdb F.A.Q.     SQL/Services, OCI Services for Rdb (SQLNet), ORDT & JDBC for Rdb        Documentation Index     OCI Services for Rdb (SQLNet), SQL/Services, ODBC, ORDT, JDBC for Rdb        Release Notes & Readme Index     Oracle Rdb JDBC Driver User's Guide, Release 7.3.0.0  Troubleshooting     Remote Diagnostic Agent (RDA) for the Rdb Product Family on OpenVMS  White Papers / Frequently Used NotesOracle ODBC Driver for Rdb  Documentation     SQL/Services, OCI Services for Rdb (SQLNet), ODBC, and OEM Versions List     RDBPROD: Oracle ODBC Driver for Rdb F.A.Q.     OCI Services for Rdb (SQLNet), SQL/Services, ODBC, ORDT, JDBC for Rdb        Release Notes & Readme Index     RDBPROD: ODBC ReadMe 3.3.2.0  Troubleshooting     RDBPROD: Logging and Tracing Options Available for the Rdb ODBC Driver  White Papers / Frequently Used Notes     RDBPROD: Problems After Installation of ODBC Driver for Rdb 3.3.x     RDBPROD: Using the Oracle ODBC Driver for Rdb with 64 bit Microsoft Windows x64Oracle Rdb Developer Tools for Visual Studio .NET (ORDP & DDEX)  Documentation     RDBPROD: Oracle Rdb Data Provider for .Net and JDBC for Rdb Compatibility Matrix     RDBPROD: Oracle Rdb Data Provider for .Net F.A.Q.     SQL/Services, OCI Services for Rdb (SQLNet), ORDT & JDBC for Rdb        Documentation Index     OCI Services for Rdb (SQLNet), SQL/Services, ODBC, ORDT, JDBC for Rdb        Release Notes & Readme Index     Oracle Rdb Developer Tools for Visual Studio, Development Guide, Release 7.3.2.0     ORDP for .NET Development Guide, Release 7.3.2.0  Troubleshooting  White Papers / Frequently Used NotesOracle Rdb Extension for SQL Developer  Documentation     Oracle� Rdb Extension for SQL Developer User's Guide V7.3  Troubleshooting     RDBPROD: SQL Developer and Oracle Rdb  White Papers / Frequently Used NotesOracle CDD/Repository  Documentation     RDBPROD: CDD/Repository Supported Versions     RDBPROD: Oracle CDD/Repository Documentation Index     Oracle CDD/Repository Release Notes & Readme Index     Oracle CDD/Repository version 7.0 Desupport Notice     Oracle CDD/Repository version 6.1 Desupport Notice  Troubleshooting     Remote Diagnostic Agent (RDA) for the Rdb Product Family on OpenVMS  White Papers / Frequently Used Notes     RDBPROD: Moving A System Or Personal Repository To New LocationOracle Trace / Expert for Rdb  Documentation     Trace & Expert for Rdb Versions     Trace & Expert Documentation Index     Trace & Expert Release Notes & Readme Index  Troubleshooting     RDBPROD: Procedure to summarize a bugcheck dump  White Papers / Frequently Used NotesOracle Rdb Extension for OEM 10g  Documentation     SQL/Services, OCI Services for Rdb (SQLNet), ODBC, and OEM Versions List     Oracle Rdb� Extension to Oracle Enterprise Manager 10g User Guide Release 7.2.1.0   Troubleshooting  White Papers / Frequently Used Notes     RDBPROD: Grid Control For Rdb Without Oracle 10gReplication Option for Rdb  Documentation     RDBPROD: Replication Option for Rdb Documentation Index     Replication Option for Rdb Release Notes & Readme Index  Troubleshooting     RDBPROD: Replication Option for Rdb Log Files - Definitions and Locations  White Papers / Frequently Used Notes     RDBPROD: How to Move RDB$CHANGES Out of the RDB$SYSTEM Storage Area     RDBPROD: Using TCP/IP with Rdb Replication Option

In this Document   What is being announced?      Current Releases      Oracle Rdb      Oracle CODASYL DBMS      Oracle SQL/Services and OCI Services for Rdb      Oracle JDBC for Rdb      Oracle ODBC...

Master Notes

Master Note for Berkeley DB Java Edition (Doc ID 1214266.1)

In this Document  What is being announced?     Oracle Berkeley DB Java EditionApplies to: Berkeley DB Java Edition - Version: 1.5.3 and later   [Release: and later ]Information in this document applies to any platform.What is being announced?Oracle Berkeley DB Java EditionGetting Started GuidesGetting Started Guide PDF | HTMLWriting Transactional Applications PDF | HTMLJava Collections Tutorial PDF | HTMLGetting Started with BDB JE High Availability PDF | HTMLProgrammatic APIs Javadoc: The javadoc covers the full set of all public JE APIs. Starting points for the different APIs are listed below.Base API Direct Persistence Layer (DPL) High Availability/Replication Collections Examples General ExamplesTranslating SQL into BDB ExampleBDB JE HA ExamplesBuild and Installation Notes Installation Notes JE and Android Monitoring JE with JConsole and JMX Using JE trace logging Change Log Release NotesDownloads and licensingRelease History and Past Change LogsLicenseCommunityOracle Technology Network Support Forums - Berkeley DB Java EditionCharles Lamb's Blog on Berkeley DB Java EditionThe Official Oracle Berkeley DB BlogBerkeley DB Java Edition Archives on the official Oracle Berkeley DB BlogProduct InformationOracle Berkeley DB Java Edition Data Sheet (PDF)Berkeley DB Java Edition High Availability (HA) (PDF)Oracle Berkeley DB Java Edition High Availability -Large Configuration and Scalability Testing (PDF)Write Forwarding Using Java RMI (PDF)Berkeley DB Java Edition Architecture (PDF)Direct Persistence Layer for Berkeley DB Java Edition ( English) ( Chinese) (PDF)Java Object Storage on Android (PDF)Oracle Berkeley DB Java Edition vs. Apache Derby: A Performance Comparison (PDF)Technical ResourcesFAQsHow to configure Oracle Berkeley DB Java Edition for use on Google Android devicesPerforming Queries in Oracle Berkeley DB Java Edition -- demonstrates how to rewrite SQL using the Direct Persistence Layer (DPL)Article: Dynamo: Amazon's Highly Available Key-Value Store, All Things Distributed (Oct 2007)Article: Embedded Java Persistence ( English) ( Chinese) (PDF), Oracle Magazine (March/April 2007)Article: Oracle Berkeley DB Java Edition 3.1.0: Direct Persistence Layer, The Server Side (October 2006)Article: High-Performance Data Management in Java, Dr. Dobb's Journal (May 2005)Article: Using Berkeley DB Java Edition as a Persistence Manager for the Google Web Toolkit, OTN (Feb 2008)Presentation: Design and Implementation of a Transactional Data Manager, JavaOne 2004 (PDF)

In this Document   What is being announced?      Oracle Berkeley DB Java Edition Applies to: Berkeley DB Java Edition - Version: 1.5.3 and later   [Release: and later ]Information in this document...

Master Notes

Master Note for Berkeley DB Core (Doc ID 1217693.1)

In this Document  What is being announced?  What do you need to do?Applies to: Berkeley DB - Version: 1.85 to 5.0.26Information in this document applies to any platform.What is being announced?This document provides an overview of the Berkeley DB Core product. What do you need to do?Berkeley DB Core DocumentationGetting Started with Data StorageC             PDF | HTMLC++        PDF | HTMLJava        PDF | HTMLGetting Started with Transaction ProcessingC             PDF | HTMLC++        PDF | HTMLJava        PDF | HTMLGetting Started with Transactional Java CollectionsTutorial   PDF | HTMLGetting Started with ReplicationC             PDF | HTMLC++        PDF | HTMLJava        PDF | HTMLBerkeley DB SQL API               PDF | HTMLBerkeley D Porting Guide               PDF | HTMLReference Guide for Berkeley DB               PDF | HTMLAdditional TopicsConfiguration TipsSelecting an access methodSelecting a page sizeSelecting a cache sizeSelecting a byte orderConfiguring locking: sizing the systemConfiguring loggingTuning TipsAccess method tuningTransaction tuningBerkeley DB Consurrent Data Store ApplicationsHandling failureArchitecting applicationsBerkeley DB Transactional Data Store ApplicationsHandling failureArchitecting applicationsBerkeley DB ExtentionsUsing Berkeley DB with ApacheUsing Berkeley DB with PerlUsing Berkeley DB with PHPProgrammatic APIsC             PDF | HTMLC++        PDF | HTMLC#                     HTMLSTL        PDF |  HTMLJavadoc             HTMLTCL        PDF | HTMLBuild, Installation, and Upgrade Guide                PDF | HTMLInstallation GuideUNIX/POSIXVxWorksWindowsWindows CEChange LogBerkeley DB Library Version 11.2.5.1 Change LogUpgrading ApplicationsUpgrading Applications to Berkeley DB 11.2.5.1 Historical Upgrade           PDF | HTMLWriting In-Memory Bekeley DB Applications           PDF | HTMLStandalone utilitiesBerkeley DB FAQArticlesGeneral ArticlesMaster - Master Note for Berkeley DB (Doc ID 1217684.1)Master Note for Berkeley DB SQL (Doc ID 1216325.1)Master Note for Berkeley DB High Availability / Replication (Doc ID 1193165.1) PerformancePre-warming the Berkeley DB cache (Doc ID 781726.1)Solaris Options for Avoiding High fsflush Daemon I/O Rates (Doc ID 969711.1)How are the Berkeley DB flags impacting performance (Doc ID 785259.1)  Trouble Shooting "Logging region out of memory" when upgrading Berkeley DB to 4.7.25 or higher (Doc ID 781780.1)  BadImageFormatException raised when 32bit Windows Applicaton Executes in 64bit Environment (Doc ID 1197054.1)  Environment Panic Due to a Database Corruption Where Dirty Pages are on the Free List (Doc ID 972978.1)  BuildHow to Build Berkeley DB with Visual C++ Express Edition (2005/2008) (Doc ID 605597.1)  General TipsHow to Understand BTree Overflow Pages (Doc ID 1087395.1)  How to Use the Environment set_thread_count Method Properly (Doc ID 1105756.1)  Aspects of Moving Database Files From One Environment To Another (Doc ID 399797.1)  How to Handle Corrupt Environments Which Can Not be Removed Programmatically (Doc ID 968807.1)  CommunitiesOracle Technology Support Forums -- Berkeley DBDownloads and Licensing

In this Document   What is being announced?   What do you need to do? Applies to: Berkeley DB - Version: 1.85 to 5.0.26 Information in this document applies to any platform.What is being announced? This...

Master Notes

Master Note for Oracle Lite Syncronization Errors (Doc ID 1111228.1)

Applies to: Oracle Lite - Version: 10.3.0.1.0 to 10.3.0.3.0 - Release: 10.3 to 10.3Information in this document applies to any platform.GoalThis Note contains most of the Note where you have a Synchronization Error like CNS-XXXXXThe goal of this article is to give a global picture about the most usefull Notes where you have a Synchronization Error like CNS-XXXXX and appropriate actions depending where the error is found. In addition to the Support Notes below, you can get additional information here: My Oracle Support ComunitySolutionDuring a Synchronization The error CNS-9025 An Internal Error Has Occured 2/00 Note 1069307.1 CNS-9026 Wrong Username or Password When Syncing Note 464054.1 Sync Fails Because of Internal Error: CNS-9025, ORA-00001: unique constraint Mobileadmin.CMP$225A Note 468022.1 Error On Client During Syncronization: CNS-9019 ''Error During Compression''  Note 338836.1 Getting Error Message ''Invalid Object'' While Synchronizing. Cns-9029  Note 367602.1 OLITE 10.3.0.1.0: The Time On The Mobile Computer Changed (+1 Hour), Error: CNS-9026 Note 465211.1 On Window2003 Using Msync.Exe Cannot Sync Over SSL Connection, Cns-9107 Note 561474.1 CNS-9024 Network disconnected during sync, check network is active. Note 404394.1 CNS-9025 An Internal Error Has Occured ORA-1438 During Sync And Sync Always Fails Note 452952.1 CNS-9024 with OAS 10.1.3.1.0 + Mobile Server 10.3.0.1.0 Note 461298.1 ERR.LOG: TIMED OUT WAITING FOR SESSION LIMIT and CNS-9024 From Client Note 438041.1 CNS-9024 During The Processing Phase Note 468764.1 Avoid Cns-9025 An Internal Error Has Occurred. Server Busy Note 376750.1 Synchronization Fails with the Error Cns-9025 ORA-22275 Note 298391.1 ERROR CNS-9026,"Wrong username or password" WITH SSO/OID. Note 290144.1 Synchronization Error: Cns-9025 And Cons-10042 Note 460608.1 After Upgrade Of Rdbms Sync Fails with CNS-9025 CONS-10042 Cons Cache Reset Note 553281.1 Autosync Stopped, CNS-9024 and CNS-9032 in OL_SYNC.log Note 859854.1 CNS-9021 Table Class Name Mismatch Note 802102.1 "Error",Cns-9107"Network Disconnected During Sync.:12055 Note 804573.1 ERROR:",CNS-9024,"RESEND:Network disconnected during sync" Note 403805.1 CNS-9009 "Unable To Create Okapi Object" during Synchronization Note 754704.1 Device Gets CNS-9000 An Internal Error Has Ocurred During Sync Note 737412.1 Msync Returns the Error: CNS-9028 Compressed file access error, With No Admin Permissions Note 735016.1 Cns-9019 Error During Compression During A Large Sync Operation Note 339348.1 Bug 5092084 - CNS-9044 internal error Note 5092084.8 "SYNC ERROR: CNS-9005 "Could not create new file/PDB.:-1816 Not enough quota is available to process Note 735591.1 Invalid User On Syncing: CNS-9026 Using External Authentification. Note 734125.1 CNS-9028"Compressed file access error. ","MARC" CreateFile error 32 tmp_send.dat Note 564791.1 CNS-9025 Error when Syncing After Upgrade from 5.0.2.x to 10GR1 Note 313786.1 Msync Timeout With Message CNS-9024 using OAS 10.1.3.0.0 Note 472819.1 CNS-9025 AN INTERNAL ERROR WITH java.lang.NullPointerException Note 406079.1 SYNCHRONIZATION FAILS WITH ''NETWORK DISCONNECTED DURING SYNC.'' CNS-9107 Note 399779.1 Error CNS-9024 or CNS-9017Using OAS 10.1.2.0.2 With Mobile Server 10.2.0.2.0  Note 412115.1 Error 87, 6 or CNS-9024 During Sync because a lot of Device Manager Client Rebooted Note 405860.1 CNS-9070 Client Error During Sync Note 1086442.1 Java.Sql.SQLException: ORA-01489 Error While Syncing Note 421648.1 How To Avoid that Olsyncagent will start a new Sync just after the Previous Long Sync Note 743818.1 Cons-9312 Failed To Stop Sync Agent Note 472807.1 Unknown Syncprogresslistener Status = 14 Note 750751.1 Error: Java.Util.Zip.Zipexception: Invalid Stored Block Lengths During Synchronization Note 395392.1 Troubleshooting 'Client Out of Sync' Message on Oracle Lite Mobile Server: Note 331542.1How to workaround java.lang.OutOfMemoryError: Note 358294.1Server Error: java.lang.OutOfMemoryError Using OAS 10.1.2.0.2.0 <> Can Not Allocate More Memory Pol-2000 : Note 368901.1

Applies to: Oracle Lite - Version: 10.3.0.1.0 to 10.3.0.3.0 - Release: 10.3 to 10.3 Information in this document applies to any platform.Goal This Note contains most of the Note where you have...

Master Notes

TimesTen Product Documentation Library : Master Note Child (Doc ID 806197.1)

In this Document  Goal  SolutionApplies to: TimesTen Data Server - Version: 6.0.0 and later   [Release: 6.0 and later ]Information in this document applies to any platform.This note applies to all documentation for Oracle TimesTen In-Memory DatabaseGoalAbout:This is the Master Note Child for Oracle TimesTen In-Memory Database Documentation.Purpose:This note will list Documentation Notes in My Oracle Support for Oracle TimesTen In-Memory Database. SolutionUsage:To view or download a document please click on the most appropriate note below.Click Note 1088128.1 for the Oracle TimesTen In-Memory Database Master Note Parent.TimesTen 11x Documentation:Note 839628.1 TimesTen 11x Documentation Set - Install Reference GuideNote 978060.1 TimesTen 11x Documentation Set - Database Cache GuideNote 987726.1 TimesTen 11x Documentation Set - Database Reference GuideNote 1076502.1 TimesTen 11x Documentation Set - IntroductionNote 1076557.1 TimesTen 11x Documentation Set - Database Operations GuideNote 1076572.1 TimesTen 11x Documentation Set - Troubleshooting Procedures GuideNote 1076606.1 TimesTen 11x Documentation Set - TimesTen to TimesTen Replication GuideNote 1080489.1 TimesTen 11x Documentation Set - PLSQL Developers GuideTimesTen 7x Documentation:Note 784526.1 TimesTen 7x Documentation Set - Introduction Reference GuideNote 784518.1 TimesTen 7x Documentation Set - Install Reference GuideNote 784502.1 TimesTen 7x Documentation Set - Operations Reference GuideNote 784492.1 TimesTen 7x Documentation Set - Sql Reference GuideNote 784482.1 TimesTen 7x Documentation Set - Cache Connect Reference GuideNote 784258.1 TimesTen 7x Documentation Set - Api Reference GuideNote 784486.1 TimesTen 7x Documentation Set - Replication Reference GuideNote 783899.1 TimesTen 7x Documentation Set - Error Reference GuideNote 841128.1 TimesTen 7x Documentation Set - Troubleshooting GuideNote 430383.1 TimesTen 7x Release Notes Set - 7.0.2.0.0Note 468759.1 TimesTen 7x Release Notes Set - 7.0.2.4.0Note 468756.1 TimesTen 7x Release Notes Set - 7.0.3.1.0Note 468757.1 TimesTen 7x Release Notes Set - 7.0.3.2.0Note 469957.1 TimesTen 7x Release Notes Set - 7.0.3.3.0Note 552596.1 TimesTen 7x Release Notes Set - 7.0.4.0.0Note 562669.1 TimesTen 7x Release Notes Set - 7.0.5.0.0Note 753865.1 TimesTen 7x Release Notes Set - 7.0.5.4.0Note 845178.1 TimesTen 7x Release Notes Set - 7.0.5.9.0Note 845161.1 TimesTen 7x Release Notes Set - 7.0.5.10.0TimesTen 6x Documentation:Note 858271.1 TimesTen 6x Documentation Set - Installation Reference GuideNote 836494.1 TimesTen 6x Documentation Set - Api And Sql Reference GuideNote 858220.1 TimesTen 6x Documentation Set - Replication Reference GuideNote 373618.1 TimesTen 6x Release Notes Set - 6.0.3.0.0Note 394285.1 TimesTen 6x Release Notes Set - 6.0.5.0.0Note 394287.1 TimesTen 6x Release Notes Set - 6.0.7.0.0Note 406618.1 TimesTen 6x Release Notes Set - 6.0.8.0.0Note 430392.1 TimesTen 6x Release Notes Set - 6.0.10.0.0Note 836517.1 TimesTen 6x Release Notes Set - 6.0.16.0.0

In this Document   Goal   Solution Applies to: TimesTen Data Server - Version: 6.0.0 and later   [Release: 6.0 and later ] Information in this document applies to any platform.This note applies to all...

Master Notes

Master Note For Oracle Database Client Installation

    For the most current information, please refer to :(Doc ID 1157463.1) Master Note For Oracle Database Client Installation APPLIES TO:Oracle Database - Standard Edition - Version 9.2.0.1 and laterOracle Universal Installer - Version 2.2.0.18 and laterOracle Database - Enterprise Edition - Version 9.2.0.1 and laterInformation in this document applies to any platform.PURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes with respect to Oracle Database Client installation. This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest, within a Database Client Installation.  This includes the following categories:CertificationClient / Server / Interoperability Support Between Different Oracle VersionsOS RequirementsSilent Installation using response file/without using response fileInstalling 32-bit Client on 64-bit OSDiagnostic ToolsKnown IssuesGeneral InformationUsing My Oracle Support EffectivelyGeneric LinkSCOPEThis note serves as a guide through the various stages of Oracle Database Client installation .It is applicable for client installations via OUI or Silent using a response file or without a response file.The intended audience is from junior DBA's to senior DBA'sThis will cover the following versions of these products:Oracle Database Client - Version: 8.0.6.x though 11.2.0.xDETAILSCommunity DiscussionsWould you like to explore this topic further with other Oracle Customers and Specialists?Not finding the right information and considering opening a low priority Service Request?Give the Database Install/Upgrade Community a try first to save time and connect with your peers and Oracle experts.Database Install Community.CertificationFor the latest Database Certification always refer to My Oracle Support (MOS) Certify section.At the same time the documentation listed below will assist in understanding the Oracle Database Certifications:Note 1298096.1 Master Note For Database and Client CertificationNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryNote 207303.1 Client / Server / Interoperability Support Between Different Oracle VersionsNote 1307745.1 Oracle Database Support on the Itanium Processor ArchitectureNote 464754.1 Certified Software on Oracle VMNote 753601.1 SUPPORT FOR ORACLE DATABASE ON WPARS UNDER AIX 6.1Installation GuidesOracle Database documentation, 12c Release 1 (12.1)Installation manuals and platform-specific books are included in this link.Oracle Database documentation, 11g Release 2 (11.2)Installation manuals and platform-specific books are included in this link.Oracle Database documentation, 11g Release 1 (11.1)Installation manuals and platform-specific books are included in this link.Oracle Database documentation, 10g Release 2 (10.2)Installation manuals and platform-specific books are included in this link.Oracle Database documentation, 10g Release 1 (10.1)Installation manuals and platform-specific books are included in this link.Oracle9i documentation, Release 2 (9.2)Installation manuals and platform-specific books are included in this linkOracle9i documentation, Release 1 (9.0.1)Installation manuals and platform-specific books are included in this linkInstalling 32-bit Client on 64-bit OSNote 1151394.1 Running 32-bit and 64-bit Oracle Software On 64-bit WindowsNote 781432.1 Considerations Using The Oracle Client With Microsoft Products on 64 bit Operating SystemNote 870253.1 32-bit Client Install on 64-bit Windows PlatformNote 1075717.1 Installing 32-bit RDBMS Client software on x86_64 LinuxOS General Information and Known IssuesMost of the installation and relinking issues appear if you don't satisfy the minimum requirements for the installation. Therefore if you are raising any SR for installation issue it is better to ensure that you have all these requirements mentioned in the below articles before raising the SR.The documentation listed below help you in understanding the OS Requirements for Oracle Client Installation:GenericSolarisLinuxAIXHPWINDOWSGenericGeneralNote 1520299.1 Master Note For Oracle Database 12c Release 1 (12.1) Database/Client Installation/Upgrade/Migration Standalone Environment (Non-RAC)Note 1555012.1 Install/Deinstall/Cloning Assistant: Oracle Database/ClientNote 1476075.1 FAQ on downloading 9i, 10g, 11g software Note 169706.1 Oracle Database on Unix AIX,HP-UX,Linux,Mac OS X,Solaris,Tru64 Unix Operating Systems Installation and Configuration Requirements Quick Reference (8.0.5 to 11.2)Note 1351051.1 Information Center: Install and Configure Database Server/Client InstallationsNote 458893.1 Oracle Universal Installer (OUI) FAQ        Note 1467060.1 Relinking Oracle Home FAQ ( Frequently Asked Questions)Note 454442.1 11g Install : Understanding about Oracle Base, Oracle Home and Oracle Central/Global Inventory locationsNote 131321.1 How to Relink Oracle Database Software on UNIXNote 883299.1 Oracle 11gR2 Relink New FeatureNote 444595.1 Is It Necessary To Relink Oracle Following OS Upgrade?Note 556834.1 Steps To Recreate Central Inventory(oraInventory) In RDBMS HomesNote 750540.1 What are the requirements for an Oracle Home Name in the Inventory?Note 881063.1 Key RDBMS Install Differences in 11gR2Note 884232.1 11gR2 Install (Non-RAC): Understanding New Changes With All New 11.2 InstallerNote 1189783.1 Important Changes to Oracle Database Patch Sets Starting With 11.2.0.2Note 1212156.1 How To Identify What Type of Oracle11g Release 2 (11.2) Client Software Is Installed?Note 1397346.1 What Are The Different Oracle Client Components Installed With Different Installation Type In 11.2.0.2 or 11.2.0.3Note 429074.1 Can You Run Different Releases Of The Oracle Database On The Same Computer At The Same Time?Note 389678.1 Are Compilers Necessary for Oracle RDBMS Installation?Note 401332.1 How To Identify A Server Which Has Itanium2 (Montecito, Montvale, Tukwila....) Processors InstalledNote 820135.1 What Does -ignoreSysPrereqs Ignore When Invoking runInstaller?Note 434295.1 How To Find If The Oracle Client Software Is 32-Bit Or 64-Bit?                Note 1474891.1 Is It Supported to Change Uid/Gid Of Oracle Database Software Owner After Installation ?Note 1493121.1 What Are OrainstRoot.sh And Root.sh Scripts In A Standalone RDBMS Installation? Known Issues GenericNote 883702.1 $ORACLE_HOME/lib32 does not exist in Oracle database 11gR2. Why?Note 1245784.1 Installing Component using Installer from 11.2 ORACLE_HOME Fails with OUI-10150Note 1091063.1 sqlplus /nolog is failing with errors : SP2-1503 SP2-0152Note 1207849.1 Install Error: String Index Out Of Range: -1 Error MessagesNote 1367658.1 11.2 OUI does not use the $TEMP variable for CV_DESTLOC during a single instance installationNote 1307782.1 Oracle 11.2 Installation Fails If OS Users Password Contains '%'Note 1162824.1 Installing Oracle 10.2.0.1> On AIX Hangs At Prereqs CheckNote 1127135.1[INS-00001] Unknown irrecoverable error installing 11.2Note 763143.1 Explanation and Options To Handle "Checking operating system version: must be ... Failed" OUI errorsNote 1268391.1 Installing 11gR2 Fails With [INS-30060] Check For Group Existence FailedNote 1301113.1 "Invalid Entry Size" Error During Oracle Database InstallationNote 1265321.1 Custom Client Installation 11.2 fails with java.io.IOException: : not foundNote 1292347.1 [Ins-10002] Inventory Initialization Failed During 11gr2 Client InstallNote 1403568.1 Install Of 11GR2 Client : Error In EqualPathStrs :The Filename, Directory Name, Or Volume Label Syntax Is IncorrectNote 849336.1 Unable to launch OUINote 1555453.1 Executing "relink all" resets permission of extjob, jssu, oradism, externaljob.oraSolaris GeneralNote 1307041.1 Certification Information for Oracle Database on Solaris SPARC 64Note 1307389.1 Certification Information for Oracle Database on Oracle Solaris x86-64Note 743042.1 Requirements for Installing Oracle 11gR1 RDBMS on Solaris 10 SPARC 64-bitNote 971464.1 FAQ - 11gR2 requires Solaris 10 update 6 or greaterNote 964976.1 Requirements for Installing Oracle 11gR2 RDBMS on Solaris 10 SPARCNote 429191.1 Kernel setup for Solaris 10 using project filesNote 317257.1 Running Oracle Database in Solaris 10 Containers - Best PracticesKnown Issues on SolarisNote 848700.1 11gR1 on Solaris failing at relink phase with errors like "ld fatal relocation error" and then "ld fatal library -lclntsh not found"Note 966929.1 11gR2 INSTALL ERRORS on Solaris 10Note 989727.1 11GR2 INSTALL ON SOLARIS FAILS MANY PREREQUISITE CHECKSNote 969497.1 During An 11gR2 (11.2) Installation On Solaris, OUI Performs A Prerequisite Check For The Optional Patch 124861-15 And Fails If It Is Not InstalledNote 1325973.1 [INS-30060] Check for group existence failed - While installing 11gR2 on SolarisNote 1064838.1 Prerequisite Checks Fail With PRVF-7532 And PRVF-9809 Errors When Installing Oracle11gR2 (11.2) 32-bit Client Software On SPARC SolarisNote 974371.1 After Installing The 11.2.0.1> 32-bit Client Software On SPARC Solaris, OPatch and OUI Fail With ELFCLASS32 Errors For liboraInstaller.soNote 867562.1 11gR1 OUI Error "fatal libCrun.so.1> open failed No such file or directory"Note 852241.1 Installation of 11G fails on relink with INFO ld fatal library -lumem not foundNote 1521751.1 11.2.0.3 Installation On Solaris Fails During Prerequisite Checks Reporting STATUS=OPERATION_FAILEDNote 1539888.1 PRVF-7530 Error During Oracle 11.2.0.3 Installation on SolarisNote 1551669.1 11.2.0.3 installation fails with : /usr/ccs/bin/as: not found [No such file or directory] on Solaris 11 or Solaris 10LinuxGeneralORACLE Validated ConfigurationsNote 1307056.1 Certification Information for Oracle Database on Linux x86Note 1304727.1 Certification Information for Oracle Database on Linux x86-64Note 851598.1 Master Note 386391.1 Defining a "default RPMs" installation of the SLES OSNote 401167.1 Defining a "default RPMs" installation of the Oracle Enterprise Linux (OEL) OSNote 728346.1 Linux OS Installation with Reduced Set of Packages for Running Oracle Database Server Note 787705.1 How to identify correct Oracle Database software to install on Linux platformNote 1315914.1 Install 11gr2 On Sles11 - Packages Required Not Mentioned In Installation GuideNote 1350000.1 Database Client or Database Server Install on Red Hat 6 (RHEL6)Note 444084.1 Multiple gcc / g++ Versions in LinuxNote 761261.1 Recommended metalink articles for Oracle Database 10gR2 installation on zLinux (s390x)Note 415182.1 DB Install Requirements Quick Reference - zSeries based LinuxNote 341507.1 Oracle Database Server on Linux on IBM POWERNote 786995.1 Troubleshooting The Relink Errors When Missing RPMs on Linux x86-64Note 377217.1 What should the value of LD_ASSUME_KERNEL be set to for Linux?Known Issues On LinuxNote 1071005.1 HOTSPOT ERROR DURING 32-BIT 11GR2 CLIENT INSTALL ON 64-BIT (X86_64) SUSE (VMWARE)Note 1313504.1 Error in invoking 'ntcontab.o' of makefile while installaing 11gr2 client on Linux x86-64 bit serverNote 308788.1 While Installing Oracle Database on SUSE SLESx x86-64 Linking Errors Occur and NetCA/DBCA Fail With Error: UnsatisfiedLinkError exception loading native library: njni10Note 468977.1 UnsatisfiedLinkError exception loading native library: njni11Note 1063952.1 11gR2 Database Server install fails with [INS-13001] Environment does not meet minimum requirementsNote 1220843.1 Getting __stack_chk_fail@GLIBC_2.4 makefile errors during Oracle 11.2 Install on LinuxNote 443617.1 "Exception java.lang.UnsatisfiedLinkError: ../jre/1.4.2/lib/i386/libawt.so: libXp.so.6: cannot open shared object file: No such file or directory occurred" on RHEL5/OEL5 for 10.2 x86 or x86_64 installNote 1543787.1 11.2.0.3 Installation failed on RHEL 6 with the error libraries: libmpfr.so.1: cannot open shared object file: No such file or directoryAIXGeneralNote 1307544.1 Certification Information for Oracle Database on IBM AIX on Power systemsNote 282036.1 Minimum Software Versions and Patches Required to Support Oracle Products on IBM Power SystemsNote 889464.1 Is Oracle 10g And 11g Database Compatible With AIX Version 6.1>Note 417451.1 How To Determine Whether an APAR has been Fixed in AIX Version/Maintenance LevelNote 739963.1 Using AIX commands genld, genkld and slibclean to avoid library file locking errors(libjox)Note 458403.1 Why AIXTHREAD_SCOPE should be set to 'S' on AIXNote 201622.1 Is It Still NecessaryTto Run the Rootpre.sh Script When Installing Oracle on AIX 5L and above?Known Issues On AIXNote 832074.1 AIX 6.1> "Dependent module /usr/lib/libXpm.a(shr_64.o) could not be loaded" Error Attempting To Execute 11.1>.0 runInstallerNote 879102.1 Stand Alone 11.1>.0.6 Silent Installation on AIX 5.3 Fails On Prerequisite Check For rsct.basic.rte(0.0) and rsct.compat.clients.rte(0.0)Note 980602.1 The Installer (OUI) Detects Processes Running on ORACLE_HOME on AIX 6.1> TL04 SP1Note 985563.1 AIX: ERROR IN INVOKING TARGET 'LINKS PROC GEN_PCSCFG' OF MAKEFILE, ld: 0711-593 SEVERE ERROR: Symbol C_BSTATNote 1327608.1 INS-13001 Environment does not meet minimum requirements when installing 11.2.0.2 on AIX 7.1>Note 859842.1 Database Server Installation on AIX 6.1> Error: OUI-18001: 'AIX Version 6100.01' is not supportedNote 1364239.1 runInstaller Failed With "OUI: 0403-006 Execute permission denied" While Installing 11.2.0.x On AIXNote 747664.1 Oracle Client 11.1>.0 NETCA Error "UnsatisfiedLinkError exception loading native library: njni11" While Installing On Aix 5.3Note 1525845.1 11gR2 Install fails with error OUI-10164:The Ownership For The File GeneralQueries.jar Could Not Be Changed To .HPGeneralNote 1307026.1 Certification Information for Oracle Database on HP-UX PA-RISC PlatformNote 1307745.1 Oracle Database Support on the Itanium Processor ArchitectureKnown Issues on HPNote 469569.1 Failure On Prerequisite Checks For 11gR2 Installation on HP-UXNote 725913.1 Error OUI-15038 At Pre-Requisite Check Stage During Oracle 11g InstallNote 1183545.1 OUI 10.2.0.4 and OUI 10.2.0.5 raise OUI-15038 "Unable to execute rule 'CheckCompatibility' .. [For input string: "11_11"]Note 471476.1 32 Bit Libraries Are Not Installed With Oracle11g (11.1>.0.6) On HP-UX ItaniumNote 287428.1 Oracle Universal Installer has detected that there are processes running in the currently selectedNote 819394.1 HP-UX: Sqlplus: ORA-12549: TNS:operating system resource quota exceededNote 1194973.1 Failure On Prerequisite Checks For 11gR2 Installation on HP-UXWINDOWSGeneralNote 1307189.1 Certification Information for Oracle Database on Microsoft Windows x86 (32-bit)Note 1307195.1 Certification Information for Oracle Database on Microsoft WindowsNote 952302.1 Is Microsoft Windows 7 certified to install/run Oracle Database Server/Client ? Note 1316079.1 How To Install Oracle 10.2.0.5 Client Software on Windows 7 (32 or 64-bit) and Windows 2008 R2 (64-bit) Note 1199616.1 11.1>.0.7 Client can not be installed on Windows 2008 R2 or Windows 7Note 1139705.1 Install the Oracle Client software into a directory that contains spaces in the nameNote 1356362.1 List Of Components Included With Administrator And Runtime Installation Options Of The 11.2.0.2 Database Client On WindowsNote 1318795.1 How to know if Oracle client, DB, or JAVA installed is 32-bit or 64-bit on Windows OSNote 1433144.1 What Is The Windows Service "Oracleremexecservice"?Note 443813.1 Check List For Oracle On Windows.Note 137200.1 Checklist when Oracle Universal Installer (OUI) fails for WindowsNote 436299.1 Common reasons for OUI failures on the MS Windows platformsNote 161549.1 Oracle Database Server and Networking Patches for Microsoft PlatformsNote 816893.1 How to Recreate the Windows Registry for Oracle, After reloading the Windows OSNote 432713.1 How To Create a Service using oradim in Windows VistaKnown Issues on WindowsNote 1313463.1 OUI crashes while installing Oracle Client 11gR2 64 bit on Window 7 SP1 Note 1312587.1 Cannot Install Oracle Client 11.2.0.1> On Windows 7 PCNote 1292785.1 Oracle Client 11gR2 32bit Installation Crashes on Microsoft Windows x64 (64-bit)Note 464265.1 SQLPLUSW.EXE Missing In 11.1>.0.6 Windows Client Installation Using Administrator OptionNote 1328667.1 Psapi.Dll Error When Installing 32 bit 10.2.0.1>Oracle Client on Windows 7 64 Bit Note 1310680.1 Problems Installing Oracle 10g Client On Windows Operating SystemNote 1185831.1 Unable To Install 10.2 64-Bit Client On Windows 7 X64Note 810374.1 Oracle Client 10.2.0.1> Installation Fails With Directory Not Writable ErrorNote 1292921.1 While Installing 11.2, OUI Fails With "Error in starting the service. The service OracleMTSRecoveryService was not found"Note 221545.1 Oracle Installation Issues in Windows PlatformsNote 418479.1 file in use Errors, when applying patches on WindowsNote 1133495.1 Installation of 11gR2 on Windows Fails Checking RequirementsNote 1088324.1 11gR2 Deinstall fails on Windows 7Note 1530206.1 Cannot Install Oracle Client 11.2 64 Bit And Oracle Client 32 Bit On Same Windows 64 Bit ServerScriptsNote 189256.1 UNIX Script to Verify Installation Requirements for Oracle 8.0.5 to 9.2 versions of RDBMSNote 250262.1 RDA 4 - Health Check / Validation Engine GuideSilent InstallNote 136660.1 Working Response files for a silent install of Oracle Server on MS-Windows- Part 1 of 3Note 136661.1 Database Configuration Response file for performing silent install - Part 2 of 3Note 136662.1 Network Configuration Reponse file for performing silent install - Part 3 of 3Note 388451.1 How to create a Response File for Silent Installation?Note 1225194.1 How To Create a Response File and Perform a Silent Install with the 11.2 Oracle Client InstallerNote 1233026.1 Unconditional Exit During 11gR2 Silent Client Install Using Custom ResponsefileNote 434315.1 How Can You Install Oracle Without Using The Oracle Universal Installer( runInstaller) GUI and Xserver especially When Doing A Remote Installation ?Note 405124.1 How to Prevent a Silent Install from Prompting for User Input to Run the 'root.sh' ScriptNote 808275.1 How to install Oracle Database Patchset software in Silent mode without using response fileNote 782919.1 How to install Oracle Database Server software silently with customized listener configurationNote 885643.1 How to install 11gR2 Database/Client Software in Silent mode without using response fileNote 782918.1 How to install/deinstall Oracle Database Server software silently without response fileNote 782923.1 How to install/deinstall Oracle Database Client software silently without response fileNote 782924.1 How to install Oracle Database Client software along with netca in silent modeNote 435680.1 Response File Installation of Database Server 10.2 (Suppressed or Silent mode)Deinstall/Removing InstallationsGeneral InformationNote 208116.1 How to remove all traces of ORACLE_HOME on UNIXNote 275493.1 Removing 10g Database and Software from AIX, HP-UX, Linux, Sun and Tru64 PlatformsNote 1363753.1 Behaviour of the Oracle De-install/Deinstall/Uninstall Utility in 11gR2Note 886749.1 Oracle 11gR2 Deinstall And Deconfig Tool OptionsNote 883743.1 How To Deinstall/Uninstall Oracle Home In 11gR2 Note 1070610.1 How To De-install Oracle Home Using runinstallerNote 435219.1 How To Clean Up The Inventory After Deleting The Oracle Home Manualy Using OS Commands("rm-rf " or other)?Note 208256.1 WIN: How to Remove a Single ORACLE_HOME and Its Traces on Microsoft Windows PlatformsNote 124353.1 Manually Removing all Oracle Components on Microsoft Windows PlatformsNote 1069034.1 11.2: How to Manually Remove Oracle Server Software on Microsoft Windows PlatformsKnown ProblemsNote 885609.1 11gR2 OUI warning: Please run the command '/deinstall/deinstall' to deinstall this OracleHomeNote 1069028.1 Deinstall utility on 11.2 removes multiple ORACLE_HOMEs Note 1271661.1 Deinstall 11.2 32-bit Home Inadvertanly Removes 64-bit Homes and Vice VersaNote 1067622.1 11.2 DEINSTALLER REMOVING LISTENERS RUNNING FROM OTHER ORACLE_HOMESNote 1322509.1 11.2 Database Corrupted After Use of $ORACLE_HOME/deinstall/deinstall ScriptNote 753929.1 Errors During Silent 11.1>.0.6 DeinstallationNote 1128464.1 Uninstall Of Oracle 11gR2 Home On Windows Does Not Recognize Non-English ResponseNote 1374044.1 When deinstalling 11gR1 ,OUI greyed out the Remove button and getting message - You cannot deinstall at this timeNote 1519494.1 The deinstall tool cannot determine the home type needed to deconfigure the selected homeDiagnostic ToolsNote 780551.1 How To Diagnose Linking Errors During Base Install of Oracle RDBMSNote 757964.1 Oracle Universal Installer ( OUI ) Error Messages and Solution Reference ListNote 1812.1 TECH: Getting a Stack Trace from a CORE fileGeneral InformationNote.738771.1 How to Log Good Service Request for Relink issues?Note.736819.1 How to Log Good Service Request for Oracle Universal Installer (OUI) issuesUsing My Oracle Support EffectivelyNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support ServiceGeneric LinksNote 1351051.1 Information Center: Install and Configure Database Server/Client InstallationsNote 1194734.1 Where do I find that on My Oracle Support (MOS) [Video]Note 742060.1 Release Schedule of Current Database ReleasesNote 268895.1 Oracle Database Server Patchset Information, Versions: 8.1>. 7 to 11.2.0Note 783141.1 Reference List of Critical Patch Update Availability(CPU) and Patch Set Update (PSU) Documents For Oracle Database and Fusion Middleware ProductNote 854428.1 Patch Set Updates for Oracle ProductsNote 1061295.1 Patch Set Updates - One-off Patch Conflict ResolutionInformation on Critical Patch Updates (CPU's)Oracle Database documentation, 11g Release 2 (11.2)Installation manuals and platform-specific books are included in this link.

For the most current information, please refer to : (Doc ID 1157463.1) Master Note For Oracle Database Client Installation APPLIES TO: Oracle Database - Standard Edition - Version 9.2.0.1 and laterOracle...

Master Notes

Master Note of Linux OS Requirements for Database Server

For the most current information, please refer to :(Doc ID 851598.1) Master Note of Linux OS Requirements for Database Server APPLIES TO:Oracle Database - Enterprise Edition - Version 9.2.0.1 and laterOracle Database - Standard Edition - Version 9.2.0.1 and laterLinux x86-64Linux ItaniumIBM: Linux on System zLinux x86Generic LinuxIBM: Linux on POWER SystemsPURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support  articles with respect to Linux OS Requirements for Oracle Database Software Installation.The vast majority of software installation and relinking problems on Linux are caused by missing OS requirements.If you are encountering a software installation or relinking problem, it is vital that you ensure your OS meets all of the minimum requirements documented in the appropriate article below before creating a new SR.SCOPEOnly Linux articles are referenced in this article.  For an "all-in-one" article that covers OS requirements for most major OS platforms, please refer to...Document 169706.1  Oracle Database ... Installation and Configuration Requirements Quick ReferenceWhere a My Oracle Support article for a given version/platform is "Not available", you should refer to the Installation Guide and the article referenced above.DETAILS Red Hat Enterprise Linux (RHEL)Document 376183.1 Defining a "default RPMs" installation of the RHEL OSQuick Links for RHELRHEL6:   x86_64 zLinuxRHEL5:   x86 x86_64 Itanium zLinux Power RHEL4:   x86 x86_64 Itanium zLinux Power RHEL3:   x86 x86_64 Itanium zLinux PowerRHEL6 x86_64: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Not certified, not supported, not planned.11.1.0 - Not certified, not supported, not planned.11.2.0 - Document 1441282.1 Requirements for Installing Oracle 11gR2 RDBMS on RHEL6 or OL6 64-bit (x86-64)12.1.0 - Document 1529864.1 Requirements for Installing Oracle Database 12.1 on RHEL6 or OL6 64-bit (x86-64)RHEL6 zLinux: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Not certified, not supported, not planned.11.1.0 - Not certified, not supported, not planned.11.2.0 - Document 1470834.1 Requirements for Installing Oracle 11.2.0.3/11.2.0.4 RDBMS on RHEL 6 on zLinux (s390x)12.1.0 - Document 1574413.1 Requirements for Installing Oracle Database 12c Release 1 on RHEL 6 on IBM: Linux on System z (s390x)RHEL5 x86: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 419646.1 Requirements For Installing Oracle 10gR2 On RHEL 5 (x86)11.1.0 - Document 438765.1 Requirements for Installing Oracle 11gR1 32bit RDBMS on RHEL 511.2.0 - Document 880936.1 Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on 32-bit x86RHEL5 x86-64: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 421308.1 Requirements For Installing Oracle10gR2 On RHEL 5 (x86_64)11.1.0 - Document 438766.1 Requirements for Installing Oracle 11gR1 RDBMS on RHEL 5 on AMD64/EM64T11.2.0 - Document 880989.1 Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 5 on AMD64/EM64T12.1.0 - Document 1529433.1 Requirements for Installing Oracle Database 12.1 on RHEL5 or OL5 64-bit (x86-64)RHEL5 Itanium: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 748378.1 Requirements for Installing Oracle 10gR2 RDBMS on RHEL 5 on Linux Itanium (ia64)RHEL5 zLinux:9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 741646.1 Requirements for Installing Oracle 10gR2 RDBMS on RHEL 5 on zLinux (s390x)11.2.0 - Document 1306889.1 Requirements for Installing Oracle 11gR2 on RHEL 5 on IBM: Linux on System z (s390x)RHEL5 Power:10.2.0 - Document 341507.1 Oracle Database Server on Linux on IBM POWERRHEL4 x86: 9.2.0 - Document 303859.1 Requirements for Installing Oracle 9iR2 on RHEL 410.1.0 - Document 392940.1 Requirements for Installing Oracle 10.1.0.x RDBMS on RHEL 4 x86 platform10.2.0 - Document 343431.1 Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 x86 platform11.1.0 - Document 430653.1 Requirements for Installing Oracle 11gR1 32-bit on RHEL 411.2.0 - Document 880211.1 Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 4 x86RHEL4 x86-64: 9.2.0 - Document 353529.1 Requirements for Installing Oracle 9iR2 64-bit on RHEL 4 x86-64 (AMD64/EM64T)10.1.0 - Document 390900.1 Requirements for Installing Oracle 10g (10.1.0.x) RDBMS on RHEL 4 on AMD64/EM64T (Linux x86-64)10.2.0 - Document 339510.1 Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 on AMD64/EM64T11.1.0 - Document 437123.1 Requirements for Installing Oracle 11gR1 RDBMS on RHEL 4 on AMD64/EM64T11.2.0 - Document 880942.1 Requirements for Installing Oracle 11gR2 RDBMS on RHEL (and OEL) 4 on AMD64/EM64TRHEL4 Itanium: 9.2.0 - Not available10.1.0 - Not available10.2.0 - Not availableRHEL4 zLinux: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 420382.1 Requirements for Installing Oracle 10gR2 RDBMS on RHEL 4 on zLinux (s390x)RHEL4 Power:10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 341507.1 Oracle Database Server on Linux on IBM POWERRHEL3 x86: 9.2.0 - Document 252217.1 Requirements for Installing Oracle 9iR2 32-bit on RHEL 310.1.0 - Document 394360.1 Requirements for Installing Oracle 10g 32-bit on RHEL 310.2.0 - Not available11.1.0 - Not certified, not supported, not planned.11.2.0 - Not certified, not supported, not planned.RHEL3 x86-64: 9.2.0 - Document 308588.1 Requirements for Installing Oracle 9iR2 x86_64 on RHEL 310.1.0 - Document 351679.1 Requirements for RPM Arch. for 10g x86_64 on RHEL 310.2.0 - Document 353735.1 Requirements for RPM Arch. for 10gR2 x86_64 on RHEL 311.1.0 - Not certified, not supported, not planned.11.2.0 - Not certified, not supported, not planned.RHEL3 Itanium: 9.2.0 - Not available10.1.0 - Not available10.2.0 - Not available11.1.0 - Not certified, not supported, not planned.11.2.0 - Not certified, not supported, not planned.RHEL3 zLinux: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Not certified, not supported, not planned.11.1.0 - Not certified, not supported, not planned.11.2.0 - Not certified, not supported, not planned.RHEL3 Power:10.2.0 - Not certified, not supported, not planned.11.1.0 - Not certified, not supported, not planned.11.2.0 - Not certified, not supported, not planned. SuSE Linux Enterprise Server (SLES)Document 386391.1 Defining a "default RPMs" installation of the SLES OSQuick Links for SLESSLES11:   x86 x86_64 zlinuxSLES10:   x86 x86_64 zLinux Power SLES 9:   x86 x86_64 zLinux Power SLES11 x86: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 842649.1 Requirements for Installing Oracle 10gR2 32-bit (x86) on SLES 1111.1.0 - Document 849583.1 Requirements for Installing Oracle 11gR1 32-bit (x86) on SLES 1111.2.0 - Document 881025.1 Requirements for Installing Oracle 11gR2 32-bit (x86) on SLES 11SLES11 x86_64: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 956194.1 Requirements for Installing Oracle 10gR2 64-bit (AMD64/EM64T) on SLES 1111.1.0 - Document 1081555.1 Requirements for Installing Oracle 11gR1 64-bit (AMD64/EM64T) on SLES 1111.2.0 - Document 881044.1 Requirements for Installing Oracle 11gR2 64-bit (AMD64/EM64T) on SLES 1112.1.0 - Document 1519770.1 Requirements for Installing Oracle Database 12.1 64-bit (AMD64/EM64T) on SLES 11SLES11 zLinux:10.2.0 - Document 1290360.1 Requirements for Installing Oracle 10gR2 RDBMS on SLES 11 on zLinux (s390x)11.2.0 - Document 1290644.1 Requirements for Installing Oracle 11gR2 on SLES 11 on IBM: Linux on System z (s390x)12.1.0 - Document 1574414.1 Requirements for Installing Oracle Database 12c Release 1 on SLES 11 on IBM: Linux on System z (s390x)SLES10 x86: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 387137.1 Requirements for Installing Oracle 10gR2 32-bit (x86) on SLES 1011.1.0 - Document 452818.1 Requirements for Installing Oracle 11gR1 32-bit on SLES 1011.2.0 - Document 763386.1 Requirements for Installing Oracle 11gR2 32-bit on SLES 10 (x86)SLES10 x86-64: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Document 373681.1 Requirements for Installing Oracle 10gR2 64-bit (AMD64/EM64T) on SLES 1011.1.0 - Document 457143.1 Requirements for Installing Oracle 11gR1 64-bit (AMD64/EM64T) on SLES 1011.2.0 - Document 884435.1 Requirements for Installing Oracle 11gR2 64-bit (AMD64/EM64T) on SLES 10SLES10 s390x (zLinux): 9.2.0 - Not available10.1.0 - Not available10.2.0 - Document 1082253.1 Requirements for Installing Oracle 10gR2 RDBMS on SUSE SLES 10 on zLinux (s390x)11.1.0 - Not certified, not supported, not planned.SLES10 Power: 10.2.0 - Document 341507.1 Oracle Database Server on Linux on IBM POWERSLES9 x86: 9.2.0 - Document 427976.1 Requirements for Installing Oracle 9iR2 32-bit on SLES 910.1.0 - Not available10.2.0 - Document 400429.1 Requirements for Installing Oracle 10gR2 32-bit on SLES 911.1.0 - Not certified, not supported, not planned.SLES9 x86-64: 9.2.0 - Document 361169.1 Requirements for Installing Oracle 9iR2 64-bit on SuSE SLES 9 x86_64 (AMD64/EM64T)10.1.0 - Not available10.2.0 - Document 365607.1 Requirements for Installing Oracle 10gR2 RDBMS on SLES 9 on AMD/EM64T11.1.0 - Not certified, not supported, not planned.SLES9 s390x (zLinux): 9.2.0 - Document 270577.1 Installing Oracle 9i on IBM z/Series - SLES8/910.1.0 - Not available10.2.0 - Document 431443.1 Requirements for Installing Oracle 10gR2 RDBMS on SLES 9 on zLinux (s390x)11.1.0 - Not certified, not supported, not planned.SLES9 Power: 10.2.0 - Document 341507.1 Oracle Database Server on Linux on IBM POWER Oracle Enterprise Linux (OEL)Document 401167.1 Defining a "default RPMs" installation of the Oracle EL OSQuick Links for OELOL6:    x86_64OEL5:   x86 x86_64 OEL4:   x86 x86_64 OL6 x86-64:11.2.0 - Identical to RHEL6 x86-64.  See "RHEL6 x86-64:" section above.OEL5 x86: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Identical to RHEL5 x86. See "RHEL5 x86:" section above.11.1.0 - Identical to RHEL5 x86. See "RHEL5 x86:" section above.11.2.0 - Identical to RHEL5 x86. See "RHEL5 x86:" section above.OEL5 x86-64: 9.2.0 - Not certified, not supported, not planned.10.1.0 - Not certified, not supported, not planned.10.2.0 - Identical to RHEL5 x86-64. See "RHEL5 x86-64:" section above.11.1.0 - Identical to RHEL5 x86-64. See "RHEL5 x86-64:" section above.11.2.0 - Identical to RHEL5 x86_64. See "RHEL5 x86_64:" section above.OEL4 x86: 9.2.0 - Identical to RHEL4 x86. See "RHEL4 x86:" section above.10.1.0 - Identical to RHEL4 x86. See "RHEL4 x86:" section above.10.2.0 - Identical to RHEL4 x86. See "RHEL4 x86:" section above.11.1.0 - Identical to RHEL4 x86. See "RHEL4 x86:" section above.11.2.0 - Identical to RHEL4 x86. See "RHEL4 x86:" section above.OEL4 x86-64: 9.2.0 - Identical to RHEL4 x86-64. See "RHEL4 x86-64:" section above.10.1.0 - Document 436800.1 Requirements for Installing Oracle 10g RDBMS on OEL 4 on AMD64/EM64T 10.2.0 - Document 418890.1 Requirements for Installing Oracle 10gR2 RDBMS on OEL 4 update 4 on AMD64/EM64T       - Document 435900.1 Requirements for Installing Oracle 10gR2 RDBMS on OEL 4 update 5 on AMD64/EM64T11.1.0 - Identical to RHEL4 x86-64. See "RHEL4 x86-64:" section above.11.2.0 - Identical to RHEL4 x86_64. See "RHEL4 x86_64:" section above.Using My Oracle Support Effectively Document 868955.1 My Oracle Support Health Checks Catalog Document 166650.1 Working Effectively With Global Customer Support Document 199389.1 Escalating Service Requests with Oracle Support ServicesGeneric LinksDocument 854428.1 Patch Set Updates for Oracle Products Document 1061295.1 Patch Set Updates - One-off Patch Conflict Resolution Document 881382.1 Critical Patch Update October 2009 Patch Availability Document for Oracle Products Document 967472.1 Critical Patch Update January 2010 Patch Availability Document for Oracle Products Document 1060989.1 Critical Patch Update April 2010 Patch Availability Document for Oracle Products Document 756671.1 Oracle Recommended Patches -- Oracle Database Document 268895.1 Oracle Database Server Patchset Information, Versions: 8.1. 7 to 11.2.0 Document 161549.1 Oracle Database Server and Networking Patches for Microsoft Platforms

For the most current information, please refer to : (Doc ID 851598.1) Master Note of Linux OS Requirements for Database Server APPLIES TO: Oracle Database - Enterprise Edition - Version 9.2.0.1...

Master Notes

Master Note For Oracle Database Downgrade

    For the most current information, please refer to :(Doc ID 1151427.1) Master Note For Oracle Database Downgrade APPLIES TO:Oracle Database - Enterprise Edition - Version 8.1.7.1 and laterOracle Database - Standard Edition - Version 8.1.7.1 and laterInformation in this document applies to any platform.PURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes with respect to Oracle Database Downgrade.This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest, within DOWNGRADE.This includes the following categories:General InformationHow toDiagnostic ToolsKnown IssuesUsing My Oracle Support EffectivelyGeneric LinkSCOPEThis note guides you through the process of downgrading a database back to the previous Oracle Database release.This note applies to the following versions of these products:Oracle Server - Enterprise Edition - Version: 8.1.7.1 to 8.1.7.xOracle Server - Enterprise Edition - Version: 9.0.1.1 to 9.0.1.xOracle Server - Enterprise Edition - Version: 9.2.0.1 to 9.2.0.xOracle Server - Enterprise Edition - Version: 10.2.0.1 to 10.2.0.xOracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.1.0.xOracle Server - Enterprise Edition - Version: 11.2.0.1 to 11.2.0.xOracle Server - Enterprise Edition - Version: 12.1.0.1 to 12.1.0.xDETAILSGeneral InformationThe documentation listed below help you in understanding the compatibility, concepts and usage of Oracle Database Downgrade:Note 551141.1 Database Server Upgrade/Downgrade Compatibility MatrixNote.444709.1 COMPATIBLE Initialization Parameter and Upgrade/Downgrade in 11gDowngrade Guides:Oracle Database Upgrade Guide 12c Release 1 (12.1)Part Number E17642-13Oracle Database Upgrade Guide 11g Release 2 (11.2)Part Number E10819-02Oracle Database Upgrade Guide 11g Release 1 (11.1)Part Number B28300-03Oracle Database Upgrade Guide 10g Release 2 (10.2)Part Number B14238-02Oracle Database Upgrade Guide 10g Release 1 (10.1)Part Number B10763-02Oracle9i Database Migration Release 2 (9.2)Part Number A96530-02How toThe documentation listed below help you in understanding the concepts and usage of Oracle Database Downgrade:Note 334598.1 How to rollback a patchsetNote 208237.1 How to Downgrade a Database Using Export/Import?Note 1516622.1 How To Downgrade Oracle Database 12c Release 1 (12.1) To Previous VersionsNote 883335.1 How To Downgrade From Database 11.2 To Previous Release (includes 11.2.0.3-11.2.0.1)Note 443890.1 Complete Checklist For Downgrading The Database From 11.1 To Lower ReleasesNote 398372.1 How to Downgrade from Oracle RDBMS 10gR2?Note 345123.1 How to Downgrade from Oracle RDBMS 9.2.0.8/9.2.0.7 to Lower Patchset Releases?Note 391879.1 How to Rollback Oracle Database Patchset 9.2.0.8.0 (applicable to both Single instance and RAC) ?Note 208201.1 Downgrading from 9.2.0.1 to 9.0.1/8.1.7Note 876353.1 How To Restore The Oracle Enterprise Manager Data To Downgrade The Single Instance Database To Previous/Source Release ?Note 783643.1 Downgrading Database Without Executing catdwgrd.sql (11.1.0.x to 10.2.0.x/10.2.0.x to 10.2.0.x)Note 871665.1 How To Restore A Database Back To The Source Release After Upgrading It To Oracle 11gR2 Using DBUANote 883855.1 Overview of the New Feature 'emdwgrd' Utility in DBControlKnown IssuesThe articles listed below will help you solving the Downgrade issues:Note 1520209.1 QT_*BUFFER Views Invalid after downgrade from Oracle Database 12.1 (12c)Note 1107285.1 catdwgrd.sql fails with xds_acl ORA-00942 table or view does not existNote 435536.1 "ORA-01722 invalid number" During Upgrade/DowngradeNote 1446710.1 Catrelod.sql failed with ORA-00600: internal error code, arguments: [qcisSetPlsqlCtx:tzi init] And Alert log showing : Failed to find timezone data file # 14 (DST_4)Note 1367333.1 Downgrade From 11.2.0.3 To 11.2.0.2 Fails At Catrelod.Sql with error: ORA-20000: Upgrade from version 11.2.0.2.0 cannot be downgraded to version 11.2.0.1Note 1292035.1 Ora-20000 : Direct Downgrade Of EM Database Control Not Supported Despite Sysman User Is DroppedNote 391656.1 Downgrade Not Supported With ORA-20000 And ORA-06512 At "SYS.VERSION_SCRIPT"Note 388604.1 ORA-00201 while downgrading from 10gR2 to 10gR1 or 9iR2Note 948710.1 Troubleshooting ORA-201 ORA-202Note 433354.1 After Downgrading From 10.2 To 9.2 Startup Migrate Returns ORA-402 ORA-405 ErrorsNote 338199.1 Downgrading A 10.1.0 DatabaseNote 1016089.102 ORA-00402, ORA-00405, when Downgrading from 8.1.5 to 8.0.5Note 9967451.8 Bug 9967451 - Downgrade catrelod fails (ORA-600 [qcisSetPlsqlCtx:tzi init])Using My Oracle Support EffectivelyNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support ServicesNote 732897.1 How to Log Good Service Requests for Database Upgrade/Migration IssuesGeneric LinksNote 854428.1 Patch Set Updates for Oracle ProductsNote 1061295.1 Patch Set Updates - One-off Patch Conflict ResolutionNote 268895.1 Oracle Database Server Patchset Information, Versions: 8.1.7 to 11.2.0Note 161549.1 Oracle Database Server and Networking Patches for Microsoft PlatformsNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryInformation on Critical Patch Updates (CPU's)

For the most current information, please refer to : (Doc ID 1151427.1) Master Note For Oracle Database Downgrade APPLIES TO: Oracle Database - Enterprise Edition - Version 8.1.7.1 and laterOracle...

Master Notes

Master Note For Oracle Database Server Installation

    For the most current information, please refer to :(Doc ID 1156586.1) Master Note For Oracle Database Server Installation PURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes with respect to Oracle Database Installation.This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest, within INSTALLATION.This includes the following categories:* Certification* Installation Guides* OS General Information and Known Issues* Scripts* Silent Install* Diagnostic Tools* General Information* Using My Oracle Support Effectively* Generic LinkSCOPEThis note guides you through the process of Oracle Database Installation.This note applies to the following versions of these products:Oracle Server - Enterprise Edition - Version: 8.1.7.1 to 8.1.7.xOracle Server - Enterprise Edition - Version: 9.0.1.1 to 9.0.1.xOracle Server - Enterprise Edition - Version: 9.2.0.1 to 9.2.0.xOracle Server - Enterprise Edition - Version: 10.2.0.1 to 10.2.0.xOracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.1.0.xOracle Server - Enterprise Edition - Version: 11.2.0.1 to 11.2.0.xDETAILSAsk Questions, Get Help, And Share Your Experiences With This ArticleWould you like to explore this topic further with other Oracle Customers, Oracle Employees, and Industry Experts?Click here to join the discussion where you can ask questions, get help from others, and share your experiences with this specific article.Discover discussions about other articles and helpful subjects by clicking here to access the main My Oracle Support Community page for Database Install/Upgrade.CertificationFor the latest Database Certification always refer to My Oracle Support (MOS) Certify.At the same time the documentation listed below assists in understanding the Oracle Database Certification:Note 1298096.1 Master Note For Database and Client CertificationNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryNote 207303.1 Client / Server / Interoperability Support Between Different Oracle VersionsNote 1307745.1 Oracle Database Support on the Itanium Processor ArchitectureNote 464754.1 Certified Software on Oracle VMNote 1306539.1 Core Oracle Database Certification InformationInstallation GuidesOracle Database 12c Release 1 (12.1)Installation manuals and platform-specific books are included in this link.Oracle Database 11g Release 2 (11.2)Installation manuals and platform-specific books are included in this link.Oracle Database 11g Release 1 (11.1)Installation manuals and platform-specific books are included in this link.Oracle Database 10g Release 2 (10.2)Installation manuals and platform-specific books are included in this link.Oracle Database 10g Release 1 (10.1)Installation manuals and platform-specific books are included in this link.Oracle Database 9i Release 2 (9.2)Installation manuals and platform-specific books are included in this linkOracle Database 9i Release 1 (9.0.1)Installation manuals and platform-specific books are included in this linkOS General Information and Known IssuesMost of the installation and relinking issues appear if you don't satisfy the minimum requirements for the installation. Therefore if you are raising any SR for installation issue it is better to ensure that you have all these requirements mentioned in the below articles before raising the SR.The documentation listed below help you in understanding the OS Requirements for Oracle Database Installation:GenericSolarisLinuxAIXHPWINDOWSGenericGeneralNote 1520299.1 Master Note For Oracle Database 12c Release 1 (12.1) Database/Client Installation/Upgrade/Migration Standalone EnvironmentNote 1483380.1 Changes For Oracle Database 12.1 Standalone InstallationNote 1555012.1 Install/Deinstall/Cloning Assistant: Oracle Database/ClientNote 1351051.1 Information Center: Install and Configure Database Server/Client InstallationsNote 1476075.1 FAQ on downloading 9i, 10g, 11g softwareNote 169706.1 Oracle Database on Unix AIX,HP-UX,Linux,Mac OS X,Solaris,Tru64 Unix Operating Systems Installation and Configuration Requirements Quick Reference (8.0.5 to 11.2)Note 1360790.1 Roadmap of Oracle Database Patchset ReleasesNote 458893.1 Oracle Universal Installer (OUI) FAQNote 403212.1 Location Of Logs For OPatch And OUINote 454442.1 11g Install : Understanding about Oracle Base, Oracle Home and Oracle Central/Global Inventory locationsNote 1467060.1 Relinking Oracle Home FAQ ( Frequently Asked Questions)Note 131321.1 How to Relink Oracle Database Software on UNIXNote 883299.1 Oracle 11gR2 Relink New FeatureNote 444595.1 Is It Necessary To Relink Oracle Following OS Upgrade? Note 556834.1 Steps To Recreate Central Inventory(oraInventory) In RDBMS Homes Note 750540.1 What are the requirements for an Oracle Home Name in the Inventory?Note 881063.1 Key RDBMS Install Differences in 11gR2Note 884232.1 11gR2 Install (Non-RAC): Understanding New Changes With All New 11.2 InstallerNote 948061.1 How to Check and Enable/Disable Oracle Binary OptionsNote 429074.1 Can You Run Different Releases Of The Oracle Database On The Same Computer At The Same Time?Note 389678.1 Are Compilers Necessary for Oracle RDBMS Installation?Note 401332.1 How To Identify A Server Which Has Itanium2 (Montecito, Montvale, Tukwila....) Processors InstalledNote 400227.1 How To Install Oracle RDBMS Software On Itanium Servers With Montecito ProcessorsNote 1474891.1 Is It Supported to Change Uid/Gid Of Oracle Database Software Owner After Installation?Note 1469625.1 Can root.sh Be Run While Standalone Database Instances Are Up And Running?Note 1475077.1 Can the $ORACLE_HOME/jdk and $ORACLE_HOME/jdk/jre Directories Be Removed?Note 1449674.1 Is It Supported to Update /Upgrade the Default JDK/JRE in Oracle Home?Note 1429678.1 How to interpret OS system tracesKnown Issues GenericNote 468771.1 Installing 11G on Any Platform : File Not Found Errors Running RunInstaller/Setup.exeNote 883702.1 $ORACLE_HOME/lib32 does not exist in Oracle database 11gR2. Why?Note 1245784.1 Installing Component using Installer from 11.2 ORACLE_HOME Fails with OUI-10150Note 1091063.1 sqlplus /nolog is failing with errors : SP2-1503 SP2-0152Note 1207849.1 Install Error: String Index Out Of Range: -1 Error MessagesNote 1325924.1 INS-32025 while installing 11.2 examplesNote 1367658.1 11.2 OUI does not use the $TEMP variable for CV_DESTLOC during a single instance installationNote 1307782.1 Oracle 11.2 Installation Fails If OS Users Password Contains '%'Note 1162824.1 Installing Oracle 10.2.0.1 On AIX Hangs At Prereqs CheckNote 1127135.1[INS-00001] Unknown irrecoverable error installing 11.2Note 744512.1 Ora-12547: Tns:Lost Contact Creating Database After Clean InstallationNote 763143.1 Explanation and Options To Handle "Checking operating system version: must be ... Failed" OUI errorsNote 1268391.1 Installing 11gR2 Fails With [INS-30060] Check For Group Existence FailedNote 1301113.1 "Invalid Entry Size" Error During Oracle Database InstallationNote 1471777.1 11.2.0.3 OUI Screen Not Coming Up Through Xmanager (Exceed)Note 1475425.1 Oracle Universal Installer - Make issue While Installing 11gR2 DatabaseNote 1478832.1 Instaling 11.2 fails with error Error in invoking target 'rat_on part_on dm_on olap_on sdo_on' of makefile and echodo: Execute permission denied.Note 1475482.1 ./runInstaller results in: [FATAL] [INS-32012] Unable to create directory: <directory path>Note 1489892.1 OUI Screen Disconnected Before Prompts To Run Scripts As 'Root' User Have Been RespondedNote 1474762.1 WARNING:OUI-67200:Make failed to invoke "/usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=<path>"....'ld: fatal: file /<path to O_H>/lib/prod/lib/v9/crti.o: open failed: Permission deniedNote 1511859.1 OUI:Attachhome fails with the Error "SEVERE: Abnormal program termination. An internal error has occured.  "Please provide the following files to Oracle Support"Note 1536056.1 Execution of oraenv fails with error "0403-009 Invalid number for this command"Note 1555453.1 Executing "relink all" resets permission of extjob, jssu, oradism, externaljob.oraNote 1292347.1 [INS-10002] Inventory Initialization Failed or [INS-10102] Installer initialization failed During InstallationNote 1569962.1 "[INS-30131] The initial setup that is required to run the installation program validation was not successful" Error While Installing 12.1Solaris GeneralNote 1525614.1 Requirements for Installing Oracle Database 12.1 on Solaris 11 SPARCNote 1517948.1 Requirements for Installing Oracle Database 12.1 on Solaris 10 SPARCNote 1496618.1 Requirements to Install Oracle 11gR2 RDBMS on Solaris 11 SPARC / X86_64 Note 1307041.1 Certification Information for Oracle Database on Solaris SPARC 64Note 1307389.1 Certification Information for Oracle Database on Oracle Solaris x86-64Note 743042.1 Requirements for Installing Oracle 11gR1 RDBMS on Solaris 10 SPARC 64-bitNote 971464.1 FAQ - 11gR2 requires Solaris 10 update 6 or greaterNote 964976.1 Requirements for Installing Oracle 11gR2 RDBMS on Solaris 10 SPARCNote 429191.1 Kernel setup for Solaris 10 using project filesNote 317257.1 Running Oracle Database in Solaris 10 Containers - Best PracticesNote 1557986.1 Defining Solaris 11 Standard InstallationKnown Issues on SolarisNote 1496746.1 Linking Of Oracle 12c Binaries on Solaris Failed with " ld: fatal: library -ldtrace: not found" ErrorNote 1518334.1 INFO: ld: warning: symbol '_init' not foundNote 848700.1 11gR1 on Solaris failing at relink phase with errors like "ld fatal relocation error" and then "ld fatal library -lclntsh not found"Note 966929.1 11gR2 INSTALL ERRORS on Solaris 10Note 989727.1 11GR2 INSTALL ON SOLARIS FAILS MANY PREREQUISITE CHECKSNote 969497.1 During An 11gR2 (11.2) Installation On Solaris, OUI Performs A Prerequisite Check For The Optional Patch 124861-15 And Fails If It Is Not InstalledNote 1325973.1 [INS-30060] Check for group existence failed - While installing 11gR2 on SolarisNote 1477021.1 make: Fatal error: Command failed for target $ORACLE_HOME/precomp/lib/procNote 1471229.1 Installation of Oracle 11gR2 fails while checking for runlevelNote 1477327.1 ROOT.SH Errors with mkdir: "/usr/local/bin": Read-only file system Solaris ContainerNote 1446945.1 ld: warning: symbol `_start' has differing types: While Installing Or Patching 11gR2 On Oracle Solaris PlatformNote 1521751.1 11.2.0.3 Installation On Solaris Fails During Prerequisite Checks Reporting STATUS=OPERATION_FAILEDNote 1539888.1 PRVF-7530 Error During Oracle 11.2.0.3 Installation on SolarisNote 1551669.1 11.2.0.3 installation fails with : /usr/ccs/bin/as: not found [No such file or directory] on Solaris 11 or Solaris 10LinuxGeneralORACLE Validated ConfigurationsNote 1307056.1 Certification Information for Oracle Database on Linux x86Note 1304727.1 Certification Information for Oracle Database on Linux x86-64Note 851598.1 Master Note of Linux OS Requirements for Database ServerNote 376183.1 Defining a "default RPMs" installation of the RHEL OSNote 386391.1 Defining a "default RPMs" installation of the SLES OSNote 401167.1 Defining a "default RPMs" installation of the Oracle Enterprise Linux (OEL) OSNote 728346.1 Linux OS Installation with Reduced Set of Packages for Running Oracle Database ServerNote 1508516.1 Is It Mandatory To Install UEK Kernel Under RHEL?Note 266043.1 Support of Linux and Oracle Products on LinuxNote 787705.1 How to identify correct Oracle Database software to install on Linux platformNote 1075717.1 Installing 32-bit RDBMS Client software on x86_64 Linux.Note 1315914.1 Install 11gr2 On Sles11 - Packages Required Not Mentioned In Installation GuideNote 1350000.1 Database Client or Database Server Install on Red Hat 6 (RHEL6)Note 444084.1 Multiple gcc / g++ Versions in LinuxNote 567506.1 Maximum SHMMAX values for Linux x86 and x86-64Note 761261.1 Recommended metalink articles for Oracle Database 10gR2 installation on zLinux (s390x)Note 415182.1 DB Install Requirements Quick Reference - zSeries based LinuxNote 341507.1 Oracle Database Server on Linux on IBM POWERNote 786995.1 Troubleshooting The Relink Errors When Missing RPMs on Linux x86-64Note 377217.1 What should the value of LD_ASSUME_KERNEL be set to for Linux?Known Issues On LinuxNote 308788.1 While Installing Oracle Database on SUSE SLESx x86-64 Linking Errors Occur and NetCA/DBCA Fail With Error: UnsatisfiedLinkError exception loading native library: njni10Note 317627.1 Error in invoking target 'agent emsubagent nmo nmb' of makefile '$ORACLE_HOME/sysman/lib/ins_sysman.mk'Note 362285.1 Error in Invoking Target 'Install' of Makefile $ORACLE_HOME/ctx/lib/ins_ctx.mkNote 468977.1 UnsatisfiedLinkError exception loading native library: njni11Note 1063952.1 11gR2 Database Server install fails with [INS-13001] Environment does not meet minimum requirementsNote 1220843.1 Getting __stack_chk_fail@GLIBC_2.4 makefile errors during Oracle 11.2 Install on LinuxNote 1313504.1 Error in invoking 'ntcontab.o' of makefile while installaing 11gr2 client on Linux x86-64 bit serverNote 1073332.1 /usr/bin/ld: Cannot Find -lxml11, Genclntsh: Failed To Link libclntsh.so.11.1Note 460969.1 /usr/bin/ld: Cannot Find -lxml10, Genclntsh: Failed To Link Libclntsh.so.10.1Note 443617.1 "Exception java.lang.UnsatisfiedLinkError: ../jre/1.4.2/lib/i386/libawt.so: libXp.so.6: cannot open shared object file: No such file or directory occurred" on RHEL5/OEL5 for 10.2 x86 or x86_64 install Note 301830.1 Upon startup of Linux database get ORA-27102: out of memory Linux-X86_64 Error: 28: No space left on deviceNote 454196.1 ./sqlplus: error on libnnz11.so: cannot restore segment prot after relocNote 1454982.1 Installing 11.2.0.3 32-bit (x86) or 64-bit (x86-64) on RHEL6 Reports That Packages "elfutils-libelf-devel-0.97" and "pdksh-5.2.14" are missing (PRVF-7532)Note 1543787.1 11.2.0.3 Installation failed on RHEL 6 with the error libraries: libmpfr.so.1: cannot open shared object file: No such file or directoryNote 1569369.1 libXi.so.6: cannot open shared object file: No such file or directoryAIXGeneralNote 1307544.1 Certification Information for Oracle Database on IBM AIX on Power systemsNote 282036.1 Minimum Software Versions and Patches Required to Support Oracle Products on IBM Power SystemsNote 889464.1 Is Oracle 10g And 11g Database Compatible With AIX Version 6.1Note 417451.1 How To Determine Whether an APAR has been Fixed in AIX Version/Maintenance LevelNote 739963.1 Using AIX commands genld, genkld and slibclean to avoid library file locking errors(libjox)Note 458403.1 Why AIXTHREAD_SCOPE should be set to 'S' on AIXNote 201622.1 Is It Still NecessaryTto Run the Rootpre.sh Script When Installing Oracle on AIX 5L and above?Note 753601.1 SUPPORT FOR ORACLE DATABASE ON WPARS UNDER AIX 6.1Note 1524569.1 Is IBM AIX Active Memory Expansion (AME) Certified or supported for Oracle Databases?Known Issues On AIXNote 832074.1 AIX 6.1 "Dependent module /usr/lib/libXpm.a(shr_64.o) could not be loaded" Error Attempting To Execute 11.1.0 runInstallerNote 879102.1 Stand Alone 11.1.0.6 Silent Installation on AIX 5.3 Fails On Prerequisite Check For rsct.basic.rte(0.0) and rsct.compat.clients.rte(0.0)Note 1439940.1 11gR2 OUI On AIX Pre-Requisite Check Gives Error "Patch IZ97457, IZ89165 Are Missing"Note 980602.1 The Installer (OUI) Detects Processes Running on ORACLE_HOME on AIX 6.1 TL04 SP1Note 985563.1 AIX: ERROR IN INVOKING TARGET 'LINKS PROC GEN_PCSCFG' OF MAKEFILE, ld: 0711-593 SEVERE ERROR: Symbol C_BSTATNote 1327608.1 INS-13001 Environment does not meet minimum requirements when installing 11.2.0.2 on AIX 7.1Note 859842.1 Database Server Installation on AIX 6.1 Error: OUI-18001: 'AIX Version 6100.01' is not supportedNote 1364239.1 runInstaller Failed With "OUI: 0403-006 Execute permission denied" While Installing 11.2.0.x On AIXNote 1506934.1 Oracle 11gR2 Install : Executing rootpre.sh fails with the error cldomain: file not foundNote 1525845.1 11gR2 Install fails with error OUI-10164:The Ownership For The File GeneralQueries.jar Could Not Be Changed To .HPGeneralNote 1307026.1 Certification Information for Oracle Database on HP-UX PA-RISC PlatformNote 1307745.1 Oracle Database Support on the Itanium Processor ArchitectureKnown Issues on HPNote 725913.1 Error OUI-15038 At Pre-Requisite Check Stage During Oracle 11g InstallNote 1183545.1 OUI 10.2.0.4 and OUI 10.2.0.5 raise OUI-15038 "Unable to execute rule 'CheckCompatibility' .. [For input string: "11_11"]Note 471476.1 32 Bit Libraries Are Not Installed With Oracle11g (11.1.0.6) On HP-UX ItaniumNote 287428.1 Oracle Universal Installer has detected that there are processes running in the currently selectedNote 819394.1 HP-UX: Sqlplus: ORA-12549: TNS:operating system resource quota exceededNote 1194973.1 Failure On Prerequisite Checks For 11gR2 Installation on HP-UXNote 552358.1 Installation Of Oracle 10.2 Client Fails On HPUX Itanium Servers With Montecito ProcessorsWINDOWSGeneralNote 1529702.1 FAQ: Oracle Database 12.1 Oracle Home User On WindowsNote 1307189.1 Certification Information for Oracle Database on Microsoft Windows x86 (32-bit)Note 1307195.1 Certification Information for Oracle Database on Microsoft WindowsNote 952302.1 Is Microsoft Windows 7 certified to install/run Oracle Database Server/Client ? Note 443813.1 Check List For Oracle On Windows.Note 137200.1 Checklist when Oracle Universal Installer (OUI) fails for WindowsNote 436299.1 Common reasons for OUI failures on the MS Windows platformsNote 161549.1 Oracle Database Server and Networking Patches for Microsoft PlatformsNote 816893.1 How to Recreate the Windows Registry for Oracle, After reloading the Windows OSNote 432713.1 How To Create a Service using oradim in Windows VistaNote 1449665.1 How To Disable/Enable A Specific Component From A 11.1.0.7 Database Home On Windows PlatformsNote 870253.1 32-bit Client Install on 64-bit Windows PlatformNote 1151394.1 Running 32-bit and 64-bit Oracle Software On 64-bit WindowsNote 208256.1 WIN: How to Remove a Single ORACLE_HOME and Its Traces on Microsoft Windows PlatformsNote 124353.1 Manually Removing all Oracle Components on Microsoft Windows PlatformsNote 1069034.1 11.2: How to Manually Remove Oracle Server Software on Microsoft Windows PlatformsNote 1433144.1 What Is The Windows Service "Oracleremexecservice" ?Known Issues on WindowsNote 1292921.1 While Installing 11.2, OUI Fails With "Error in starting the service. The service OracleMTSRecoveryService was not found"Note 221545.1 Oracle Installation Issues in Windows PlatformsNote 418479.1 file in use Errors, when applying patches on WindowsNote 271569.1 Database creation using DBCA on WIN 2003 PDC fails with DIM-19Note 334528.1 ORA-12154 or ORA-6413 Running 32-bit Oracle Software on 64-bitNote 1133495.1 Installation of 11gR2 on Windows Fails Checking RequirementsNote 1088324.1 11gR2 Deinstall fails on Windows 7Note 1468882.1 Oracle Universal Installer Hangs While Installing Oracle 11g Client/Database On WindowsNote 958591.1 Oracle Install Crashed With Java Error in Windows EnvironmentNote 1530206.1 Cannot Install Oracle Client 11.2 64 Bit And Oracle Client 32 Bit On Same Windows 64 Bit ServerNote 1569064.1 Oracle Database 12.1 Installation on Windows Errors out with Missing File dr0ulib.sql.sbsScriptsNote.189256.1 UNIX Script to Verify Installation Requirements for Oracle 8.0.5 to 9.2 versions of RDBMSNote 250262.1 RDA 4 - Health Check / Validation Engine Guide (Doc ID 250262.1)Silent InstallGeneral InformationNote 136660.1 Working Response files for a silent install of Oracle Server on MS-Windows- Part 1 of 3Note 136661.1 Database Configuration Response file for performing silent install - Part 2 of 3Note 136662.1 Network Configuration Reponse file for performing silent install - Part 3 of 3Note 388451.1 How to create a Response File for Silent Installation?Note 434315.1 How Can You Install Oracle Without Using The Oracle Universal Installer( runInstaller) GUI and Xserver especially When Doing A Remote Installation ?Note 405124.1 How to Prevent a Silent Install from Prompting for User Input to Run the 'root.sh' ScriptNote 885643.1 How To Install 11.2/12.1 Database/Client Software In Silent Mode Without Using Response FileNote 808275.1 How to install Oracle Database Patchset software in Silent mode without using response fileNote 782919.1 How to install Oracle Database Server software silently with customized listener configurationNote 885643.1 How to install 11gR2 Database/Client Software in Silent mode without using response fileNote 782918.1 How to install/deinstall Oracle Database Server software silently without response fileNote 782923.1 How to install/deinstall Oracle Database Client software silently without response fileNote 435680.1 Response File Installation of Database Server 10.2 (Suppressed or Silent mode)Known IssuesNote 1550832.1 Oracle Database 12.1 Silent Install Report Error With Null,null,nullDeinstall/Removing InstallationsGeneral InformationNote 208116.1 How to remove all traces of ORACLE_HOME on UNIXNote 275493.1 Removing 10g Database and Software from AIX, HP-UX, Linux, Sun and Tru64 PlatformsNote 1363753.1 Behaviour of the Oracle De-install/Deinstall/Uninstall Utility in 11gR2Note 886749.1 Oracle 11gR2 Deinstall And Deconfig Tool OptionsNote 883743.1 How To Deinstall/Uninstall Oracle Home In 11gR2 Note 1070610.1 How To De-install Oracle Home Using runinstallerNote 435219.1 How To Clean Up The Inventory After Deleting The Oracle Home Manualy Using OS Commands("rm-rf " or other)?Note 888934.1 Is it possible to deinstall/remove a specific component from already installed Oracle Database Home using OUI?Note 1271072.1 Is it possible to deinstall/disable a specific licensable option from already installed Oracle Database 11gR2 Home ?Known ProblemsNote 885609.1 11gR2 OUI warning: Please run the command '/deinstall/deinstall' to deinstall this OracleHomeNote 1069028.1 Deinstall utility on 11.2 removes multiple ORACLE_HOMEs Note 1271661.1 Deinstall 11.2 32-bit Home Inadvertanly Removes 64-bit Homes and Vice VersaNote 1067622.1 11.2 DEINSTALLER REMOVING LISTENERS RUNNING FROM OTHER ORACLE_HOMESNote 1322509.1 11.2 Database Corrupted After Use of $ORACLE_HOME/deinstall/deinstall ScriptNote 753929.1 Errors During Silent 11.1.0.6 DeinstallationNote 1128464.1 Uninstall Of Oracle 11gR2 Home On Windows Does Not Recognize Non-English ResponseNote 1374044.1 When deinstalling 11gR1 ,OUI greyed out the Remove button and getting message - You cannot deinstall at this time.Note 1519494.1 The deinstall tool cannot determine the home type needed to deconfigure the selected homeDiagnostic ToolsNote 780551.1 How To Diagnose Linking Errors During Base Install of Oracle RDBMSNote 757964.1 Oracle Universal Installer ( OUI ) Error Messages and Solution Reference ListNote 1812.1 TECH: Getting a Stack Trace from a CORE fileGeneral InformationNote.738771.1 How to Log Good Service Request for Relink issues?Note.736819.1 How to Log Good Service Request for Oracle Universal Installer (OUI) issues?Using My Oracle Support EffectivelyNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support ServicesGeneric LinksNote 1351051.1 Information Center: Install and Configure Database Server/Client InstallationsNote 1194734.1 Where do I find that on My Oracle Support (MOS) [Video]Note 742060.1 Release Schedule of Current Database ReleasesNote 1476075.1 FAQ: Downloading 9i, 10g, and 11g database software mediaNote 549617.1 How To Verify The Integrity Of A Patch/Software Download? [Video]Note 268895.1 Oracle Database Server Patchset Information, Versions: 8.1. 7 to 11.2.0Note 783141.1 Reference List of Critical Patch Update Availability(CPU) and Patch Set Update (PSU) Documents For Oracle Database and Fusion Middleware ProductNote 1119703.1 Database PSU-CPU Cross-Reference ListNote 854428.1 Patch Set Updates for Oracle ProductsNote 1061295.1 Patch Set Updates - One-off Patch Conflict ResolutionInformation on Security Patch Updates (SPU's)

For the most current information, please refer to : (Doc ID 1156586.1) Master Note For Oracle Database Server Installation PURPOSE This Master Note is intended to provide an index and references to the...

Master Notes

Master Note for Query Rewrite

++++++++++++++++++++++++++++++++++++++++++++++++++++The complete and the most recent version of this article can be viewedfrom My Oracle Support Knowledge Section.Master Note for Query Rewrite (Doc ID 1215173.1)++++++++++++++++++++++++++++++++++++++++++++++++++++Applies to:Oracle Server - Enterprise Edition - Version: 9.2.0.1 to 11.2.0.2.0 - Release: 9.2 to 11.2Information in this document applies to any platform.PurposeThis article gives you a single reference point from which to quickly find answers to common questions and issues for Query Rewrite. Scope and ApplicationThis Master Note helps to understand Query Rewrite and provides assistance in using diagnostics effectively to debug/troubleshoot and resolve issues encountered.Master Note for Query RewriteGeneral InformationIf you are looking for general information about query rewrite, how it works, how it can improve performance,  please have a look at the following documents:Query Rewrite Chapter in the Oracle Database Data Warehousing GuideOracle by Example (ObE) Viewlet Using Materialized Views and Query Rewrite Capabilities (10g) White PapersOracle Materialized Views and Query RewriteImproving Performance using Query Rewrite in Oracle Database 10gTechnical articleNote 311777.1 Query Rewrite using Dimensions explainedTroubleshooting1./ Query did not rewrite to ordinary materialized viewA./ The issue is reproducible at willIf the issue is reproducible at will, then  you can use the DBMS_MVIEW.EXPLAIN_REWRITE and DBMS_MVIEW.EXPLAIN_MVIEW procedures to find the cause of the problemFor further details please look at the following article:NOTE 149815.1 How To Use DBMS_MVIEW.EXPLAIN_REWRITE and EXPLAIN_MVIEW To Diagnose Query Rewrite Problems.The description of the messages that the EXPLAIN_REWRITE produces (the QSM messages) can be found in the Oracle Database Error Messages document for the corresponding database version.To interpret the EXPLAIN_REWRITE messages, and to take corrective actions, please consult the following article:NOTE 236486.1 Manual Diagnosis & Troubleshooting for Query RewriteB./ The issue is not reproducible at willIf the issue occurs intermittently, and it is not reproducible at will to run the DBMS_MVIEW.EXPLAIN_REWRITE procedure on the non rewriting query, then on 10g and above the following can be done.You can use the REWRITE_OR_ERROR hint in the query, which causes the following error if the query failed to rewrite:ORA-30393: a query block in the statement did not rewriteThis way one can avoid non rewritten long running queries, as well as better capture the moment when the above mentioned diagnostics could be used to find the cause of the non rewrite.You can look up the cause of not sharing the rewritten plan in the V$SQL_SHARED_CURSOR. This view can be searched using the SQL_ID of the query. For further information about V$SQL_SHARED_CURSOR view can be found in Oracle Database Reference.2./ The query did not rewrite to Cube Organized Materialized ViewIf the query does not rewrite to Cube Organized Materialized View (a new type of materialized view in 11g, where the data is stored in Oracle OLAP object called Analytic Workspace), then please look at the Note 577293.1 Oracle OLAP 11g: How to ensure use of Cube Materialized Views/Query RewriteKnown issuesTo browse the most recent known issues in the Knowledge Base, search for "query rewrite" under the Oracle Database product.What is nextIf you have not found the answer to your question, or solution to your problem, then you may want to log a a service request with Oracle Support. It is a good idea to provide a testcase, where the question can be demonstrated, or the problem can be reproduced. This usually reduces the resolution time of the service request significantly. A testcase for query rewrite questions/issues should contain the following:Minimum requirements:- query that does not rewrite- DDL of the objects involved in the query including indexes, constraints- DDL of the mview that the query is expected to rewrite, and DDL of the objects referenced in the mview definition query- the messages that the EXPLAIN_REWRITE and EXPLAIN_MVIEW procedures produced.Depending on the nature of the problem, Oracle Support may also need - statistics on the involved objects- values of the session parameters where the query is executedAny of the following could help you to collect testcase for a query rewrite problem: - section about how to create testcase for query rewrite problem in NOTE 236486.1- Note 727863.1 How to create a SQL-testcase using the DBMS_SQLDIAG package- DDL of the involved objects and statistics on those, as well as session parameter values can be easily collected when running the query via the Xecute method of SQLT (SQLT is a support tool and can be downloaded from NOTE 215187.1)- Testcase Builder available via EM DB Control - Support Workbench can also be used to collect and upload testcase.ReferencesNOTE:149815.1 - How To Use DBMS_MVIEW.EXPLAIN_REWRITE and EXPLAIN_MVIEW To Diagnose Query Rewrite ProblemsNOTE:215187.1 - SQLT (SQLTXPLAIN) - Tool that helps to diagnose SQL statements performing poorlyNOTE:236486.1 - Manual Diagnosis & Troubleshooting for Query Rewrite ProblemsNOTE:311777.1 - Query Rewrite using Dimensions explainedNOTE:577293.1 - Oracle OLAP 11g: How to ensure use of Cube Materialized Views/Query RewriteNOTE:727863.1 - How to create a SQL-testcase using the DBMS_SQLDIAG package.

++++++++++++++++++++++++++++++++++++++++++++++++++++ The complete and the most recent version of this article can be viewed from My Oracle Support Knowledge Section.Master Note for Query Rewrite (Doc...

Master Notes

Master Note For Cloning Oracle Database Server ORACLE_HOME's Using the Oracle Universal Installer (OUI)

For the most current information, please refer to :(Doc ID 1154613.1) Master Note For Cloning Oracle Database Server ORACLE_HOME's Using the Oracle Universal Installer (OUI) APPLIES TO:Oracle Universal Installer - Version 2.2.0.18 and laterOracle Database - Enterprise Edition - Version 9.2.0.8 and laterOracle Database - Standard Edition - Version 9.2.0.8 and laterInformation in this document applies to any platform.PURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support notes with respect to Oracle Database Server Cloning using the Oracle Universal Installer (OUI).This includes the following categories:GeneralHow toKnown IssuesUsing My Oracle Support EffectivelyGeneric LinksSCOPEThis note is intended for junior through expert level of expertise for Oracle software administration.This note serves as a guide for Oracle Database Server Cloning using the Oracle Universal Installer (OUI).It is applicable to cloning of the Oracle Database Server ORACLE_HOMEIt is NOT applicable to cloning done via Rapid Clone (APPs) or "Enterpirse" Manager Provisioning cloning.It is NOT applicable to database cloning.It will cover the following versions:Oracle Server - Version: 9.2.0.x and higherDETAILSGeneralNote 565009.1 FAQs on RDBMS Oracle Home Cloning Using OUINote 458893.1 Oracle Universal Installer (OUI) FAQNote 757964.1 Oracle Universal Installer ( OUI ) Error Messages and Solution Reference ListNote 403212.1 Location Of Logs For OPatch And OUIOracle Universal Installer and OPatch User's Guide 11g Release 2 (11.2) for Windows and UNIXOracle Universal Installer and OPatch User's Guide 11g Release 1 (11.1) for Windows and UNIXOracle Universal Installer and OPatch User's Guide 10g Release 2 (10.2) for Windows and UNIX(References for 9.2 and 10.1 cloning will be in the "How To" section below.)How ToNote 300062.1 How To Clone An Existing RDBMS Installation Using OUINote:559301.1 "Cloning An Existing Oracle10g Release 1 (10.1.0.x) RDBMS Installation Using OUI"Note:559304.1 "Cloning An Existing Oracle10g Release 2 (10.2.0.x) RDBMS Installation Using OUI"Note:559305.1 "Cloning An Existing Oracle11g Release 1 (11.1.0.x) RDBMS Installation Using OUI"Note:1221705.1 "Cloning An Existing Oracle11g Release 2 (11.2.0.x) RDBMS Installation Using OUI"Note 1493677.1 Cloning An Existing Oracle Database 12c Release 1 (12.1.0.x) RDBMS Installation Using OUINote.559863.1 An Example Of How To Clone An Existing Oracle9i Release 2 (9.2.0.x) RDBMS Installation Using OUINote.558478.1 Cloning A Database Home And Changing The User/Group That Owns ItNote 1520299.1 Master Note For Oracle Database 12c Release 1 (12.1) Database/Client Installation/Upgrade/Migration Standalone Environment (Non-RAC)Note 1555012.1 Install/Deinstall/Cloning Assistant: Oracle Database/ClientKnown IssuesNote 1061788.1 Connect as SYSDBA on 11.2 Cloned Home Gives "ORA-1031: Insufficient Privileges" ErrorNote 1367164.1 Relink Reports "Undefined Reference" Errors While Cloning 11.2Note 836056.1 DBUA Failed With Java Not found Error In A Cloned Oracle HomeNote 1324423.1 Oracle Base Defaults To Wrong Value In OuiNote 1321352.1 Libskgxnr.So: No Such File Or Directory When Running $ORACLE_HOME/clone/rootpre.shNote 416255.1 Cloning An Oracle10gR2 Standard Edition Installation Changes It To Enterprise EditionNote 751827.1 Cloning An 11.1.0.7 Standard Edition Installation Changes It To Enterprise EditionNote 553177.1 "Can't Locate English.Pm In @Inc Message" On Clone Of Oracle_homeNote 557681.1 During Cloning, clone.pl Fails With OUI-10197Note 443733.1 Error in invoking target during Cloning A 10.2.0.2 or 10.2.0.3 Oracle_homeNote 945297.1 Java.Lang.Nullpointerexception On ccr.zip When Cloning 11.1.0.7 Client HomeNote 559859.1 While Cloning Oracle9i Release 2 (9.2.0.x), OUI Fails With "Exception in thread "main" java.lang.NoClassDefFoundError: oracle/sysman/oii/oiic/OiicInstaller"Note 435943.1 Oui Not Reading Oracle_home_name VariableNote 578805.1 The "root.sh" Script Generated While Cloning A 9.2 Home Contains Duplicate ActionsNote 560411.1 Cloning Fails With 'Java RunTime Environment was not found at ../..//apps/oracle/jre/1.4.2/bin/java'Note 740486.1 Cloning Fails With The Error: Cannot fork the command "./runInstaller -clone ..."Note 457719.1 Oracle Home cloning gives file copy error for three filesNote 415855.1 After Cloning A 9.2.0.x Installation Using A 9.2.0.8 Clonerstage Patch, The Local Inventory Of The Cloned Home Is IncorrectNote 454943.1 Install Hangs At Checking Prerequisites (30%)Note 344798.1 Cloning A 10.2 RDBMS Installation Reports The Error `Properties file /config/cs.properties was not found. Failed loading corresponding properties.'Note 455528.1 Wrong Elf Class: Elfclass32 Link Error Following Db Home CloningNote 469637.1 Clone.pl Fails With Error "Exiting With Status 255"Note 1462229.1 11gR2 Cloning Fails : "OUI-10080:The Pre-requisite For The Component Oracle Configuration Manager 10.3.3.1.1 Has Failed"Note 1475985.1 Cloning ORACLE_HOME fails: java.lang.UnsatisfiedLinkError: /tmp/OraInstall2012-03-26_02-18-59PM/oui/lib/solaris/liboraInstaller.so: ld.so.1: java: fatal: /usr/lib/libxml2.so.2: wrong ELF class: ELFCLASS32Note 1511859.1 OUI:Attachhome fails with the Error "SEVERE: Abnormal program termination. An internal error has occured. Please provide the following files to Oracle SupportUsing My Oracle Support EffectivelyNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support ServicesGeneric LinksNote 854428.1 Patch Set Updates for Oracle ProductsNote 1061295.1 Patch Set Updates - One-off Patch Conflict ResolutionNote 268895.1 Oracle Database Server Patchset Information, Versions: 8.1. 7 to 11.2.0Note 161549.1 Oracle Database Server and Networking Patches for Microsoft PlatformsNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryNote 1476075.1 FAQ: Downloading 9i, 10g, and 11g database software mediaNote 1194734 Where do I find that on My Oracle Support (MOS) [Video]Information on Security Patch Updates (SPU's)

For the most current information, please refer to : (Doc ID 1154613.1) Master Note For Cloning Oracle Database Server ORACLE_HOME's Using the Oracle Universal Installer (OUI) APPLIES TO: Oracle...

Master Notes

Master Note For Oracle Database Upgrades and Migrations

For the most current information, please refer to :(Doc ID 1152016.1) Master Note For Oracle Database Upgrades and Migrations     APPLIES TO:Oracle Database - Standard Edition - Version 8.0.6.0 and laterOracle Database - Enterprise Edition - Version 8.0.6.0 and laterInformation in this document applies to any platform.PURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support Notes with respect to Oracle Database Upgrade and Migration. This Master Note is subdivided into categories to allow for easy access and reference to notes that are applicable to your area of interest within Upgrade and Migration. This includes the following categories:GeneralHow toDiagnostic ToolsKnown Issues    Preupgrade Tool Issues    During Upgrade Issues    Post Upgrade Issues    DBUA Specific IssuesSCOPEThis note serves as a guide through the various stages of upgrading/migrating a database.It is applicable to upgrade/migration done via DBUA or the manual method.It assumes you have already Installed Oracle software.The intended audience is from junior DBA's to senior DBA'sThis will cover the following versions of these products:Oracle Server - Version: 8.0.6.x though 11.2.0.xDETAILSAsk Questions, Get Help, And Share Your Experiences With This ArticleWould you like to explore this topic further with other Oracle Customers, Oracle Employees, and Industry Experts?Click here to join the discussion where you can ask questions, get help from others, and share your experiences with this specific article.Discover discussions about other articles and helpful subjects by clicking here to access the main My Oracle Support Community page for Database Install/Upgrade.GeneralNote 1561791.1 Upgrade / Downgrade Assistant: Oracle Database/ClientNote 419550.1 Different Upgrade Methods For Upgrading Your DatabaseNote 551141.1 Database Server Upgrade/Downgrade Compatibility MatrixNote 730365.1 Oracle Database Upgrade Path Reference ListNote 455744.1 Best Practices to Minimize Downtime During UpgradeNote 739485.1 How to estimate the time required to upgrade a database ?Note 1515747.1 Oracle Database 12c Release 1 (12.1) Upgrade New FeaturesNote 1493645.1 Oracle Database 12c Release 1 (12.1) DBUA : Understanding New Changes With All New 12.1 DBUANote 757245.1 Can you Upgrade RDBMS and Convert From 32-bit to 64-bit Binaries Directly on Linux or Windows based Intel Platforms Using the Database Upgrade Assistant (DBUA)Note 1320966.1 Things to Consider Before Upgrading to 11.2.0.2 to Avoid Poor Performance or Wrong ResultsNote 1392633.1 Things to Consider Before Upgrading to 11.2.0.3 to Avoid Poor Performance or Wrong ResultsNote 733205.1 Migration Of An Oracle Database Across OS Platforms (Generic Platform)Note 785351.1 Oracle 11gR2 Upgrade CompanionNote 601807.1 Oracle 11gR1 Upgrade CompanionNote 466181.1 10g Upgrade CompanionOracle Database Upgrade Guide 12c Release 1 (12.1)Oracle Database Upgrade Guide 11g Release 2 (11.2)Oracle Database Upgrade Guide 11g Release 1 (11.1)Oracle Database Upgrade Guide 10g Release 2 (10.2)Oracle Database Upgrade Guide 10g Release 1 (10.1)Oracle Database Upgrade Guide 9i Release 2 (9.2)How ToNote 251.1 Database Upgrade Planner from 10.2 to 11.2Note 264.1 Database Upgrade Planner from 9.2 to 11.2Note 732897.1 How to Log Good Service Requests for Database Upgrade/Migration IssuesNote 1503653.1 Complete Checklist for Manual Upgrades to Oracle Database 12c Release 1 (12.1)Note 1516557.1 Complete Checklist for Upgrading to Oracle Database 12c Release 1 (12.1) using DBUANote 837570.1 Complete Checklist for Manual Upgrades to 11gR2Note 870814.1 Complete checklist to upgrade the database to 11gR2 using DBUANote.1276368.1 Complete checklist for out-of-place manual upgrade from previous 11.2.0.N version to the latest 11.2.0.N patchset.Note 1377613.1 In-Place Upgrade on UNIX For Single Instance 11gR2 DatabaseNote 429825.1 Complete Checklist for Manual Upgrades to 11gR1Note 556477.1 Complete Checklist for Upgrades to 11gR1 using DBUANote 316889.1 Complete Checklist for Manual Upgrades to 10gR2Note 263809.1 Complete checklist for manual upgrades to 10gR1 (10.1.0.x)Note 159657.1 Complete Upgrade Checklist for Manual Upgrades from 8.X / 9.0.1 to Oracle9iR2 (9.2.0)Note 117048.1 How to Upgrade from Standard to Enterprise EditionNote 139642.1 Converting An Enterprise Edition Database To Standard EditionNote 1437019.1 How To Prevent Apex From Being Installed During DB UpgradeNote 1516616.1 Oracle Database 12c Release 1 (12.1) DBUA in Silent modeDiagnostic ToolsDB Upgrade/Migration-Troubleshooting ToolNote 884522.1 How to Download and Run Oracle's Database Pre-Upgrade UtilityNote 556610.1 Script to Collect DB Upgrade/Migrate Diagnostic Information (dbupgdiag.sql)Note 212588.1 How To Investigate/Diagnose Hanging Issues When Upgrading or Migrating (includes DBMA & DBUA)Note 577775.1 Dbua or Migration Is Hanging What to do?Note 428118.1 How to enable Java code tracing for DBUA, DBCA and NETCA?Known IssuesPreupgrade Tool IssuesNote 437371.1 ERROR:" WARNING: --> Database contains stale optimizer statistics.Refer to the 10g Upgrade Guide for instructions to update"Note 407031.1 ORA-01403 no data found while running utlu102i.sql/utlu102x.sql on 8174 databaseNote 271733.1 ORA-600 [17069] During Upgrade Using Database Upgrade Assistant, Utlu101i.sql or Utlu102i.sqlNote 549276.1 Utlu102i.sql Shows "WARNING: The database has not been patched to release 10.1.0.4.2" ShowedNote 370825.1 UTLU102I.SQL Reports That Already-Installed Components Need To Be InstalledNote 797750.1 "WARNING: --> Database contains schemas with objects dependent on network packages" When Running utlu111i.sqlNote 803774.1 The Utlu111i.Sql Pre-Upgrade Script Reports Stale Sys StatisticsNote 1473478.1 Pre-upgrade script utlu112i.sql errors with ORA-06550 and ORA-00942Note 843899.1 DB Upgrade Throws ORA-25153: Temporary Tablespace Is EmptyDuring Upgrade IssuesNote 412271.1 ORA-600 [22635] and ORA-600 [KOKEIIX1] Reported While Upgrading Or Patching Databases To 10.2.0.3Note 579523.1 ORA-600 [22635], ORA-600 [KOKEIIX1], ORA-7445 [KOPESIZ] and OCI-21500 [KOXSIHREAD1] Reported While Upgrading To 11.1.0.6 Or 11.1.0.7Note 1077748.1 Upgrading to 11.1.0.7 Crashes with ORA-600, arguments: [kokeeiix3], [3]Note 1081627.1 Upgrade to 11.1.0.7 Cancels with ORA-07445. ORA-600[QCTOSTIIX1] Showed at the Alert file.Note 761711.1 Upgrade From 10.2.0.3 To 11.1.0.7 Using DBUA Throws ORA-600 [kksfbc-reparse-infinite-loop] Error During Update to SYS.AQ_SRVNTFN_TABLENote 1067072.1 ORA-600 [kzdugt], [2147483629] While Upgrading From 11gR1 To 11gR2Note 1077748.1 Upgrading to 11.1.0.7 or 11.2.0.1 Crashes with ORA-600, arguments: [kokeeiix3], [3] or ORA-07445: exception encountered: core dump [qmxdGetMatchingChildren_int]Note 464716.1 ORA-1722 when running Oracle 11g DBUA utilityNote 730057.1 Upgrading to 11g Fails with ORA-01722 invalid numberNote 859576.1 Upgrading to 11gR1 Stops With ORA-01722 invalid numberNote 744693.1 ORA-00001 For SYS.DIANA_VERSION$ During RDBMS Upgrade From 11.1.0.6 To 11.1.0.7Note 845768.1 ORA-19905 LOG_ARCHIVE_FORMAT During UpgradeNote 293658.1 10.1 or 10.2 Patchset Install Getting ORA-29558 JAccelerator (NCOMP) And ORA-06512Note 557242.1 Upgrade Gives Ora-29558 Error Despite of JAccelerator Has Been InstalledNote 472959.1 "Ora-20000 : Insufficient privileges to analyze an object in Dictionary" During Catupgrd.SqlNote 734563.1 Upgrading To 10.2.0.4 Manually Or Using DBUA Fails With ORA-01437Note 470667.1 ORA-1422 During Upgrade To 10.2 or 11.1Note 554983.1 Error During Migration To 11gR1: Database Is Using An Old Timezone File VersionNote 788087.1 When Running Catupgrd.Sql, Exits With ORA-00904/ORA-00942Note 802694.1 Catupgrd.Sql Hang While Running dbms_metadata_hack.load_xsd('kusmv.xsd')Note 1070951.1 11.1.0.6.0->11.2.0.1.0 UPGRADE FAILS WITH ORA-22371 REFERENCING XDB.XDB$PROPERTY_T, VERSION 1Note 1104643.1 Upgrading Database to 11.2 Fails with ORA-01422Note 1100828.1 11gr2 Upgrade, Re-Running catupgrd.sql Causes Ora-00001 ErrorsNote 1051368.1 Startup Upgrade Gives ORA-202 ORA-17505 ORA-15061 ORA-600 [kccgtt_1]Note 1091252.1 ORA-31114 "XDB configuration has been deleted or is corrupted" during upgrade to 11.2Note 1148285.1 "ORA-01166: File number 101 is larger than MAXDATAFILES (100)" While Upgrading To 11.1.0.7 Or 11.2Note 1326522.1 ORA-7445 [qmxarFindPartition] during database upgrade to 11.2Note 1329606.1 10.2 Upgrade to 11.2 Hangs While Running Catupgrd.sqlNote 1425763.1 During A Database Upgrade To 11.2, Gathering Dictionary Statistics Takes Too Long (catupgrd.sql, cmpupend.sql)Note 1465852.1 ORA-00942 reports during upgrade to 11.2.0.3 using manual or DBUANote 1466082.1 ORA-02095: Specified Initialization Parameter Cannot Be ModifiedNote 1486277.1 SELECT TO_NUMBER('MUST_PATCH_TIMEZONE_FILE_VERSION_ON_NEW_ORACLE_HOME')Post Upgrade IssuesNote 438210.1 Synonyms Become Invalid Since Upgrading To 10g Note 456845.1 UTLU102S.SQL May Show Different Results Than Select From DBA_REGISTRYNote 753041.1 How To Diagnose Components With NON VALID Status In DBA_REGISTRY After an Upgrade Note 578841.1 How To Find What Objects Are Keeping Catalog And Catproc In Dba_Registry InvalidNote 443706.1 After Migrate From 9.2.0.7 To 10.2.0.3 We Get Ora-04065 On PlitblmNote 730136.1 OCI-21500 When Running UTLRP After A Successful Upgrade.Note 782735.1 Oracle Database Server Component Invalid After Upgrade To 10.2.0.4, 11.1.0.6 Or 11.1.0.7Note 978040.1 ORA-04045 ORA-04064 ORA-06508 ORA-06512 ERRORS RUNNING utlu111s.sql DURING UPGRADE TO FROM 9.2.0 TO 11.1.0Note 1313429.1 Is it required to re-install the UTL_* packages after an upgrade to 11.2?Note 1543850.1 Post Upgrade Scripts - utlu112s.sql & catuppst.sql report many Errors after upgrading Database from 10.2.0.X to 11.2.0.XDBUA Specific IssuesNote 315184.1 Database Upgrade Assistant Hangs On Collecting Server InformationNote 567037.1 DBUA hangs at 82% for More than an Hour When Upgrading APPS DB to 10GR2Note 339887.1 Failed Upgrade ''Dbua Thinks This Is A Rerun Operation'' ORA-01507Note 603015.1 DBUA Fails with ORA-32004 and "Dbua Thinks This Is A Rerun Operation"Note 729909.1 Upgrading to 11.1.0 and DBUA reports ORA-4023 On SYS.STANDARDNote 789779.1 DBUA Error ORA-04031 unable to allocate 4120 bytes of shared memory, ORA-01034 ORACLE not available, ORA-00600 internal error code, arguments [504]Note 271733.1 ORA-600 [17069] During Upgrade Using Database Upgrade Assistant, Utlu101i.sql or Utlu102i.sqlNote 308377.1 "ORA-01031 Insufficient Privileges" During An Upgrade Using DBUA On Unix/LinuxNote.386855.1 'startup migrate' failed with ORA-64 while upgrading to 10.2.0.2 with DBUANote 816457.1 DBUA unable to connect to the database for upgradeNote 431437.1 10g DBUA Failed To Bring Up 9i Database. ORA-00119 ORA-00132 ORA-01078Note 604248.1 DBUA Fails with ORA-0119 ORA-0132 ORA-1078 Upgrading to 10gR2Note 391283.1 ORA-27054 NFS error while using DBUA to upgrade to 10gR2Note 444148.1 Upgrading to 10gr2 Using Dbua Stops With An Alarm Sign At Step1Note 737400.1 Ora-01408 Error Upgrading from 9i to 11g using DbuaNote 559075.1 Dbua Won't Find Init<SID>.Ora During Upgrade Although It Is At The Right LocationNote 1321814.1 Underscore (Hidden) Parameters will not be retained after upgrade using DBUANote 1078388.1 Upgrdade to 11.2 via DBUA fails on ASMCA error on a database with no RAC or ASMNote 980693.1 11gR2 DBUA States "The CEP File does not provide the version directive"Note 1066828.1 11GR2 DBUA ORA-06550 PLS-00201 IDENTIFIER SYS.DBMS_JAVA MUST BE DECLAREDNote 1067422.1 DBUA ERROR: DOES NOT PROVIDE VERSION DIRECTIVENote 1066116.1 DBUA RAISES ORA-942 WHILE UPGRADING EXPRESSION FILTER COMPONENTNote 1304849.1 DBUA: Database Is Missing From List Of Candidate Databases To Be UpgradedNote 1364808.1 DBUA fails with error "Database is not open, restart database in open mode and try again" when upgrading from 10.2 to 11.2Note 1466447.1 Database Upgrade Assistant (DBUA) Displays A Dialog Box When There Are Special Characters In The PfileNote 1477555.1 DBUA FAILS WITH "FOR INPUT STRING: 8,192" ERRORNote 1453545.1 Database Upgrade using DBUA failed with invalid specification for system parameter LOCAL_LISTENERNote 1427179.1 11.2.0.3 DBUA Hangs While Specifying ASM Disk To FRAUsing My Oracle Support EffectivelyNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support ServicesGeneric LinksNote 854428.1 Patch Set Updates for Oracle ProductsNote 1061295.1 Patch Set Updates - One-off Patch Conflict ResolutionNote 268895.1 Oracle Database Server Patchset Information, Versions: 8.1. 7 to 11.2.0Note 161549.1 Oracle Database Server and Networking Patches for Microsoft PlatformsNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryInformation on Security Patch Updates (SPU's)

For the most current information, please refer to : (Doc ID 1152016.1) Master Note For Oracle Database Upgrades and Migrations APPLIES TO: Oracle Database - Standard Edition - Version 8.0.6.0 and laterOr...

Master Notes

Master Note For OPatch

     For the most current information, please refer to :(Doc ID 293369.1) Master Note For OPatch APPLIES TO:Oracle Database - Standard Edition - Version 8.1.7.4 and laterOracle Database - Enterprise Edition - Version 8.1.7.4 and laterOracle Universal Installer - Version 1.0.0.0.50 and laterSiebel CRM - Version 8.1.1.11 SIA [23030] to 8.1.1.11 SIA [23030] [Release V8]Information in this document applies to any platform.PURPOSETo provide a comprehensive list of commonly used OPatch notes.SCOPEThis is applicable to all levels of expertise for OPatch.DETAILSOPatch User GuidesOracle OPatch User's Guide Release 12.1 for Windows and UNIXOracle Universal Installer and OPatch User's Guide 11g Release 2 (11.2) for Windows and UNIXOracle Universal Installer and OPatch User's Guide 11g Release 1 (11.1) for Windows and UNIXOracle Universal Installer and OPatch User's Guide 10g Release 2 (10.2) for Windows and UNIXGeneralNote 1561792.1 Patching Assistant: Oracle Database/ClientNote 1451669.1 FAQ : Understanding OPatch Tool And PatchingNote 224346.1 OPatch - Where Can I Find the Latest Version of OPatch?Note 453495.1 FAQs on OPatch Version : 11.1Note.334108.1 OPatch version 10.2 - FAQNote 433218.1 Why Should You Install 32-Bit JDK/JRE When You Have 64-Bit Oracle and Operating System ?Note 1475077.1 Can the $ORACLE_HOME/jdk and $ORACLE_HOME/jdk/jre Directories Be Removed?Note 1449674.1 Is It Supported to Update /Upgrade the Default JDK/JRE in Oracle Home?Note 1446582.1 Frequently Asked Questions (FAQ): Patching Oracle Database ServerNote 865255.1 Best approach for Oracle database patching sequence to latest/required patchset along with CPU/PSU/any-other-one-off patchNote 1526322.1 11g : Can Examples CD Be Installed After Interim Patches Are Applied On The Base Version?Note 1530108.1 Oracle Database 12.1 : FAQ on Queryable Patch InventoryNote 1376691.1 Composite Patches for Oracle Products [Video]Note 761111.1 RDBMS Online Patching Aka Hot PatchingNote 1463390.1 Limits Of Online PatchingNote 563656.1 Different types of Patch Conflicts - Superset/Subset/Duplicate/Bug/FileNote 1321267.1 Database Patch conflict resolutionNote 1299688.1 Patch conflict resolutionNote 1061295.1 Patch Set Updates - One-off Patch Conflict ResolutionNote 418537.1 Significance of oraInst.loc When Installing Oracle Products and Applying PatchesNote 812874.1 Understanding and Diagnosing MISSING_COMPONENT and/or SKIPPING_COMPONENT messages during opatch applyNote 551222.1 Why Does OPatch Backup The Same Library Multiple Times?Note 283367.1 OPatch for Oracle Application Server 10g (10.1.2, 10.1.3, 10.1.4)Note 403212.1 Location Of Logs For OPatch And OUINote 403218.1 Can You Delete $ORACLE_HOME/.patch_storage Directory?Note 861152.1 Does  PSU Released For Oracle 10gR2 (10.2.0.4) ,11g R1 (11.1.0.7) and 11g R2( 11.2.0.1) Change The Database Version/Fifth Digit?Note 1520299.1 Master Note For Oracle Database 12c Release 1 (12.1) Database/Client Installation/Upgrade/Migration Standalone Environment (Non-RAC)How ToNote 298906.1 How To Repair A Corrupted Inventory For RDBMSNote 556834.1 Steps To Recreate Central Inventory(oraInventory) In RDBMS HomesNote 406037.1 Using opatch -report option, how to check for potential conflicts during patch apply without Database / Listener shutdownNote 458485.1 How to find whether the one-off Patches will conflict or not?Note 550522.1 How To Avoid Disk Full Issues Because OPatch Backups Take Big Amount Of Disk Space.Note 749787.1 How to run opatch in silent mode?Note 821263.1 How to confirm that a Critical Patch Update (CPU) has been installed in Linux / UNIXNote 750350.1 How To Find The Description/Details Of The Bugs Fixed By A Patch Using OPatch?Note 729384.1 How To Find If a Specific Patch Has Been Applied On Oracle 8i or 9.0.1 (or 9i R1)?Note 470225.1 How To Use Truss With OPatch?Note 312767.1 How to Rollback a Failed Interim Patch InstallationNote 966023.1 How to create a ocm response file for opatch silent installation.Note 459360.1 How To Do The Prerequisite/Conflicts Checks Using OUI(Oracle Universal Installer) And OPatch Before Applying/Rolling Back A PatchNote.739963.1 Using AIX commands genld, genkld and slibclean to avoid library file locking errors (libjox)Note 1394537.1 How To Rollback An Online Patch If The Database Is Down Or RemovedNote 453399.1 How To Check if inventory.xml/comps.xml Or Any Oracle XML File Is Correct?Note 1474666.1 How to apply OC4J patch (13705478) on RDBMS homeNote 1543697.1 How To Use MOS Patch Planner Via Enterprise Manager 12c To Check for and Resolve Patch ConflictsDiagnostic Tools For OPatchNote 1430571.1 Script To Collect OPatch Diagnostic Information For Unix Platform (opatchdiag.sh)Note 1441309.1 Oneoffsdiag.sh:Script To Verify The Existance Of $ORACLE_HOME/inventory/oneoffs/PATCH_ID and also actions.xml and actions.xml inventory.xml in $ORACLE_HOME/inventory/oneoffs/PATCH_ID/etc/configKnown IssuesWith InventoryNote 229773.1 ALERT Applying Multiple Interim Patches Using OPatch Can Cause Inventory Corruption Note 401454.1 opatch lsinventory Raises An OPatchException While Accessing O2ONote 413829.1 OPatch Fails With 'Can not get details for given Oracle Home'Note 417275.1 opatch lsinventory Reports `Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/transform/Source` When Executed Against 9.2.0.xNote 417694.1 "OUI-67124:Inventory check failed: Patch ID is not registered in Oracle Home Inventory" while applying one-off patches using opatchNote 431783.1 opatch lsinventory Result Shows: * Warning: no filemap entries available.Note 795452.1 OPatch reports Error when using multiple oracle usersNote 864117.1 OPatch error: "Inventory load failed... OPatch cannot load inventory for the given Oracle Home."Note 821550.1 opatch lsinventory Error: " Unable To Parse The Xml File"Note 861677.1 OPatch error : "Unable to lock Central Inventory"Note 864803.1 OPatch error: "Apply Session failed: ApplySession failed to prepare the system. No detail available."Note 437026.1 ERROR :"OPatch cannot find a valid oraInst.loc file to locate Central Inventory.OPatch failed with error code 104"Note 728417.1 OPatch Fails With "LsInventorySession failed:OracleHomeInventory gets null oracleHomeInfo"Note 1086853.1 opatch lsinventory Reports "OUI-67076 : OracleHomeInventory was not able to create a lock file"Note 876343.1 Patch conflict Error OUI-67301 on 11.1.0.7.0 due to one off in inventoryNote 1185246.1 OPatch Fails With "OUI-67028:Inventory load failed"Note 1308866.1 OPatch Cannot Load Inventory For The Given Oracle HomeNote 1475147.1 OPatch - Failed to load the patch object. Possible causes are: OPatch failed with error code = 73 LsInventorySession failed:Note 1456247.1 The opatch apply failing with OPatch failed with error code 39 :This patch requires some components to be installed in the home. Either the Oracle Home doesn't have the components or this patch is not suitable for this Oracle Home.With JavaNote 332698.1 OPatch was not able to locate executable jarNote 417372.1 OPatch Error " java.lang.UnsatisfiedLinkError: ... liboraInstaller.so: ... "Note 808368.1 OPatch: cannot open shared object file.  Possible cause: architecture word width mismatchNote 974371.1 After Installing The 11.2.0.1 32-bit Client Software On SPARC Solaris, OPatch and OUI Fail With ELFCLASS32 Errors For liboraInstaller.soNote 417177.1 ERROR : "Java could not be located. OPatch cannot proceed!.OPatch failed with error code = 1"Note 1474852.1 UTE011: Active tracepoint array length for JVM is 7559; should be 7577 Could not create JVM.Note 1475894.1 OPatch commands error: JVMJ9VM019E Fatal error: Unable to find and initialize required class java/lang/ClassCircularityErrorWith Pre-PatchingNote 943436.1 opatch prereq checkapplicable Error: "Unable to create Patch Object. Check if the given patch location is correct"Note 780858.1 OPatch Fails With `Prerequisite check "CheckActiveFilesAndExecutables" failed` And `Following executables are active : $ORACLE_HOME/lib/libnmemso.so`Note 882932.1 Prerequisite check "CheckPatchApplicableOnCurrentPlatform" failed When Applied Generic Patch 8282506Note 861507.1 PSU OPatch conflict check fails with "OPatch failed with error code 73"Note 292946.1 OPatch Detecting Wrong Platform Id For Install Of PatchNote 404473.1 OPatch Reports Error "OPatch detects your platform as... this patch...supports platforms: 0 (Generic Platform)"Note 602334.1 OPatch Not Working, Reporting That It Is ObsoleteNote 729218.1 ERROR: "The Oracle Home does not meet OUI version requirement.OPatch failed with error code 73"Note 257800.1 OPatch: Interim Patch  is holding the lock from this patchNote 312926.1 opatch apply Gives: Can't Locate English.Pm In @IncNote 1050819.1 OPatch Reports Missing Component and Prerequisite Check "CheckApplicable" FailedNote 869945.1 Zop-40: The Patch(es) Has Conflicts/Supersets With Other PatchesNote 1231223.1 OPatch Fails With Error Code 255Note 747049.1 OPatch apply/rollback error: Prerequisite check "CheckActiveFilesAndExecutables" failedNote 1475206.1 Patch <patch_number> requires OPatch version <opatch version>Note 1629444.1 Opatch Fails Updating Archives with " No space left on device " ErrorWith LinkingNote 417319.1 Getting Warnings While Applying Patches on Linux x86-64Note 457126.1 OPatch Aborted After Relink Fails On Target Ioracle.Note 789053.1 Cannot find or open library file: -l pls10 when applying a patch with 10.2Note 402945.1 While installing one-off Patch on AIX systems, getting several WARNING messages: OUI-67215 - TOC overflow and/or xlC: not found / xlC: Execute permission deniedNote 1446945.1 "ld: warning: symbol `_start' has differing types:" While Installing Or Patching 11gR2 On Oracle Solaris PlatformNote 1474762.1 WARNING:OUI-67200:Make failed to invoke "/usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=<path>"....'ld: fatal: file /<path to O_H>/lib/prod/lib/v9/crti.o: open failed: Permission deniedNote 1467060.1 Relinking Oracle Home FAQ ( Frequently Asked Questions)Note 1525333.1 Relink error on target iKfed while patching Oracle 11.2.0.3 Client with 11.2.0.3.5 PSUNote 1534583.1 OPatch fails to Rollback Patch due to relink errors.Note 1539887.1 Undefined first referenced symbol ld: fatal: symbol referencing errors.With Post-PatchingNote 353150.1 OPatch Failing Validation Phase When Archiving Really Did Not FailNote 791187.1 OPatch Failing Validation Phase - OPatch versions 10.2.0.4.6 and 11.1.0.6.6Note 1361802.1 opatch auto doesn't patch the RDBMS Home if no database existsWith SQLPatching (datapatch)Note 1599479.1 Datapatch errors at "SYS.DBMS_QOPATCH"Note 1632818.1 Datapatch may fail while patching a RAC+PDB environment MiscellaneousNote 799883.1 OPatch error: OUI-67160 Unexpected end of ZLIB input streamNote 739815.1 OUI-67294 Warning During opatch applyNote 553244.1 WARNING MESSAGE "OUI-67078" and "OUI-67620" from OPatch when applying 10.2.X patchesNote 751107.1 ERROR: "Copy Action: Source file <filename> does not exist.'oracle.rdbms, <version> Cannot copy file'"Note 553263.1 OPatch Reports 'libclntsh.so.10.1 not removed. Text file busy'Note 563356.1 OPatch error 'file not found' message on 11.1.0.6 32-bitNote 1136432.1 OPatch fails with JVMXM008Note 1066937.1 Apply Patch Fails with 'Patch ID is null' and Error Code 73Note 1475147.1 OPatch - Failed to load the patch object. Possible causes are: OPatch failed with error code = 73 LsInventorySession failed:Note 1108834.1 OUI-67073:ApplySession failed: Patch ID is null occurred when applying patch 9352179Note 803097.1 OUI-67200:Make failed problem while applying a PatchNote 730754.1 OPatch Gives Exception In Thread "Main" Unsatisfiedlinkerror: HaltimplNote 603465.1 OPatch Fails With "Jar not applied ... to $ORACLE_HOME/jdbc/lib/classes12.zip... 'invalid entry compressed size (expected x but got y bytes)'"Note 1474874.1 opatch apply Fails - Jar Action: Source File "/<path to patch stage directory>/<filename> '<Oracle Product/Componet>': Cannot update file '/<path to ORACLE_HOME>/<directory>/<filename>' with '/<path>/<filename>'Note.1085753.1 OPatch Failing With Error Code 74 While Applying PSU Note 402945.1 While installing one-off Patch on AIX systems, getting several WARNING messages: OUI-67215 - TOC overflow and/or xlC: not found / xlC: Execute permission deniedNote 1055397.1 opatch napply HangingNote 1244414.1 OPatch throws: Archive Action: Source file "$ORACLE_HOME/.patch_storage/........." does not exist.Note 1269454.1 OPatch Failed With Error Code 73 Because Session Is Not Able to Create a DirectoryNote 1355040.1 Online patching errors: use "OPatch util enableOnlinePatch" to add the SID(s)Note 1462303.1 OPatch Fails With Error "No such file or directory"Note 1331900.1 Prerequisite Check "CheckRollbackable" On Auto-rollback Patches FailedNote 1474805.1 OPatch Log Shows - INFO:Skip patch patch number from list of patches to apply: This patch is not needed.Note 1476964.1 OPatch Logging "Generic Conflict" Between Sub-Patches Of A Composite PatchNote 1468097.1 Argument (s) Error ... Illegal File Format For Composite Patch. There Is No Composite.xml File.Note 1435438.1 Multiple OUI Messages In OPatch Log : OUI-67161, OUI-67085, OUI-67047, OUI-67049, OUI-67048, OUI-67085 and OUI-6705Note  244241.1 Rolling Patch - OPatch Support for RACWindows SpecificNote 390761.1 Using OPatch On Windows When ORACLE_HOME Contains Space CharactersNote 416318.1 HOW TO Use Process Explorer to obtain a list of DLLs loaded by a Windows processNote 418479.1 Files in Use errors, when running OPatch to apply patches on WindowsNote 791005.1 OPatch permissions problem on 64-bit windows 2008 and Oracle 10.2.0.3 and 10.2.0.4Note 1446273.1 OPatch 11.2.0.3.0 LOCKS SRVM.JAR WHEN APPLYING ANY 11.2.0.3 BUNDLE PATCH ON WINDOWS X64Note 1530132.1 Applied Bundle patch 16 (16167942/16167941) but opatch lsinventory incorrectly showing as Bundle patch 15.Note 1568694.1 OPatch throwing "Unable to lock Central Inventory" on Windows 7Using My Oracle Support EffectivelyNote 732697.1 What Information Oracle Support Need To Work On OPatch Service Requests?Note 735257.1 What information Oracle Support needs to work on Critical Patch Update (CPU) Installation Service RequestsNote 868955.1 My Oracle Support Health Checks CatalogNote 166650.1 Working Effectively With Global Customer SupportNote 199389.1 Escalating Service Requests with Oracle Support ServicesNote 1194734.1 Where do I find that on My Oracle Support (MOS) [Video]Generic LinksNote 1454618.1 Quick Reference To Patch Numbers For Database PSU, CPU And Bundle PatchesNote 1119703.1 Database PSU-CPU Cross-Reference ListNote 854428.1 Patch Set Updates for Oracle ProductsNote 268895.1 Oracle Database Server Patchset Information, Versions: 8.1. 7 to 11.2.0Note 161549.1 Oracle Database Server and Networking Patches for Microsoft PlatformsNote 161818.1 Oracle Database (RDBMS) Releases Support Status SummaryInformation on Security Patch Updates (SPU's)

For the most current information, please refer to : (Doc ID 293369.1) Master Note For OPatch APPLIES TO: Oracle Database - Standard Edition - Version 8.1.7.4 and laterOracle Database - Enterprise...

Master Notes

Oracle Support Master Note on Oracle Universal Installer (OUI) for installing Oracle Database Software

    For the most current information, please refer to :(Doc ID 1157464.1) Master Note on Oracle Universal Installer (OUI) for installing Oracle Database Software APPLIES TO:Oracle Universal Installer - Version 2.2.0.18 and laterOracle Database - Enterprise Edition - Version 8.1.7.4 and laterOracle Database - Standard Edition - Version 8.1.7.4 and laterInformation in this document applies to any platform.PURPOSEThis Master Note is intended to provide an index and references to the most frequently used My Oracle Support notes with respect to Oracle Database Server and this note in particular provides general information on Oracle Universal Installer (OUI).This includes the following categories:GeneralHow toKnown IssuesUsing My Oracle Support EffectivelyGeneric LinksSCOPEThis note is intended for junior through expert level of expertise for Oracle software administration.This note serves as a guide for Oracle Universal Installer (OUI) with respect to Oracle Database Software.It is applicable only to Oracle Database Server or Client.It is NOT applicable to Application Server or E-Business Suite or any other Oracle Product.It will cover the following versions:Oracle Server - Version: 9.2.0.x though 11.2.0.xDETAILSOUI User GuidesOracle Universal Installer and OPatch User's Guide 11g Release 2 (11.2) for Windows and UNIXOracle Universal Installer and OPatch User's Guide 11g Release 1 (11.1) for Windows and UNIXOracle Universal Installer and OPatch User's Guide 10g Release 2 (10.2) for Windows and UNIXGeneralNote 458893.1 Oracle Universal Installer (OUI) FAQNote 757964.1 Oracle Universal Installer ( OUI ) Error Messages and Solution Reference ListNote 403212.1 Location Of Logs For Opatch And OUINote 454442.1 11g Install : Understanding about Oracle Base, Oracle Home and Oracle Central/Global Inventory locationsNote 820135.1 What Does -ignoreSysPrereqs Ignore When Invoking runInstaller?Note 787598.1 Standalone Prerequisite Check using OUI to Avoid Failures during 10g and 11g InstallationNote 884232.1 11gR2 Install (Non-RAC): Understanding New Changes With All New 11.2 InstallerHow ToNote 269837.1 How to trace OUINote 400227.1 How To Install Oracle RDBMS Software On Itanium Servers With Itanium2 (Montecito, Montvale, Tukwila....) ProcessorsNote 459360.1 How To Do The Prerequisite/Conflicts Checks Using OUI(Oracle Universal Installer) And Opatch Before Applying/Rolling Back A PatchNote 885643.1 How to install 11gR2 Database/Client Software in Silent mode without using response fileKnown IssuesNote 258137.1OUI hangs while installing 9.2.0.4 patchset on Linux clusterNote 266617.1Oracle Universal Installer (OUI) may appear and then disappear on Windows 2000 with Service Pack 4Note 390803.1Oracle Universal Installer Fails Due To `line 2: cd:/oui/bin/linux: No such file or directory` ErrorNote 435943.1 OUI Not Reading Oracle_home_name VariableNote 757964.1 Color problems using Exceed for OUI (Oracle Universal Installer)Note 468051.1 OUI-15038 Occurs During Oracle Universal Installer's Product Specific Prerequisite ChecksNote 559859.1 While Cloning Oracle9i Release 2 (9.2.0.x), OUI Fails With "Exception in thread "main" java.lang.NoClassDefFoundError: oracle/sysman/oii/oiic/OiicInstaller"Note 885609.1 11gR2 OUI warning: Please run the command '/deinstall/deinstall' to deinstall this OracleHomeNote 969497.1 During An 11gR2 (11.2) Installation On Solaris, OUI Performs A Prerequisite Check For The Optional Patch 124861-15 And Fails If It