OCI Full Stack DR Changes Required for Multiple Standby Database Support on ExaDB-D and ExaDB-C@C

Oracle Exadata Database Service on Dedicated Infrastructure (ExaDB-D) and Oracle Exadata Database Service on Cloud@Customer (ExaDB-C@C) now supports multiple standby databases using Data Guard automation. Before this new feature, customers were restricted to only a single standby DB (local or remote) per primary database via the OCI Console/API/SDKs or Terraform scripts. With this new capabiilty, customers now have the ability to configure multiple local and remote standby databases (up to 6) per primary database via the OCI Console/API/SDK or Terraform scripts increasing the data protection and disaster recovery with more flexible options that can include both local and remote standby databases.

Data Guard Automation Models with ExaDB-D and ExaDB-C@C:

With the introduction of these new features, the existing Data Guard Cloud automation tooling in ExaDB-D and ExaDB-C@C has been updated.  Previously, with a single Data Guard standby configuration, details could be found listed under the Data Guard Association section. Once multiple standby databases are configured, details can instead now be viewed under a new Data Guard Group Resource section.

Considerations for the new feature

• This feature is only supported for Oracle database release 19c and later.
• The new Data Guard group resource model allows the creation of up to 6 standby databases for a primary database.
• If you have existing Data Guard configurations that were created manually not using Cloud automation, you will need to recreate those configurations within the OCI console.
• With this release, support for both the existing Data Guard Association and the new Data Guard Group API and associated user interfaces will be available providing backwards compatibility but only Data Guard Groups can be used with multiple standby databases. The Data Guard Association API and resource will be deprecated after providing advance notification to customers as per the API deprecation policy.

Let’s understand the differences between the Data Guard Association model and the Data Guard Group resource model.

Data Guard Association:  This allows the creation of one standby DB with the primary database using cloud automation. Throughout the blog, we will refer to this configuration as the old Data Guard automation model.

Data Guard Group Resource: This allows the creation of multiple standby databases for a given primary database using cloud automation. Throughout the blog, we will refer to this configuration as the new Data Guard automation model.

Regardless of whether you have a single or multiple standby databases, it is highly recommended that you move to the new Data Guard automation model, as the old model is expected to be deprecated soon. Switching from the Old Data Guard automation model to the New Data Guard automation model is reliatively straightforward. For a step-by-step guidance on how to do this, please refer to the ExaDB-D and ExaDB-C@C documentation.

Impact on the OCI Full Stack DR Configuration with Multiple Standby Databases for ExaDB-D and ExaDB-C@C:

 OCI Full Stack DR supports both ExaDB-D and ExaDB-C@C database member resource types in the DR protection group, along with other resources like the Autonomous Database (Serverless and Dedicated), ExaScale, Base DB, Compute, Storage, Load Balancer, and Oracle Kubernetes Engine. For more details regarding adding these different member resource types to a DR protection group, please refer to the OCI Full Stack DR documentation. For the rest of the blog, we will focus specifically on the ExaDB-D and ExaDB-C@C databases in the context of a Full Stack DR configuration.

You can add ExaDB-D and ExaDB-C@C databases with multiple standby databases to the Full Stack DR protection groups. Full Stack DR supports an association between two DR protection groups (as a pair) with primary and standby roles. It typically includes a cross-region primary database and standby database in the respective DR protection groups. If the database role changes to a local standby database outside of the Full Stack DR, the Full Stack DR configuration will break, and we must take care of additional considerations, which we will discuss below.

Full Stack DR automatically creates built-in plan groups for switchover and failover plans for ExaDB-D and ExaDB-C@C databases. However, it currently requires that user-defined plan groups be created and included as part of the drill plans.The good news is that OCI Full Stack DR was created specifically with this type of extensibility in mind. Before we dive into the specifics, here is the summary of all four DR plan types.

Let’s use the below configuration with Ashburn as the primary region and Phoenix as the standby region as an example. The databases are created using ExaDB-D with one primary and two standby databases using the new Data Guard automation model. The primary database is created in Ashburn-AD1, local standby database is created in Ashburn-AD2, and remote standby database is created in Phoenix-AD1 regions.

• Use Case 1: Existing Full Stack DR configuration with single standby DB with the old Data Guard automation model
  • The primary database is Ashburn-AD1 and standby database is Phoenix-AD1. There should be no impact on the existing Full Stack DR protection groups and DR plans (Switchover and Failover)
• Use Case 2: Creating a single standby DB with the new Data Guard automation model 
  • ​​​​​​​The primary database is Ashburn-AD1 and standby database is Phoenix-AD1. Once the Data Guard configuration is complete, configure the Full Stack DR protection groups, associate roles, adds the database members to the protection groups, and create and customise the DR plans in the standby DR protection group region. Full Stack DR plans ( Switchover and Failover) will use the new model APIs. 
• Use Case 3: Migration of single standby DB from the old Data Guard automation model to the new Data Guard automation model 
  • ​​​​​​​The primary database is Ashburn-AD1 and standby database is Phoenix-AD1. You then need to migrate the Data Guard automation model. There should be no impact on the existing Full Stack DR protection groups and DR plans (switchover and failover). It is recommended that you run the pre-checks for all the Full Stack DR plans and verify that they are successful.
• Use Case 4: Create multiple standby databases using the new Data Guard automation model
  • ​​​​​​​The primary database is Ashburn-AD1, the local standby database is Ashburn-AD2, and the remote standby database is Phoenix-AD1. With this configuration, you can create Full Stack DR switchover and failover plans in the PHX-AD1 DR protection group. Full Stack DR will create the built-in steps for both plans. Because you can only add the primary database (Ashburn-AD1) to the primary DR protection group, but cannot add the local standby database (Ashburn-AD2) to the DR protection group, Full Stack DR service will be aware of only between primary database (Ashburn-AD1) and the remote standby database (Phoenix-AD1), but it will be unaware of the standby database in the local standby DB (Ashburn-AD2).  Let’s see how the switchover DR and failover DR plans work.
    • Switchover DR plan
      • If the database’s switchover from the primary database (Ashburn-AD1) to the local standby (Ashburn-AD2) is done outside of the Full Stack DR configuration, the database in Ashburn-AD2 will become primary, and Ashburn-AD1 will become standby.
      • This creates a conflict between the database role and Full Stack DR protection group role.
      • Assuming this role reversal was done temporarily and you intend to switch back to the original primary database (Ashburn-AD1), no reconfiguration is required in Full Stack DR. However, during this time period, Full Stack DR plan pre-checks will fail because of the configuration conflict.  This also means that you cannot use the Full Stack DR configuration for switchover or drill operations.
      • If you intend to use the new primary database in Ashburn-AD2 for a longer period, you should reconfigure and repair the Full Stack DR configuration by performing the below.   
        • Remove the original primary database member (Ashburn-AD1) from the primary DR protection group IAD-AD1
        • Add the new primary database member (Ashburn-AD2) to the primary DR protection group IAD-AD1
        • Refresh the DR plans in the standby DR protection group PHX-AD1
        • Run the prechecks and verify they are successful. 
    • Failover DR plan
      • If the database’s switchover from the primary database (Ashburn- AD1) to the local standby (Ashburn- AD2) is done outside of the Full Stack DR configuration, the database in Ashburn-AD2 will become primary, and Ashburn-AD1 will become standby.
      • During this scenario, if there is a regional outage to Ashburn and you would like to failover the entire stack from Ashburn-AD1 to Phoenix-AD1, you can run the failover DR plan in PHX-AD1. This will trigger a full stack failover, and the database in Phoenix-AD1 will become primary, you must reinstate the database outside of Full Stack DR based on the outage.

Conclusion:

In conclusion, the introduction of multiple standby databases using Data Guard automation in Oracle Exadata Database Service on Dedicated Infrastructure (ExaDB-D) and Oracle Exadata Database Service on Cloud@Customer (ExaDB-C@C) significantly enhances data protection, flexibility, and return on investment for users. The transition to the new Data Guard automation model is encouraged, as it offers improved integration with OCI Full Stack DR configurations. However, as discussed in this blog post, administrators should be mindful of the necessary adjustments and considerations in the Full Stack DR configuration when performing local region database role transitions outside of the Full Stack DR configuration, and if they do so, they should take appropriate actions to ensure smoother functioning of the Full Stack DR plans.