We are very excited about two new disaster recovery (DR) drill plan types that were just added to OCI Full Stack Disaster Recovery (Full Stack DR) last week. DR drills deliver the power to validate recovery for an entire application stack at the click of a button.
You now have the power to perform a complete dry run of your failover plan without having to call out an army of technical experts for a weekend full of heroics trying to get things running again.
New Full Stack DR plan types include start and stop DR drills, as seen from the OCI console in the screenshot below.
I’d like to explain these powerful tools for recovering and validating your business recovery systems.
Full Stack DR provides a fully automated and comprehensive disaster recovery orchestration solution for all the layers of a full-stack cloud application, including infrastructure, database, and application. Full Stack DR orchestrates recovery for your application stack across OCI regions or availability domains within the same region. Use Full Stack DR to quickly turn your manual DR runbook or workflow into automated DR plans. Full Stack DR offers four distinctly different types of DR plans shown in the list below:
- Failover
- Switchover
- Start DR drill (new)
- Stop DR drill (new)
You organize the resources you want Full Stack DR to manage during a recovery into logical containers called DR protection groups. Create a DR protection Group in each OCI region that you want to act as primary and standby peers, then add one or more member resource types shown in the list below. The member resources inform Full Stack DR what OCI resources must be managed during a recovery operation. DR plans offered in the list above define what actions or steps Full Stack DR needs to execute during a recovery.
Full Stack DR currently supports the following member resource types, and we will be adding more in the future:
- Compute – VM and dedicated virtual machine host
- Oracle Database – Base Database Service, Exadata Database on Dedicated Infrastructure Service, Autonomous Database Serverless
- Volume Group
- Load Balancer (new)
- File Storage (new)
Based on the members in the DR protection groups, Full Stack DR will generate the type of DR plan you choose, automatically prepopulating the plan with built-in steps organized into DR Plan Groups. DR plan groups are groupings of similar actions or tasks that must be executed in parallel during a recovery.
A failover plan is used for unplanned recovery during disasters or outages. It immediately activates services in the standby region whenever the primary is entirely inaccessible.
A switchover plan allows for a planned and controlled transition of services from a primary to a standby region. It involves shutting down services in the primary region and bringing them up in the standby region.
Now, let’s explore the two new DR drill plan types that were just added
DR drills
Performing DR drills is an essential DR service capability that allows users to exercise and validate their business continuity configuration and plans without disrupting their production stack. DR Drill plans to allow the creation and termination of a copy of the production stack in a standby region for testing and validation. The best part of DR drills is that the dry run of failover has no impact on your production application stack.
Creating a Start DR drill plan generates a plan similar to a failover since the steps performed during a drill are only executed in the standby region. Creating a Stop DR drill plan generates a plan that breaks down whatever the start DR plan created at the standby.
Full Stack DR does not currently provide built-in DR Drill support for Base or Exadata Database resources because these services lack the necessary APIs for converting physical standby to snapshot standby and vice versa. However, you can implement DR Drill capability using the Snapshot Standby Database feature for Base Database Service and Exadata Database on Dedicated Infrastructure Service by customizing DR plans with your own user-defined DR plan groups and steps.
Executing a Start DR drill plan stands up a replica of the production application stack in the standby region just like an actual failover, with the caveat that production is not disrupted. Full Stack DR uses features like refreshable clones for Oracle Autonomous Database Serverless, OCI storage replication for block storage, and file storage to create new VMs and file systems during a DR drill. The following illustrates some essential points about Start DR drill plan types:
- A DR drill creates a replica of the production application stack in the standby region, as mentioned above.
- The status of the primary and standby DR protection groups will be set to “Inactive (Drill in Progress)” until you execute a stop DR drill plan.
- Changes to the DR protection groups are restricted while a DR drill is in progress.
- You cannot use any Switchover or Failover plans while a DR drill is in progress; all other plans in the DR protection group will be in an “Inactive” state.
Executing a Stop DR drill plan tears down the replicated application stack created by the Start DR drill plan. The following illustrates some critical points about Stop DR drill plan types:
- A “Stop DR Drill” plan can only be created after a “Start DR Drill” plan has been executed, and the primary and standby DR protection groups are in an “Inactive (Drill in Progress)” state.
- Stop DR drill plans remove the replica of the production stack created during the “Start DR drill.”
- The status of Primary and Standby DR protection groups is set to Active after a stop DR drill has been completed.
- All other plans become Active, and changes to DR protection groups are again allowed after a stop DR drill has been completed.
A step-by-step example
To be clear, you only need to create start DR drill plans once. Your IT staff use the Start DR drill plan repeatedly as required. IT governance policies in your organization drive the frequency with which DR drills are executed.
In this example, Ashburn (IAD) is the primary region, and Phoenix (PHX) is the DR region. We will use a Cold VM Standby topology with an Autonomous (ADB) Serverless Database. Virtual machines will run only in the Primary region. The primary database will run in the IAD region, and the standby database will run in the PHX region. Create the below OCI resources.
- Two VM’s (instance-1 and instance-2) in the IAD region
- Cross-region (IAD to PHX) volume groups as vg-inst in the IAD region for the boot volumes of the virtual machines instance-1 and instance-2
- Autonomous Transaction Processing- Serverless (fsdratp) in the IAD region and set up the cross-region autonomous Data Guard (fsdratp_Remote) in the PHX region
Please take a look at the OCI Compute, Block Storage, and ADB Serverless documentation for more information.
Let’s create DR Drill plans for this DR topology.
1. Create & associate DR protection groups and add members.
- In the Ashburn (IAD) region, create a DR protection group as “Drill-IAD.”
- In the Phoenix (PHX) region, create a DR protection group as “Drill-PHX.”
- Associate the “Drill-IAD” DR protection group as the Primary role and the “Drill-PHX” DR protection group as the Standby role
- In the “Drill-IAD” DR protection group, add the two compute instances (instance-1 and instance-2) as members with the moving-instance option
- In the “Drill-IAD” DR protection group, add the volume group (vg-inst) as a member
- In the “Drill-IAD” DR protection group, add the primary database (fsdratp) as a member
- In the “Drill-PHX” DR protection group, add the standby database (fsdratp_Remote) as a member
Please refer to OCI Full Stack DR documentation and Full Stack DR move VM tutorial for more information.
2. Steps for creating a Start Drill plan.
Navigate to the DR Protection Group page in the Phoenix region.
- In the “Drill-PHX” DR protection group, click the Create Plan dialog.
- Provide “Start Drill-PHX“ as the name of the plan.
- Select “Start Drill“ as the plan type.
- Click on “Create“ to finish creating the Start Drill plan.
Similarly, create switchover and failover plans; you can see the switchover, failover, and start drills plans are available.
You will get an error if you try to create a “Stop drill” plan. You can create a “Stop drill” only after executing a start drill plan, and the primary and standby DR protection groups are in an ” Inactive (Drill in Progress”) state.
3. Let’s explore the “Start Drill-PHX” plan we created.
We have built-in plan groups to restore volume groups, create ADB refreshable clones, and launch compute instances. We can customize the plan and add our user-defined plan groups. However, we will not be customizing this Start Drill plan in this example.
4. Let’s run a precheck for the “Start Drill-PHX” plan.
After a while, the “Start Drill – PHX” prechecks plan execution succeeds.
5. Now, execute the “Start Drill-PHX” plan and monitor the status.
After a while, the “Start Drill – PHX” plan execution succeeds.
The “Start drill” plan will create the resources (block volumes, compute instances, ADB refreshable clone) in the standby region without any impact on the production region. These resources, created by the drill in the standby region, can be used for validating and testing the business application. Once the validation and testing are over, the drill can be ended by creating and executing a “Stop Drill” plan.
As soon as the “Start Drill-PHX“ plan succeeds, the DR protection group state changes to “Inactive (Drill in progress),” and changes to the DR protection groups are restricted. All DR plans ( switchover/failover/start drill) will change to “Inactive.”
6.Steps for creating a Stop Drill Plan.
Navigate to the DR Protection Group page in the Phoenix region.
- In the “Drill-PHX” DR protection group, click the Create Plan dialog.
- Provide “Stop Drill-PHX” as the name of the plan.
- Select “Stop Drill” as the plan type.
- Click on “Create” to finish creating the Stop Drill plan.
7.Let’s explore the “Stop Drill-PHX“ plan which we have created.
We have built-in plan groups for stopping and terminating compute instances, terminating volume groups, deleting ADB refreshable clones, and other plan groups for removing members from the DR protection group. We can add user-defined groups, if any; we are not adding user-defined plan groups. The stop drill plan will remove all the necessary resources created by the start drill plan.
8.Let’s run a precheck for the “Stop Drill-PHX” plan
After a while, the “Stop Drill – PHX” prechecks plan execution succeeds.
9.Let’s execute the “Stop Drill-PHX“ plan and monitor the status. After a while, the “Stop Drill – PHX” plan execution succeeds.
All the resources (compute, ADB refreshable clones) created during the “Start drill” plan are terminated in the standby region.
10. As soon the Stop drill plan succeeds, other DR plans ( switchover/failover/start drill), which were inactive, will become active. Stop drill will become inactive.
You can watch the demo of the DR Drills using OCI Full Stack DR – Drill Plans
Conclusion
Using disaster recovery(DR) drill plans is highly valuable for fulfilling regulatory requirements, conducting DR testing, and validating disaster recovery processes. Different types of DR drill plans within Full Stack DR are essential for efficiently carrying out DR drills.
Additional resources
Here are some additional resources to help you get started with Full Stack Disaster Recovery:
- Full Stack Disaster Recovery product page
- Full Stack Disaster Recovery User Guides
- Full Stack Disaster Recovery API Reference Guide
- OCI Base DB documentation
- OCI Exadata Database on Dedictated Infrastructure documentation
- OCI Autonomous Database Serverless documentation
- OCI Compute documentation
- OCI Block Storage documentation
- Try out Full Stack Disaster Recovery using our hands-on lab
Feel free to connect with me directly on Twitter and LinkedIn