ASM Cluster File System Snapshots on Exadata

Database snapshots are a regularly required feature of any application development project. With Oracle Database and on Oracle Exadata, there are many ways of creating and maintaining database clones or copies, including PDB Clones, Full Database Clones, Exadata Sparse Clones, and now ACFS Snapshot Clones on Exadata!

Database clones are needed to allow organizations to test and develop against production-like database environments. Sometimes the volume of data is important, other times less so. Sometimes each developer should have (or wants) a separate environment for their work before code is moved into system test, integration, performance test environments. Some use cases for database clones may have nothing to do with development or testing - at least not in the IT sense of these terms - some organizations may want to furnish their analysts with a database for what/if analysis or model training, tuning, etc without impacting their production database. There are many reasons why a database clone may be required, and as I mentioned above, there are many different ways of supporting this requirement on Exadata.

The ASM Cluster File System (ACFS) is a POSIX compliant general-purpose file system that can also be used to house database files on Exadata. ACFS supports advanced snapshot capabilities similar to third-party copy-on-write style filesystem snapshots. Of particular note when using ACFS Snapshots, customers can create read-write test masters spanning multiple timelines.

Conceptually, something like the following is possible:

Let's unpack that a little.

Firstly, to use ACFS Snapshots, you need to have a database - say a Data Guard Standby Database - on ACFS. You could also use gDBClone or use RMAN to create a duplicate database on ACFS. ACFS Snapshots are filesystem snapshots, unlike Exadata Sparse Clones which are sparse files that point back to a read-only datafile.

Once you have a "source" database, you would then stop redo apply, create an ACFS Snapshot - a Test Master in the above diagram - and then restart redo apply in the standby database.

What's happening now is the ACFS Snapshot is static, but the standby is moving forward again as the primary database sends redo to it. As the standby moves forward ACFS is copying changed blocks into the snapshot directory so the "original" image of the block is available for queries and DML. Note that the initial snapshot level is read-only.

You can then create multiple read-write ACFS Snapshots and database instances with which to use the datafiles. These can in turn can be a read-write "Test Master" for subsequent read-write snapshots and databases.

If you have started with a Data Guard Standby Database, you can repeat the process of stopping redo apply, creating a new ACFS Snapshot - at the current time - and restarting redo apply, allowing multiple timelines to exist simultaneously using the same Standby as the ultimate source of data blocks.

Another way of presenting this is as follows. The opaque database at the top is our Standby Database on ACFS that continually moves forward in time as it receives and applies redo. At various intervals, ACFS Snapshots are created to act as a read-only Test Master, and then multiple snapshots at varying levels are created for use as read-only or read-write snapshot databases.

A few important points to make:

• ACFS on Exadata supports only the Exadata Flash Cache - features such as (but not limited to) Smart Scan, Storage Indexes, Persistent Memory Data and Commit Accelerators are not available to databases located on ACFS
• ACFS Snapshots on Exadata are for Dev/Test only
• Oracle Grid Infrastructure 19.10 (with performance patches) or higher is required. Grid Infrastructure 19.11 is recommended
• ACFS Encryption is NOT supported for database data files - if you need to encrypt data in any Oracle Database, you should use Transparent Data Encryption.
• ACFS Snapshots support all types of Oracle Database:
  • CDB Databases and all Plugable Databases in said CDB
  • Individual Pluggable Databases
  • Non-CDB Databases
• If you're using Data Guard with ACFS, make sure you have a "true" DR Data Guard Standby Database on ASM. If you need to failover to DR, you don't want to lose all the Exadata performance features.
• For highest availability, the ASM diskgroups that the ACFS Filesystems are created from should be High Redundancy diskgroups
• Do not use ACFS for database homes, diagnostics, or audit destinations
• Do not co-locate dev/test databases with production databases on the same RAC or VM Cluster

You can find more on ACFS Snapshots and Exadata Sparse Clones in this presentation from the MAA Team

These MOS Notes and documentation links are also well worth checking out as you get stuck into creating database clones.

• Oracle ACFS Support on Oracle Exadata Database Machine (Linux only) (Doc ID 1929629.1)
• Oracle ACFS Snapshot Use Cases on Exadata (Doc ID 2761360.1)
• Oracle Automatic Storage Management Cluster File System - Administrator's Guide
• Oracle System Software – User’s Guide for Exadata Sparse for comparison
• Exadata Snapshots: Simple, Secure, Space Efficient

I'd like to thank the MAA, Exadata, and of course ACFS development teams for all the work that has gone into delivering and testing ACFS Snapshots on Exadata.

Let us know what you think about ACFS Snapshots and Exadata Sparse Clones in the comments below.

Happy snapshotting!