X

Maximum Availability Architecture – Oracle’s industry-leading set of database high availability capabilities

Physical migration of an AWS RDS Oracle Database to Oracle Cloud Infrastructure via RMAN backup and restore

Marco Calmasini
Sr. Principal Product Manager

Part 2 of our AWS to OCI migration blog series is about a complete physical migration of an AWS RDS Oracle instance to Oracle Cloud Infrastructure Database Cloud Service or Exadata Cloud Service instance using Recovery Manager (RMAN) backups and Oracle DB Backup Cloud Service.

In part 1 we covered how to copy backups from AWS S3 to OCI.

 

OCI offers several advantages for Oracle Database workloads over AWS:

  • All Database options are available
  • Instances can run on Virtual Machines, Bare Metal or Exadata Infrastructure
  • Support for Real Application Clusters
  • Hybrid deployments are easier

 

Environments used

To perform this exercise we used an AWS db.t3.small instance running Oracle Database EE 19.6. Our target environment is a OCI Database Cloud Service VM.Standard2.1 instance running Oracle Database EE 19.6. The two main differences to consider are that the AWS RDS instance is a non-Multitenant DB and not TDE-encrypted. In OCI DBCS and ExaCS, Multitenant DB  and TDE are the mandatory deployment model. The procedure will first migrate the AWS Oracle DB as-is to OCI, then convert it to multitenant and finally implement TDE.

 

The Storage for both environments is LVM. If migrating to ASM, this conversion can be done at the PDB adoption step.

 

AWS RDS Configuration

 

Engine version 19.0.0.0.ru-2020-01.rur-2020-01.r1

Character set AL32UTF8

Option groups s3-integration-db2

Parameter group default.oracle-ee-19 (in-sync)

Instance class db.t3.small

Multi-Tenant   NON-CDB

Storage Encryption Enabled (NO TDE)

 

 

Oracle Database Cloud Service Configuration

 

Shape: VM.Standard2.1

Oracle Database Software Edition: Enterprise Edition High Performance

Storage Management Software: Logical Volume Manager

Database Version: 19.6.0.0.200114

Patch History: Jan 2020 19c Database patch

TDE Encryption: ON (Mandatory)

Multi-Tenant: YES (Mandatory)

 

 

 

Migration Steps

Migration Steps: 1. Prepare the AWS instance  2.Take a full backup to the  local block storage 3.Copy the backup from the AWS Block Storage to AWS S3 4.Copy the backup from S3 to OCI Object Storage 5.Restore the non-CDB backup to a temporary instance in DBCS/ExaCS  6.Adopt the restored non-CDB as a PDB in the DBCS/EcaCS instance 7.Encrypt the PDB tablespaces

 

These are the steps to follow to perform the migration

  1. Prepare the AWS instance for local RMAN backup
  2. Take a full backup to the  local AWS block storage
  3. Copy the backup from the AWS Block Storage to AWS S3
  4. Copy the backup from AWS S3 to OCI Object Storage
  5. Restore the non-CDB backup to a temporary instance on the DBCS/ExaCS instance
  6. Adopt the restored non-CDB as a PDB in the DBCS/EcaCS instance
  7. Encrypt the PDB tablespaces

 

  1. Prepare the AWS Instance

AWS RDS does not provide access to RMAN or the Operating System. All operations must be performed via sqlplus using the RDS provided tooling

 

Create a backup destination on AWS RDS local disk

SQL> exec rdsadmin.rdsadmin_util.create_directory(p_directory_name => 'BACKUP');

 

If archived log retention is not enabled, enable it

SQL> exec  rdsadmin.rdsadmin_util.set_configuration( name  => 'archivelog retention hours', value => '48');

 

Create the S3 bucket and perform additional configuration steps via the AWS Console or CLI in order to grant the RDS instance write privileges on the S3 bucket

  • Create the bucket using the RDS CLI or the S3 Console
  • Create an IAM Policy to allow RDS to access the S3 bucket with the appropriate action
  • Create an IAM role
  • Associate the IAM role with the RDS instance
  • Associate the RDS instance with an option group including the S3_INTEGRATION option

Refer to the AWS documentation on how to configure the RDS – S3 integration

 

  1. Take the full backup

Use the rdsadmin_rman_util package to create a full backup plus archived logs to the local AWS block storage directory created above in step 1.

SQL>exec rdsadmin.rdsadmin_rman_util.backup_database_full ( p_owner => 'SYS', p_directory_name => 'BACKUP', p_include_archive_logs => TRUE, p_parallel => 4, p_section_size_mb => 100, p_rman_to_dbms_output => FALSE );

 

Verify that the backup pieces are created:

SQL> SELECT * FROM table(rdsadmin.rds_file_util.listdir('BACKUP')) order by mtime;

FILENAME                                                  TYPE        FILESIZE  MTIME

--------------------------------------------------------- ----------  ---------- ---------

BACKUP-2020-03-04-04-07-46-backup-20200304-03uq7aso_1_1   file          4137984  04-MAR-20

BACKUP-2020-03-04-04-07-46-backup-20200304-04uq7aso_1_1   file           522752  04-MAR-20

BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1   file         10772480  04-MAR-20

BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1   file         275087360 04-MAR-20

BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1   file         382230528 04-MAR-20

BACKUP-2020-03-04-04-07-46-backup-20200304-08uq7at9_1_1   file              9216 04-MAR-20

BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00       file           8749056 04-MAR-20

01/                                                       directory         4096 04-MAR-20

8 rows selected.

 

  1. Copy the backup set to the S3 bucket

Copy the backup pieces from the ‘BACKUP’ location to your S3 bucket. In this example, the bucket name is rman-database-2, p_prefix and p_s3_prefix are empty strings.

 

SQL> SELECT rdsadmin.rdsadmin_s3_tasks.upload_to_s3(

      p_bucket_name    => 'rman-database-2',

      p_prefix         =>'',   

      p_s3_prefix      =>'',

      p_directory_name => 'BACKUP')

   AS TASK_ID FROM DUAL;

 

This returns the task-id, which you can query to check the results

SQL> SELECT text FROM table(rdsadmin.rds_file_util.read_text_file('BDUMP','dbtask-<task-id>.log'));

Once the backup completes, use the following query to obtain a list of the backup piece handles (highlighted in red below, excluding the directory name) and save them to a notepad, as they will be needed later for restore:

 

SQL> col handle format A90

SQL> set lines 333

 

SQL> SELECT d.file#  , p.handle

  FROM v$backup_piece p, v$backup_datafile d

 WHERE d.set_stamp = p.set_stamp

   AND d.set_count = p.set_count;

 

     FILE# HANDLE

---------- -------------------------------------------------------------------------------------

        3 /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1

        4 /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1

        2 /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1

        5 /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1

        1 /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1

        0 /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00

6 rows selected.

 

 

These are the backup piece handles from the backup example above (highlighted in red)

 

BACKUP-2020-03-04-04-07-46-backup-20200304-04uq7aso_1_1

BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1

BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1

BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1

BACKUP-2020-03-04-04-07-46-backup-20200304-08uq7at9_1_1

BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00

 

  1. Copy the backup pieces from S3 to OCI Object Storage

 

Follow the instructions in part 1 of this blog series to copy the backup piece objects to an OCI bucket and then to set up the Database Backup Cloud Module on the OCI instance in preparation for restore. Note that the assumption in the blog was that the objects in S3 were written directly from RMAN using the OSB Cloud Module for AWS S3, which can only be done from databases running on AWS compute instances or on-premises, not from RDS instances. For RDS, the backups were  written to disk (local AWS block storage), then copied to S3 bucket. After transferring the backups to OCI, they will need to be converted to the “cloud object storage format” which will be done in step 5 below.

Once the backup pieces are in the OCI bucket, prepare the OCI destination instance for restore by Installing the Database Backup Cloud Module following the documentation

NOTE: When you create the DBCS or ExaCS target instance, make sure to use a name (both for the instance and the PDB) that is different from the RDS DB name.

 

  1. Restore the non-CDB backup to a temporary instance in DBCS

Set the ORACLE_SID environment variable to the instance name of the RDS DB

 

$> RMAN target /

 

Start as a dummy instance in nomount mode and set the DBID

RMAN> startup nomount

RMAN> set DBID=<RDS DBID>

 

The backup was originally sent by RMAN to DISK, so it’s missing the xml manifest files RMAN needs to be able to restore directly from Cloud Object Storage. We can use the “export backuppiece” command for each backup piece in order create manifest files.

 

RMAN> run {

               allocate channel t1 device type sbt parms='SBT_LIBRARY=/home/oracle/ocilib/libopc.so ENV=(OPC_PFILE=/home/oracle/database-2.cfg)';

               send channel t1 'export backuppiece BACKUP-2020-03-04-04-07-46-backup-20200304-04uq7aso_1_1';

                    send channel t1 'export backuppiece BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1';

                    send channel t1 'export backuppiece BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1';

                    send channel t1 'export backuppiece BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1';

                    send channel t1 'export backuppiece BACKUP-2020-03-04-04-07-46-backup-20200304-08uq7at9_1_1';

                    send channel t1 'export backuppiece BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00';

                   

          }
 

 

At this point the AWS disk backups are now converted to Cloud Object Storage format, so we can restore those backups, starting with spfile and controlfile from the controlfile autobackup (the one which includes “c-<DBID>” part)

 

RMAN> run {

               allocate channel t1 device type sbt parms='SBT_LIBRARY=/home/oracle/ocilib/libopc.so ENV=(OPC_PFILE=/home/oracle/database-2.cfg)';

               restore spfile to '/home/oracle/awsspfile.ora' from 'BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00';

          }

 

Let’s create a pfile from the spfile we just restored

 

SQL> create pfile=’/home/oracle/rdspfile.ora’ from spfile ‘/home/oracle/awsspfile.ora’;

 

Edit the pfile to delete the spfile line and note controlfile, datafiles, audit and logs file names and locations. Prepare the OCI DB environment by creating the appropriate directories.

 

In this example, we will use the same paths and filenames as the RDS DB, if you want to follow this same approach make sure the target filesystem is large enough for a complete DB restore.

 

Shutdown the dummy instance

 

RMAN>shutdown immediate

 

Copy the pfile to the appropriate $ORACLE_HOME/dbs/init<SID>.ora

 

Start the instance, specifying the pfile:

 

RMAN> Startup nomount pfile=’<pfilename>’;

 

Now, restore the controlfile

run {

               allocate channel t1 device type sbt parms='SBT_LIBRARY=/home/oracle/ocilib/libopc.so ENV=(OPC_PFILE=/home/oracle/database-2.cfg)';

               restore controlfile from 'BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00';

          }

 

RMAN> startup mount

 

Configure  the RMAN automatic channels

 

RMAN> CONFIGURE CHANNEL DEVICE TYPE 'SBT_TAPE' MAXPIECESIZE 2 G FORMAT  '%d_%U' PARMS  'SBT_LIBRARY=/home/oracle/ocilib/libopc.so ENV=(OPC_PFILE=/home/oracle/database-2.cfg)';

RMAN> configure default device type to SBT_TAPE;

 

Catalog the backup pieces

RMAN> catalog device type sbt backuppiece 'BACKUP-2020-03-04-04-07-46-backup-20200304-03uq7aso_1_1','BACKUP-2020-03-04-04-07-46-backup-20200304-04uq7aso_1_1', 'BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1','BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1','BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1','BACKUP-2020-03-04-04-07-46-backup-20200304-08uq7at9_1_1','BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00';

 

allocated channel: ORA_SBT_TAPE_1

channel ORA_SBT_TAPE_1: SID=175 device type=SBT_TAPE

channel ORA_SBT_TAPE_1: Oracle Database Backup Service Library VER=19.0.0.1

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-03uq7aso_1_1 RECID=7 STAMP=1037318207

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-04uq7aso_1_1 RECID=8 STAMP=1037318207

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1 RECID=9 STAMP=1037318208

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1 RECID=10 STAMP=1037318209

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1 RECID=11 STAMP=1037318210

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-08uq7at9_1_1 RECID=12 STAMP=1037318210

channel ORA_SBT_TAPE_1: cataloged backup piece

backup piece handle=BACKUP-2020-03-04-04-07-46-c-1559204751-20200304-00 RECID=13 STAMP=1037318211

 

 

 

Crosscheck the backup and delete expired backup to remove the entries for the AWS block storage backups which are still in the controlfile

 

RMAN> crosscheck backup of database;

using channel ORA_SBT_TAPE_1

using channel ORA_DISK_1

crosschecked backup piece: found to be 'EXPIRED'

backup piece handle=/rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1 RECID=3 STAMP=1034136473

crosschecked backup piece: found to be 'AVAILABLE'

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1 RECID=11 STAMP=1037318210

crosschecked backup piece: found to be 'AVAILABLE'

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1 RECID=10 STAMP=1037318209

crosschecked backup piece: found to be 'AVAILABLE'

backup piece handle=BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1 RECID=9 STAMP=1037318208

crosschecked backup piece: found to be 'EXPIRED'

backup piece handle=/rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1 RECID=4 STAMP=1034136473

crosschecked backup piece: found to be 'EXPIRED'

backup piece handle=/rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1 RECID=5 STAMP=1034136473

Crosschecked 6 objects

 

Delete expired backups

RMAN> delete expired backup;

using channel ORA_SBT_TAPE_1

using channel ORA_DISK_1

 

List of Backup Pieces

BP Key  BS Key  Pc# Cp# Status      Device Type Piece Name

------- ------- --- --- ----------- ----------- ----------

3       3       1   1   EXPIRED     DISK        /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1

4       4       1   1   EXPIRED     DISK        /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1

5       5       1   1   EXPIRED     DISK        /rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1

 

Do you really want to delete the above objects (enter YES or NO)? YES

deleted backup piece

backup piece handle=/rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-07uq7asp_1_1 RECID=3 STAMP=1034136473

deleted backup piece

backup piece handle=/rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-06uq7asp_1_1 RECID=4 STAMP=1034136473

deleted backup piece

backup piece handle=/rdsdbdata/userdirs/01/BACKUP-2020-03-04-04-07-46-backup-20200304-05uq7asp_1_1 RECID=5 STAMP=1034136473

Deleted 3 EXPIRED objects

 

At this point we are ready to restore and recover the database

 

RMAN> restore database;

Starting restore at 10-APR-20

using channel ORA_SBT_TAPE_1

using channel ORA_DISK_1

 

[…]

 

Finished restore at 10-APR-20

 

RMAN> recover database until available redo;

Starting recover at 10-APR-20

using channel ORA_SBT_TAPE_1

using channel ORA_DISK_1

 

starting media recovery

 

[…]

Finished recover at 10-APR-20

 

Open the database to verify it

 

RMAN> alter database open resetlogs;

 

Statement processed    

 

 

  1. Adopt the restored non-CDB as a PDB in the DBCS instance

 

 

Follow the note: How to Convert Non-CDB to PDB Database in 12c - Testcase (Doc ID 2012448.1)

 

 

SQL>shutdown immediate

Database closed.

Database dismounted.

ORACLE instance shut down.

 

SQL> startup open read only;

ORACLE instance started.

 

Total System Global Area  1392508864 bytes

Fixed Size                   8896448 bytes

Variable Size              805306368 bytes

Database Buffers           570425344 bytes

Redo Buffers                7880704 bytes

Database mounted.

Database opened.

 

Create the manifest file that will be used to check the CDB/PDB compatibility and to create the PDB

 

SQL> BEGIN

DBMS_PDB.DESCRIBE(pdb_descr_file => '/home/oracle/awsORCL.xml');

    END;

    /

 

PL/SQL procedure successfully completed.

 

Now we can shutdown the temporary instance.

SQL> shutdown immediate;

Database closed.

Database dismounted.

ORACLE instance shut down.

 

Reset  the ORACLE_SID variable to the original DBCS instance name and launch sqlplus to connect to it as sysdba

 

Run the compatibility analysis using the manifest file

 

SQL> SET SERVEROUTPUT ON;

DECLARE compatible CONSTANT VARCHAR2(3) := CASE DBMS_PDB.CHECK_PLUG_COMPATIBILITY(pdb_descr_file => '/home/oracle/awsORCL.xml')

WHEN TRUE THEN 'YES’

ELSE 'NO’

END;

BEGIN

DBMS_OUTPUT.PUT_LINE(compatible);

END;

/

 

Check the results

SQL> col name for a20

SQL> col message for a35 word_wrapped

SQL> select time, name,cause,type,message,status from PDB_PLUG_IN_VIOLATIONS where name='ORCL';

 

Handling any messages for ERRORS or WARNINGS.

The ERRORS must be resolved. If you find patches installed on the PDB but not on the CDB, download them from MOS and install them on the DBCS or ExaCS instance.

The WARNINGS can be related to options configured on the CDB but not on the PDB, that’s fine and these can be ignored. WARNINGS related to the PDB tablespaces not being encrypted, can also be ignored for now, since encryption is performed in later step below. In the example there was a warning about a Time Zone file version mismatch. The PDB had Time Zone file version 34 and the CDB had Time Zone version 32. Even as a warning it must be resolved by searching for the patch corresponding to the required Time Zone file version and installing it on the CDB.

 

We are now ready to create the PDB

SQL> CREATE PLUGGABLE DATABASE ORCL USING '/home/oracle/awsORCL.xml'  COPY;

SQL> ALTER PLUGGABLE DATABASE OPEN;

Pluggable database altered with warnings;

 

If there was a patch mismatch and new patches were installed on the CDB, the PDB will open in restricted mode.

 

SQL> show pdbs

 

    CON_ID CON_NAME                       OPEN MODE   RESTRICTED

---------- ------------------------------ ----------  ----------

       2    PDB$SEED                      READ ONLY   NO

       3    DB0218_PDB1                   READ WRITE  NO

       4    ORCL                          READ WRITE  YES

 

 

If that’s the case exit sqlplus and run the datapatch patch script located in the OPatch directory. This will install the patches in the PDB.

 

After datapatch completes you can open the PDB successfully.

 

SQL> ALTER PLUGGABLE DATABASE OPEN READ WRITE;

Pluggable database altered.

 

 

  1. Encrypt the PDB tablespaces

 

This is the last step, our AWS RDS database is now open as a PDB in our DBCS instance, but it must be encrypted. First we must create the MASTER KEY for this PDB

 

Connect to the PDB and create the MASTER KEY (the wallet password is the same as the SYS user password)

 

SQL> alter session set container=ORCL;

Session altered.

SQL> ADMINISTER KEY MANAGEMENT SET KEY USING TAG 'RDSDBtag' FORCE KEYSTORE IDENTIFIED BY <password> WITH BACKUP USING 'backup1';

keystore altered.

 

All the tablespaces must now be encrypted similar to the example below.

SQL> alter tablespace SYSTEM encryption online using 'AES128' encrypt;

Tablespace altered.

 

  1. Conclusion

Once all the tablespaces have been encrypted the RDS DB is now up and running as a Database Cloud Service or Exadata Cloud Service in OCI.  A slight variation to this procedure, which is useful for large DBs and helps reduce the cutover time, would be to take a L1 incremental backup on the RDS DB, following the same procedure to copy the backup to OCI, and finally applying the incremental to the restored OCI DB to catch it up to more current time.

 

For more details on the AWS RDS preparation and backup operations refer to the AWS documentation.

How to create the backup destination directory in AWS RDS

How to configure the RDS-S3 integration

AWS RDS RMAN reference

Copying data from an RDS local directory to an S3 bucket

 

For more details on how to convert a non-CDB database to a PDB

How to Convert Non-CDB to PDB Database in 12c - Testcase (Doc ID 2012448.1)

After running datapatch, PDB plugin or cloned db returns violations shown in PDB_PLUG_IN_VIOLATION (Doc ID 1635482.1)

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.