Tuesday Jul 07, 2015

Snap Cloning Databases on Exadata using Enterprise Manager

Historically, Exadata has mostly been deployed for heavy, production workloads, leaving cheap commodity hardware and third-party storage to perform as infrastructure for Development and Testing. From the viewpoint of Enterprise Manager, we have seen customers clone production databases running on Exadata to secondary storage such as ZFS Storage Appliance or even third-party NAS or SAN for the purpose of testing. Customers mainly used RMAN (with or without Enterprise Manager) to clone the databases. While the master clones (often referred to as Test Master) could be further cloned via storage efficient snapshots, there were significant limitations to the approach.

  • First of all, the testing on non-like systems from Exadata (both compute and storage) often yielded erroneous inferences.
  • Second, both the compute and storage on existing Exadata racks often remained underutilized.

Most surveys establish that there are several Dev/Test copies for every Production database, and leaving Dev/Test outside the realm of Exadata can only yield partial usage of engineered systems.

Two recent advancements in Exadata break this existing barrier. First of all, the compute nodes on Exadata can now be virtualized. Consolidated environments can now use Oracle Virtual Machine (OVM) on X5-2, X4-2, X3-2, and X2-2 database servers to deliver higher levels of consolidation and isolation between workloads. The Virtual machines can be configured on demand with the appropriate number of Virtual CPUs (vCPUs) for iterative testing. Second, with Oracle Database 12c Release 1 (12.1) release BP5 or later, space-efficient database snapshots can now be quickly created for test and development purposes on Exadata.

Snapshots start with a shared read-only copy of the production database (referred to as the Test Master in Enterprise Manager Parlance) that has been masked and/or cleansed of any sensitive information. Snapshot technology as deployed on Exadata is "allocate on first write", not copy on write. As changes are made, each snapshot writes the changed blocks to a sparse disk group. Multiple users can create independent snapshots from the same base database, therefore multiple test and development environments can share space while maintaining independent databases for each task. The base database must remain read-only during the usable lifespan of snapshots that are created from that base database. If there is a need to create another set of snapshots from a more recent copy of production data, a new read-only base from a production database needs to be created.

Enterprise Manager 12cR5 leverages the capabilities of Exadata to extend the Snap Clone capabilities for Exadata Sparse clones. As shown in this tutorial, with Snap Clone, Enterprise Manager can create a Test Master using either Dataguard technology or RMAN preceded by data masking. The Test Master can be created either on the same Exadata rack or on a different one. Once the Test Master has been created, snapshots can then be created on the sparse disk groups using the Deployment Procedures. The Deployment Procedures also automate the post-cloning discovery and promotion of the cloned targets, making them fully managed right from inception. Internal testing confirms that for cloning a Terabyte of database with a complete discovery of all its components takes less than a minute.

Enterprise Manager also helps DBAs track the lineage of the clones by providing a report on the production database, the Test Master and its clones. Enterprise Manager Snap Clone on Exadata supports both regular as well as pluggable databases with optional ACFS configuration. In addition to support for Exadata sparse clones, Snap Clone continues to support NAS (ZFS Storage Appliance and NetApp) and SAN (certain EMC storage arrays), in case users want to deploy these for their Dev-Test environments. 

Further Reading Resources

  • To learn more about Database as a Service visit the otn page.
  • Prerequisites for setting up Exadata Snapshots are documented here.
  • Watch the video, Snap Clone Multitenant (Pluggable Database) on Exadata here.
--Subhadeep Sengupta (@Subh_here)

Thursday Nov 13, 2014

Will the REAL Snap Clone functionality please stand up?

Possibly one of the least known pieces of functionality that is provided as part of the Enterprise Manager Cloud Control 12c product suite is the Snap Clone product. In simple terms, Snap Clone is a storage agnostic self service approach to rapidly creating space efficient clones of large databases (and by large, we're talking terabytes or more).  I blogged on that a few months back, but a few months can be an eternity in software development terms, so here's an update that covers not just the functionality in the product but also addresses some of the misleading statements some of our competitors have been making about it of recent times.

Tuesday Jul 01, 2014

Limit Self Service User Access to Database Self Service Portal

When implementing database as a service and/or snap clone, a common request was for a way to hide the other service types like IaaS, MWaaS, etc from the self service portal for the end users. Before EM12c R4, there was no way to restrict the portal view. Essentially, any user with the EM_SSA_USER role would be directed to the self service portal and would then be able to see all service types supported by EM12c.

Of course, you could always set Database as your default self service portal from the 'My Preferences' pop up, but this only helps with their post-login experience. The end user still gets to see all the options as shown in screen above.

In EM12c R4, a new out of the box role called EM_SSA_USER_BASE has been introduced. This role, by default, does not give access to any portal, that is an explicit selection. Here is how you use this role:

1. Create a custom role and add the EM_SSA_USER_BASE role to it.

 2. Now in the Resource Privileges step, select the Resource Type 'Cloud Self Service Portal for Database', and edit it

3. Check the 'Access the Cloud Self Service Portal for Database.' privilege. Finish the rest of the wizard.

 Now, when a user with this custom role accesses the self service portal, they can only do so for databases and nothing else.

While the EM_SSA_USER role will continue to work, we recommend you start using the new EM_SSA_USER_BASE role. For more details on DBaaS or Snap Clone roles, refer to the cloud admin guide chapter on roles and users.

-- Adeesh Fulay (@AdeeshF)

Wednesday Jun 04, 2014

EM12c Release 4: Database as a Service Enhancements

Oracle Enterprise Manager (or simply put EM12c R4) is the latest update to the product. As previous versions, this release provides tons of enhancements and bug fixes, attributing to improved stability and quality. One of the areas that is most exciting and has seen tremendous growth in the last few years is that of Database as a Service. EM12c R4 provides a significant update to Database as a Service. The key themes are:

  1. Comprehensive Database Service Catalog (includes single instance, RAC, and Data Guard)
  2. Additional Storage Options for Snap Clone (includes support for Database feature CloneDB)
  3. Improved Rapid Start Kits
  4. Extensible Metering and Chargeback
  5. Miscellaneous Enhancements

1. Comprehensive Database Service Catalog

Before we get deep into implementation of a service catalog, lets first understand what it is and what benefits it provides. Per ITIL, a service catalog is an exhaustive list of IT services that an organization provides or offers to its employees or customers. Service catalogs have been widely popular in the space of cloud computing, primarily as the medium to provide standardized and pre-approved service definitions. There is already some good collateral out there that talks about Oracle database service catalogs. The two whitepapers i recommend reading are:

Service Catalogs: Defining Standardized Database Service

High Availability Best Practices for Database Consolidation: The Foundation for Database as a Service [Oracle MAA]

EM12c comes with an out-of-the-box service catalog and self service portal since release 1. For the customers, it provides the following benefits:

  • Present a collection of standardized database service definitions,
  • Define standardized pools of hardware and software for provisioning,
  • Role based access to cater to different class of users,
  • Automated procedures to provision the predefined database definitions,
  • Setup chargeback plans based on service tiers and database configuration sizes, etc

Starting Release 4, the scope of services offered via the service catalog has been expanded to include databases with varying levels of availability - Single Instance (SI) or Real Application Clusters (RAC) databases with multiple data guard based standby databases. Some salient points of the data guard integration:

  • Standby pools can now be defined across different datacenters or within the same datacenter as the primary (this helps in modelling the concept of near and far DR sites)
  • The standby databases can be single instance, RAC, or RAC One Node databases
  • Multiple standby databases can be provisioned, where the maximum limit is determined by the version of database software
  • The standby databases can be in either mount or read only (requires active data guard option) mode
  • All database versions 10g to 12c supported (as certified with EM 12c)
  • All 3 protection modes can be used - Maximum availability, performance, security
  • Log apply can be set to sync or async along with the required apply lag

The different service levels or service tiers are popularly represented using metals - Platinum, Gold, Silver, Bronze, and so on. The Oracle MAA whitepaper (referenced above) calls out the various service tiers as defined by Oracle's best practices, but customers can choose any logical combinations from the table below:

 Primary  Standby [1 or more]
 EM 12cR4
 SI  -
where RON = RAC One Node is supported via custom post-scripts in the service template

A sample service catalog would look like the image below. Here we have defined 4 service levels, which have been deployed across 2 data centers, and have 3 standardized sizes. Again, it is important to note that this is just an example to get the creative juices flowing. I imagine each customer would come up with their own catalog based on the application requirements, their RTO/RPO goals, and the product licenses they own. In the screenwatch titled 'Build Service Catalog using EM12c DBaaS', I walk through the complete steps required to setup this sample service catalog in EM12c.

2. Additional Storage Options for Snap Clone

In my previous blog posts, i have described the snap clone feature in detail. Essentially, it provides a storage agnostic, self service, rapid, and space efficient approach to solving your data cloning problems. The net benefit is that you get incredible amounts of storage savings (on average 90%) all while cloning databases in a matter of minutes. Space and Time, two things enterprises would love to save on. This feature has been designed with the goal of providing data cloning capabilities while protecting your existing investments in server, storage, and software. With this in mind, we have pursued with the dual solution approach of Hardware and Software. In the hardware approach, we connect directly to your storage appliances and perform all low level actions required to rapidly clone your databases. While in the software approach, we use an intermediate software layer to talk to any storage vendor or any storage configuration to perform the same low level actions. Thus delivering the benefits of database thin cloning, without requiring you to drastically changing the infrastructure or IT's operating style.

In release 4, we expand the scope of options supported by snap clone with the addition of database CloneDB. While CloneDB is not a new feature, it was first introduced in patchset, it has over the years become more stable and mature. CloneDB leverages a combination of Direct NFS (or dNFS) feature of the database, RMAN image copies, sparse files, and copy-on-write technology to create thin clones of databases from existing backups in a matter of minutes. It essentially has all the traits that we want to present to our customers via the snap clone feature. For more information on cloneDB, i highly recommend reading the following sources:

Blog by Tim Hall: Direct NFS (DNFS) CloneDB in Oracle Database 11g Release 2

Oracle OpenWorld Presentation by Cern: Efficient Database Cloning using Direct NFS and CloneDB

The advantages of the new CloneDB integration with EM12c Snap Clone are:

  • Space and time savings
  • Ease of setup - no additional software is required other than the Oracle database binary
  • Works on all platforms
  • Reduce the dependence on storage administrators
  • Cloning process fully orchestrated by EM12c, and delivered to developers/DBAs/QA Testers via the self service portal
  • Uses dNFS to delivers better performance, availability, and scalability over kernel NFS
  • Complete lifecycle of the clones managed by EM12c - performance, configuration, etc

3. Improved Rapid Start Kits

DBaaS deployments tend to be complex and its setup requires a series of steps. These steps are typically performed across different users and different UIs. The Rapid Start Kit provides a single command solution to setup Database as a Service (DBaaS) and Pluggable Database as a Service (PDBaaS). One command creates all the Cloud artifacts like Roles, Administrators, Credentials, Database Profiles, PaaS Infrastructure Zone, Database Pools and Service Templates. Once the Rapid Start Kit has been successfully executed, requests can be made to provision databases and PDBs from the self service portal. Rapid start kit can create complex topologies involving multiple zones, pools and service templates. It also supports standby databases and use of RMAN image backups.

The Rapid Start Kit in reality is a simple emcli script which takes a bunch of xml files as input and executes the complete automation in a matter of seconds. On a full rack Exadata, it took only 40 seconds to setup PDBaaS end-to-end. This kit works for both Oracle's engineered systems like Exadata, SuperCluster, etc and also on commodity hardware. One can draw parallel to the Exadata One Command script, which again takes a bunch of inputs from the administrators and then runs a simple script that configures everything from network to provisioning the DB software.

Steps to use the kit:

  • The kit can be found under the SSA plug-in directory on the OMS: EM_BASE/oracle/MW/plugins/oracle.sysman.ssa.oms.plugin_12.
  • It can be run from this default location or from any server which has emcli client installed
  • For most scenarios, you would use the script dbaas/setup/database_cloud_setup.py
  • For Exadata, special integration is provided to reduce the number of inputs even further. The script to use for this scenario would be dbaas/setup/exadata_cloud_setup.py
  • The database_cloud_setup.py script takes two inputs:
    • Cloud boundary xml: This file defines the cloud topology in terms of the zones and pools along with host names, oracle home locations or container database names that would be used as infrastructure for provisioning database services. This file is optional in case of Exadata, as the boundary is well know via the Exadata system target available in EM.
    • Input xml: This file captures inputs for users, roles, profiles, service templates, etc. Essentially, all inputs required to define the DB services and other settings of the self service portal.
  • Once all the xml files have been prepared, invoke the script as follows for PDBaaS:
    emcli @database_cloud_setup.py -pdbaas 

         The script will prompt for passwords a few times for key users like sysman, cloud admin, SSA admin, etc. Once complete, you can simply log into EM as the self service user and request for databases from the portal.

More information available in the Rapid Start Kit chapter in Cloud Administration Guide

4. Extensible Metering and Chargeback

 Last but not the least, Metering and Chargeback in release 4 has been made extensible in all possible regards. The new extensibility features allow customer, partners, system integrators, etc to :

  • Extend chargeback to any target type managed in EM
  • Promote any metric in EM as a chargeback entity
  • Extend list of charge items via metric or configuration extensions
  • Model abstract entities like no. of backup requests, job executions, support requests, etc

 A slew of emcli verbs have also been added that allows administrators to create, edit, delete, import/export charge plans, and assign cost centers all via the command line.

More information available in the Chargeback API chapter in Cloud Administration Guide.

5. Miscellaneous Enhancements

There are other miscellaneous, yet important, enhancements that are worth a mention. These mostly have been asked by customers like you. These are:

  • Custom naming of DB Services
    • Self service users can provide custom names for DB SID, DB service, schemas, and tablespaces
    • Every custom name is validated for uniqueness in EM
  • 'Create like' of Service Templates
    • Now creating variants of a service template is only a click away. This would be vital when you publish service templates to represent different database sizes or service levels.
  • Profile viewer
    • View the details of a profile like datafile, control files, snapshot ids, export/import files, etc prior to its selection in the service template
  • Cleanup automation - for failed and successful requests
    • Single emcli command to cleanup all remnant artifacts of a failed request
    • Cleanup can be performed on a per request bases or by the entire pool
    • As an extension, you can also delete successful requests
  • Improved delete user workflow
    • Allows administrators to reassign cloud resources to another user or delete all of them
  • Support for multiple tablespaces for schema as a service
    • In addition to multiple schemas, user can also specify multiple tablespaces per request

I hope this was a good introduction to the new Database as a Service enhancements in EM12c R4. I encourage you to explore many of these new and existing features and give us feedback.

Good luck!


Cloud Management Page on OTN

Cloud Administration Guide [Documentation]

-- Adeesh Fulay (@adeeshf)

Friday Apr 04, 2014

Webcast: Database Cloning in Minutes using Oracle Enterprise Manager 12c Database as a Service Snap Clone

April 10 — 7:00 a.m. PT | 10:00 a.m. ET | 4:00 p.m. CET
April 24 — 10:00 a.m. PT | 1:00 p.m. ET
May 8 — 7:00 a.m. PT | 10:00 a.m. ET | 4:00 p.m. CET
May 22 — 10:00 a.m. PT | 1:00 p.m. ET

Since the demands from the business for IT services is non-stop, creating copies of production databases in order to develop, test and deploy new applications can be labor intensive and time consuming. Users may also need to preserve private copies of the database, so that they can go back to a point prior to when a change was made in order to diagnose potential issues. Using Snap Clone, users can create multiple snapshots of the database and “time travel” across these snapshots to access data from any point in time.

Join us for an in-depth technical webcast and learn how Oracle Cloud Management Pack for Oracle Database's capability called Snap Clone, can fundamentally improve the efficiency and agility of administrators and QA Engineers while saving CAPEX on storage. Benefits include:

  • Agile provisioning (~ 2 minutes to provision a 1 TB database)
  • Over 90% storage savings
  • Reduced administrative overhead from integrated lifecycle management

Register Now!

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Wednesday Jan 15, 2014

Oracle Enterprise Manager Snap Clone Webcast Replay and Slides

Thank you to all who attended our webcast on Enterprise Manager 12c Snap Clone last month. In this webcast, we talked about how EM12c Snap Clone can help:

  • Leverage storage copy-on-write technologies for rapid provisioning
  • Integrate cloning with other Oracle Enterprise Manager 12c Lifecycle Management features, such as data masking and sub-setting
  • “Time travel” across multiple database snapshots to restore and access past data
  • Reduced administrative overhead from integrated lifecycle management

 For those who missed this webcast, the replay is available here and the slides have been uploaded to slideshare.

Feel free to reach out to us if you have any questions on Snap Clone or Database as a Service.

- Adeesh Fulay (@adeeshf)


Latest information and perspectives on Oracle Enterprise Manager.

Related Blogs


« October 2015