Wednesday Jan 15, 2014

Oracle Enterprise Manager Snap Clone Webcast Replay and Slides

Thank you to all who attended our webcast on Enterprise Manager 12c Snap Clone last month. In this webcast, we talked about how EM12c Snap Clone can help:

  • Leverage storage copy-on-write technologies for rapid provisioning
  • Integrate cloning with other Oracle Enterprise Manager 12c Lifecycle Management features, such as data masking and sub-setting
  • “Time travel” across multiple database snapshots to restore and access past data
  • Reduced administrative overhead from integrated lifecycle management

 For those who missed this webcast, the replay is available here and the slides have been uploaded to slideshare.

Feel free to reach out to us if you have any questions on Snap Clone or Database as a Service.

- Adeesh Fulay (@adeeshf)

Tuesday Jan 07, 2014

Create a Database Instance without DB Control Schema

When using Database Configuration Assistant (DBCA) to create a database instance to house the repository of Enterprise Manager Cloud Control, people often end up with an instance containing DB Control schema, and they have to remove the schema from the instance before installing EM12c. To create an instance without DB Control or SYSMAN schema in the first place, make the following selections in the DBCA database creation process.

1. In the Database Template step, select the Custom Database option.

2. In the Management Options step, uncheck the Configure Enterprise Manager option.

Note: If DBCA locates an agent installation on the host, it will provide an option to register the database instance that you are creating with the corresponding Management Service. If you choose that option, DBCA will add the database instance to the Enterprise Manager site as a managed target upon completion of instance creation. This option, however, does not create SYSMAN schema in the database instance.

3. In the Database Content step, uncheck the Enterprise Manager Repository option.

Remember to make the above-mentioned selections, and you will have an instance without DB Control, which is fit for housing an EM repository.

Note: From EM12c Release 2, there is an option to create an database instance with a pre-configured EM repository. For instructions see:

Creating a Database Instance with Preconfigured Repository Using Database Templates

Installing Oracle Database and Creating a Database

Thursday Jan 02, 2014

What is EM 12c DBaaS Snap Clone?

Happy New Year to all! Being the first blog post of the new year, lets look at a relatively new feature in EM that has gained significant popularity over the last year - EM 12c DBaaS Snap Clone.

The ‘Oracle Cloud Management Pack for Oracle Database’ a.k.a the Database as a Service (DBaaS) feature in EM 12c has grown tremendously since its release two years ago.  It started with basic single instance and RAC database provisioning, a technical service catalog, an out of box self service portal, metering and chargeback, etc. But since then we have added provisioning of schemas and pluggable databases, full clones using RMAN backups, and Snap Clone. This video showcases the various EM12c DBaaS features.

This blog will cover one of the most exciting and popular features – Snap Clone. In one line, Snap Clone is a self service way of creating rapid and space efficient clones of large (~TB) databases.

Self Service - empowers the end users (developers, testers, data analysts, etc) to get access to database clones whenever they need it.
Rapid - implies the time it takes to clone the database. This is in minutes and not hours, days, or weeks.
Space Efficient - represents the significant reduction in storage (>90%) required for cloning databases

Customer Scenario

To best explain the benefits of Snap Clone, let’s look at a Banking customer scenario:

  • 5 production databases total 30 TB of storage
  • All 5 production databases have a standby
  • Clones of the production database are required for data analysis and reporting
  • 6 total clones across different teams every quarter
  • For security reasons, sensitive data has to be masked prior to cloning

Based on the above scenario, the storage required, if using traditional cloning techniques, can be calculated as follows:

5 Prod DB                  = 30 TB
5 Standby DB            = 30 TB
5 Masked DB             = 30 TB (These will be used for creating clones)
6 Clones (6 * 30 TB) = 180 TB
Total                           = 270 TB
Time = days to weeks

As the numbers indicate, this is quite horrible. Not only 30 TB turn into 270 TB, creating 6 clones of all production databases would take forever. In addition to this, there are other issues with data cloning like:

  • Lack of automation. Scripts are good but often not a long term solution.
  • Traditional cloning techniques are slow while, existing storage vendor solutions are DBA unfriendly
  • Data explosion often outpaces storage capacity and hurts ITs ability to provide clones for dev and testing
  • Archaic processes that require multiple users to share a single clone, or only supports fixed refresh cycles
  • Different priorities between DBAs and Storage admins

Snap Clone to the Rescue

All of the above issues lead to slow turnaround times, and users have to wait for days and weeks to get access to their databases. Basically, we end up with competing priorities and requirements, where the user demands self service access, rapid cloning, and the ability to revert data changes, while IT demands standardization, better control, reduction in storage and administrative overhead, better visibility into the database stack, etc.

EM 12c DBaaS Snap Clone tries to address all these issues. It provides:

  • Rapid and space efficient cloning of databases by leveraging storage copy-on-write (or similar) technology
  • Supports all database versions from 10g to 12c
  • Supports various storage vendors and configurations NAS and SAN
  • Lineage and association tracking between clone master and its various clones and snapshots
  • 'Time Travel' capability to restore and access past data
  • Deep visibility into storage, OS, and database layer for easy triage of performance and configuration issues
  • Simplified access for end user via out-of-the-box self service portal
  • RESTful APIs to integrate with custom portals and third party products
  • Ability to meter and charge back on the clone databases

So how does Snap Clone work?

The secret sauce lies in the Storage Management Framework (SMF) plug-in. This plug-in sits between the storage system and the DBA, and provides the much needed layer of abstraction required to shield DBAs and users from the nuances of the different storage systems. At the storage level, Snap Clone makes use of storage copy-on-write (or similar) technology. There are two options in terms of using and interacting with storage:

1. Direct connection to storage: Here storage admins can register NetApp and ZFS storage appliance with EM, and then EM directly connects to the storage appliance and performs all required snapshot and clone operations. This approach requires you to license the relevant options on the storage appliance, but is the easiest and the most efficient and fault tolerant approach.

2. Connection to storage via ZFS file system: This is a storage vendor agnostic solution and can be used by any customer. Here instead of connecting to storage, the storage admin mounts the volumes to a Solaris server and format it with ZFS file system. Now all snapshot and clone operations required on the storage are conducted via ZFS file system,. The good thing about this approach is that it does not require thin cloning options to be licensed on the storage since ZFS file system provides these capabilities.

For more details on how to setup and use Snap Clone, refer to a previous blog post.

Now, lets go back to our Banking customer scenario and see how Snap Clone helped then reduce their storage cost and time to clone.

5 Prod DB                      = 30 TB
5 Standby DB                 = 30 TB
5 Masked DB                 = 30 TB
6 Clones (6 * 30 TB)      = 180 TB
6 Clones (6 * 5 * 2 GB) = 60 GB
Total                               = 270 TB 90 TB
Time = days to weeks minutes

Assuming the clone databases will have minimal writes, we allocate about 2GB of write space per clone. For 5 production databases and 6 clones, this totals to just 60GB in required storage space. This is a whopping 99.97% savings in storage. Plus, these clones are created in matter of minutes and not the usual days or weeks. The product has out-of-the-box charts that show the storage savings across all storage devices and cloned databases. See the screenshot below.

Snap Clone Savings

Where can you use Snap Clone databases?

As i said earlier, Snap Clone is most effective when cloning large databases  (~TBs). Common scenarios we see our customers best use Snap Clone are:

  • Application upgrade testing. For example, EBusiness suite upgrade to R12.
  • Functional testing. For example, testing using production datasets.
  • Agile development. For example, run parallel development sprints by giving each sprint its own cloned database.
  • Data Analysis and Reporting. For example, stock market analysis at the close of market everyday.

Its obvious that Snap Clone has a strong affinity to applications, since its application data that you want to clone and use. Hence it is important to add that the Snap Clone feature when combined with EM12c middleware-as-a-service (MWaaS) can provide a complete end-to-end self service application deployment experience. If you have existing portals or need to integrate Snap Clone with existing processes, then use our RESTful APIs for easy integration with third party systems.

In summary, Snap Clone is a new and exciting way of dealing with data cloning challenges. It shields DBAs from the nuances of different storage systems, while allowing end users to request and use clones in a rapid and self service fashion. All of this while saving storage costs. So try this feature out today, and your development and test teams will thank you forever.

In subsequent blog posts, we will look at some popular deployment models used with Snap Clone.

-- Adeesh Fulay (@adeeshf)

Additional References

Cloud Management Page on OTN

Cloud Administration Guide (Documentation)

Enterprise Manager 12c Database-as-a-Service Snap Clone Overview (Presentation)

Saturday Nov 02, 2013

Sending notification after an event has remained open for a specified period

Enterprise Manager (EM) 12c allows you to create an incident rule to send a notification and/or create an incident after an event has been open for a specified period. Such an incident rule will help prevent premature alerts on issues that may correct themselves within a certain amount of time.

For example, there are some agents in an unstable network area, and often there are communication failures between the agents and the OMS lasting three, four minutes at a time. In this scenario, you may only want to receive alerts after an agent in that area has been in the Agent Unreachable status for at least five minutes.

Note: Many non-target availability metrics allow users to specify the “number of occurrences” or the number of consecutive times metric values reach thresholds before a notification is sent. It is best to use the feature for such metrics.

This article provides a step-by-step guide for creating an incident rule set to cater for the above scenario, that is, to create an incident and send a notification after the Agent Unreachable event has remained open for a five-minute duration.

Steps to create the incident rule

1.     Log on to the console and navigate to Setup -> Incidents -> Incident Rules.

Note: A non-super user requires the Create Enterprise Rule Set privilege, which is a resource privilege, to create an incident rule.

The Incident Rules - All Enterprise Rules page displays.

2.     Click Create Rule Set …

The Create Rule Set page displays.

3.     Enter a name for the rule set (e.g. Rule set for agents in flaky network areas), optionally enter a description, and leave everything else at default values, and click + Add.

The Search and Select: Targets page pops up.

Note:  While you can create a rule set for individual targets, it is a best practice to use a group for this purpose.

4.     Select an appropriate group, e.g. the AgentsInFlakyNework group. The Select button becomes enabled, click the button.

The Create Rule Set page displays.

5.     Leave everything at default values, and click the Rules tab.

The Create Rule Set page displays.

6.     Click Create…

The Select Type of Rule to Create page pops up.

7.     Leave the Incoming events and updates to events option selected, and click Continue.

The Create New Rule : Select Events page displays.

8.     Select Target Availability from the Type drop-down list.

The page shows more options for Target Availability.

9.     Select the Specific events of type Target Availability option, and click + Add.

The Select Target Availability events page pops up.

10.   Select Agent from the Target Type dropdown list.

The page expands.

11.   Click the Agent unreachable checkbox, and click OK.

Note: If you want to also receive a notification when the event is cleared, click the Agent unreachable end checkbox as well before clicking OK.

The Create New Rule : Select Events page displays.

12.   Click Next.

The Create New Rule : Add Actions page displays.

13.   Click + Add.

The Add Actions page displays.

14.   Do the following:

a.     Select the Only execute the actions if specified conditions match option (You don’t want the action to trigger always).

The following options appear in the Conditions for Actions section.

b.     Select the Event has been open for specified duration option.

The Conditions for actions section expands.

c.     Change the values of Event has been open for to 5 Minutes as shown below.

d.     In the Create Incident or Update Incident section, click the Create Incident checkbox as following:

e.     In the Notifications section, enter an appropriate EM user or email address in the E-mail To field.

f.     Click Continue (in the top right hand corner).

The Create New Rule : Add Actions page displays.

15.   Click Next.

The Create New Rule : Specify name and Description page displays.

16.   Enter a rule name, and click Next.

The Create New Rule : Review page appears.

17.   Click Continue, and proceed to save the rule set.

The incident rule set creation completes.

After one of the agents in the group specified in the rule set is stopped for over 5 minutes, EM will send a mail notification and create an incident as shown in the following screenshot.

In conclusion, you have seen the steps to create an example incident rule set that only creates an incident and triggers a notification after an event has been open for a specified period. Such an incident rule can help prevent unnecessary incidents and alert notifications leaving EM administrators time to more important tasks.

- Loc Nhan

Thursday Oct 17, 2013

IOUG SIG Webcast on October 30th : Performance Tuning your DB Cloud

The Oracle Enterprise Manager Special Interest Group (SIG) is a growing body of IOUG members who manage or are interested in all aspects of Oracle Enterprise Manager. This IOUG SIG is managed by volunteers and supported by Oracle Enterprise Manager product managers and developers. The purpose of the SIG is to bring relevant information and education through webcasts, discussions and networking to users interested in learning more about the product, and to share user experiences.

On October 30th at 10 AM pacific time, Oracle Enterprise Manager SIG is hosting a webcast on "Performance Tuning your DB Cloud in OEM 12c Cloud Control - 360 Degrees". In this webcast, Tariq Farooq , CEO, BrainSurface and Mike Ault, Oracle  will provide a tutorial on how to monitor and perform performance tuning of the Oracle database cloud environment.

You will learn how to leverage Oracle Enterprise Manager for tuning, trouble-shooting & monitoring your Oracle Database Cloud Ecosystem. The session covers lessons learned, tips/tricks, recommendations, best practices, gotchas and a whole lot more on how to effectively use Oracle Enterprise Manager Cloud Control 12c for quick, easy & intuitive performance tuning of your Oracle Database Cloud.

Session Objectives:
• Leveraging OEM12c Cloud Control for Oracle DB Tuning/Monitoring
• Limited Deep-Dive on AWR
• Oracle DB Cloud Performance Tuning
• Best Practices for DB Cloud Maintenance/Monitoring

Register Now !

Stay Connected:

Twitter |  Face book |  You Tube |  Linked in |  Google+ Newsletter

Wednesday Jul 24, 2013

Understanding Agent Resynchronization

Agent Resynchronization (resync) is an important topic but often misunderstood or misused. In this Q&A styled blog, I discuss how and when it is appropriate to use agent resynchronization.

What is Agent Resynchronization?

Management Agent can be reconfigured using target information present in the Management Repository. Resynchronization pushes all targets and related information from the Management Repository to the Management Agent and then unblocks the Agent.

 Why do agents need to be re-synchronized?

Read More

Thursday Jul 11, 2013

Oracle Enterprise Manager 12c Release 3: What’s New in EMCLI

If you have been using the classic Oracle Enterprise Manager Command Line interface ( EMCLI ), you are in for a treat. Oracle Enterprise Manager 12c R3 comes with a new EMCLI kit called ‘EMCLI with Scripting Option’. Not my favorite name, as I would have preferred to call this EMSHELL since it truly provides a shell similar to bash or cshell. Unlike the classic EMCLI, this new kit provides a Jython-based scripting environment along with the large collection of verbs to use. This scripting environment enables users to use established programming language constructs like loops (for, or while), conditional statements (if-else), etc in both interactive and scripting mode.

Benefits of ‘EMCLI with Scripting Option’

Some of the key benefits of the new EMCLI are:

  • Jython based scripting environment
  • Interactive and scripting mode
  • Standardized output format using JSON
  • Can connect to any EM environment (no need to run EMCLI setup …)
  • Stateless communication with OMS (no user data is stored with the client)
  • Generic list function for EM resources
  • Ability to run user-defined SQL queries to access published repository views

Before we go any further, there are two topics that warrant some discussion – Jython and JSON.

Read More

Tuesday Jul 02, 2013

Oracle Enterprise Manager 12c R3 introduces advancements in cloud lifecycle and operations management

Oracle Enterprise Manager 12c Release 3 (R3) was announced ( Press Release ) earlier today. It is now available for download at  OTN .

This latest release features improvements in several areas, including:

  • Improvements to Private Cloud and Engineered Systems Management
  • Expanded Middleware and Application Management Capabilities
  • Efficiency Gains for Enterprise manager Users in EM’s Enterprise-Ready Framework

You can learn more about what's new in the Oracle Enterprise Manager 12c R3 in the Enterprise Manager 12c documentation . You will see more blogs and details about the new features during the next few weeks. Please let us what

On July 18th, you can join us at a webcast to hear Thomas Kurian, EVP of Product Development on what Oracle Engineering has achieved with Oracle Enterprise Manager 12c Release 3 to address these challenges. Later, during this webcast, Oracle experts will discuss the latest capabilities in Oracle Enterprise Manager 12c Release 3 for cloud lifecycle and operations management.

The presentation will be followed by a live Q&A session with Oracle experts. You can also join us online on Twitter to get your specific questions answered. Please use hash tag #em12c to join the conversation.

Register Now for the Webcast!

Stay Connected:

Twitter |  Face book |  You Tube |  Linked in |  Newsletter

Tuesday Jun 25, 2013

Planning Database as a Service Implementation Project

People, process and planning are the three key elements to success in a private cloud journey. Some common questions i hear from field/customers often relates to tasks involved in setting up Database-as-a-Service(DBaaS) using Oracle Enterprise Manager 12c from scratch and how these tasks are mapped to current IT roles and responsibilities - overall list of activities that needs to be performed to setup and achieve DBaaS and who does what during the implementation process.

In terms of roles, responsibilities and users, you will map to these categories

  • Project Manager is responsible for working with all stakeholders, project team and coordinating in planning and delivering DBaaS implementation project
  • EM Admin sets ups Cloud management infrastructure and is responsible for overall EM 12c installation/configuration and Administration
  • Cloud Admins are responsible for managing the Cloud infrastructure(Servers, Network, Storage) and related Administration
  • SSA Admins or Cloud DBAs are responsible for Self Service portal setup, managing quotas, policies, target DBs S/W provisioning/ongoing Administration
  • SSA Users or End users of Cloud who actually requests for DB instances to be rapidly provisioned on-demand from Self Service portal

Here is a sample Work-breakdown structure(WBS) listing down tasks involved and with individual owners mapped. Note - we have all type of use cases covered in the plan (DBCA, RMAN, Schema as a Service, Snap Clone) and you can pick up any or mix or all of these tasks in your plan depending on your use cases and requirements for implementing DBaaS -

 Activity Description
A. Initiation and Startup

Determine scope of implementation
Determine and document scope of DBaaS Cloud implementation together with customer and end users. Determine requirements and goals. Introduction meeting implementation team. Project Manager
Coordination and quality management Coordination and quality management on throughout all phases of the implementation Project Manager
Define High availability Needs
Determine the level of High availability required Project Manager
Technical intake Technical intake meeting; Infrastructure requirements (Hardware, OS, network/firewall, s/w requirement) for cloud setup Project Manager
Functional intake Functional intake meeting; Monitoring /Reporting/Alerting requirement. User/Roles/Groups/Rules model Project Manager
Deliver implementation plan Document requirements and goals. Document required functional and technical configuration. Build architecture Diagram. Update estimate planning based on implementation plan. Project Manager
B. Infrastructure Setup

Install hardware systems Plan EM12c compatible hardware systems EM Admin
Operating system installation
Installation of certified OS platform [Refer Certify on MOS] EM Admin
Network configuration of hardware systems General network configuration of hardware systems I.e. TCP/IP, routing, NTP, firewall access, SSH, HTTP and HTTPS access. Set up network access between systems and work place of end users. Optionally implementation of remote access EM Admin
C. Production EM 12c Cloud Control Installation

Install of Oracle Database 11gR2 Installation of Oracle Database 11gR2 for Repository EM Admin
Installation of EM 12c Cloud Control Release 2 Update 1 and latest PSU Installation of EM 12c Cloud Control OMS/Agents EM Admin
Review HA, Security, Sizing Best practices Setup as per EM 12c Admin guide EM Admin
Review recommended patches Review latest PSU and DBaaS patches as per MOS Note 1549855.1 EM Admin