Monday Mar 16, 2015

Oracle Private Database Cloud: Defining Database Sizes in the Service Catalog

Latest release of cloud plug-in (part of EM12c R4 Plug-in Update 1) brings the ability to define sizes for database cloud services (Schema and PDB services already support definition of size). Prior to this ability, customers were required to define a new template for each size – small, medium, large, etc. This will help in significantly reducing the number of templates required.

So lets see how to use them.

Read More

Discover and Promote Oracle Homes as EM Targets

Typically, Oracle Homes are discovered and promoted as targets automatically along with guided flows for addition of primary targets like databases, weblogic domains, etc, but there might be instances (not very often) where you need to discover the Homes standalone.

There are two ways to do this – from the GUI and using EMCLI verbs.

Read More

Friday Feb 27, 2015

Oracle Private Database Cloud REST APIs

While we spend a lot of time developing GUI for our private cloud features – like the self service portal, setup screens, etc, a large percentage of our customers use these features via our APIs, both EMCLI and REST based.

Q: So why would anyone not use the out of the box cloud (self service) portal and use the APIs instead?

A: Well, there are many reasons:

  1. You already have a custom cloud portal that you use to deploy non-Oracle products and now you would like Oracle products to be part of it
  2. You want cloud related actions to be part of a larger orchestration flow. For example, prior to provisioning, you want to integrate an approval workflow, and then post-provision update your asset management system with the service details.
  3. You want to integrate the cloud actions with our IT service desk or ticketing systems
  4. Integrate actions like cloning of databases (Snap Clone), or provisioning of middleware and applications with your continuous integration or devops process.

Read More

Monday Feb 02, 2015

Snap Clone using EMC SAN and ASM

Recently we announced the latest update to Enterprise Manager Cloud Control 12c Release 4.  One of the enhancements in that release is support for Snap Clone on Automatic Storage Management (ASM) and EMC Storage.  Before we examine the details of this specific enhancement, let's look at a quick refresher on what Snap Clone provides for you.

What is Snap Clone?

Snap Clone is a storage agnostic and self service approach to creating rapid and space efficient clones of large databases (and by large, we’re talking terabytes or more). Now that’s probably more buzz words in one sentence than anyone’s brain can deal with without exploding, so let’s explain some of those terms more:
  • Storage agnostic – by that I mean Snap Clone supports all storage vendors, both NAS and SAN. It can leverage storage layer APIs or layer a ZFS filesystem on top to provide copy on write.
  • Self service – in the XaaS world – where X can be any of I, MW, P and DB :) – one of the key features is empowering the end user to do the work, rather than waiting on some techie to find time in their otherwise busy schedules. So it’s the end user who makes the adhoc clones here, not the storage admin
  • Rapid – People simply don’t have the time to wait weeks for provisioning to happen any more, so you have to support the functionality to clone databases in minutes rather than the days or weeks things used to take.
  • Space efficient – When you’re working with terabyte or larger databases, you simply may not have the storage to create full-sized clones, so you have to significantly reduce the storage footprint to start with.
Over the various EM releases,  more and more functionality has been added to Snap Clone:
  • EM12cR2 provided Snap Clone for NAS storage (NetApp and Sun ZFSSA).  It provided RMAN backup based clones, and included the Snap Clone Analyzer to show you the storage savings you could make using Snap Clone
  • EM12cR3 added in support for Snap Clone using the Solaris File System (ZFS) and admin flows for Snap Clone for PDB’s (pluggable databases)
  • EM12cR4 added a lot more:
    • Snap Clone using CloneDB – this is the biggie, as it means Snap Clone can now be used with ANY Oracle database release that supports CloneDB, regardless of what storage it’s on
    • Data Guard standby as a test master – allows offloading the impact of creating the test master from your Production environment
    • NetApp Ontap 8.x cluster mode support
    • Certification for engineered systems, with I/O over Infiniband
    • Support for NFSv4
  • And in the latest plugin update that's just been shipped, we added:
    • Integrated data lifecycle management
    • Snap Clone using EMC SAN and ASM
    • Admin flows for test master creation
    • Integration with masking, patching, upgrades etc.

Snap Clone using EMC SAN and ASM

Most NAS technologies offer storage efficient clones in the form of Snapshots. The snapshots make use of underlying volumes, knows as flexvols (Netapp) or shares (ZFS). Unfortunately, SAN storage does not provide native snapshotting capability unless a file is created on it by leveraging TCP/IP over iSCSI over Ethernet. However this defeats the purpose of having high speed fiber channel fabric, not to mention that it makes little sense to overlay SAN with a filesystem. Another complaint we heard from our customers is that cloning is a data intensive operation that could flood the corporate IT backbone if Ethernet is used. Consequently, lot of our customers want native support for SAN for cloning purposes, especially, the ones who run ASM on SAN. And they are quite a lot in number.

Using Snap Clone on ASM and EMC storage provides the ability to create ‘live’ thin clones of databases that are on ASM. A live clone is NOT snapshot based but rather a live copy of the database, residing on copy-on-write storage technology, that can be within the same cluster or indeed another one. Both single instance and RAC are supported – supported versions are 10.2.0.5 or higher of the database, 11.2 and higher of the Grid Infrastructure code. This functionality works on both EMC VMAX (with Time Finder VPSnap) and VNX storage appliances.

Diagrammatically, the configuration looks like this:



Why Use Snap Clone with EMC SAN and ASM

There are a number of major challenges that Snap Clone can be used to address:
  1. Lack of automation - Manual tasks such as provisioning and cloning of new databases (for example, for test or development systems) is one area that many DBA’s complain is too time consuming. It can take days to weeks, often because of the need to coordinate the involvement of different groups, as shown in the image below:

  2. When an end user, be it a developer or a QA engineer, needs a database he or she typically has to go through an approval process like this, which then translates into a series of tasks for the DBA, the sysadmin and storage admin. The sysadmin has to provide the compute capacity while the storage admin has to provide the space on a filer. Finally, the DBA would install the bits, create the database (optionally on Real Application Clusters), and deliver that to the user. Clearly, this is a cumbersome and time-consuming process that needs to be improved on.

  3. Database unfriendly solutions – Obviously, when there is a need looking for a solution, different people take different approaches to solving that need. There are a variety of point solutions and storage solutions out there, but the vast bulk of them are not database aware. They tend to clone storage volumes rather than databases and have no visibility into the database stack, which of course makes it hard to triage performance issues as a DBA. They also lack the ability to track configuration, compliance and data security issues, as well as having limited or no lifecycle capabilities. As mentioned before, DBAs would like to leverage the native FDDI protocol of SAN for cloning.  This will make cloning fast and efficient without disrupting regular network traffic.

  4. Storage issues and archaic processes – Of course, one of the main issues is storage. Data volumes are ever increasing, particularly in these Big Data days, and the growth can often outpace your storage capacity. You can throw more disks at the problem, but it never seems to be enough, and you can end up with degraded performance if you take the route of sharing clones between users. There can also be different processes and different priorities between the storage team and the DBA team, and you may still have fixed refresh cycles, making it difficult to clone on an adhoc basis.
So the end result of all of this is that far too often, there are competing priorities at odds. Users want flexibility – simplified self service access, rapid cloning, and the ability to revert data changes. IT, on the other hand, want standardization and control, which allows a reduction in storage use, reduction in administrative overhead, visibility into the complete database stack and lineage tracking.  

Snap Clone on EMC storage helps you to address all these competing priorities, using hardware you may already own.  Indeed, EMC is well established as an Oracle database storage vendor over many years, and that integration has become tighter and tighter over the past few years.  In addition to that, the actual setup and configuration can be simpler than is the case when using other hardware, as you do not need to create Database Profiles in this configuration.  Service Templates are created directly on either a single instance or RAC database that resides on ASM.  Because you're using this combination of ASM and EMC SAN storage, the database is already Snap Clone enabled as it resides on copy-on-write storage technology.

In my next post, I'll discuss more details of what else is new in the Snap Clone product in this latest release, so stay tuned for more details on that soon!

For More Information

You can see more details on how you actually set Snap Clone up on EMC storage by viewing the following screenwatches:

For more details on using Enterprise Manager Cloud Control 12c to provide Database as a Service functionality, visit the OTN page located here.

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Tuesday Jan 27, 2015

Top Oracle Enterprise Manager 12c Questions

Guest contributors: Courtney Llamas and David Wolf

What are the steps involved for upgrading from Oracle Enterprise Manager 12c Release 3 to Release 4? Is it an upgrade in place or a new install/migration
Answer: Yes, Oracle Enterprise Manager 12c R3 to R4 is an out-of-place upgrade, which means the installer will install in a new Oracle Home and migrate the application over. You will need the additional space for a new Oracle Home. You can reduce downtime of the upgrade by performing a “Software Only Install”, and then upgrade later. See the upgrade documentation here.

Do the patch and upgrade functionality work on Oracle Database 11g targets or only on Oracle Database 12c targets?
A: Yes, patching and upgrading can be performed on any certified target version. For more details on database patching and lifecycle management visit this page.

Where can I find a full list of new enhancements compared to Release 3?
A: The new features are listed in Oracle Enterprise Manager Cloud Control Introduction Guide.

Is there a list available of new features by management pack?
A: The new features are listed in Oracle Enterprise Manager Cloud Control Introduction Guide. They are listed by plug-in, not management pack. For the license information read this page.

What are the best hardware configuration and setup in order to provide high availability capabilities for database as a service (i.e. Oracle RAC, Oracle Exadata)?
A: The more you rely on Oracle Enterprise Manager, the more you need to think about high availability. The best recommendation is to have a multi-OMS system with a standby for disaster recovery. You can scale up to this as the environment grows, but starting with a multi-OMS system will give you the availability and scalability you need. A standby database with Oracle Data Guard and a software replicated standby OMS would be the next step. Please read this whitepaper Deploying a Highly Available Oracle Enterprise Manager 12c Cloud Control for more information.

Can Snap Clone be used with EMC storage or is Oracle ZFS storage required?
A: Yes. Snap Clone can be used with either EMC VMAX and/or VNX Block Storage, both are supported. More details can be found on this page under the section: Database Cloning and Dynamic Data Refresh.

Does Oracle Enterprise Manager support monitoring of hypervisors?
A: Oracle Enterprise Manager supports monitoring of Oracle VM natively and VMWare via a Blue Medora third party plug in. More details on managing physical and virtual host can be found here.

Is the metering and chargeback functionality part of the base product or does it requires an additional management pack?
A: Metering and Chargeback for Oracle Database is part of the Oracle Cloud Management Pack for Oracle Database. Metering and Charge back for guest virtual machines (VMs) and hosts are included with the base product and does not require a separate license. Read this whitepaper for more on Metering and Chargeback with Oracle Enterprise Manager.

Does Enterprise Data Governance require additional licensing?
A: Use of the Enterprise Data Governance capabilities within Oracle Enterprise Manager requires an Oracle Database Lifecycle Management Pack license.

How is the new AWR Warehouse feature different from the existing AWR report in Oracle Enterprise Manager 12c Release 3?
A: The existing AWR report in Oracle Enterprise Manager pulls AWR data from the source database and relies on the AWR retention setting of that database. This is typically only saved for 8 days. There’s been a big demand for saving this data for 30-60 days and even longer. AWR Warehouse, automates the extract of the source AWR data and loading to a warehouse database so that data can be retained without affecting the performance and storage of the source database. The reports are the same, it just allows you to dig back further in time. Read this article for more details.

Does this new release require more disk space for the AWR Warehouse? More CPU and memory?
A: Not for the OMS. The recommendation is to setup the AWR Repository in a separate database, outside of the Oracle Enterprise Manager repository database. Hosting on the same server would be fine, so long as there is sufficient memory and CPU for both instances. Read this article on AWR Warehouse for more details.

Does AWR Warehouse also support Oracle Database 10g and/or 11g AWR?
A: AWR Warehouse must be installed on Oracle Database 12.1.0.2 or higher or version 11.2.0.4 with the appropriate patch. It also must be an equal or higher database version of the source databases it accommodates. Check out this demo to understand AWR Warehouse. Read the AWR Warehouse documentation.

Is there a list of 3rd party technologies that can be managed by Oracle Enterprise Manager 12c?
A: Check out the Oracle Enterprise Manager Extensibility Exchange for a list of the available 3rd party plug-ins and connectors.

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Friday Jan 09, 2015

Innovating to Optimize: Creating Room in the IT budget for Initiatives that Boost Business

Read the latest Economist Intelligence Unit whitepaper called Innovating to Optimize: Creating Room in the IT budget for Initiatives that Boost Business. The paper outlines new ways executives can use technology to improve IT operational processes in order to drive innovation and optimize their business. The Economist interviews industry experts and thought leaders, revealing candid insights and new emerging trends.

Download the whitepaper

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Wednesday Jan 07, 2015

Webcast: Oracle Database 12c Performance Analysis

IOUG Webcast: Long Term and Cross-Database Performance Analysis for Oracle Databases Using Automatic Workload Repository Warehouse

Thursday, January 15, 2015
1:00 p.m. EST | 11:00 a.m. PST

The Automatic Workload Repository (AWR) feature in Oracle Database has been the standard used by DBAs to diagnose database performance problems.

In this session, you will learn about the new AWR Warehouse capability, which enables DBAs with capacity planning and performance analysis for Oracle Databases without the constraint of the retention period and without any overhead on critical databases.

Learn how AWR Warehouse can tell why performance was slow this month compared to the same period last year or give administrators an annual report of problems in your mission-critical databases.

Featured Speaker: Jagan Athreya, Senior Director of Product Management, Oracle

Registered Now

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Friday Dec 19, 2014

Business Brief: Optimize Your Database Performance - Spend Less Time Keeping the Lights on and More Time Innovating

How are you keeping up with the challenges and demands from the business? Are you able to innovate or just keeping the lights on?

Read the latest business brief on Oracle Database Performance Management to get better insight into the challenges facing IT and find out ways to overcome them.

Read the business brief.

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Monday Nov 24, 2014

Download Urls for Self Update Entities in Offline Mode

Many enterprises run their EM 12c environments in the offline mode (no direct internet connection). These customers have the additional task of downloading the plug-ins, connectors, agents, etc on a machine with access to internet, and then upload them to EM. The current solution makes the task of fetching the download url rather cumbersome, so in this blog i discuss a quick hack to list all download urls for plug-ins and agents.

Read More

Thursday Nov 13, 2014

Will the REAL Snap Clone functionality please stand up?

Possibly one of the least known pieces of functionality that is provided as part of the Enterprise Manager Cloud Control 12c product suite is the Snap Clone product. In simple terms, Snap Clone is a storage agnostic self service approach to rapidly creating space efficient clones of large databases (and by large, we're talking terabytes or more).  I blogged on that a few months back, but a few months can be an eternity in software development terms, so here's an update that covers not just the functionality in the product but also addresses some of the misleading statements some of our competitors have been making about it of recent times.

Wednesday Jul 23, 2014

Enterprise Manager Ops Center 12c R2 U1 Released

We are happy and excited to announce that on July 20, 2014, Oracle Enterprise Manager Ops Center 12c Release 2 Update 1 was released for all platforms including Oracle Solaris SPARC/x86 and Linux.

Ops Center 12cR2 PSU1 is an update release containing improvements and enhancements in the following areas: performance, new hardware support and general quality improvements.

In the performance area we have made improvements in core Ops Center components such as the Enterprise Controller, Proxy Controller and Virtualization Agent. We have reduced the Enterprise Controller memory footprint and enhanced start up times for the Enterprise Controller and agents. We have looked at areas such as deployment wizards and the management of LUN's and made improvements in the performance of these areas.

For new hardware we support the discovery, monitoring and provisioning of both OS and Firmware for: X4-4, X4-8, M4000 and M10. We also made improvements for firmware management of the X4-2 and introduced enhanced support for add / modify hardware configurations for Oracle SuperCluster. 

In the general quality area we improved security, refined logging, made improvements to OS provisioning and enhanced areas such as LDAP and the UI.

For customers with a support contract historically any new updates to the Ops Center components would automatically appear in the Download window of the UI. However, we have noticed a bug which prevents this new version appearing. A fix / IDR is in place and is described in the release notes for this version available here. There is also a MOS note 1908726.1 which describes the IDR and procedure to enable the download.

Software downloads outside of this automatic process will be available on the OTN and edelivery in the coming days.

Before upgrading your Enterprise Controller please check the upgrade guide available here.

There is also an excellent blog which gives advice on pre upgrade tasks to ensure a smooth and successful upgrade experience here.

Friday Jun 27, 2014

Convert Crontab to Enterprise Manager Jobs

Surprisingly, a popular question posted on our internal forum is about the possibility of using the Enterprise Manager (EM) Job System to replace customer’s numerous cron jobs. The answer is obviously YES! I say surprisingly because the EM Job system has been in existence for around 10 years (I believe since EM 10.2.0.1), and my hope was that, by now, customers would have moved to using more enterprise class job schedulers instead of cron. So, here is a quick post on how to get started with this conversion from cron to EM Jobs for some of our new users.

Benefits of EM Job System

 Before we learn about the how, let’s look at the why. The EM job system is:

  • Free - (Yes, I said free) It is included with the base EM at no cost.
  • Flexible - It supports multiple options for scheduling, notification, authentication, etc
  • Infinitely scalable - the job system seamlessly scales to every new Oracle Management Server (OMS). In fact, in case of OMS failures, the job steps are automatically picked up by the next available OMS without affecting the job execution.
  • General purpose - General purpose since it provides numerous out-of-the-box job types like run OS command, start/stop, backup, SQL Script, patch, etc that span multiple target types. As of today, there are over 50 job types available in the product.
  • Enterprise grade - It allows users to automate multiple administrative tasks like backup, patching, cloning, etc across multiple targets. Customers have not only converted their cron jobs to EM, but have also replaced other enterprise tools like Autosys and migrated 1000s of jobs to EM Job System.
  • APIs - Jobs can be scheduled and managed from the UI and using EMCLI (the command line interface).

Now back to our topic.

The Conversion Process

Let’s start with a sample crontab that we want to convert.

Sample Crontab

A cron expression consists of 6 fields, where the first 5 fields represent the schedule, while the last field represents the command or script to run.

 Field Name
Mandatory?  Allowed Values
 Allowed special characters
 Minutes  Yes 0-59  * / , -
 Hours  Yes  0-23  * / , -
 Day of month
Yes  1-31 * / , - ? L W
 Month Yes  1-12 or JAN-DEC
* / , -
 Day of week
Yes  0-6 or SUN-SAT
* / , - ? L #

Cron jobs run on the operating system, often using the native shell or other tools installed on the operating system. The equivalent of this capability in Enterprise Manager is the ‘OS Command’ job type. Here are the steps required to convert the first entry in the crontab to an EM job:

1. Navigate to the Job Activity page
Job activity menu

2. Select the ‘OS Command’ job and click Go
OS Command

A 5-tab wizard will appear. Let’s step through this one by one.

3. Select the first tab called ‘General’. Here provide a meaningful name and description for the job. Since this job will be run on the Host target, keep the target type selection as ‘Host’. Next, select all host targets in EM that you wish to run this script against.

While cron jobs are defined on a per host bases, in EM a job definition can be run and managed across multiple hosts or groups of hosts. This avoids having to maintain the same crontab across multiple hosts.

General

4. Select the ‘Parameters’ tab. Here enter the command or script as specified in the last field of the crontab entry. When constructing the command, you can make use of the various target properties.
Parameters tab

5. Next select ‘Credentials’. Here we provide the credentials required to connect to the host and execute the required commands or scripts. Three options are presented to the user:

  • Preferred – default credential set for the host
  • Named - Credentials are stored within Enterprise Manager as "named" entities. Named credentials can be a username/password, or a public key-private key pair. Here we choose pre-created named credentials
  • New – This allows us to create and use new named credential

Note: If your OS user does not have the required privileges to execute the set command, Named Credentials also support use of sudo, powerbroker, sesu, etc.

Credentials tab

6. Next, we set the schedule and this is where it gets interesting. As discussed before, crontab uses a textual representation for the schedule, while EM Job system has a graphical representation for the schedule.

Our sample schedule in the crontab is ‘00 0 * * Sun’. This translates to a weekly job at 12 midnight on every Sunday. To set this in EM, choose the ‘Repeating’ schedule type. The screenshot below shows all the other selections.
Schedule tab

 The key here is to select the correct ‘Frequency Type’, the rest of the selections are quite obvious. This also lets you choose the desired timezone for the schedule. Your options are to either start the job w.r.t a fixed timezone, or start it in individual target's timezone. The latter is very popular, for example, I want to start a job at 2 AM local time in every region around the world.

Another selection of note is that for ‘Grace Period’. This is an extremely powerful feature, but often not used by many customers. Typically, we expect jobs to be started within a few seconds or minutes (based on the load on the system and number of jobs scheduled) of the start time, but a job might not start on time for many reasons. The most common reasons are the Agent being down or due to a blackout. The grace period controls the latest start time for the job in case the job is delayed, else its is marked as skipped. By default, jobs are scheduled with indefinite grace periods, but I highly recommend setting a value for it. In the sample above, I set a 3 hr limit which may seem large but given the weekly nature of the job seems reasonable. So the job system will wait until 3 am (the job start time is 12 am) to start the job, after which the iteration will be skipped. For repeating schedules, the grace period should always be less than the repeat interval. If the job starts on time, the grace period is ignored.

7. Finally, we navigate to the ‘Access’ tab. This tab has two parts:

  • Privilege assignment to roles and users: this allows you to control job level access for other users
  • Email notifications for the Job owner: this allows you to control the events for which you wish to receive notifications. Note, this only sets notification for the job owner, the other users can subscribe to emails by setting up notification and/or incident rules.

To prevent EM from sending deluge of emails, I recommend the following settings in the notifications region:

  • Match status and severity: Both
  • Select severity of status: Critical
  • Select status: Problems & Action Required

       You can always come back and modify these settings to suit your needs.

Access tab

Not all cron jobs need to be converted to OS command. For example, if you are taking Oracle database backups using cron, then you probably want to use the out-of-the-box  job type for RMAN scripts. Just provide the RMAN script, list of databases to run this against, and the credentials required to connect to the database. Similarly, if you run sqls on numerous databases, you can leverage the SQL Script job type for this purpose. There are over 50 job types available in EM12c, all available for use from the UI and EMCLI.

Finally, the best way to learn more about the EM Job System is to actually play with it. I also recommend blogs from Maaz, Kellyn, and other users on this topic. Good luck!!

References

Maaz Anjum: http://maazanjum.com/2013/12/30/create-a-simple-job-for-a-host-target-in-em12c/
Kellyn Pot'vin: http://dbakevlar.com/

-- Adeesh Fulay (@adeeshf)

Monday Jun 23, 2014

Patch automation of Data Guard Databases takes a leap with EM12cR4


Patching has always been one of the many joys (or dreads) of Oracle DBAs. 
The effort compounds as the Database sprawl grows and the architecture complexity increases.  Additionally,  there is an increasing pressure to patch the systems with minimal or zero downtime. EM12c's Database patch automation functionality provides  an end-to-end management starting with proactive patch recommendations (like quarterly PSUs and CPUs), complete pre-flight checker to ensure predictable patching during maintenance window and  automation for patching the comprehensive list of products under the Oracle Database family.  ( EM12c Patch Automation overview

Patching has always been one of the many joys (or dreads) of Oracle DBAs. The effort compounds as the Database sprawl grows and the architecture complexity increases. Additionally, there is an increasing pressure to patch the systems with minimal or zero downtime. 

EM12c's Database patch automation functionality provides an end-to-end management starting with proactive patch recommendations (like quarterly PSUs and CPUs), complete pre-flight checker to ensure predictable patching during maintenance window and  automation for patching the comprehensive list of products under the Oracle Database family.  ( EM12c Patch Automation overview)

With the introduction of "Out of Place" patching in EM12c, the automation feature got a complete overhaul. Over the past 3 years I have seen customers moving to this mode to achieve their automation goals.  ( What is Out of Place?)

Patching Data guard environments (read as Primary and its corresponding Standby Databases) has always been a challenging task for the DBAs. 
In addition to handling the differences in steps needed for the configuration, the very nature of its distributed environment and incorporating any additional custom tasks makes it more demanding. Till now in EM we only supported automation in disparate steps and in “In Place” mode.

Starting 12.1.0.4 (EM12c R4), the support is enhanced to be more tighter, can be done in 'Out of Place' mode, and has new supplemental features to manage the process along with its additional tasks.

In this post we will take a closer look of the data guard stand by patching feature and then dive into a use case, you could try out in your environment.

[Read More]

Tuesday Jun 17, 2014

EM12c Release 4: Job System Easter Eggs - Part 2

This is part 2 of my two part blog post on Job System Easter Eggs (hidden features). The two features being covered are:

  1. New Job progress tracking UI
  2. Import/Export of job definitions

In the previous post, I talked about the new UI while in this post I will cover import and export for job definitions. 


2.  Import/Export of job definitions

The ability to export and import job definitions across EM environments has been a popular enhancement request for quite some time. There are 3 primary drivers:

  • Test to Production: Moving active and library job definitions from test site to production
  • Migration: Move job definitions across EM versions. This is often used as an alternative to upgrade.
  • As Failsafe: For that extra protection, in addition to repository backups. This can be useful for selective restoration of job definition.

In the first iteration, we are exposing these functions as emcli verbs.  Since these emcli verbs are not formally supported in the release, no documentation is included in the EMCLI Reference Guide. So to learn more about them, we will have to look at emcli help. The two verbs are called export_jobs, and import_jobs.

>> ./emcli help | grep _jobs


Some salient points about the import & export verbs are.

Export:

  • Rich filter criteria to form the list of jobs to export
  • Active and Library jobs can be exported
  • Preview mode, to view results before exporting them
  • Job definitions are exported as zip files for ease of transfer
  • Contextual information like targets, credentials, access settings, etc is exported, but not imported. In future, we may be able to support import for this information as well.
  • System jobs and nested jobs are not exported

Import:

  • Special Preview mode to view contents of the exported file, and to determine possible conflicts during import
  • Rich filter criteria to selectively import job definitions from the exported file
  • Job definitions are imported to the library ONLY. This is true even if an active job was exported
  • Two failure modes - skip on error, and rollback on error
  • Import only works on same or higher version of EM12c. The patch set number matters.


Export Job Definitions

Let's walk through an example.

1. We start with export_jobs. The help for this verb will show you all the available options. Most of the filters are self explanatory, so i will skip the explanation. The most important option of all is the -preview flag. This when used in conjunction with other filters, will show results without exporting the job definitions.

>> ./emcli help export_jobs

emcli export_jobs
   [-name="job name1;job name2;..."]
   [-type="job type1;job type2;..."]
   [-targets="tname1:ttype1;tname2:ttype2;..."]
   [-owner="owner1;owner2;..."]
   [-preview]
  -export_file=<Zip filename that will be created>"
   -libraryjobs

2.  Now lets play with this verb. If we invoke the verb with just the -preview flag, it will list all job definitions that can be exported both that are active, and from the library. Note: system jobs and nested jobs are skipped from this output.

>> ./emcli export_jobs -preview

Not all job types are exportable. To determine the list of job types supported via the import and export verbs, use the get_job_types verb.

>>  ./emcli get_job_types

Currently, there are over 50 job types supported, and this list will continue to grow with every release.

3. From the list above, I am primarily interested in jobs that are created by the user AFULAY. So I apply the -owner filter.

>> ./emcli export_jobs -owner=AFULAY -preview


In this output i see 2 jobs, 'Library Job' which is a simple OS command job stored in the job library, while the 'Test Job' is an active OS command job scheduled against a bunch of targets.

Note: if multiple options are specified, like -name and -owner, then these are ANDed together.

4. Since i am the lazy kind, i would rather export every thing and then later decide what i wish to import. So here it goes. The -export_file option takes the location and file name for the export file. Note the actual output file is an xml that contains job metadata, but the export file is always archived and stored in a zip format. At this time, I am sure most would instinctively unzip the file and start perusing through its contents, but doing so would be analogous to removing the warranty void sticker of your new TV or Blue Ray player. Basically, attempts to manually modify the contents of the exported zip file is highly discouraged.

>> ./emcli export_jobs -export_file=/tmp/afulay_alljobs.zip


Note how the status column reports success or failure for each job being exported. With the file exported, we now move on to the import verb.

.

Import Job Definitions

In the previous section, we exported a file with all job definitions in it. Now lets say we share this file with bunch of other admins and ask them to import whatever job definitions that make sense or are relevant to their environment.

1. To understand the capabilities of the import verb, we take a look at the help content.

>> ./emcli help import_jobs

emcli import_jobs
   [-name="job name1;job name2;..."]
   [-type="job type1;job type2;..."]
   [-targets="tname1:ttype1;tname2:ttype2;..."]
   [-owner="owner1;owner2;..."]
   [-preview]
   [-force]
   [-stoponerror]
   -import_file=<Zip filename that contains job definitions>"

 Most of the options are quite similar to export_jobs barring a few. The -force flag allows the admin to update an existing job definition. Typically, you will run into these situations when a conflicting job is found in the new environment, and you want to either update its definition with the new version in the import file or overwrite the localized changes. The -stoponerror flag, when specified, will stop the import process on first encountered failure, and then rollback all jobs imported in the session. We will likely change this label to rollbackonerror to correctly represent its behavior. The default behavior is to skip failed jobs and continue importing others.

2. Before our admins import job definitions, they first need to view the contents of the exported file. This again can be done using the -preview option.

>> ./emcli import_jobs -preview -import_file=/tmp/afulay_alljobs.zip


The -preview option in the import verb is very special. It not only just lists the contents of the exported file, but also connects to the new EM environment and looks for potential conflicts during import. So this is a deep validation test. As seen in the above screenshot, there are two sections in the output, first is just a listing of all job definitions from the import file, while the second is a list of all conflicts. Note: for demo sake, i am exporting and importing to the same EM site, and hence every job shows up as a conflict. To address this issue, i will eventually delete the 'Library Job' from my job library, and import it from the import file.

Disclaimer:

In the interest of full disclosure, i should mention that there are few known bugs for the import verb, hence the rationale for not releasing these verbs formally with EM12c R4. Some bugs i ran into when writing this blog were:

  • you cannot export an active job, delete it, and import it back to the same EM environment, this currently is only possible with library jobs.  This is an obscure case though.
  • The -force flag is a little flaky, so sometimes it wouldn't force import even if you want it to
  • The -owner flag does not work on the import file, it instead will throw an exception

That said, when the job does get imported, it does so properly, so there is never any risk of metadata corruption.


3. If i try to import the 'Library Job', the verb will fail and give me an error message.

>> ./emcli import_jobs -name='LIBRARY JOB' -import_file=/tmp/afulay_alljobs.zip


The Status column reports Error, while the Notes column gives the reason as 'job already exists'.

4. Now lets delete the library job and try to import it.

>> ./emcli delete_library_job -name='LIBRARY JOB'

>> ./emcli import_jobs -name='LIBRARY JOB' -import_file=/tmp/afulay_alljobs.zip


Success!!We were able to delete the library job, and import it back from the import file.


In summary, there two very useful enhancements made in EM12c R4. Unfortunately, due to time constraints and our inability to meet the set quality standards, we decided to ship these features in disabled state. This ensures that production sites and users are not impacted, while still giving the few brave souls the opportunity to test these features. In my assessment, i have found the new UI to be fairly robust as i have been using this exclusively for a while. On the other hand, there are few known bugs with the import and export emcli verbs, so use these with caution.

-- Adeesh Fulay (@adeeshf)  

Monday Jun 16, 2014

EM12c Release 4: Job System Easter Eggs - Part 1

So you just installed a new EM12c R4 environment or upgraded your existing EM environment to EM12c R4. Post upgrade you go to the Job System activity page (via Enterprise->Job->Activity menu) and view the progress details  of a job. Well nothing seems to have changed, its the same UI, the same multi-page drill down to view step output, same no. of clicks, etc. Wrong! In this two part blog post, i talk about two Job System Easter Eggs (hidden features) that most of you will find interesting. These are:

  1. New Job progress tracking UI
  2. Import/Export of job definitions

So before i go any further, let me address the issue of why are these features hidden? As we were building these features, we realized that we would not be ready to ship the desired quality of code by the set dates. Hence, instead of removing the code, it was decided to ship it in a disabled state so as not to impact customers, but still allowing a brave few to experiment with it and provide valuable feedback.

1.  New Job Progress Tracking UI

The job system UI hasn't changed much since its introduction almost 10 years ago. It is a daunting task to update all the job system related UIs in a single release, and hence we decided to take a piecemeal approach instead. In the first installment, we have revamped the job progress tracking page.

Old Job Progress Tracking UI

The current UI, as shown above, while being very functional, is also very laborious. Multiple clicks and drill downs are required to view the step output for a particular target. Also, any click leads to complete page refresh, which leads to wastage of time and resources. The new UI tries to address all these concerns. It is a single page UI, which means no matter where you click, you never leave the page and thus never lose context of the target or step you where in. It also significantly reduces the no. of clicks required to complete the same task as in the current UI. So lets take a look at this new UI.

 First, as i mentioned earlier, you need to enable this UI. To do this, you need to run the following emctl command from any of the OMS:

./emctl set property -name oracle.sysman.core.jobs.ui.useAdfExecutionUi -value true

 This command will prompt for the sysman password, and then will enable the new UI.

NOTE: This command does not require a restart of the OMS. Once run, the new UI will be enabled for all user across all OMSes.

EMCTL Output

Now revisit the job progress tracking page from before. You will be directed to the new UI.

New Job Progress Tracking UI

There are in all 6 key regions on this new single page job progress tracking UI. Starting from top left, these are:

  1. Job Run Actions - These are actions that can be performed on the job run like suspend resume, retry, stop, edit, etc.
  2. Executions - This region displays all the executions in the job run. An execution, in most cases, represents a single target and hence runs independently from other executions. This region thus shows the progress and status of all executions in a single view. The best part of this region is the column titled 'Execution Time'. The cigar chart in this column represents two things - one, the duration of the execution, and two, the difference in start times. The visual representation helps in identifying runaway executions, or just compare execution times across different targets. The Actions menu allows various options like start, stop, debug, delete, etc.
  3. Execution Summary - Clicking on an execution in the above region, paints the area on the right. This specific region shows execution summary with information like status, start & end date, execution id, command, etc
  4. Execution Steps - This region lists the steps that make up the execution.
  5. Step Output - Clicking on a step from the above region, paints this region. This shows the details of the step. This includes the step output and the ability to download it to a text file.
  6. Page Options - We imagine that learning any new UI takes time, and hence this final region provides the option to switch between the new and the classic view. Additionally, this also allows you to set the auto refresh rate for the page.

Essentially, considering that jobs have two levels - executions and steps, we have experimented with a multi-master style layout. EM has never used such a layout and hence there were concerns raised when we chose to do so.

Master 1 (region 2) -> Detail 1 (regions 3, 4, & 5)

Master 2 (region 4) -> Detail 2 (region 5)


In summary, with this new UI, we have been able to significantly reduce the no. of clicks required to track job progress and drill into details. We have also been able to show all relevant information in a single page, thus avoiding unnecessary page redirection and reloads. I would love to hear from you if this experiment has paid off and if you find this new UI useful.

In the next part of this blog i talk about the new emcli verbs to import and export job definitions across EM environments. This has been a long standing enhancement request, and we are quite excited about our efforts.

-- Adeesh Fulay (@adeeshf)  

About

Latest information and perspectives on Oracle Enterprise Manager.

Related Blogs




Search

Categories
Archives
« March 2015
SunMonTueWedThuFriSat
1
2
3
4
5
7
8
9
10
11
12
13
14
15
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    
       
Today