Wednesday Feb 25, 2015

Editing EM12c Jobs in Bulk

I recently received requests for suggestions on how to edit EM12c jobs in bulk. The two examples that were presented to me were:

  1. Change the start time of 70+ jobs
  2. Change the oracle home path in 80+ RMAN script jobs

So how do we do this?

Read More

Monday Jun 16, 2014

EM12c Release 4: Job System Easter Eggs - Part 1

So you just installed a new EM12c R4 environment or upgraded your existing EM environment to EM12c R4. Post upgrade you go to the Job System activity page (via Enterprise->Job->Activity menu) and view the progress details  of a job. Well nothing seems to have changed, its the same UI, the same multi-page drill down to view step output, same no. of clicks, etc. Wrong! In this two part blog post, i talk about two Job System Easter Eggs (hidden features) that most of you will find interesting. These are:

  1. New Job progress tracking UI
  2. Import/Export of job definitions

So before i go any further, let me address the issue of why are these features hidden? As we were building these features, we realized that we would not be ready to ship the desired quality of code by the set dates. Hence, instead of removing the code, it was decided to ship it in a disabled state so as not to impact customers, but still allowing a brave few to experiment with it and provide valuable feedback.

1.  New Job Progress Tracking UI

The job system UI hasn't changed much since its introduction almost 10 years ago. It is a daunting task to update all the job system related UIs in a single release, and hence we decided to take a piecemeal approach instead. In the first installment, we have revamped the job progress tracking page.

Old Job Progress Tracking UI

The current UI, as shown above, while being very functional, is also very laborious. Multiple clicks and drill downs are required to view the step output for a particular target. Also, any click leads to complete page refresh, which leads to wastage of time and resources. The new UI tries to address all these concerns. It is a single page UI, which means no matter where you click, you never leave the page and thus never lose context of the target or step you where in. It also significantly reduces the no. of clicks required to complete the same task as in the current UI. So lets take a look at this new UI.

 First, as i mentioned earlier, you need to enable this UI. To do this, you need to run the following emctl command from any of the OMS:

./emctl set property -name oracle.sysman.core.jobs.ui.useAdfExecutionUi -value true

 This command will prompt for the sysman password, and then will enable the new UI.

NOTE: This command does not require a restart of the OMS. Once run, the new UI will be enabled for all user across all OMSes.

EMCTL Output

Now revisit the job progress tracking page from before. You will be directed to the new UI.

New Job Progress Tracking UI

There are in all 6 key regions on this new single page job progress tracking UI. Starting from top left, these are:

  1. Job Run Actions - These are actions that can be performed on the job run like suspend resume, retry, stop, edit, etc.
  2. Executions - This region displays all the executions in the job run. An execution, in most cases, represents a single target and hence runs independently from other executions. This region thus shows the progress and status of all executions in a single view. The best part of this region is the column titled 'Execution Time'. The cigar chart in this column represents two things - one, the duration of the execution, and two, the difference in start times. The visual representation helps in identifying runaway executions, or just compare execution times across different targets. The Actions menu allows various options like start, stop, debug, delete, etc.
  3. Execution Summary - Clicking on an execution in the above region, paints the area on the right. This specific region shows execution summary with information like status, start & end date, execution id, command, etc
  4. Execution Steps - This region lists the steps that make up the execution.
  5. Step Output - Clicking on a step from the above region, paints this region. This shows the details of the step. This includes the step output and the ability to download it to a text file.
  6. Page Options - We imagine that learning any new UI takes time, and hence this final region provides the option to switch between the new and the classic view. Additionally, this also allows you to set the auto refresh rate for the page.

Essentially, considering that jobs have two levels - executions and steps, we have experimented with a multi-master style layout. EM has never used such a layout and hence there were concerns raised when we chose to do so.

Master 1 (region 2) -> Detail 1 (regions 3, 4, & 5)

Master 2 (region 4) -> Detail 2 (region 5)


In summary, with this new UI, we have been able to significantly reduce the no. of clicks required to track job progress and drill into details. We have also been able to show all relevant information in a single page, thus avoiding unnecessary page redirection and reloads. I would love to hear from you if this experiment has paid off and if you find this new UI useful.

In the next part of this blog i talk about the new emcli verbs to import and export job definitions across EM environments. This has been a long standing enhancement request, and we are quite excited about our efforts.

-- Adeesh Fulay (@adeeshf)  

Tuesday Jun 10, 2014

EM12c: Using the LIST verb in emcli

Many of us who use EM CLI to write scripts and automate our daily tasks should not miss out on the new list verb released with Oracle Enterprise Manager 12.1.0.3.0. The combination of list and Jython based scripting support in EM CLI makes it easier to achieve automation for complex tasks with just a few lines of code. Before I jump into a script, let me highlight the key attributes of the list verb and why it’s simply excellent!

1. Multiple resources under a single verb:
A resource can be set of users or targets, etc. Using the list verb, you can retrieve information about a resource from the repository database.

Here is an example which retrieves the list of administrators within EM.
Standard mode
$ emcli list -resource="Administrators"


Interactive mode
emcli>list(resource="Administrators")
The output will be the same as standard mode.

Standard mode
$ emcli @myAdmin.py
Enter password :  ******

The output will be the same as standard mode.

Contents of myAdmin.py script
login()
print list(resource="Administrators",jsonout=False).out()


To get a list of all available resources use
$ emcli list -help

With every release of EM, more resources are being added to the list verb. If you have a resource which you feel would be valuable then go ahead and contact Oracle Support to log an enhancement request with product development. Be sure to say how the resource is going to help improve your daily tasks.

2. Consistent Formatting:
It is possible to format the output of any resource consistently using these options:

  –column

  This option is used to specify which columns should be shown in the output.

Here is an example which shows the list of administrators and their account status
$ emcli list -resource="Administrators" -columns="USER_NAME,REPOS_ACCOUNT_STATUS"

To get a list of columns in a resource use:
$ emcli list -resource="Administrators" -help


You can also specify the width of the each column. For example, here the column width of user_type is set to 20 and department to 30.
$ emcli list -resource=Administrators -columns="USER_NAME,USER_TYPE:20,COST_CENTER,CONTACT,DEPARTMENT:30"

This is useful if your terminal is too small or you need to fine tune a list of specific columns for your quick use or improved readability.

  –colsize
  This option is used to resize column widths.
Here is the same example as above, but using -colsize to define the width of user_type to 20 and department to 30.
$ emcli list -resource=Administrators -columns="USER_NAME,USER_TYPE,COST_CENTER,CONTACT,DEPARTMENT" -colsize="USER_TYPE:20,DEPARTMENT:30"

The existing standard EMCLI formatting options are also available in list verb. They are:
-format="name:pretty" | -format="name:script” | -format="name:csv" | -noheader | -script

There are so many uses depending on your needs. Have a look at the resources and columns in each resource. Refer to the EMCLI book in EM documentation for more information.

3. Search:
Using the -search option in the list verb makes it is possible to search for a specific row in a specific column within a resource. This is similar to the sqlplus where clause. The following operators are supported:
          =
           !=
           >
           <
           >=
           <=
           like
           is (Must be followed by null or not null)

Here is an example which searches for all EM administrators in the marketing department located in the USA.
$emcli list -resource="Administrators" -search="DEPARTMENT ='Marketing'" -search="LOCATION='USA'"

Here is another example which shows all the named credentials created since a specific date. 
$emcli list -resource=NamedCredentials -search="CredCreatedDate > '11-Nov-2013 12:37:20 PM'"
Note that the timestamp has to be in the format DD-MON-YYYY HH:MI:SS AM/PM

Some resources need a bind variable to be passed to get output. A bind variable is created in the resource and then referenced in the command. For example, this command will list all the default preferred credentials for target type oracle_database.

Here is an example
$ emcli list -resource="PreferredCredentialsDefault" -bind="TargetType='oracle_database'" -colsize="SetName:15,TargetType:15"


You can provide multiple bind variables.

To verify if a column is searchable or requires a bind variable, use the –help option. Here is an example:
$ emcli list -resource="PreferredCredentialsDefault" -help


4. Secure access
When list verb collects the data, it only displays content for which the administrator currently logged into emcli, has access.

For example consider this usecase:
AdminA has access only to TargetA.
AdminA logs into EM CLI
Executing the list verb to get the list of all targets will only show TargetA.

5. User defined SQL
Using the –sql option, user defined sql can be executed. The SQL provided in the -sql option is executed as the EM user MGMT_VIEW, which has read-only access to the EM published MGMT$ database views in the SYSMAN schema.

To get the list of EM published MGMT$ database views, go to the Extensibility Programmer's Reference book in EM documentation. There is a chapter about Using Management Repository Views. It’s always recommended to reference the documentation for the supported MGMT$ database views.  Consider you are using the MGMT$ABC view which is not in the chapter. During upgrade, it is possible, since the view was not in the book and not supported, it is likely the view might undergo a change in its structure or the data in it. Using a supported view ensures that your scripts using -sql will continue working after upgrade.

Here’s an example
  $ emcli list -sql='select * from mgmt$target'

6. JSON output support   
JSON (JavaScript Object Notation) enables data to be displayed in a collection of name/value pairs. There is lot of reading material about JSON on line for more information.

As an example, we had a requirement where an EM administrator had many 11.2 databases in their test environment and the developers had requested an Administrator to change the lifecycle status from Test to Production which meant the admin had to go to the EM “All targets” page and identify the set of 11.2 databases and then to go into each target database page and manually changes the property to Production. Sounds easy to say, but this Administrator had numerous targets and this task is repeated for every release cycle.

We told him there is an easier way to do this with a script and he can reuse the script whenever anyone wanted to change a set of targets to a different Lifecycle status.

Here is a jython script which uses list and JSON to change all 11.2 database target’s LifeCycle Property value.

If you are new to scripting and Jython, I would suggest visiting the basic chapters in any Jython tutorials. Understanding Jython is important to write the logic depending on your usecase.
If you are already writing scripts like perl or shell or know a programming language like java, then you can easily understand the logic.

Disclaimer: The scripts in this post are subject to the Oracle Terms of Use located here.

 1 from emcli import *
 2 
search_list = ['PROPERTY_NAME=\'DBVersion\'','TARGET_TYPE=
 \'oracle_database\'','PROPERTY_VALUE LIKE \'11.2%\'']
 3 if len(sys.argv) == 2:
 4    print login(username=sys.argv[0])
 5    l_prop_val_to_set = sys.argv[1]
 6  
   l_targets = list(resource="TargetProperties", search=search_list,
   columns="TARGET_NAME,TARGET_TYPE,PROPERTY_NAME")
 7    for target in l_targets.out()['data']:
 8       t_pn = 'LifeCycle Status'
 9      print "INFO: Setting Property name " + t_pn + " to value " +
      l_prop_val_to_set + " for " + target['TARGET_NAME']
 10      print  set_target_property_value(property_records=
      target['TARGET_NAME']+":"+target['TARGET_TYPE']+":"+
      t_pn+":"+l_prop_val_to_set)
 11 
else:
 12   print "\n ERROR: Property value argument is missing"
 13
  print "\n INFO: Format to run this file is filename.py <username>
  <Database Target LifeCycle Status Property Value>"

You can download the script from here. I could not upload the file with .py extension so you need to rename the file to myScript.py before executing it using emcli.

A line by line explanation for beginners:
Line

 1 Imports the emcli verbs as functions
 2 search_list is a variable to pass to the search option in list verb. I am using escape character for the single quotes. In list verb to pass more than one value for the same option, you should define as above comma separated values, surrounded by square brackets.
 3 This is an “if” condition to ensure the user does provide two arguments with the script, else in line #15, it prints an error message.
 4 Logging into EM. You can remove this if you have setup emcli with autologin. For more details about setup and autologin, please go the EM CLI book in EM documentation.
 5 l_prop_val_to_set is another variable. This is the property value to be set. Remember we are changing the value from Test to Production. The benefit of this variable is you can reuse the script to change the property value from and to any other values.
 6 Here the output of the list verb is stored in l_targets. In the list verb I am passing the resource as TargetProperties, search as the search_list variable and I only need these three columns – target_name, target_type and property_name. I don’t need the other columns for my task.
 7 This is a for loop. The data in l_targets is available in JSON format. Using the for loop, each pair will now be available in the ‘target’ variable.
 8 t_pn is the “LifeCycle Status” variable. If required, I can have this also as an input and then use my script to change any target property. In this example, I just wanted to change the “LifeCycle Status”.
 9 This a message informing the user the script is setting the property value for dbxyz.
 10 This line shows the set_target_property_value verb which sets the value using the property_records option. Once it is set for a target pair, it moves to the next one. In my example, I am just showing three dbs, but the real use is when you have 20 or 50 targets.

The script is executed as:
$ emcli @myScript.py subin Production



The recommendation is to first test the scripts before running it on a production system. We tested on a small set of targets and optimizing the script for fewer lines of code and better messaging.

For your quick reference, the resources available in Enterprise Manager 12.1.0.4.0 with list verb are:
$ emcli list -help


Watch this space for more blog posts using the list verb and EM CLI Scripting use cases. I hope you enjoyed reading this blog post and it has helped you gain more information about the list verb. Happy Scripting!!

Disclaimer: The scripts in this post are subject to the Oracle Terms of Use located here.


Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
mt=8">Download the Oracle Enterprise Manager 12c Mobile app

Monday Jun 09, 2014

EM12c Release 4: New EMCLI Verbs

Here are the new EM CLI verbs in Enterprise Manager 12c Release 4 (12.1.0.4). This helps you in writing new scripts or enhancing your existing scripts for further automation.

Basic Administration Verbs
 invoke_ws - Invoke EM web service.

ADM Verbs
 associate_target_to_adm - Associate a target to an application data model.
 export_adm - Export Application Data Model to a specified .xml file.
 import_adm - Import Application Data Model from a specified .xml file.
 list_adms - List the names, target names and application suites of existing Application Data Models
 verify_adm - Submit an application data model verify job for the target specified.

BI Publisher Reports Verbs
 grant_bipublisher_roles - Grants access to the BI Publisher catalog and features.
 revoke_bipublisher_roles - Revokes access to the BI Publisher catalog and features.

Blackout Verbs
 create_rbk - Create a Retro-active blackout.

CFW Verbs
 cancel_cloud_service_requests -  To cancel cloud service requests
 delete_cloud_service_instances -  To delete cloud service instances
 delete_cloud_user_objects - To delete cloud user objects.
 get_cloud_service_instances - To get information about cloud service instances
 get_cloud_service_requests - To get information about cloud requests
 get_cloud_user_objects - To get information about cloud user objects.

Chargeback Verbs
 add_chargeback_entity - Adds the given entity to Chargeback.
 assign_charge_plan - Assign a plan to a chargeback entity.
 assign_cost_center - Assign a cost center to a chargeback entity.
 create_charge_entity_type - Create  charge entity type
 export_charge_plans - Exports charge plans metadata to file
 export_custom_charge_items -  Exports user defined charge items to a file
 import_charge_plans - Imports charge plans metadata from given file
 import_custom_charge_items -  Imports user defined charge items metadata from given file
 list_charge_plans - Gives a list of charge plans in Chargeback.
 list_chargeback_entities - Gives a list of all the entities in Chargeback
 list_chargeback_entity_types - Gives a list of all the entity types that are supported in Chargeback
 list_cost_centers - Lists the cost centers in Chargeback.
 remove_chargeback_entity - Removes the given entity from Chargeback.
 unassign_charge_plan - Un-assign the plan associated to a chargeback entity.
 unassign_cost_center - Un-assign the cost center associated to a chargeback entity.

Configuration/Association History
 disable_config_history - Disable configuration history computation for a target type.
 enable_config_history - Enable configuration history computation for a target type.
 set_config_history_retention_period - Sets the amount of time for which Configuration History is retained.

ConfigurationCompare
 config_compare - Submits the configuration comparison job
 get_config_templates - Gets all the comparison templates from the repository

Compliance Verbs
 fix_compliance_state -  Fix compliance state by removing references in deleted targets.

Credential Verbs
 update_credential_set

Data Subset Verbs
 export_subset_definition - Exports specified subset definition as XML file at specified directory path.
 generate_subset - Generate subset using specified subset definition and target database.
 import_subset_definition - Import a subset definition from specified XML file.
 import_subset_dump - Imports dump file into specified target database.
 list_subset_definitions - Get the list of subset definition, adm and target name

Delete pluggable Database Job Verbs
 delete_pluggable_database - Delete a pluggable database

Deployment Procedure Verbs
 get_runtime_data - Get the runtime data of an execution

Discover and Push to Agents Verbs
 generate_discovery_input - Generate Discovery Input file for discovering Auto-Discovered Domains
 refresh_fa - Refresh Fusion Instance
 run_fa_diagnostics - Run Fusion Applications Diagnostics

Fusion Middleware Provisioning Verbs
 create_fmw_domain_profile - Create a Fusion Middleware Provisioning Profile from a WebLogic Domain
 create_fmw_home_profile - Create a Fusion Middleware Provisioning Profile from an Oracle Home
 create_inst_media_profile - Create a Fusion Middleware Provisioning Profile from Installation Media

Incident Rules Verbs
 add_target_to_rule_set - Add a target to an enterprise rule set.
 delete_incident_record - Delete one or more open incidents
 remove_target_from_rule_set - Remove a target from an enterprise rule set.

 Job Verbs
 export_jobs - Export job details in to an xml file
 import_jobs - Import job definitions from an xml file
 job_input_file - Supply details for a job verb in a property file
 resume_job - Resume a job or set of jobs
 suspend_job - Suspend a job or set of jobs

 Oracle Database as Service Verbs
 config_db_service_target - Configure DB Service target for OPC

Privilege Delegation Settings Verbs
 clear_default_privilege_delegation_setting - Clears the default privilege delegation setting for a given list of platforms
 set_default_privilege_delegation_setting - Sets the default privilege delegation setting for a given list of platforms
 test_privilege_delegation_setting - Tests a Privilege Delegation Setting on a host

SSA Verbs
 cleanup_dbaas_requests - Submit cleanup request for failed request
 create_dbaas_quota - Create Database Quota for a SSA User Role
 create_service_template - Create a Service Template
 delete_dbaas_quota - Delete the Database Quota setup for a SSA User Role
 delete_service_template - Delete a given service template
 get_dbaas_quota - List the Database Quota setup for all SSA User Roles
 get_dbaas_request_settings - List the Database Request Settings
 get_service_template_detail - Get details of a given service template
 get_service_templates -  Get the list of available service templates
 rename_service_template -  Rename a given service template
 update_dbaas_quota - Update the Database Quota for a SSA User Role
 update_dbaas_request_settings - Update the Database Request Settings
 update_service_template -  Update a given service template.

 SavedConfigurations
 get_saved_configs  - Gets the saved configurations from the repository

 Server Generated Alert Metric Verbs
 validate_server_generated_alerts  - Server Generated Alert Metric Verb

Services Verbs
 edit_sl_rule - Edit the service level rule for the specified service

Siebel Verbs
 list_siebel_enterprises -  List Siebel enterprises currently monitored in EM
 list_siebel_servers -  List Siebel servers under a specified siebel enterprise
 update_siebel- Update a Siebel enterprise or its underlying servers

SiteGuard Verbs
 add_siteguard_aux_hosts -  Associate new auxiliary hosts to the system
 configure_siteguard_lag -  Configure apply lag and transport lag limit for databases
 delete_siteguard_aux_host -  Delete auxiliary host associated with a site
 delete_siteguard_lag -  Erases apply lag or transport lag limit for databases
 get_siteguard_aux_hosts -  Get all auxiliary hosts associated with a site
 get_siteguard_health_checks -  Shows schedule of health checks
 get_siteguard_lag -  Shows apply lag or transport lag limit for databases
 schedule_siteguard_health_checks -  Schedule health checks for an operation plan
 stop_siteguard_health_checks -  Stops all future health check execution of an operation plan
 update_siteguard_lag -  Updates apply lag and transport lag limit for databases

Software Library Verbs
 stage_swlib_entity_files -  Stage files of an entity from Software Library to a host target.

Target Data Verbs
 create_assoc - Creates target associations
 delete_assoc - Deletes target associations
 list_allowed_pairs - Lists allowed association types for specified source and destination
 list_assoc - Lists associations between source and destination targets
 manage_agent_partnership - Manages partnership between agents. Used for explicitly assigning agent partnerships

Trace Reports
 generate_ui_trace_report  -  Generate and download UI Page performance report (to identify slow rendering pages)

VI EMCLI Verbs
 add_virtual_platform - Add Oracle Virtual PLatform(s).
 modify_virtual_platform - Modify Oracle Virtual Platform.

To get more details about each verb, execute
$ emcli help <verb_name>
Example: $ emcli help list_assoc

New resources in list verb
These are the new resources in EM CLI list verb :
Certificates
  WLSCertificateDetails

Credential Resource Group
  PreferredCredentialsDefaultSystemScope - Preferred credentials (System Scope)
  PreferredCredentialsSystemScope - Target preferred credential

Privilege Delegation Settings
  TargetPrivilegeDelegationSettingDetails  - List privilege delegation setting details on a host
  TargetPrivilegeDelegationSetting - List privilege delegation settings on a host
  PrivilegeDelegationSettings  - Lists all Privilege Delegation Settings
  PrivilegeDelegationSettingDetails - Lists details of  Privilege Delegation Settings

To get more details about each resource, execute
$ emcli list -resource="<resource_name>" -help
Example: $ emcli list -resource="PrivilegeDelegationSettings" -help

Deprecated Verbs:
Agent Administration Verbs
 resecure_agent - Resecure an agent

To get the complete list of verbs, execute:
$ emcli help

Update (6/11):- Please note that the "Gold Agent Image Verbs" and "Agent Update Verbs" verbs shown under "emcli help" are not supported yet.

Stay Connected:
Twitter |
Facebook | YouTube | Linkedin | Newsletter
Download the Oracle Enterprise Manager 12c Mobile app

Thursday Jul 11, 2013

Oracle Enterprise Manager 12c Release 3: What’s New in EMCLI

If you have been using the classic Oracle Enterprise Manager Command Line interface ( EMCLI ), you are in for a treat. Oracle Enterprise Manager 12c R3 comes with a new EMCLI kit called ‘EMCLI with Scripting Option’. Not my favorite name, as I would have preferred to call this EMSHELL since it truly provides a shell similar to bash or cshell. Unlike the classic EMCLI, this new kit provides a Jython-based scripting environment along with the large collection of verbs to use. This scripting environment enables users to use established programming language constructs like loops (for, or while), conditional statements (if-else), etc in both interactive and scripting mode.

Benefits of ‘EMCLI with Scripting Option’

Some of the key benefits of the new EMCLI are:

  • Jython based scripting environment
  • Interactive and scripting mode
  • Standardized output format using JSON
  • Can connect to any EM environment (no need to run EMCLI setup …)
  • Stateless communication with OMS (no user data is stored with the client)
  • Generic list function for EM resources
  • Ability to run user-defined SQL queries to access published repository views

Before we go any further, there are two topics that warrant some discussion – Jython and JSON.

Jython

Jython is the Java implementation of the Python programming language. I have been working with Python (or CPython) and Jython for the last 10 years, and to me it is the best scripting language ever. It is fun, easy to learn, the syntax is simple, is self formatted, and is dynamically typed. This comic from XKCD summarizes it the best:

Python

There are numerous tutorials for Python/Jython on the web, so feel free to pick anyone you like but just remember that the Jython version supported by the new kit is v2.5.1.

JSON

JSON stands for JavaScript Object Notation. It is a data interchange format, much like XML, which is easier to read and write for both humans and machines, but unlike XML it contains very little metadata (elements and attribute names). JSON format is quite simple; it basically represents data as a collection of name/value pairs. These pairs can be contained within arrays, lists, or maps. Here is a sample:

{"menu": {

"id": "file",

"value": "File",

"popup": {

"menuitem": [

{"value": "New", "onclick": "CreateNewDoc()"},

{"value": "Open", "onclick": "OpenDoc()"},

{"value": "Close", "onclick": "CloseDoc()"}

]

}

}}

JSON is quite popular. You will often find it used with REST based web services APIs or even with some modern databases like MongoDB. Most programming languages provide libraries to work with JSON.

The EMCLI kit uses JSON as its output format as well. Many of the verbs return output in JSON format for ease of programmatic use. I say many, since there are still some verbs that don’t, but this is only matter of time.

Now let’s get back to EMCLI.

Steps to setup the kit for ‘EMCLI with Scripting Option’

1. To download the new EMCLI kit, go to Setup->Command Line Interface. Here you will notice the new section for ‘EMCLI with Scripting Option’. Click on the link to download the kit to your desktop or desired server.

Download

You can also download the kit directly from the following url:

http(s)://<host>:<port>/em/public_lib_download/emcli/kit/emcliadvancedkit.jar
 

2. Copy the kit (emcliadvancedkit.jar) to a directory where you wish to install EMCLI

kit

3. To install, run the following command. Note that we need the Java version to be 1.6.0_43 or greater.

java -jar emcliadvancedkit.jar client -install_dir=<emcli_client_dir>

Verify Java version 

4. The last step to complete the setup is to run ‘sync’. Before using EMCLI you have to connect to the OMS to install all verb-related command line help. In classic EMCLI, this happens automatically when you run the ‘setup’ command. But in the new EMCLI, since we do not run setup, we run the ‘sync’ command instead.

The ‘sync’ verb now accepts some additional arguments. Run the following command:
emcli sync -url=http(s)://<host>:<port>/em -username=<user> -trustall

It will prompt for the user password and then take a few minutes to download and install all the help content.

emcli sync

5. Now confirm the setup with a simple test. We do this using the interactive mode. Just run ‘emcli’, and once you see the prompt run ‘help()’. This will print list all verbs along with their description.

emcli interactive mode

With the setup complete, let’s now have some fun.

Using the ‘EMCLI with Scripting Option’

Connect to the interactive mode by running ‘emcli’ from the command prompt. Now try the following commands:

1. Basic Jython: Since EMCLI is built using the Jython interpreter, you can run Jython commands at the EMCLI prompt. For example, you can try the following:

>>1+2

>>print “Hello Jython”

>>mylist = [1,2,3]

>>print mylist

Jython test

2. EMCLI Status: Next, print the status of the EMCLI session using the ‘status()’ command.

emcli status

You will notice that the EM url and user are not set. To do this we have to set the client_properties. Run ‘help('client_properties')’ for more details.

client properties

The help text instructs us to set the client properties to connect to a specific EM environment. The 4 properties of interest to us are the following:

Name

Details

EMCLI_OMS_URL

The EM url

EMCLI_USERNAME

The EM user to connect as. We will use the login() function to set this.

EMCLI_TRUSTALL

I like to set this to true, but the default is false.

EMCLI_OUTPUT_TYPE

I like to set this to JSON even for interactive mode

To set these properties run the following:

>>set_client_property('EMCLI_OMS_URL','http(s)://<host>:<port>/em')

>>set_client_property('EMCLI_TRUSTALL','TRUE')

>>set_client_property('EMCLI_OUTPUT_TYPE', 'JSON')

>>login(username="<em_user>",password="<password>")

You should see the message on successful login. Now we are connected to EM.

login

3. Understanding help and verb invocations: Most of the help text presented in EMCLI is tailored towards the classic interface. Since Jython is a programming language, verb invocations are done in the function form. There is a simple mechanism for converting the classic invocation format for use in both interactive and scripting mode. Let’s use the login() verb as an example.

The EMCLI help for login is as follows:

>>help('login')

emcli login

-username=<EM Console Username>

[-password=<EM Console Password>]

[-force]

This means, when using classic EMCLI, you would invoke it as follows:

emcli login –username=”foo” –password=”bar” -force

Instead, in the interactive or script mode, the invocation would look like:

login(username="<em_user>",password="<password>",force=True)

Essentially, all verbs are now functions, and all arguments to the verb are now parameters passed to the function. Since the –force argument does not take any value, it is treated as a Boolean in Jython and takes the values of True or False.

Note: The -force parameter in the login() function is not applicable to the interactive or script mode, but is being used in this example to explain the concept of passing Boolean values. Again, you should never use the -force parameter in the interactive or script mode.

Another such conversion that you may come across is for list of values. For example,

In classic EMCLI, some verbs will ask for the same attribute to be repeated with varying values to represent a list.

emcli grant_privs -name='jan.doe' 
         -privilege="USE_ANY_BEACON"
         -privilege="FULL_TARGET;TARGET_NAME=host1.acme.com:TARGET_TYPE=host"

In interactive or script mode, you can use native Jython listes instead and pass it as parameters. In Jython, lists are represented within square brackets ([]).

>>priv_list = ['USE_ANY_BEACON','FULL_TARGET;TARGET_NAME=host1.acme.com:TARGET_TYPE=host']
>>grant_privs(name='jan.doe',privilege=priv_list)

4. Sample Use Case: Let’s take a very simple use case to demonstrate the interaction with EMCLI in the interactive mode. So our sample use case is to ‘List all targets of type oracle_database and those whose name starts with the characters ‘db’”.

For this use case, we will make use of the new generic ‘list’ verb. Traditionally, each feature in EM provided its own verbs for list, get, show, and describe. Rather than working with multiple such variants, the new generic ‘list’ verb takes a page from the REST web service specification and provides a generic action that can work against different EM resources.

To learn more about this verb, we ru:

>>help('list')

help

The help text shows us the format of this verb. Essentially, there are 3 parameters that we care about:
  • resource = the EM resource which is to be queried
  • columns = specify the different resource attributes to display
  • search = filters to narrow down the result

First, we need to know the list of resources that are supported by this verb. For this we run

>>List(‘help’)

list help

From the output, it is obvious that for our sample use case we want to query the Targets resource.

Second, we need to know which columns are supported by the Targets resource. For this, we run

>>list('help',resource="Targets")

help resourcesl

From the output, we can determine that we need the column related to target name and type. With this we have all the information we need to construct the final function call for our sample use case.

For ease of explanation, I will break down the process of determining the final function call into small incremental steps. Once you gain proficiency, you will be able to define this function in a single pass.

       1. List all targets in the EM environment. For this we run,

>>list(resource="Targets")

This command will spew a lot of text on your screen as there are likely to be numerous targets in your EM environment. So instead of listing all of them on the screen, let’s just get a count. For this, we need to understand the output format of this verb.

Any function that you run in the interactive or script mode returns an object of class Response (<class 'emcli.response.Response'>). The Response class has 4 key methods:

Function

Description

out()

Provides the verb execution output. The output can be text, or the JSON.

 isJson() method on the Response object can be used to determine whether the output is JSON.

error()

Provides the error text (if any) of the verb execution if there are any errors or exceptions during verb execution.

exit_code()

Provides the exit code of the verb execution. The exit code is zero for a successful execution and non-zero otherwise.

isJson()

Provides details about the type of output. It returns True if response.out() can be parsed into a JSON object.

So let’s look at a code snippet.

snippet

For the first function call to list all targets in EM, we store the results into a variable called ‘all_tgts’. This variable contains the response object. ‘all_tgts.out()’ will give us the actual output. The output returned is in JSON format which automatically gets converted into a Jython dictionary (collection of name-value pairs represented by curly brackets). The output dictionary has a key name called ‘data’ which contains all search results in the form of a Jython list as its value. Finally, len() is a native Jython function which returns the number of elements in a Jython list. As seen in the output, we found 878 targets in the EM environment which is clearly not what we desire.

 2. Now we add search parameters to filter our results. We add two search filters, first the target type should be equal to oracle_database, and second the target name be like db%. You can add multiple search filters to the function call, but all these filters should be encapsulated in a Jython list. The search filter supports various operators: =, !=, >, <, >=, <=, like, null, and not null. Similar to a SQL query, you can also control which columns are to be displayed in the output.

So let’s run our final function.

>>search_filters=["TARGET_TYPE ='oracle_database'","TARGET_NAME like 'db%'"]

>>list(resource="Targets", columns="TARGET_NAME,TARGET_TYPE", search=search_filters)

The formatted output looks like this. As mentioned before it is in the form of a Jython dictionary which can be easily accessed programmatically. The value of the ‘data’ key is a Jython list that contains all search results, while the other keys provide other metadata related to the result.

{

'exceedsMaxRows': False,

'columnHeaders': ['TARGET_NAME', 'TARGET_TYPE'],

'columnLength': [256, 64],

'columnNames': ['TARGET_NAME', 'TARGET_TYPE'],

'data':

[

{'TARGET_NAME': 'db9328.acme.com', 'TARGET_TYPE': 'oracle_database'},

{'TARGET_NAME': 'db3092.acme.com', 'TARGET_TYPE': 'oracle_database'},

],

'filler': '\n\n\n'}

You must have noticed that I hardly talk about the scripting mode. This is on purpose, as I believe that interactive mode is the best interface to learn the new EMCLI. Once you master the interactive mode, converting your code snippets into a script is fairly easy. In future blog posts, I will cover scripting mode and numerous other use cases that seem like a perfect fit for the new EMCLI.

In summary, ‘EMCLI with Scripting Option’ is a new kit that is built on top of a Jython interpreter. It is much superior to the classic EMCLI, as it provides a complete programming environment with the ability to use native Jython functions and primitives. The output is presented in the JSON format which is both human and machine readable, and avoids the need for parsing text output. The client is completely stateless, which means no user data is stored with the client. This means numerous sessions can be launched from a single client, each connecting to a different EM environment, and as a different user.

I encourage you to play around with this new EMCLI kit, and post the different use cases that you found interesting and would benefit the community. You can reach me on twitter @AdeeshF.

Additional Reading:

The EMCLI Documentation Guide

Stay Connected:

Twitter |  Face book |  You Tube |  Linked in |  Newsletter

Monday Apr 29, 2013

Creating a generic service with emcli ( Oracle Enterprise Manager Command Line Interface )

This blog walks you through the steps to create an Enterprise Manager Cloud Control 12c generic service using the command line API emcli.  At the end of the walk through you will have created a service consisting of;
  • A Host, an Oracle Fusion Middleware Farm, a Database and a Listener
  • HTTP and JDBC Service Test

  • System based performance and usage metrics

  • Service based performance metrics

The following emcli verbs will be used during the exercise;

For this exercise we need to create an XML file consisting of the service test definitions and substitution variables.  The documentation for the create_service verb implies that we can specify separate files in the -input_file parameter, one for the tests and one for the variables; however the documentation is wrong and this is being addressed by (Bug 16329952).

The XML schema definition transaction-template.xsd, contains the definition for both variables and template files, however we recommend that you create a template from an existing service definition using the UI and then use the emcli verb extract_template_tests command to extract a pre-existing template, however if you are comfortable with XML file creation then you can review the examples template.xml and variables.xml.

Creating a template from an existing service definition

Using the UI navigate to Enterprise > Monitoring > Monitoring Templates

Click on the Create button

Click on Target for the Copy Monitoring Settings using option button and enter the name of an existing service that you want to copy and then click Continue.

In the next screen provide the Name of the template and click OK; this will accept the defaults that were provided from the target being copied.


Extract the service test definition from the template using emcli.

emcli extract_template_tests \
    -templateName="myTemplate" \
    -templateType=generic_service \
    -output_file=myTemplate.xml

At this point you need to edit the file myTemplate.xml and ensure that you have the correct values for the substitution variables at the top of the file.

<variables>
   <variable name="HOST1" value="myserver.company.com"/>
   <variable name="PASSWORD1" value="thepassword"/>
   <variable name="PORT1" value="7799"/>
   <variable name="PROTOCOL1" value="http"/>
</variables>

Create the System and Service definition

Now we can start by creating the system definition.

emcli create_system \
    -name="mySystem" \
    -add_members="EMGC_GCDomain:oracle_ias_farm" \
    -add_members="emrep.company.com:oracle_database:key_member" \
    -add_members="LISTENER_myserver.company.com:oracle_listener:key_member" \
    -add_members="myserver.company.com:host:key_member" \
    -timezone_region="PST8PDT" \
    -availabilty_type="ALL"

Before we can create the service we must first ensure that we have a beacon to replay the tests.  Create a beacon on an existing agent.

emcli add_target \
    -name="myBeacon" \
    -type="oracle_beacon" \
    -host="myserver.company.com"

Now we are ready to create the service.

emcli create_service \
    -name="myService" \
    -type="generic_service" \
    -availType="test" \
    -availOp="or" \
    -timezone_region="PST8PDT" \
    -systemname="mySystem" \
    -systemtype="generic_system" \
    -input_file=template:"/u01/app/oracle/home/myTemplate.xml" \
    -beacons="myBeacon:Y" \
    -keycomponents="emrep.company.com:oracle_database"

NOTE: Although -timezone_region is optional if it is not specified then the availability computation of the targets will fail and the service and its tests will always show status pending.  Therefore you MUST specify the timezone_region parameter (Bug 16344350).

Additionally if you try to create a service with a name that has been used before it will fail with the error;

Oracle Error :ORA-20233: Target with guid CB9DED6762A117EC708709489C883230 does not exist

This is fixed by Patch 10096491.

We can now create a bunch of Usage and Performance metrics for the service definition that we can alert on.  The key to these metrics is the MGMT$METRIC_COLLECTION view, from this view you will use the METRIC_NAME and METRIC_COLUMN field for the metric definitions for each target type. First we will create the Usage metrics which can only be based on the system definition and not the service.

Let’s add a Usage metric based on the Active HTTP Requests across all targets on the system definition.

emcli set_metric_promotion \
    -name="myService" \
    -type="generic_service" \
    -category=Usage \
    -basedOn=system \
    -aggFunction=AVG \
    -promotedMetricKey="Active HTTP Requests" \
    -metricName="ohs_server" \
    -column="request.active" \
    -depTargetType="oracle_apache" \
    -depTargets="/EMGC_GCDomain/instance1/ohs1" \
    -threshold="125;100;GT" \
    -mode=CREATE

Now one for the Average Active Sessions for the database targets in the system.

emcli set_metric_promotion \
    -name="myService" \
    -type="generic_service" \
    -category=Usage \
    -basedOn=system \
    -aggFunction=AVG \
    -promotedMetricKey="Average Active Sessions" \
    -metricName="instance_throughput" \
    -column="avg_active_sessions" \
    -depTargetType="oracle_database" \
    -depTargets="emrep.company.com" \
    -threshold="150;125;GT" \
    -mode=CREATE

Next the Performance metrics, the first one based on the system definition for CPU Utilization (%).

emcli set_metric_promotion \
    -name="myService" \
    -type="generic_service" \
    -category=Performance \
    -basedOn=system \
    -aggFunction=AVG \
    -depTargetType="host" \
    -depTargets="myserver.company.com" \
    -metricName="Load" \
    -column="cpuUtil" \
    -promotedMetricKey="CPU Utilization (%)" \
    -threshold="80;70;GE" \
    -mode=CREATE

The second based on the Total Time (ms) for the JDBC test

emcli set_metric_promotion \
    -name="myService" \
    -type="generic_service" \
    -category=Performance \
    -basedOn=test \
    -aggFunction=AVG \
    -testname="Database Login" \
    -testtype="JDBC" \
    -beacons="myBeacon" \
    -promotedMetricKey="Total Time (ms)" \
    -column="total_time" \
    -metricName="jdbc_response" \
    -metricLevel=TXN \
    -threshold="200;100;GT" \
    -mode=CREATE

The third based on the Perceived Total Time (ms) for the Homepage test

emcli set_metric_promotion \
    -name="myService" \
    -type="generic_service" \
    -category=Performance \
    -basedOn=test \
    -aggFunction=AVG \
    -testname="Homepage" \
    -testtype="HTTP" \
    -beacons="myBeacon" \
    -promotedMetricKey="Perceived Total Time (ms)" \
    -column="avg_response_time" \
    -metricName="http_response" \
    -metricLevel=TXN \
    -threshold="12000;6000;GT" \
    -mode=CREATE

At this point the Service homepage in Cloud Control looks like;

You will notice it does not show a chart, the metrics do not appear however if you navigate to Usage or Performance metrics page and click OK the metrics appear on the chart and you have to do this for both Usage and Performance.


About

Latest information and perspectives on Oracle Enterprise Manager.

Related Blogs




Search

Archives
« February 2015
SunMonTueWedThuFriSat
1
3
4
5
6
7
8
10
12
13
14
15
16
17
19
20
21
22
26
27
28
       
       
Today