Tuesday Aug 06, 2013

Answers to Your Common Questions on Enterprise Manager 12.1.0.3 Agent deployment using Add Host Targets Wizard (Agent Push)

Deploying EM agents through Add Host Targets Wizard (Agent Push) is one of the most used deployment method and oracle recommended method for deploying multiple agents in a single go.  In this blog we will address some frequently asked questions:

Question 1) Can I perform mass agent deployment using command line?

Ans) Yes, starting EM 12.1.0.3 you can now perform mass agent deployment using EM CLI. This functionality will allow you to push multiple agents from command line.  We can divide EM CLI verbs into 2 categories:


A. Perform agent deployment

a) submit_add_host -- Submit an Add Host session
b) list_add_host_platforms -- List the supported Add Host platforms
c) retry_add_host -- Retry a failed Add Host session

d) continue_add_host - Resume a failed Add Host session


B. Tracking Add Host verbs

a) get_add_host_status        -- Display the latest status of a submitted Add Host session.
b) list_add_host_sessions         -- List the submitted Add Host sessions

You can watch short video on how to perform agent deployment using EMCLI from here:
Screenwatch: How to perform mass agent deployment using Add host wizard (Agent Push) through EM CLI (New in EM 12.1.0.3)

Question 2) Can I use my SSH-key credential while deploying agent through Add Host Targets Wizard (Agent Push) ? 

 Answer 2) Starting EM 12.1.0.3 named credentials support SSH public key authentication and password based authentication. So you can use an existing SSH public key authentication without exposing your passwords. There are 2 use cases here.

i.  If you have setup SSH between the OMS and the target hosts and has the ssh keys handy the he has to follow only steps 4 and 5

ii. If you have not setup SSH between the OMS and the target hosts and does not have the ssh keys then he has to follow steps 1 to 5.

1. Navigate to the following location in the OMS home: $<OMS_HOME>/oui/prov/resources/scripts 

For example, /home/software/em/middleware/oms/oui/prov/resources/scripts

2. Run the following script, and pass the OMS user name you used for installing the OMS and the fully qualified name of the target hosts.

sshUserSetup.sh -setup -user <user_name> -hosts <target_hosts>

3. The SSH keys are generated under $OMS_USER_HOME/.ssh/id_rsa and $OMS_USER_HOME/.ssh/id_rsa.pub

4. Upload the keys to EM

5. Select the keys during agent deployment

Question 3) In my company we use Tectia instead of SSH, can I still use Add Host Targets Wizard (Agent Push) to deploy agents? 

Answer 3)  Yes, Add Host Targets Wizard (Agent Push ) supports SSH Vendors are OpenSSH (SSH2), Tectia. If you have any of these setup between your OMS and target machine you don’t have to do anything extra, application is intelligent enough to pick up the configuration and perform the agent deployment.

Question 4) Does Add Host Targets Wizard (Agent Push) support RSH?  

Answer 4) We don’t support RSH as we need secure shell to perform remote execution and remote copy. Even if RSH is setup Agent push application will not honor it.

Question 5)  I am not sure on which port my SSH is running and how to change SSH port for Add Host Targets Wizard (Agent Push)?

Answer 5) SSH default port is 22 but if you want to cross check or its running on different port then you can cat /etc/ssh/sshd_config to find out what port ssh is running on. Another way to find the port used by SSH you can run:

netstat -anp | grep -i sshd

Output of this command  will be like- 

tcp      0 0 0.0.0.0:22         0.0.0.0:*         LISTEN         3188/sshd

So in the above output 22 is the ssh port.

Once you know the ssh port used, you can update the SSH_PORT property in $<OMS_HOME>/oui/prov/resources/Paths.properties as below: SSH_PORT=<port number>

Question 6) Cygwin is required for deploying agents on Windows host using Add Host Targets Wizard (Agent Push). Once 12c agent is properly installed, is there a need to maintain a copy of Cygwin on the Windows server?  Can it be deleted?

Answer 6) Cygwin 1.7  is required to just deploy EM 12c agent on remote host and can be remove once agents are deployed. Cygwin is not required for any other EM lifecycle operations. 

Follow the steps to configure cygwin for agent deployment from here: http://docs.oracle.com/cd/E24628_01/install.121/e22624/preinstall_req_cygwin_ssh.htm#CBHCDFCH

Question 7)  What is the purpose of user "cyg_server" getting created, while Installing Cygwin, from 12c Cloud Control side. Is this user used during authentication? What if we provide some other username instead of "cyg_server"?

Answer 7) This user is an internal user created by ssh for privilege delegation and not used for agent install.

Question 8) I don’t want to deploy agents using EM login user: SYSMAN. Can I perform role based agent deployment? 

 Answer 8) Yes you can deploy agent from UI and EM CLI with non-sysman user having CREATE_TARGET privilege. If user tries to access the add host page with a user who does not have this privilege, he will get the below error.

Question 9) What all authentication utilities does Add Host Targets Wizard (Agent Push) support?

Answer 9) We support for all standard pdp utilities, sudo, pbrun, sesu , su. 

Question 10) What are the sudo requirements for deploying agent using Add Host Targets Wizard (Agent Push)?

Answer 10)  If your install user can become root then make sure that the /etc/sudoers file have following permission: 

<install_user> ALL=(root) /usr/bin/id, <agent_home>/*/agentdeployroot.sh

For example, oracle ALL=(root) /usr/bin/id, /home/oracle/agentibd/*/agentdeployroot.sh.

Here, oracle is the installing user, and /home/oracle/agentibd is the Management Agent home, that is, the agent base directory.

Question 11) Why do we need install user to have permission to run  /usr/bin/id command?

Answer 11) We use this command to check if the install user has the privilege to switch to root user using the privilege delegation tool.  If you are have locked account usecase then the user that you will switch to should have the permission to run this command. More details on locked account usecase refer to : http://docs.oracle.com/cd/E24628_01/install.121/e22624/install_agent.htm#CACJEFJI

Question 12)  In EM 12.1.0.3 do we still need to set the visiblepw in the /etc/sudoers file?

Answer 12) From EM 12.1.0.3 its not mandatory to set this parameter in /etc/sudoers file instead user can just  pass –enablePty in the additional parameters and set the global property oracle.sysman.prov.agentpush.enablePty=true in $OMS_HOME/sysman/prov/agentpush/agentpush.properties.

You need to set the above property so that the privilege delegation tools such as pbrun, sesu, and su can use pseudo terminal for remote command execution over SSH.

More Information:

What's New in Enterprise Manager 12.1.0.3 Agent Deployment

http://www.oracle.com/technetwork/oem/install-upgrade/em-12103-agent-deployment-1967206.pdf

EM 12.1.0.3 Add Host Targets Wizard (Agent Push) chapter

http://docs.oracle.com/cd/E24628_01/install.121/e22624/install_agent.htm#CACJEFJI 

Screenwatch: How to perform mass agent deployment using Add host wizard (Agent Push) through EM CLI (New in EM 12.1.0.3)  

https://apex.oracle.com/pls/apex/f?p=44785:24:0:::24:P24_CONTENT_ID,P24_PREV_PAGE:7714,1 

    Secure, Subset and Self-Service Database provisioning in Cloud


    One-step inline data masking and data subsetting is a very innovative solution in Oracle Enterprise Manager 12c that enables enterprises to provision secure and reduced size test systems directly from the production or standby database without the need for a full production database copy. So you are able to create reduced size copies of production database keeping the referential integrity of data set intact saving on storage costs and more importantly also ensure your sensitive database never creeps into any of your development or test systems in compliance with data privacy policies.

    With Oracle Enterprise Manager 12c Release 3 customers have now started adopting this one-step on-the-fly masking and subsetting solution along with Database-as-a-Service(DBaaS) self-service provisioning capabilities and the outcome is one of the most comprehensive and unique solution to realize self-service provisioning of secured, subset copies of production like databases for all of testing and development needs in a private cloud model.




    In this blog post I will walk through the steps required to achieve this comprehensive solution for test and development systems provisioning in enterprise private cloud.


    1) Prepare source database

    One of the ideal option in case customer has physical standby is to convert physical standby into a snapshot standby for one-step data subsetting and masking operation. A snapshot standby receives and archives but does not apply redo from primary until converted back to physical standby. So once you have created subsetted and secure dump from snapshot standby you can convert snapshot standby back to physical standby using Oracle Enterprise Manager. Refer to Oracle Data Guard guide for considerations in using snapshot standby.  In-line data subsetting and masking can also be performed on production database in case there is no physical standby.

    2) Perform Inline data subsetting and masking workflow

    Here are the steps :

    a) Create Application Data Model(ADM)
    ADM is a knowledge base entity in Oracle Enterprise Manager that captures application metadata, referential relationships, sensitive columns and is used by both test data management and data masking. There are pre-defined drivers to capture data relationships from application metadata tables for Oracle Applications such as Fusion Apps and eBusiness Suite. Security Administrator requires EM_ALL_OPERATOR privilege to create ADM and Data Masking and Subsetting definition.

    b) Create Masking and Subsetting definition

    As a best practice, Security Administrator can create masking formats for all regulated information in the enterprise. There are out-of-box Oracle supplied default masking formats also that can be used from format library. Next step is to create masking definition that includes information regarding table columns and the format for each column. You can choose which columns to mask. Data Masking workflow can be referred here.
    Next step is to create Data subset definition and use the same ADM and database used in creating the masking definition. In this step you can define the table rules,rule parameters, include 'Ancestor and Descendant tables' or 'Ancestor tables only' option this ensuring referential integrity is maintained. At this step Space estimates can also be reviewed and depending on the result, you can modify or add new rules and review back on space estimates value. Pre/Post subset scripts can also be included in definition.

    c) Generate subset using export option

    Using the definition subset can be created by writing the subset data to export file. Specify directory where to save the export dump and schedule the subset job. Once the job is complete, the subset Data pump file of production data with sensitive data masked is ready. The overall flow is described here
    Some key tips -
    - Parallel degree can be used for faster export. Start with twice the number of CPUs and tune from there
    - %U and max file size parameter can be used to ensure optimum use of parallelism
    - Rule parameter in where clause can be used for actual subset generation with different values
    - Column rules can be used to set large-sized columns to null or fix value to reduce database size further

    3) Prepare secure test/reference database

    Once the Data pump export file is generated it can be imported in a test/staging database . For details refer here.This test database can be used as a reference for all development and testing copies and hosted in Self-Service portal for end users to request from service catalog.

    4) Self-Service Portal Setup

    Following two options are available to cloud-enable this secure reference database for self-service provisioning of future copies of this database via Oracle Enterprise Manager 12c Database-as-a-Service (DBaaS) solution :

    a) Snap Clone option that leverages storage level copy-on-write technologies for cloning and is suited best for functional testing requirements and for short lived databases.With this option you can clone terabytes of data in few minutes and storage saving is enormous with this option. you need to review the supported storage options for this method. As of time of writing the post Netapp and ZFS is supported though there are plans to support more storage options in near future.

    b)  RMAN based cloning from backup is one of the favorite option for DBAs. The restore process is completely automated with EM 12c DBaaS and this option is best suited for performance and load testing, development requirements and for databases used for significant updates.
    Once you decide on the option, steps to follow are documented in one of my recent blog post around Planning Database as a Service Implementation Project

    Refer to Private Cloud Setup and Administration guide here for details on DBaaS Setup and Snap Clone, RMAN Profile options. The secure database prepared in previous step will be used as reference target by SSA Administrator while creating database provisioning profile using RMAN backup or Snapshot option. Please also ensure you have reviewed this MOS Note for DBaaS related patches -
    "Enterprise Manager Cloud Control 12c Recommended Plug-Ins and Patches for Database as a Service (DBaaS) (Doc ID 1549855.1)"




    5) Self-Service database provisioning in Cloud

    End users can now request for secure subset copies of production database via Self-service portal for all kind of functional QA, load and performance testing, development requirements. All the databases provisioned from this approach are also enabled by default with EM 12c powered monitoring and diagnostics, lifecycle and cloud management capabilities.




    Solution References

    1) Oracle Enterprise Manager 12c Packs
    Oracle Test Data Management Pack
    Oracle Data Masking Pack
    Oracle Database Lifecycle Management Pack
    Oracle Database Cloud Management Pack

    2) Oracle Enterprise Manager 12c Platform and Plug-ins
    Enterprise Manager Cloud Control 12c Release 3 Base Platform (12.1.0.3)
    Enterprise Manager for Oracle Database (DB) plug-in 12.1.0.4
    Enterprise Manager for Oracle Virtualization (VT) plug-in 12.1.0.5
    Enterprise Manager Storage Management Framework (SMF) plug-in 12.1.0.2
    Enterprise Manager for Oracle Cloud (SSA) plug-in 12.1.0.6


    Additional information

    Stay Connected:

    Twitter |  Face book |  You Tube |  Linked in |  Newsletter

    Friday Aug 02, 2013

    Managing Oracle Database 12c

    Oracle recently announced new management capabilities for Oracle Database 12c, providing customers with even more enhancements to improve productivity and manageability. The new Oracle Enterprise Manager 12c features are supported out-of-the-box with Oracle Database 12c. So whether you want to take advantage of the new Data Redaction capabilities at the security layer, or whether you want to take a look at some of the heat map capabilities in our Information Lifecycle Management or leverage the new performance management features, such as, Database Operations Monitoring or Enhanced Real-Time ADDM, you can use Oracle Enterprise Manager 12c to manage them all.

    These new capabilities can be integrated with your standard monitoring and management that you already have in your environment without having to undertake any additional tools or any additional training. Simply install or upgrade to Oracle Enterprise Manager 12c and Oracle Database 12c and together these two products will let you plug into the Cloud.

    Here is a quick summary of all the new Oracle Enterprise Manager 12c capabilities for Oracle Database 12c:

    • Complete lifecycle management of multitenant databases: Support for the entire lifecycle operations of multitenant databases that comprise migration, creation, cloning, patching and tracking for compliance
    • Consolidated database replay: Allows customers to test and plan their consolidation using Oracle’s multitenant container and pluggable database technology
    • At-source data masking: Allows customers to maximize compliance and security by masking data in-flight as it is being copied from production
    • Enhanced diagnostics: Improvements to Automatic Database Diagnostic Monitor (ADDM) for real-time and proactive problem detection, especially designed for catching short, transient performance issues
    • Real-time business operations monitoring: Monitors composite business operations such as; batch jobs and ETL operations, providing real-time reports on all SQLs and PL/SQLs executed inside the job
    • Performance hub: Newly redesigned performance home page that provides administrators with a single, at-a-glance view of all performance data and advisors
    • New Oracle Enterprise Manager Database Express 12c: An out-of-the-box, lightweight, easy-to-use web-based management solution for managing individual databases
    • Schema as a Service: This ultimate consolidation of databases lets administrators host multiple application schemas within a single database and offer database as a service to hundreds of application users without creating database sprawl
    • Snap Clone: Instant database cloning allows administrators to create fully functional copies of databases using the copy-on-write capabilities of the underlying storage layer

    To learn more about all the latest capabilities, be sure to register for our upcoming webcast: Managing Oracle Database 12c with Oracle Enterprise Manager 12c.

    Stay Connected:
    Twitter |
    Facebook | YouTube | Linkedin | Newsletter
    Download the Oracle Enterprise Manager Cloud Control12c Mobile app

    Wednesday Jul 31, 2013

    IOUG SIG Webcast on August 7th : SQL Monitoring and ASH Analytics

    The Oracle Enterprise Manager Special Interest Group (SIG) growing body of IOUG members who manage or are interested in all aspects of Enterprise Manager. This IOUG SIG is managed by volunteers and supported by Oracle EM product managers and developers. The purpose of the SIG is to bring relevant information and education through webcasts, discussions and networking to users interested in learning more about the product, and to share user experiences.

    On August 7th at 10 AM pacific time, Oracle Enterprise Manager SIG is hosting a webcast on SQL monitoring and ASH analytics. In this webcast, Edward Whalen, Chief Technologist, Performance Tuning Corporation will provide a tutorial on how to take advantage of two of the powerful performance monitoring and debugging tools; SQL Monitoring and ASH Analytics.

    After attending the session, the attendee should have a better understanding of:

    1. Oracle Enterprise Manager Cloud Control 12c and how to take advantage of some of the key performance monitoring features.

    2. The usage and basic properties of SQL Monitoring; how to invoke it and how to use it to identify long running SQL statements.

    3. What ASH Analytics is, what is ASH data and how to use ASH Analytics for determining which sessions, modules or SQL statements are consuming system resources.


    Stay Connected:

    Twitter |  Face book |  You Tube |  Linked in |  Newsletter

    Tuesday Jun 25, 2013

    Planning Database as a Service Implementation Project

    People, process and planning are the three key elements to success in a private cloud journey. Some common questions i hear from field/customers often relates to tasks involved in setting up Database-as-a-Service(DBaaS) using Oracle Enterprise Manager 12c from scratch and how these tasks are mapped to current IT roles and responsibilities - overall list of activities that needs to be performed to setup and achieve DBaaS and who does what during the implementation process.

    In terms of roles, responsibilities and users, you will map to these categories

    • Project Manager is responsible for working with all stakeholders, project team and coordinating in planning and delivering DBaaS implementation project
    • EM Admin sets ups Cloud management infrastructure and is responsible for overall EM 12c installation/configuration and Administration
    • Cloud Admins are responsible for managing the Cloud infrastructure(Servers, Network, Storage) and related Administration
    • SSA Admins or Cloud DBAs are responsible for Self Service portal setup, managing quotas, policies, target DBs S/W provisioning/ongoing Administration
    • SSA Users or End users of Cloud who actually requests for DB instances to be rapidly provisioned on-demand from Self Service portal


    Here is a sample Work-breakdown structure(WBS) listing down tasks involved and with individual owners mapped. Note - we have all type of use cases covered in the plan (DBCA, RMAN, Schema as a Service, Snap Clone) and you can pick up any or mix or all of these tasks in your plan depending on your use cases and requirements for implementing DBaaS -


     Activity Description
     Owner
    A. Initiation and Startup

    Determine scope of implementation
    Determine and document scope of DBaaS Cloud implementation together with customer and end users. Determine requirements and goals. Introduction meeting implementation team. Project Manager
    Coordination and quality management Coordination and quality management on throughout all phases of the implementation Project Manager
    Define High availability Needs
    Determine the level of High availability required Project Manager
    Technical intake Technical intake meeting; Infrastructure requirements (Hardware, OS, network/firewall, s/w requirement) for cloud setup Project Manager
    Functional intake Functional intake meeting; Monitoring /Reporting/Alerting requirement. User/Roles/Groups/Rules model Project Manager
    Deliver implementation plan Document requirements and goals. Document required functional and technical configuration. Build architecture Diagram. Update estimate planning based on implementation plan. Project Manager
    B. Infrastructure Setup

    Install hardware systems Plan EM12c compatible hardware systems EM Admin
    Operating system installation
    Installation of certified OS platform [Refer Certify on MOS] EM Admin
    Network configuration of hardware systems General network configuration of hardware systems I.e. TCP/IP, routing, NTP, firewall access, SSH, HTTP and HTTPS access. Set up network access between systems and work place of end users. Optionally implementation of remote access EM Admin
    C. Production EM 12c Cloud Control Installation

    Install of Oracle Database 11gR2 Installation of Oracle Database 11gR2 for Repository EM Admin
    Installation of EM 12c Cloud Control Release 2 Update 1 and latest PSU Installation of EM 12c Cloud Control OMS/Agents EM Admin
    Review HA, Security, Sizing Best practices Setup as per EM 12c Admin guide EM Admin
    Review recommended patches Review latest PSU and DBaaS patches as per MOS Note 1549855.1 EM Admin
    D. Setting up High Availability

    Configure Repository For High Availability Enable Repository as per database high availability best practices.
    Configure Standby database @ DR site for EM cloud control repository.
    Configure repository database as RAC database with 2 or more nodes
    Configure OMS for RAC/standby repository
    EM Admin
    Configure OMS for High Availability See Enterprise Manager Cloud Control Administrator's Guide for Configuring OMS for high availability EM Admin
    Configure Agents for High availability See Enterprise Manager Cloud Control Administrator's Guide for Configuring Agents for high availability EM Admin