A common request from customers is the need to roll up data from multiple Enterprise Manager (EM) instances into a single view, to monitor and evaluate the status of databases across regions, applications, or configured groups. The EM federation capability is designed for that purpose and has been included since version 13.4 Release Update 4 (RU4). The good news is, there are no additional plug-ins or software installations needed to use it. Federation dashboards provide users a single pane of glass to monitor and drill down into details to triage and manage across all Federated EM sites by geographies or business units.

See more in the webinar. Realtime Federation For Extreme Database Monitoring (Youtube)

EM Federation Dashboard
Figure 1:  EM Federation Dashboard

 

There are two out-of-box EM Federation Dashboards present in Dashboard listing page, as shown in the screenshot below. The Dashboard listing page can be accessed via the hamburger menu. Details are discussed in the EM Configuration section below.

Fed Summary
Figure 2: Dashboard Listing Page

 

Dashboard 1-2
Figure 3:  Enterprise Manager Dashboard Federation Summary

 

Federation Dashboards Overview

The EM Federation Dashboard aggregates EM instances and summarizes target status, incidents, problems, and jobs across them in real-time. Dashboards are displayed by site or by a group. Each dashboard also provides context-specific drill-downs.

 

Federation Summary
Figure 4: EM Federation Dashboard Aggregate EM instance Summary Target Status

 

Federation Dashboard – By Sites

Selecting the Federation Dashboard – By Sites gives a bird’s eye view of your whole enterprise tab gives a bird’s-eye view of your whole enterprise.  In some cases, it is desirable to filter the data by a group you are interested in. Let’s say you are a DB admin and interested in only a few select groups. Use the filter dropdown menu to choose the interested group where you want to drill down further. Using the “By Group” tab, you can now click to view the group you created.  The configuration of groups is described below in the configuration section.

In the above screenshot, there are 4 EMs “IN-South”, “US-North”, “US-South” & “US-West” and 3 groups are registered i.e. “Prod Databases”, “DevOps Databases” & “ERP Databases”. Say you want to view just the “Prod Database” group. You select it from the dropdown and the dashboard refreshes with “Prod Database” as shown below.

 

Production Databases
Figure 5: Product Database Group

 

To triage the 1 critical incident in “IN-South”, click on the “1” taking you into the “IN-South” EM site in the context of your selected group and selected incident severity to see more details about this incident. The screenshot below shows a tablespace incident above the critical threshold for the “SalesDB” database. Now you can manage this important incident to avoid any outages to this database and eventually the application.

 

Incident Manager
Figure 6: Tablespace incident

 

This use case highlights how we started with the Federated bird’s-eye view of the whole enterprise, then double-clicked on the group with a critical incident to triage and eventually managed that specific incident.

Federation Dashboard- By Groups

We have another dashboard “By Groups”, if you are mainly interested in consolidated information of all configured groups across EMs then this is the dashboard for you.

Targets with status
Figure 7:  Consolidated Configured Groups Across EM

 

From this dashboard, one can click on the Group name to view the dashboard in the context of the selected group.

Role-Based Access Control

The Federation Dashboard respects the role-based access control configured at the individual EM instance for the users, so the user will only see the data they have access to.

How to Configure Federated EM Sites and Groups

To use Federation, an EM instance must be set as a Primary site, then instances must be registered to that Primary site. Configuring the Primary EM is a one-time activity and registering Federated EMs into the Primary EM is done by a Super Administrator. Primary EM and Federated EMs trust must be established, this enables primary EM to get data from federated EM using same user.

emctl set property -name oracle.sysman.federation.masterEMAuthNProducerName   -value EMFederation

 

configure primary site
Figure 8:  Configuring a Primary Enterprise Manager Site

 

Configuring a Primary EM Site

To set up the Primary Enterprise Manager site, you need to enable the Enterprise Manager Federation and implicitly make the site the Primary Enterprise Manager site for Federation by running the following:

emctl set property -name oracle.sysman.federation.masterEMAuthNProducerName    -value EMFederation”

To create an AuthN Provider on the Primary Enterprise Manager site, run the following:

“emctl config apiauth -create_provider -name EMFederation -useDefaults -out EMFederation.json

Establishing trust between primary EM and Federated EM

Do the following on each secondary Enterprise Manager site to be federated to establish the trust relationship between the primary Enterprise Manager site and the respective secondary site:

  • Copy the generated JSON file from the primary site to each secondary site.

This operation is designed to be performed out-of-band from Enterprise Manager.

Typically, the file is copied using a secure copy command like the following:

        scp <generated_json_file> <em_secondary_site>
  • Login to each secondary site and create an AuthN Asserter using the JSON file transferred to the respective secondary EM site by running the following:
emctl config apiauth -create_asserter  -name EMFederation  –in EMFederation.json

Any EM 13.4 and above sites can be added as a Federated EM. To add a list of Enterprise Manager sites, you can use any REST API tool. The below examples use cURL command line tool to add “EU-Site” into the Primary EM:

curl -X POST https://primaryEM.oracle.com/em/websvcs/restful/fed/emSites -u ‘sysman’ -H ‘content-type: application/json’ -d ‘

{“siteURL”:”https://eu-site.oracle.com”,

“name”: “EU-Site”}’

The response to the above command will return a unique ID for the EM site.  This ID will be needed to add a certificate or link credentials to a specific EM.  For the above example, the ID is  “422cf85c13354336874a1971c1d57a70” for the “EU-Site”.

Importing a Certificate in the Primary EM Trust Store (Optional)

To allow HTTPS connectivity to other Enterprise Manager sites, the Enterprise Manager administrator imports the Federated Enterprise Manager sites certificates into the Primary Enterprise Manager site’s trust store. This step is required for self-signed certificates. The administrator needs to ensure the certificates provided are PEM encoded (DER encoded certificates are not supported).

The following is an example of using cURL command line tool to import a certificate:

curl -X POST https://primaryEM.oracle.com/em/websvcs/restful/fed/emSites/422cf85c13354336874a1971c1d57a70/certificates -u ‘user1’ -H ‘content-type: multipart/form-data’ -F file=@em_site1.pem

Adding a Group to Federation

The group-based functionality needs the Primary EM to be on the upcoming version 13.5 Release Update 1 (RU1) and above and any Federated EM to be version 13.4 Release Update 9 (RU9). To add a group “Finance DB”, use the following cURL command :

curl -X POST https://primaryEM.oracle.com/em/websvcs/restful/fed/composites -u ‘superadmin’ -H ‘content-type: application/json’ -d ‘

{ “targetName”:”Finance DB”,

    “targetType”:”composite”}

That covers common use cases when using Federation.

Resources: