By user9127559 on Jul 23, 2014
Now that Enterprise Manager 12cR4 has been out for a little while, more people are getting around to upgrading their agents. Since the monthly Patch bundles were released we already have a few Agent side patches that we want to apply to our newly upgraded agents. I’ve written about simplifying your agent patching before, but this feature still seems to fly under the radar. It’s days like these that I miss running a massive Enterprise Manager with thousands of databases, because this is one of the things that would have made me dance in my cubicle.Let's say, you have 100 188.8.131.52.0 agents (50 with Database plug-in, 50 with Middleware plug-in). In my previous blog on EM patches, I explained the different types of patches available for EM, so I’m not going to go into detail here. What I'm going to illustrate is how we can upgrade those 100 agents, and patch them with the following patches in one step (current as of today):
this is just a quick note for those looking to upgrade to the new relase of Ops Center (184.108.40.206.0).
It is avalable for download:
Before you can see the new version in an existing 12.2.0 (or earlier) version of Ops Center you will need to apply a quick patch to to make the new version visable to downlaod. Details of how to access this IDR and how to apply it can be found in the MOS Note (Doc ID 1908726.1)
With the release of Enterprise Manager Ops Center 12.2.1, it is time to go through the upgrade cycle. I thought I would share the pre-upgrade checks I go through when I upgrade to a new Ops Center build. As part of the development team, I get involved in pre-release Quality Assurance testing, which means I end up doing hundreds of upgrades as part of the testing process.
Update releases come out regularly and contain enhancements and bug fixes. As with any other application in your environment, you should upgrade Ops Center to the current release/update in a timely and controlled manner. For those of you who are long time sys-admins, there is no rocket science here. It is the same sort of planning you would do for any other Enterprise level application.
In my test environments, I have my Enterprise Controller (EC) and Proxy Controllers (PC) inside Solaris Zones (Solaris 11), so I have a couple of extra checks I do, but the process as a whole is still valid if your EC/PC are on their own separate hardware.
Yes, those release notes/README files are important and you should spend the time reading them. They will contain the latest information about the update and any known issues and workarounds.
Confirm that there is enough disk space to unpack and install the upgrade. How much is enough space is the ultimate question. It will vary with each different upgrade and will depend on how you have configured your underlying filesystems and your actual environment. Here are some guidelines. Please note that the numbers I quote tend to be a little generous as it is always better to have more free space than not enough.
root@ec:/ec_backup# du -hs * 1.3G sat-backup-pre-12.2.1-upgrade.20140702root@ec:/ec_backup# du -hs /var/opt/sun/xvm/oracle/oradata/OCDB 14G /var/opt/sun/xvm/oracle/oradata/OCDB root@ec:/ec_backup#
Although more space is actually used during the backup before it is packed up, I would allow for about 4 GB of space.
Before commencing any upgrade, you should make sure you can roll back if something goes horribly wrong. Years of history in administration and support have made me a paranoid person. I believe you can never have too many backups, so I do the following:
Of course, copy the generated backup file to somewhere safe on another system.root@ec:/# /opt/SUNWxvmoc/bin/ecadm backup -d pre-12.2.1-upgrade -o /ec_backup/sat-backup-pre-12.2.1-upgrade.20140702 ecadm: using logFile = /var/opt/sun/xvm/logs/sat-backup-2014-07-02-11:52:16.log ecadm: *** PreBackup Phase ecadm: *** Backup Phase ecadm: *** PostBackup Phase ecadm: *** Backup complete ecadm: *** Output in /ec_backup/sat-backup-pre-12.2.1-upgrade.20140702 ecadm: *** Log in /var/opt/sun/xvm/logs/sat-backup-2014-07-02-11:52:16.log root@ec:/#
### Take a zfs snapshot ### root@ec:/# zfs list NAME USED AVAIL REFER MOUNTPOINT rpool 156G 41.1G 31K /rpool rpool/ROOT 134G 41.1G 31K legacy rpool/ROOT/solaris 134G 41.1G 24.6G / rpool/ROOT/solaris-backup-1 174K 41.1G 1.37G / rpool/ROOT/solaris-backup-1/var 110K 41.1G 27.9G /var rpool/ROOT/solaris-backup-2 296K 41.1G 24.2G / rpool/ROOT/solaris-backup-2/var 232K 41.1G 48.4G /var rpool/ROOT/solaris/var 109G 41.1G 77.2G /var rpool/VARSHARE 88K 41.1G 66.5K /var/share rpool/ec_backup 1.29G 41.1G 1.29G /ec_backup rpool/export 161K 41.1G 32K /export rpool/export/home 111K 41.1G 32K /export/home rpool/export/home/ocadmin 61K 41.1G 40.5K /export/home/ocadmin rpool/oracle 20.7G 41.1G 20.7G /var/opt/sun/xvm/oracle root@ec:/# root@ec:/# zfs snapshot -r rpool@pre-OC-12.2.1-install.20140702 root@ec:/#
It is good practice to clear/enable/disable any broken SMF services, but there are a few key ones to check.
Make sure all the Ops Center services that should be running are running and the ones that should not are not. A classic example here is when you have an EC running without a collocated PC. The PC shows as disabled, but still shows in a "svcs -xv" output.
root@ec:/var/tmp/downloads# svcs -xvsvc:/application/management/common-agent-container-1:scn-proxy (Cacao, a common Java container for JDMK/JMX based management solution) State: disabled since June 12, 2014 08:07:08 AM ESTReason: Disabled by an administrator. See: http://support.oracle.com/msg/SMF-8000-05 See: man -M /usr/share/man -s 1M cacaoadm See: man -M /usr/share/man -s 5 cacao Impact: 1 dependent service is not running: svc:/application/scn/proxy-available:default root@ec:/var/tmp/downloads#
In this case, our EC did not have a collocated PC, so we should ensure that these services are really disabled and don't try to start-up during the upgrade process.
root@ec:/var/tmp/downloads# svcadm disable svc:/application/scn/proxy-available:defaultroot@ec:/var/tmp/downloads# svcadm disable svc:/application/management/common-agent-container-1:scn-proxy
root@t4-1-syd04-b:~# svcs svc:/application/pkg/zones-proxyd:default STATE STIME FMRI online Jul_02 svc:/application/pkg/zones-proxyd:default root@t4-1-syd04-b:~#
root@ec:~# svcs svc:/application/pkg/zones-proxy-client:default STATE STIME FMRI online 8:54:47 svc:/application/pkg/zones-proxy-client:default root@ec:~#
root@ec:/var/tmp/downloads# svcs -xvroot@ec:/var/tmp/downloads#
To be able to do a successful upgrade, you need the pkg publisher for a system to be working. In a zones environment, that means the publishers in the GZ and all the NGZ should be working. Publishers that don't resolve when a package links into a zone will cause the whole upgrade to stop.
So here are a couple things to look for when you are using an EC in a zone.
# pkg unset-publisher Publisher-Name
The aim here is to clear all the local publishers in the zone and just use the proxied publishers in the GZ.
root@t4-1-syd04-b:~# pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F https://oracle-oem-oc-mgmt-pc217:8002/IPS/
cacao origin online F https://oracle-oem-oc-mgmt-pc217:8002/IPS/
mp-re (non-sticky) origin online F https://oracle-oem-oc-mgmt-pc217:8002/IPS/
opscenter origin online F https://oracle-oem-oc-mgmt-pc217:8002/IPS/
root@t4-1-syd04-b:~# pkg unset-publisher opscenter
root@t4-1-syd04-b:~# pkg unset-publisher mp-re
root@t4-1-syd04-b:~# pkg unset-publisher cacao
root@t4-1-syd04-b:~# pkg set-publisher -G '*' -g http://ec:11000/ solaris
root@t4-1-syd04-b:~# pkg publisher
PUBLISHER TYPE STATUS P LOCATION
solaris origin online F http://ec:11000/
# /var/opt/sun/xvm/utils/install_ips_ac.sh -P PC_IP_AddressUse 127.0.0.1 as the IP address for the EC/PC when it is pointing too itself
Run the OCDoctor troubleshoot script over your EC and PC's before an upgrade. It is a good sanity check to look for and fix underlying problems before you start the upgrade process. If you are in connected mode, your EC should already have the latest version of OCDoctor downloaded. Otherwise, you can update it by running "OCDoctor.sh --update" or downloading from https://java.net/projects/oc-doctor/downloads/download/OCDoctor-4.36.zip
Note: The error "'root' should not be a role" can be safely ignored as it was only required for earlier versions of Ops Center.
root@ec:/var/tmp/downloads# /var/opt/sun/xvm/OCDoctor/OCDoctor.sh -t Ops Center Doctor 4.34 [OC 220.127.116.1163,SunOS11] [Read only] [02-Jul-2014 11:25AM EST] ======================== Checking Enterprise Controller...============================== OK: Total number of OSes: 12 Total LDOMs:7 Total Zones: ERROR: User 'root' should not be a role. You should convert it to a normal user before the installation. This can be done by running: # rolemod -K type=normal root OK: Files in /var/opt/sun/xvm/images/agent/ have the right permissions OK: Files in /var/opt/sun/xvm/osp/web/pub/pkgs/ have the right permissions OK: both pvalue and pdefault in systemproperty are equal to false (at id 114) OK: Found only 285 OCDB*.aud files in oracle/admin/OCDB/adump folder OK: Found no ocdb*.aud files in oracle/admin/OCDB/adump folder OK: No auth.cgi was found in cgi-bin OK: User 'oracleoc' home folder points to the right location OK: User 'allstart' home folder points to the right location OK: Apache logs are smaller than 2 GB OK: n1gc folder has the right permissions OK: All agent packages are installed properly OK: All Enterprise Controller packages are installed properly OK: Enterprise Controller status is online OK: the version is the latest one (18.104.22.16863) OK: satadm timeouts were increased OK: tar command was properly adjusted in satadm OK: stclient command works properly OK: Colocated proxy status is 'disabled' OK: Local Database used space is 19%, 6G out of 32G (local DB, using 1 files) OK: Debug is disabled in .uce.rc OK: Debug is disabled for cacao instance oem-ec OK: no 'conn_properties_file_name' value in .uce.rc OK: 30G available in / OK: 30G available in /var OK: 30G available in /var/tmp OK: 30G available in /var/opt/sun/xvm OK: 30G available in /opt OK: DNS does not unexpectedly resolve hostname '_default_' OK: Found the server .uce.rc at /var/opt/sun/xvm/uce/opt/server/cgi-bin/.uce.rc OK: Server .uce.rc has the correct file permissions OK: Server .uce.rc has the correct ownership OK: Connectivity to the KB public servers works properly (using download_large.cgi) OK: Grouplock file doesn't exist OK: package email@example.com is not installed OK: package driver/x11/xsvc is not installed OK: Cacao facet is set to False OK: All Solaris 11 agent bundles in /var/opt/sun/xvm/images/agent are imported properly to the repository OK: Disconnected mode is not configured OK: Locales are OK ("en_US.UTF-8") OK: No need to check for Solaris 11 agent bundle issue as this EC is newer than Update 1 OK: No partially installed packages OK: UCE 'private' folder exists OK: No http_proxy is set in the user profile files OK: 'public' folder has the right ownership OK: 'public' folder is writable for uce-sds OK: 'private' folder has the right ownership OK: 'private' folder is writable for uce-sds OK: '/var/tmp' folder is writable for uce-sds OK: No old jobs rerun (CR 6990675) OK: No need to adjust SEQ_COUNT (MAXID:2986 SEQCOUNT:2986) OK: no row with ssh.tunnel.info found in DB table HD_RESOURCE_PARAMETER NOTICE: Can't perform cryptoadm test inside a zone. Run --troubelshoot from the global zone as well to test the crypto services. OK: System time is not in the past OK: User uce-sds is part of all the proper groups OK: oracleoc user ulimit -Sn is 1024 OK: oracleoc user ulimit -Hn is 65536 OK: FC Libraries do not contain duplicate LUNs OK: 'update-saved-state' folder exists and has the right permissions OK: verify-db does not return 'Invalid pad value' message OK: No credential issues found =========== Proxy controller is installed but not configured, skipping ================== =========== Agent controller is installed but not configured, skipping ================== root@ec:/var/tmp/downloads#
Choose whichever upgrade method you like. Both the BUI and CLI methods will give you the same end result. The Ops Center upgrade is not a difficult upgrade and following some simple pre-work checks will maximize your chance of a straightforward and successful upgrade.
The core Enterprise Manager system is typically patched with the quarterly PSU patches (released Jan, Apr, July, Oct) or a one-off when directed by support for a critical issue. PSU patches will be cumulative, so you need not apply each of them, just apply the latest. The OMSes must be shutdown during patching, however some patches are being released with rolling patch instructions for multi-OMS systems. These patches must be applied at the host level, and cannot be automated via EM. ALWAYS read the readme, yes every time. The patching steps can change from patch to patch so it's critical to read the readme. OPatch or OPatchauto will be used to apply these patches. Did I mention to read the readme for every patch? It's also important to note that there may be additional steps when patching in a multi-OMS or standby environment, so read the output of OPatchauto carefully.
Always download the latest OPatch release for the appropriate version. If you read the readme, you already know this! Download patch 6880880 for 11.1 (the OPatch version used by EM) and unzip into the $ORACLE_HOME. Most errors in patching are related to not updating OPatch.
For more information on PSU Patches and patching EM:
Oracle Enterprise Manager Cloud Control Administrators Guide - Chapter 16 Patching Oracle Management Server and the Repository
EM 12c Cloud Control: List of Available Patch Set Updates PSU (Doc ID 1605609.1)
How to Determine the List of Patch Set Update(PSU) Applied to the Enterprise Manager OMS and Agent Oracle Homes? (Doc ID 1358092.1)
Each plug-in has binaries that will require patches as well. Same downtime requirements apply for plug-in patches as the quarterly PSUs. Starting in 22.214.171.124, the plug-in patches are being released as a monthly bundle. This means that if you have 6 plug-ins, you may have 6 OMS side patches to apply - 1 for each plug-in. Bundles are not always released for every plug-in every month. They are cumulative, so pick the latest.
Starting with 126.96.36.199, the individual OMS-side plug-in bundles are being grouped into a System Patch each month. So for example, in June 2014 the System patch includes MOS, Cloud, DB, FA, FMW, SMF, and Siebel plug-ins. Non-required patches will be skipped.
For more information on the EM Patch Bundles and Patching EM:
Enterprise Manager 188.8.131.52.0 (PS3) Master Bundle Patch List (Doc ID 1900943.1)
Enterprise Manager 184.108.40.206 Bundle Patch Master Note (Doc ID 1572022.1)
Agent patches are applied to each agent. They can be applied via EM using the MOS patch plans, which makes it a lot easier when you have 100s or 1000s of Agents to patch! The Patch Plans will start a blackout, validate prerequisites, check for conflicts, and update OPatch for you. If you don't use the Patch Plan you can patch manually with OPatch, don't forget to read the readme! The Agent must be shutdown during the patch application. There are 4 main types of Agent patches you will see:
You can apply the latest Agent bundle, JDBC patch and the plug-in bundles in one patch plan. If there's a conflict, you'll be notified. If the Agents you've selected don't have specified plug-ins, you'll also receive notice during the analyze step. As of now, for my 220.127.116.11 agents, I would apply the 18.104.22.168.1 patch (18873338) and the two available plug-in agent patches DB monitoring (19002534) and FMW monitoring (18953219) and the latest JDBC patches (18502187,18721761) all in one patch plan.
I discovered a new feature in 22.214.171.124 while testing this. Normally you had to have Normal Oracle Home preferred credentials set for all Agent targets to patch, or select Override and specify the Normal Oracle Home credentials. In 126.96.36.199, the Agent uses it's internal credentials to Patch itself, so setting preferred credentials or specifying at run-time is not required. The user patching would require the Manage Target Patch and Patch Plan privileges.
The OMS and Agent are the key components, and my main focus here. However it's important to keep the infrastructure stack up to date as well. This includes the Oracle Fusion Middleware and Oracle Database that are used for EM. The recommendation is to follow the best practices for each of these components, and regularly update with the PSU patches available. The following reference notes will help in identifying the current PSU patches. The WebLogic Server version used by EM 12c is 10.3.6.
Hopefully this will help you understand the various types of components involved with keeping EM up to date. Obviously, you may not want to patch each month and maybe not every quarter, but the patches are available to keep the software up to date and make things easier to apply in bundles. You'll want to setup a plan for planned software maintenance in your environment. There's a whitepaper Oracle Enterprise Manager Software Planned Maintenance that will help guide you through the best practices.
When implementing database as a service and/or snap clone, a common request was for a way to hide the other service types like IaaS, MWaaS, etc from the self service portal for the end users. Before EM12c R4, there was no way to restrict the portal view. Essentially, any user with the EM_SSA_USER role would be directed to the self service portal and would then be able to see all service types supported by EM12c.
Of course, you could always set Database as your default self service portal from the 'My Preferences' pop up, but this only helps with their post-login experience. The end user still gets to see all the options as shown in screen above.
In EM12c R4, a new out of the box role called EM_SSA_USER_BASE has been introduced. This role, by default, does not give access to any portal, that is an explicit selection. Here is how you use this role:
1. Create a custom role and add the EM_SSA_USER_BASE role to it.
2. Now in the Resource Privileges step, select the Resource Type 'Cloud Self Service Portal for Database', and edit it
3. Check the 'Access the Cloud Self Service Portal for Database.' privilege. Finish the rest of the wizard.
Now, when a user with this custom role accesses the self service portal, they can only do so for databases and nothing else.
While the EM_SSA_USER role will continue to work, we recommend you start using the new EM_SSA_USER_BASE role. For more details on DBaaS or Snap Clone roles, refer to the cloud admin guide chapter on roles and users.
-- Adeesh Fulay (@AdeeshF)
Surprisingly, a popular question posted on our internal forum is about the possibility of using the Enterprise Manager (EM) Job System to replace customer’s numerous cron jobs. The answer is obviously YES! I say surprisingly because the EM Job system has been in existence for around 10 years (I believe since EM 10.2.0.1), and my hope was that, by now, customers would have moved to using more enterprise class job schedulers instead of cron. So, here is a quick post on how to get started with this conversion from cron to EM Jobs for some of our new users.
Before we learn about the how, let’s look at the why. The EM job system is:
Now back to our topic.
Let’s start with a sample crontab that we want to convert.
A cron expression consists of 6 fields, where the first 5 fields represent the schedule, while the last field represents the command or script to run.
| Field Name
||Mandatory?|| Allowed Values
|| Allowed special characters
|Minutes||Yes||0-59||* / , -|
|Hours||Yes||0-23|| * / , -
| Day of month
||Yes||1-31|| * / , - ? L W
|Month||Yes|| 1-12 or JAN-DEC
||* / , -|
| Day of week
||Yes|| 0-6 or SUN-SAT
|| * / , - ? L #
Cron jobs run on the operating system, often using the native shell or other tools installed on the operating system. The equivalent of this capability in Enterprise Manager is the ‘OS Command’ job type. Here are the steps required to convert the first entry in the crontab to an EM job:
1. Navigate to the Job Activity page
2. Select the ‘OS Command’ job and click Go
A 5-tab wizard will appear. Let’s step through this one by one.
3. Select the first tab called ‘General’. Here provide a meaningful name and description for the job. Since this job will be run on the Host target, keep the target type selection as ‘Host’. Next, select all host targets in EM that you wish to run this script against.
While cron jobs are defined on a per host bases, in EM a job definition can be run and managed across multiple hosts or groups of hosts. This avoids having to maintain the same crontab across multiple hosts.
4. Select the ‘Parameters’ tab. Here enter the command or script as specified in the last field of the crontab entry. When constructing the command, you can make use of the various target properties.
5. Next select ‘Credentials’. Here we provide the credentials required to connect to the host and execute the required commands or scripts. Three options are presented to the user:
Note: If your OS user does not have the required privileges to execute the set command, Named Credentials also support use of sudo, powerbroker, sesu, etc.
6. Next, we set the schedule and this is where it gets interesting. As discussed before, crontab uses a textual representation for the schedule, while EM Job system has a graphical representation for the schedule.
Our sample schedule in the crontab is ‘00 0 * * Sun’. This translates to a weekly job at 12 midnight on every Sunday. To set this in EM, choose the ‘Repeating’ schedule type. The screenshot below shows all the other selections.
The key here is to select the correct ‘Frequency Type’, the rest of the selections are quite obvious. This also lets you choose the desired timezone for the schedule. Your options are to either start the job w.r.t a fixed timezone, or start it in individual target's timezone. The latter is very popular, for example, I want to start a job at 2 AM local time in every region around the world.
Another selection of note is that for ‘Grace Period’. This is an extremely powerful feature, but often not used by many customers. Typically, we expect jobs to be started within a few seconds or minutes (based on the load on the system and number of jobs scheduled) of the start time, but a job might not start on time for many reasons. The most common reasons are the Agent being down or due to a blackout. The grace period controls the latest start time for the job in case the job is delayed, else its is marked as skipped. By default, jobs are scheduled with indefinite grace periods, but I highly recommend setting a value for it. In the sample above, I set a 3 hr limit which may seem large but given the weekly nature of the job seems reasonable. So the job system will wait until 3 am (the job start time is 12 am) to start the job, after which the iteration will be skipped. For repeating schedules, the grace period should always be less than the repeat interval. If the job starts on time, the grace period is ignored.
7. Finally, we navigate to the ‘Access’ tab. This tab has two parts:
To prevent EM from sending deluge of emails, I recommend the following settings in the notifications region:
You can always come back and modify these settings to suit your needs.
Not all cron jobs need to be converted to OS command. For example, if you are taking Oracle database backups using cron, then you probably want to use the out-of-the-box job type for RMAN scripts. Just provide the RMAN script, list of databases to run this against, and the credentials required to connect to the database. Similarly, if you run sqls on numerous databases, you can leverage the SQL Script job type for this purpose. There are over 50 job types available in EM12c, all available for use from the UI and EMCLI.
Finally, the best way to learn more about the EM Job System is to actually play with it. I also recommend blogs from Maaz, Kellyn, and other users on this topic. Good luck!!
Maaz Anjum: http://maazanjum.com/2013/12/30/create-a-simple-job-for-a-host-target-in-em12c/
Kellyn Pot'vin: http://dbakevlar.com/
-- Adeesh Fulay (@adeeshf)
Contributed by Juergen Fleischer and Mahesh Sharma.
The purpose of this blog is to show you how to create a Secondary I/O domain. The First I/O domain is commonly known as Control Domain (CDOM). There are various terms that are used for a Secondary domain, like alternative I/O domain or redundant I/O domain.
The secondary I/O domain will have been assigned some physical I/O devices, which may be a PCIe bus root complex, a PCI device, or a SR-IOV (Single Root I/O Virtualization) virtual function.
Within Ops Center when creating a Secondary Domain we also use the terms Physical I/O Domain and Root Domain. The Physical I/O Domain maps PCI device end points, and the Root Domain maps PCIe buses, which also has an option to create SR-IOV functions.
In this blog we will show you how to create a Root Domain by assigning PCIe buses, so that we have a redundant I/O domain which will enable us to shutdown the CDOM without affecting any of our guests.Our host a T5-2 (we’re using the same host as in the previous blogs) has two free PCIe buses that have not been assigned to domains (pci_2 and pci_3). So let’s create a Secondary I/O domain (Root Domain) with these buses and give the domain two whole cores with 4 GB of Memory.
We’ll start by creating a Logical Domain Profile. This is created from:
Navigation -> Plan Management -> Profiles and Polices, then click on Logical Domain. On the right under Actions select Create Profiles
From the Identify Profile screen, give the Profile a name (Secondary-I/O in our case) and select Root Domain in the Sub-type, as shown above. Click Next.
Step 2: is where we provide a name for our Secondary I/O domain, we called ours secondary and click Next to continue.
The next screen, show below, we entered the amount of cores and memory.
Step 4: is where we specify how many PCIe buses we will assign to this secondary domain. In our cases we have specified two for pci_2 and pci_3.
The next few steps are optional and are not required for this example.
Step 7: Is the Summary, if everything looks correct click Finish.
Note: The Metadata is on the local disk (file://guest). This is fine for the Secondary Domain as it will not be migrated. It’s just the Logical domains and their Guests that will get migrated, therefore making it mandatory to have the metadata on shared storage for these – if you want migration to succeed!
Now that we have created the Profile, we will create our Secondary domain (Root domain).
This is done from Navigation -> Plan Management -> Deployment Plans – Create Logical Domain and select the plan we have just created. From the Actions panel on the right select "Apply Deployment Plan".
From the Select Target Asset pop-up, select the CDOM and move it to the Target list. Select Next.
Complete Step 1, by specifying the secondary name.
Step 2, we pick the PCIe Buses that will be assigned to the secondary domain.
In step 3: we kept the default Virtual Disk Sever (vds) name.
The next few screens are not required in this example.
From the Summary screen click Finish. This will create the Secondary Domain.
Once the Secondary Domain has been created we can check if it’s built as we specified. We can check via the Ops Center BUI and also command line.
Let’s do both:
And from the command line
While on the command line we can also check if the buses have been assigned correctly.