Friday Oct 18, 2013

MSR Issue on 12.1 Enterprise Controllers

We've noticed a problem with MSR initialization and synchronization on Enterprise Controllers that are using Java 7u45. If you're running into the issue, these jobs fail with Java errors. Java 7u45 is bundled with Oracle Solaris 11.1 SRU 12, so if you're using that version or if you plan to use it, you should be aware of this issue.

There's a simple fix. You can do the fix before upgrading to SRU 12, but you can't do it before you install the Enterprise Controller.

First, log on to the Enterprise Controller system and stop the EC using the ecadm command. This command is in the /opt/SUNWxvmoc/bin directory on Oracle Solaris systems and in the /opt/sun/xvmoc/bin directory on Linux systems:

ecadm stop -w

Then run this command to fix the issue:

cacaoadm set-param java-flags=`cacaoadm get-param -v java-flags -i oem-ec | sed 's/Xss256k/Xss384k/'` -i oem-ec

And then restart the EC:

ecadm start -w

Once you apply this fix, you should be set.

Thursday Oct 10, 2013

Using a RAC Database

I got a question about the Ops Center database:

"What versions of Oracle Database (with or without Real Application Clusters) are supported in Ops Center 12.1.4?"

The answer is 11gR1 (without RAC) or 11gR2 (with or without RAC). There's more information in the Certified Systems Matrix.

Thursday Oct 03, 2013

Installing Ops Center on OVM

I got a question about installing Ops Center:

"Can I install an Enterprise Controller or a Proxy Controller on an LDOM guest?"

Yes. You can also install on Oracle VM Server for x86, or in Oracle Solaris Zones. There are a few catches with installing in zones (Proxy Controllers can't be installed in S10 zones, only S11 zones, for instance). The specifics of installing in a zone are covered in the Oracle Solaris Installation Guide.

For a full listing of the supported platforms for the Enterprise Controller and Proxy Controllers, take a look at the Certified Systems Matrix.

Thursday Sep 26, 2013

Custom Monitoring Policies

In Ops Center, you can create custom monitoring policies to set different monitoring thresholds for different groups of assets. I got a question about how to implement this:

"I'm trying to create a custom monitoring policy that'll be used for all global zones. So I copied the default one, tweaked a threshold and clicked 'Set as Default Policy'. However, even after checking that option, all of my global zones are still using the old monitoring policy. How do I get them to use the new one?"

So, setting a monitoring policy as the default just means that it'll be used for any new assets of that type. So, in this case, new global zones would use your custom policy, but global zones that already have another policy won't automatically switch. To switch your existing global zones over to your new policy, you need to select them (either individually or in a group) and apply the new policy to them. There's an action to do this in the actions pane.

Thursday Sep 19, 2013

Moving the DB to a new location

I got a question about the database location:

"Ops Center installs an embedded Oracle database in /var, but we'd like to move it to another directory. Is this possible?"

So, theoretically you could change the location on an embedded database, by stopping the EC and doing some ZFS juggling to move the /var/opt/sun/xvm/oracle directory contents to a new directory and mounting it to /var/opt/sun/xvm/oracle, but there's an easier (and officially supported) way to do this.

The officially supported way is to switch to a customer-managed database. This database can be located anywhere you want, so you can give it as much space as you need.

Thursday Sep 12, 2013

Backup and Restore issue

I got a question about the backup and restore process for the Enterprise Controller:

"I upgraded my Enterprise Controller from 12.1.3 to 12.1.4 a while back, but I haven't run the 'ecadm backup' command since then. So the only backups I have are from before the upgrade and the automatic one made during the upgrade.

"Well, we just had a crash on our EC system. However, when I try to restore from either of those two backups, it fails, and says that it's looking for the 12.1.3 packages. How do I get this system back up?"

So, in order to restore from a backup file, the Enterprise Controller version needs to be identical. So, the backup from before the upgrade expects 12.1.3, and so does the backup made during the upgrade. (The automatic backup is only made so that, if the upgrade fails, we can roll back to the prior version.)

In this case, you'd have to uninstall 12.1.4, then reinstall 12.1.3, and finally restore from your backup file. The Backup and Recovery chapter explains this process and the requirements.

It's a good general practice to do regular backups so that a system failure doesn't cost you much data, and we also recommend doing a backup after completing an upgrade.

Thursday Sep 05, 2013

Connecting Ops Center to an Enterprise Management Framework

I got a question about making Ops Center work with other tools:

"My environment uses CA Unicenter. Is there a way for me to forward alerts from Ops Center over to CA Unicenter?"

There are two ways to do this.

One way is to use Halcyon's Neuron Integration, which can take alerts from Ops Center and pass them on to CA Unicenter or other Enterprise Management Frameworks.

Another way is to use Oracle Enterprise Manager Cloud Control to take data from Ops Center and then send it on to CA Service Desk.

1. Make sure that any asset that you want to pass along alerts for is Agent-managed in Ops Center.

2. Use the System Monitoring Plug-in to connect Ops Center to Cloud Control.

3. Use the CA Service Desk Connector to connect Cloud Control to CA Unicenter.

Cloud Control has connectors for a variety of Enterprise Management Frameworks, including BMC Service Desk, IBM Tivoli, and HP Service Manager.

Thursday Aug 29, 2013

Oracle Solaris 11 Zone support

I got a question about Oracle Solaris 11 Zones:

"I manually created an Oracle Solaris 11 Zone. I can discover it and bring it into Ops Center, but the Migrate Zone action is greyed out, whereas other zones created through Ops Center have all of their actions enabled. What's the difference? I thought that all zones were supposed to be equal in 12c."

All zones are equal in 12c, but sometimes it seems like some zones are more equal than others.

What's going on here is that, with a manually created zone, the zone metadata is stored locally and the zone storage isn't fully managed. To enable the Migrate Zone action, you need to use the Move Storage action to move the zone metadata and storage. Once you've done that, the storage will be managed and, as long as another global zone is available, the Migrate Zone action should be enabled.

Thursday Aug 22, 2013

Ops Center CLI functions

I see questions from time to time about the Ops Center CLI and what you can do with it, so I thought I'd mention a couple of resources.

The Ops Center CLI lets you connect to an Enterprise Controller and perform many tasks from the command line. These tables from the CLI chapter show the features that can be accessed through the CLI; many but not all of the product's features are CLI-accessible.

That same chapter also explains how to log in, how to use the different CLI modes, and how to run scripted jobs through the CLI.

Tuesday Aug 13, 2013

Database locations

I got a question about customer-managed database locations:

"We are preparing to install Ops Center with ECHA. The install docs say that ECHA uses a customer-managed database on a separate system. What if I partitioned a system, and had the EC node and DB node on separate partitions? Could that work?"

Theoretically yes, although I don't think we've tested that kind of setup.

The reason we say that the database should be on a separate system is because an Enterprise Controller failover doesn't include failover for the database. We assume that, with a customer-managed database, you'll have your own methods for protecting the DB from system failures. ECHA, then, just focuses on the Enterprise Controller, and assumes that the database will still be accessible when the new EC node comes up.

Thursday Aug 08, 2013

Upgrade question

I got a question about upgrading to 12.1.4:

"I'm trying to do a manual upgrade from 12.1.3 to 12.1.4. I've upgraded the Enterprise Controller, but now I can't find the Proxy Controller upgrade bundle. Where is that?"

This depends on how you're downloading the upgrade bundles.

If you're getting them through the UI, they're put in the /var/opt/sun/xvm/images/productUpdate directory. You can copy them from there to the Proxy Controller systems and do a manual upgrade (or upgrade through the UI if you're so inclined.)

You can also get the Proxy Controller bundles from either the OTN or e-Delivery, and then copy them to the Proxy Controller system.

However you do the download, we recommend that once you've started upgrading your environment, you upgrade the whole thing as soon as you can.

Thursday Aug 01, 2013

LDoms and Maintenance Mode

 I got a few questions about how maintenance mode works with LDoms.

"I have a Control Domain that I need to do maintenance on. What does being put in maintenance mode actually do for a Control Domain?"

Maintenance mode is what you use when you're going to be shutting a system down, or otherwise tinkering with it, and you don't want Ops Center to generate incidents and notification of incidents. Maintenance mode stops new incidents from being generated, but it doesn't stop polling, or monitoring, the system and it doesn't prevent alerts.

"What does maintenance mode do with the guests on a Control Domain?"

If you have auto recovery set and the Control Domain is a member of a server pool of eligible systems, putting the Control Domain in maintenance mode automatically migrates guests to an available Control Domain.  When a Control Domain is in maintenance mode, it is not eligible to receive guests and the placement policies for guest creation and for automatic recovery won't select this server as a possible destination. If there isn't a server pool or there aren't any eligible systems in the pool, the guests are shut down.

You can select a logical domain from the Assets section to view the Dashboard for the virtual machine and the Automatic Recovery status, either Enabled or Disabled.

To change the status, click the action in the Actions pane.

"If I have to do maintenance on a system and I do not want to initiate auto-recovery, what do I have to do so that I can manually bring down the Control Domain (and all its Guest domains)?"

Use the Disable Automatic Recovery action.

"If I put a Control Domain into maintenance mode, does that also put the OS into maintenance mode?"

No, just the Control Domain server. You have to put the OS into maintenance mode separately.

"Also, is there an easy way to see what assets are in maintenance mode? Can we put assets into, or take them out of, maintenance mode on some sort of group level?"

You can create a user-defined group that will automatically include assets in maintenance mode. The docs here explain how to set up these groups. You'll use a group rule that looks like this:

Thursday Jul 25, 2013

Management access point authentication failure

Some users of Ops Center 12.1.4 are seeing an issue where, after upgrading or doing a new install, their assets are getting dozens of warning incidents that say "Management access point failed authentication during login to asset." Eventually, the severity increases as the warnings pile up, and communication with the asset can be disrupted.

You can prevent this issue, or halt it if you've already hit it, by logging in to the Enterprise Controller and creating a /var/tmp/no_os_selfhealing file if it doesn't exist. For example:

touch /var/tmp/no_os_selfhealing

You can use this fix before you do the upgrade or installation, to prevent the issue from occurring at all.

If you have assets that are already having communication trouble because of the issue, then once you've added the file as described above, you can solve the issue by deleting and rediscovering the asset deleting the Access Point for the Service Processor, and then rediscovering just the hardware. If you have a Proxy Controller or Enterprise Controller that's been affected, you should restart them after applying this fix.

Edit: You can also fix the issue by downloading the latest version of the OCDoctor (4.24) and running it with the --troubleshoot option. Take a look at this bulletin for more info.

Edit 2: Updated to use a less destructive fix for affected assets.

Tuesday Jul 23, 2013

Provisioning Different Versions of Oracle Solaris 11

I got a couple of questions about the new support for multiple Automated Installer (AI) releases in 12.1.4:

"How do I set up different versions of Oracle Solaris 11 in the Solaris 11 Software Library? For example, I want to use these versions for OSP:

  • Solaris 11/11 SRU 5.5
  • Solaris 11.1 SRU 8.4

So how do I get both of them into my library and ready for OSP use?"

There are a couple of ways. The first is to configure the parent repository to use a source with both of those releases. If you haven't initialized your library, configuring the parent repository is part of the library creation; otherwise it's a separate job, which you can set to repeat according to a schedule.

The other way is to use the Add Content wizard to add a specific S11 release to your library. This action doesn't save the parent repository location, but it works well if you're adding a specific version.

Once your S11 library has the content for both releases, you'll be able to create OSP profiles for either one. There's a dropdown option that lets you pick which version of S11 you want to use:

"I have my library using a valid repository, but the new versions aren't showing up in the OSP wizard. Do I need to restart the Enterprise Controller to make them show up?"

If the new versions aren't showing up, re-running the library sync job should make them appear. You shouldn't have to restart the EC.

Once you've done these steps, you can provision these versions of S11 from any of your Proxy Controllers.

Tuesday Jul 16, 2013

DHCP and OS Provisioning

I got a question about an OS provisioning error:

"When I create an Oracle Solaris 10 OSP profile, I get a "DHCP is not configured" error on the Select Networks step. What do I do about this?"

Well, it depends on what role you have. If you're an Ops Center Admin, you can go into Administration, select a Proxy Controller, and set DHCP up. There are two ways that you can do this - you can either configure DHCP on your Proxy Controllers:

Or, alternatively, if you have an existing DHCP server that you'd prefer to use, you can configure Ops Center to use it:

These tasks both require admin privileges, so you might need to bug your admin if you don't have that role yourself. But, either way, once you have DHCP set up you'll be able to do OS provisioning.


This blog discusses issues encountered in Ops Center and highlights the ways in which the documentation can help you


« April 2014