Tuesday Nov 10, 2015

Oracle Solaris Cluster Manager: Setting Up Geo Disaster Recovery Orchestration

Oracle Solaris Cluster Manager (Cluster Manager) provides a way to set up and administer disaster recovery (DR) orchestration starting in Oracle Solaris Cluster 4.3. Prior to this version, DR orchestration could be administered only by using the command line interface (CLI). The Cluster Manager browser interface makes it easier by providing a user-friendly way of doing it, with either clicks or by typing in the user-defined fields, eliminating the need to memorize the commands and the sub-commands. Providing inline and online help, Cluster Manager also eliminates potential errors by doing appropriate checks.

Prerequisites for DR Orchestration to Be Usable Under Oracle Solaris Cluster Manager 4.3

  • Geographic Edition software is installed.
    # pkg info ha-cluster/group-package/ha-cluster-geo-full

  • Geographic Edition infrastructure is enabled and the Geographic Edition resource groups and resources are running.
    # /usr/cluster/bin/geoadm show
    # /usr/cluster/bin/clresourcegroup status geo-clusterstate geo-infrastructure
    # /usr/cluster/bin/clresource status -g geo-clusterstate,geo-infrastructure

Access the DR Orchestration

In version 4.3, the DR orchestration feature is accessed via the Sites folder in the navigation pane.

Access Sites Details

The site details can be accessed in one of the following ways:

  • Click the site name in the tree under Sites folder in the navigation pane.
  • Click the Sites folder. Then, click the site name in the Sites table in the main panel.

Access Multigroup Details

Browse to the site details page as mentioned above, then click the multigroup name in the Multigroups table.

Access the Details of a Protection Group In a Multigroup

Browse to the Multigroup details page as mentioned above, then click the protection group name in the Protection Groups table.

Creating a Site and a Multigroup

Creating a Site

  • Click the Sites folder. Click the Create button in the Sites table. A wizard will pop up.
  • In the Initial Setup step, enter the Site Name. The role of the current cluster is controller by default. Click the Add Cluster button to add more clusters. Enter the name of the cluster in the table below the button and the role of the cluster. To skip the next step and use the default site properties, check the checkbox.
    Note: In version 4.3, the cluster to be added to the current cluster must have the same password for root user as that of the current cluster.

  • In the Site Properties step, enter the different site property fields.

  • In the Review step, review the already entered values and click Finish.

Creating a Multigroup

  • Browse to the Site details page as mentioned earlier.
  • Click the Create button in the Multigroups table.
  • In the pop-up, enter the multigroup name. The other fields are optional. Then click Create.

DR Orchestration Operations Available in the Cluster Manager Browser Interface

Site Operations

In All Sites Table

  • Create site
  • Join Site
  • Leave Site
  • Validate Site

In Site Details Page

  • Add Cluster to the site
  • Set Role for clusters in the site
  • Accept another cluster in the site as controller
  • Remove a cluster from the site
  • Update site information from another cluster in the site

Multigroup Operations

In the Multigroups Table

  • Create a multigroup
  • Delete a multigroup
  • Update a multigroup information from another cluster in the site
  • Validate a multigroup with another cluster in the site
  • Start Multigroup
  • Stop Multigroup
  • Switch Over Multigroup
  • Take Over Multigroup

In the Protection Groups Table of the Multigroup Details Page

  • Add Protection Group
  • Remove Protection Group

Editing Site and Multigroup Properties

  • Browse to the Site or Multigroup details page.
  • Click the Properties tab at the top below the breadcrumb.
  • Click the Edit button.

  • The area with the Edit button now have Save and Cancel button. The value column changes into editable fields wherever applicable(encircled in blue). Make the changes and click the Save button.

To see more on configuring, administering and CLI options of Oracle Solaris Cluster 4.3 Geographic Edition, follow the links below:

Friday Nov 06, 2015

New choices for Oracle Solaris Cluster Public Networking: Link Aggregation (Trunk & DLMP) and VNIC

From Oracle Solaris Cluster 4.3 and on, in addition to IPMP, cluster is able to use IP over link aggregation and IP over VNIC over link aggregation for public networking.

[Read More]

Tuesday Nov 03, 2015

Configuring a Data Service for Oracle VM Server for SPARC by Using a Graphical Wizard

In earlier releases of Oracle Solaris Cluster,  you were able to configure a data service (sometimes also referred to as an agent) for Oracle VM Server for SPARC only manually using a command line interface. This was tedious, requiring the administrator to run commands on multiple nodes.

With Oracle Solaris Cluster 4.3 comes a new graphical wizard that simplifies the configuration considerably. The wizard, using a few clicks, makes a guest domain, as a black box entity, highly available.

[Read More]

Friday Oct 30, 2015

Introducing the New Cluster Configuration Wizard

Oracle Solaris Cluster users have been using the scinstall command-line interface to configure new clusters in the past. Version 4.3 of the software is introducing a new browser-based wizard that makes this process easier. Users will be able to navigate through a series of panels in a browser to configure a new cluster.

The new cluster configuration wizard in the Oracle Solaris Cluster Manager browser interface has a typical mode and a custom mode. They provide the same degree of customization to the user when configuring a new cluster as the existing interactive scinstall utility, all in a more intuitive interface.

As prerequisites to use this new feature, you will need Oracle Solaris Cluster packages installed on all nodes that are planned to form a new cluster and you need to run the clauth command on all non-control nodes to authenticate the control node, if you haven’t done so previously.

Then you can open a browser and navigate to address “hostname:8998/scm/faces/main”. The “hostname” here stands for the host of the webapp server that Cluster Manager runs on. Note that the host could be the control node (just make sure the Cluster Manager packages are installed on that node) or any machine hosting Cluster Manager that could access the control node. The address will lead you to the login page (shown below). Login as the root role to the control node on this page and you can now start configuring a new cluster.

The panels guide you through the process of configuring a new cluster. You might find the online help pane handy on the right side of the page. At the end of the process, the wizard invokes a script to configure and reboot each node in a sequence automatically. The control node will be the last one in the sequence that is configured and rebooted. When all nodes are rebooted, the new cluster configuration process is completed.

Wednesday Oct 28, 2015

Oracle Solaris Cluster Manager: Getting Started

Oracle Solaris Cluster Manager (Cluster Manager) is a browser interface available starting in Oracle Solaris Cluster 4.2 running on Oracle Solaris 11.

Like its predecessor in Oracle Solaris Cluster 3.3 for Oracle Solaris 10, Cluster Manager displays all cluster objects with their status and details and allows all management actions including creating and deleting objects.

Getting Started

Making Sure Cluster Manager Is Installed

The browser interface is delivered as two IPS packages:

These packages are automatically installed as part of incorporation ha-cluster-full or can be manually installed later if necessary with this command:

   # pkg install ha-cluster/system/manager

After package installation these two SMF services:
    online         Aug_06   svc:/system/cluster/manager-glassfish3:default
    online         Aug_06   svc:/system/cluster/manager:default

will be started automatically and will come online. See Troubleshooting below if either service does not come online.

Running Cluster Manager

Point your browser at a cluster node where the browser interface services are online:

Only https is supported, which means that the first time accessing Cluster Manager you'll be given a certificate warning. Take a moment to assure yourself that you really are talking to the correct machine, then allow your browser to add this name to its exceptions list for certificates.


By default, the target machine to authenticate into is "localhost".

You may enter any machine name
where at least Oracle Solaris Cluster 4.2 is installed, even if it's in a different cluster than the machine you browsed to. In version 4.3, if the target machine isn't yet a cluster node but cluster software is present on that machine you'll be able to connect and then bring up the cluster configuration wizard.

A future post will go into detail on using this wizard.


Cluster Manager in version 4, as in version 3, is laid out with a navigation pane to the left and the details pane in the center. New
in version 4 is a right-side online help pane with links to more documentation.

Both the navigation and online help panes can be pushed aside by clicking on the little black triangle halfway down the center border, and restored by clicking it again. Alternatively, the panes can be resized wider or narrower by grabbing that border and dragging.

in version 4 is a "cluster landing page" showing an overview of the state of important cluster objects, such as resource groups and nodes as a group, as well as "grid boxes", which display information on individual objects. Hovering over the pie graphs and grid boxes brings up tool tips with information. Clicking on the pie graphs or grid boxes is a speedy way to navigate to overview or detail pages for these objects.

Each folder in the navigation pane (except for Tasks, which opens a page of buttons for launching various data service configuration wizards) is dedicated to a specific part of Oracle Solaris Cluster. The folders open to show lists of objects such as resource groups or zone clusters by name, as well as the "landing page" for that type of object.

in version 4, Geographic Edition partnerships are included in the main browser interface, and in version 4.3 the "disaster recovery orchestration" feature is supported via the Sites folder in the navigation pane.

Each folder of the navigation pane has an overview page with tables listing instances of the objects along with their status and other details.

Most of the tables contain buttons for management actions that can be taken on one selected row in the table, and sometimes multiple selected rows. Click a row in the leftmost column to select it. Shift and control clicks allow selection of multiple rows. Most tables can be re-sorted by clicking in the column headers. Other customization is available by clicking on the View drop-down list in the upper left of each table. Many tables have a Create or New button, which will launch wizards for creating new objects such as resource groups, resources, zone clusters, and more.

Clicking an individual object's name in the table drills into the details for that object. Some of these details pages have a Properties tab as well as the default Status tab. The Properties tabs start in read-only mode and can be converted to edit mode by clicking on the Edit button.


In addition to the various object-creation wizards available on some tables, the Tasks page hosts wizards for data services ranging from HA Storage and Logical Hostname to HA for Oracle Database. Cluster Manager 4.3 also offers wizards to create HA for Oracle Solaris Zone and HA for Oracle VM Server for SPARC. Future posts will describe these new wizards in detail.

Resource Groups Topology View

First available in 4.2 is a completely new topology view of resource groups with their resources and how they are laid out, both across global and zone cluster nodes and within global and zone clusters as entities. Navigate to the Resource Groups folder/overview/Status page, then click the Topology tab.

Multi-Cluster Management

Another feature first delivered in 4.2 is the ability to authenticate into multiple clusters at once, then switch back and forth quickly between them. At the top of the navigation pane, next to the "Cluster" heading, is a drop-down list that, when closed, shows the name of the "current cluster." This drop-down list contains names of all clusters authenticated into during this session and also contains the names of
Geographic Edition partners. 

At the bottom is the "Other..." choice, which pops up a dialog to allow authenticating into additional clusters. In a Partnership detail page, clicking on the name of a partner allows authenticating into that cluster.


1) Is the Cluster Manager service available?

The browser interface depends upon two SMF services:
   online         Aug_06   svc:/system/cluster/manager-glassfish3:default
   online         Aug_06   svc:/system/cluster/manager:default

If either of these services is not online, the browser interface will not be available. The very first time cluster/manager is started takes a bit longer than subsequent times because the Cluster Manager application is being deployed into the application server. Sometimes following an uncivilized shutdown, the manager-glassfish3 service might take longer than usual to start.

If either service goes into the
maintenance state, first check the SMF logs:

to see if any useful error messages are present. Often, simply clearing the service will be enough to bring it online. Make sure the manager-glassfish3 service is online first, and if not, clear the service
     # svcadm clear -s svc:/system/cluster/manager-glassfish3:default

Then check that the cluster/manager service is online, and if not, clear that service:
     # svcadm clear -s svc:/system/cluster/manager:default

2) Sometimes an error might cause a loss of connection to the application server. If this should happen, the easiest recovery is to remove the browser cookie for the machine you browsed to, then log in again. In some rare cases, it might be necessary to cycle the application server by performing:
     # svcadm restart -s svc:/system/cluster/manager-glassfish3:default

If the cluster/manager service does not come online automatically, start it as well:
     # svcadm enable -s svc:/system/cluster/manager-glassfish3:default

If you ever need to contact Oracle Support about an issue with Cluster Manager, please provide copies of the server log: /var/cluster/ClusterManager/glassfish3/domains/domain1/logs/server.log, and possibly the next older log as well if the log has rolled over recently.

3) Cluster Manager depends upon the common agent container. This service
will come online when packages are installed. Some additional configuration will be done at first cluster boot. If an error message indicates that there may be a problem with the common agent container, you can check it this way:
     # svcs -a | grep common-agent-container

     # /usr/sbin/cacaoadm status

If the service is not online, use the svcdm command to start, clear, or restart it as appropriate.

See also the Oracle Solaris Cluster documentation Troubleshooting Oracle Solaris Cluster Manager.


Oracle Solaris Cluster Engineering Blog


« November 2015