Tuesday Apr 16, 2013

Oracle Solaris Cluster 3.3 3/13 Update2 patches

The Oracle Solaris Cluster 3.3 3/13 Update2 is released. Click here for further information.

The package versions of the Oracle Solaris Cluster 3.3 3/13 Update2 are the same for the core framework and the agents as for Oracle Solaris Cluster 3.3 or Oracle Solaris Cluster 3.3 05/11 Update1. Therefore it's possible to patch up an existing Oracle Solaris Cluster 3.3 or 3.3u1. But Oracle Solaris 10 8/11 Update10 is a minimum requirement for Oracle Solaris Cluster 3.3 3/13 Update2.

The package versions of the Oracle Solaris Cluster Geographic Edition 3.3 3/13 Update2 are NOT the same as Oracle Solaris Cluster Geographic Edition 3.3 or 3.3u1. But it's possible to upgrade the Geographic Edition without interruption of the service. See upgrade documentation for details.

The following patches (with the mentioned revision) are included/updated in Oracle Solaris Cluster 3.3 3/13 Update2 release. If these patches are installed on Oracle Solaris Cluster 3.3 or higher release, then the features for framework & agents are identical with Oracle Solaris Cluster 3.3 3/13 Update2. It's always necessary to read the "Special Install Instructions of the patch" but I made a 'note' behind some patches where it's very important to read the "Special Install Instructions of the patch" (I'm using the shortcut SIIOTP in this blog).

Included/updated patch revisions of Oracle Solaris Cluster 3.3 3/13 Update2 for Oracle Solaris 10 8/11 Update10 or higher
x 145333-20 Oracle Solaris Cluster 3.3: Core Patch for Oracle Solaris 10 Note: See SIIOTP
x 145335-16 Oracle Solaris Cluster 3.3: HA-Oracle Patch for Oracle Solaris 10 Note: See SIIOTP
x 145337-05 Oracle Solaris Cluster 3.3: HA-SunONE Message Queue Patch for Oracle Solaris 10
x 145341-05 Oracle Solaris Cluster 3.3: HA-Tomcat Patch for Oracle Solaris 10 Note: See SIIOTP
x 145343-06 Oracle Solaris Cluster 3.3: HA-BEA WebLogic Patch for Oracle Solaris 10 Note: See SIIOTP
x 145345-04 Oracle Solaris Cluster 3.3: HA-SunONE App Server Patch for Oracle Solaris 10 Note: See SIIOTP
x 145636-03 Oracle Solaris Cluster 3.3: HA-JavaWebServer Patch for Oracle Solaris 10 Note: See SIIOTP
x 145638-09 Oracle Solaris Cluster 3.3: SC Checks / sccheck Patch for Oracle Solaris 10 Note: See SIIOTP
x 145644-04 Oracle Solaris Cluster 3.3: HA-PostgreSQL Patch for Oracle Solaris 10 Note: See SIIOTP
x 145646-05 Oracle Solaris Cluster 3.3: HA-MySQL Patch for Oracle Solaris 10 Note: See SIIOTP
x 146085-07 Oracle Solaris Cluster 3.3: SDS/SVM Mediator patch for Oracle Solaris 10 Note: See SIIOTP
x 146087-03 Oracle Solaris Cluster 3.3: HA-Apache Patch for Oracle Solaris 10
x 146091-07 Oracle Solaris Cluster 3.3: HA-SAP WEB AS for Oracle Solaris 10 Note: See SIIOTP
x 146093-07 Oracle Solaris Cluster 3.3: HA-NFS for Oracle Solaris 10
x 146241-05 Oracle Solaris Cluster 3.3: SWIFTAlliance patch for Oracle Solaris 10
x 146760-06 Oracle Solaris Cluster 3.3: HA-Oracle E-business suite for Oracle Solaris 10 See SIIOTP
x 146762-07 Oracle Solaris Cluster 3.3: HA-xVM Patch for Oracle Solaris 10
x 146764-07 Oracle Solaris Cluster 3.3: HA-Sybase Patch for Oracle Solaris 10
x 147073-04 Oracle Solaris Cluster 3.3: Man Pages Patch for Oracle Solaris 10
x 147077-02 Oracle Solaris Cluster 3.3: HA-Containers Patch for Oracle Solaris 10
x 147079-02 Oracle Solaris Cluster 3.3: HA-OracleBIE Patch for Oracle Solaris 10
x 147080-03 Oracle Solaris Cluster 3.3: HA-Kerberos Patch for Oracle Solaris 10
x 147082-02 Oracle Solaris Cluster 3.3: HA-DB patch for Oracle Solaris 10
x 147084-02 Oracle Solaris Cluster 3.3: HA-DNS patch for Oracle Solaris 10
x 147086-05 Oracle Solaris Cluster 3.3: HA-Siebel Patch for Oracle Solaris 10
x 147087-03 Oracle Solaris Cluster 3.3: HA-LiveCache Patch for Oracle Solaris 10
x 147089-03 Oracle Solaris Cluster 3.3: HA-SAP Patch for Oracle Solaris 10
x 147091-04 Oracle Solaris Cluster 3.3: HA-SAPDB Patch for Oracle Solaris 10
x 147095-02 Oracle Solaris Cluster 3.3: HA-SUNWscTimesTen Patch for Oracle Solaris 10
x 147141-02 Oracle Solaris Cluster 3.3: Localization patch for Oracle Solaris 10
x 147358-02 Oracle Solaris Cluster 3.3: HA-Agfa Patch for Oracle Solaris 10
x 147490-02 Oracle Solaris Cluster 3.3: HA-Websphere MQ Patch for Oracle Solaris 10
x 147492-02 Oracle Solaris Cluster 3.3: HA Websphere Message Broker patch for Oracle Solaris 10
x 147494-02 Oracle Solaris Cluster 3.3: HA-Informix Patch for Oracle Solaris 10
x 147497-02 Oracle Solaris Cluster 3.3: HA-DHCP Patch for Oracle Solaris 10
x 147499-02 Oracle Solaris Cluster 3.3: Ha-Samba Patch for Oracle Solaris 10 See SIIOTP
x 147501-02 Oracle Solaris Cluster 3.3: HA-Oracle Application Server Patch for Oracle Solaris 10
x 148636-03 Oracle Solaris Cluster 3.3: HA-PeopleSoft Patch for Oracle Solaris 10
x 149432-02 Oracle Solaris Cluster 3.3: JFreeChart patch for Oracle Solaris 10
x 150051-01 Oracle Solaris Cluster 3.3: HA for Oracle External Proxy patch for Oracle Solaris 10
x 150100-01 Oracle Solaris Cluster 3.3: JCommon patch for Oracle Solaris 10
x 150150-01 Oracle Solaris Cluster 3.3: HA for N1 Grid Engine patch for Oracle Solaris 10


Included/updated patch revisions of Sun Cluster 3.3 3/13 Update2 for Oracle Solaris 10 x86 8/1 Update10 or higher
x 145334-20 Oracle Solaris Cluster 3.3: Core Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145336-16 Oracle Solaris Cluster 3.3: HA-Oracle Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145338-04 Oracle Solaris Cluster 3.3: HA-SunONE Message Queue Patch for Oracle Solaris 10_x86
x 145342-05 Oracle Solaris Cluster 3.3: HA-Tomcat Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145344-06 Oracle Solaris Cluster 3.3: HA-BEA WebLogic Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145346-04 Oracle Solaris Cluster 3.3: HA-SunONE App Server Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145637-03 Oracle Solaris Cluster 3.3: HA-JavaWebServer Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145639-09 Oracle Solaris Cluster 3.3: SC Checks / sccheck Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145645-04 Oracle Solaris Cluster 3.3: HA-PostgreSQL Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 145647-05 Oracle Solaris Cluster 3.3: HA-MySQL Patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 146086-07 Oracle Solaris Cluster 3.3: SDS/SVM Mediator patch for Oracle Solaris 10_x86 Note: See SIIOTP
x 146088-02 Oracle Solaris Cluster 3.3: HA-Apache Patch for Oracle Solaris 10_x86
x 146092-07 Oracle Solaris Cluster 3.3: HA-SAP WEB AS for Oracle Solaris 10_x86 Note: See SIIOTP
x 146094-07 Oracle Solaris Cluster 3.3: HA-NFS for Oracle Solaris 10_x86
x 146766-05 Oracle Solaris Cluster 3.3: HA-Sybase patch for Oracle Solaris 10_x86
x 147074-04 Oracle Solaris Cluster 3.3: Man Pages Patch for Oracle Solaris 10_x86
x 147078-02 Oracle Solaris Cluster 3.3: HA-Containers Patch for Oracle Solaris 10_x86
x 147081-03 Oracle Solaris Cluster 3.3: HA-Kerberos Patch for Oracle Solaris 10_x86
x 147083-02 Oracle Solaris Cluster 3.3: HA-DB patch for Oracle Solaris 10_x86
x 147085-02 Oracle Solaris Cluster 3.3: HA-DNS patch for Oracle Solaris 10_x86
x 147088-03 Oracle Solaris Cluster 3.3: HA-LiveCache Patch for Oracle Solaris 10_x86
x 147090-03 Oracle Solaris Cluster 3.3: HA-SAP Patch for Oracle Solaris 10_x86
x 147092-04 Oracle Solaris Cluster 3.3: HA-SAPDB Patch for Oracle Solaris 10_x86
x 147096-02 Oracle Solaris Cluster 3.3: HA-SUNWscTimesTen Patch for Oracle Solaris 10_x86
x 147142-02 Oracle Solaris Cluster 3.3: Localization patch for Oracle Solaris 10_x86
x 147491-02 Oracle Solaris Cluster 3.3: HA-Websphere MQ Patch for Oracle Solaris 10_x86
x 147493-02 Oracle Solaris Cluster 3.3: HA Websphere Message Broker patch for Oracle Solaris 10_x86
x 147495-02 Oracle Solaris Cluster 3.3: HA-Informix Patch for Oracle Solaris 10_x86
x 147498-02 Oracle Solaris Cluster 3.3: HA-DHCP Patch for Oracle Solaris 10_x86
x 147500-02 Oracle Solaris Cluster 3.3: HA-Samba Patch for Oracle Solaris 10_x86 See SIIOTP
x 147502-02 Oracle Solaris Cluster 3.3: HA-Oracle Application Server Patch for Oracle Solaris 10_x86
x 149433-02 Oracle Solaris Cluster 3.3: JFreeChart patch for Oracle Solaris 10_x86
x 150052-01 Oracle Solaris Cluster 3.3: HA for Oracle External Proxy patch for Oracle Solaris 10_x86
x 150101-01 Oracle Solaris Cluster 3.3: JCommon patch for Oracle Solaris 10_x86
x 150151-01 Oracle Solaris Cluster 3.3: HA for N1 Grid Engine patch for Oracle Solaris 10_x86


The Solaris Cluster quorum server is an alternative to the traditional quorum disk. The quorum server is outside of the Oracle Solaris Cluster and will be accessed through the public network. Therefore the quorum server can be a different architecture.
Note: The Solaris Cluster quorum server software is only required on the quorum server and NOT on the Solaris Cluster nodes which are using the quorum server.

Included/updated patch revisions in Oracle Solaris Cluster 3.3 3/13 Update2 for quorum server feature:
x 146089-06 Oracle Solaris Cluster 3.3: Quorum Server patch for Oracle Solaris 10
x 150049-01 Oracle Solaris Cluster 3.3: Quorum Man Pages Patch for Oracle Solaris 10
x 146090-06 Oracle Solaris Cluster 3.3: Quorum Server patch for Oracle Solaris 10_x86
x 150050-01 Oracle Solaris Cluster 3.3: Quorum Man Pages Patch for Oracle Solaris 10_x86


Beware of the "Install Requirements" of the Oracle Solaris Cluster Framework patches.
Further details refer to Overview of Patching Oracle Solaris Cluster in Oracle Solaris Cluster System Administration Guide 3.3 3/13

Last but not least, the whole actual patch lists can be found at:
Oracle Solaris Cluster 3.3/SPARC patches for Solaris 10
Oracle Solaris Cluster 3.3/x86 patches for Solaris 10

Wednesday Jul 04, 2012

How to configure a zone cluster on Solaris Cluster 4.0

This is a short overview on how to configure a zone cluster on Solaris Cluster 4.0. This is a little bit different as in Solaris Cluster 3.2/3.3 because Solaris Cluster 4.0 is only running on Solaris 11. The name of the zone cluster must be unique throughout the global Solaris Cluster and must be configured on a global Solaris Cluster. Please read all the requirements for zone cluster in Solaris Cluster Software Installation Guide for SC4.0.
For Solaris Cluster 3.2/3.3 please refer to my previous blog Configuration steps to create a zone cluster in Solaris Cluster 3.2/3.3.

A. Configure the zone cluster into the already running global cluster
  • Check if zone cluster can be created
    # cluster show-netprops
    to change number of zone clusters use
    # cluster set-netprops -p num_zoneclusters=12
    Note: 12 zone clusters is the default, values can be customized!

  • Create config file (zc1config) for zone cluster setup e.g:

  • Configure zone cluster
    # clzc configure -f zc1config zc1
    Note: If not using the config file the configuration can also be done manually # clzc configure zc1

  • Check zone configuration
    # clzc export zc1

  • Verify zone cluster
    # clzc verify zc1
    Note: The following message is a notice and comes up on several clzc commands
    Waiting for zone verify commands to complete on all the nodes of the zone cluster "zc1"...

  • Install the zone cluster
    # clzc install zc1
    Note: Monitor the consoles of the global zone to see how the install proceed! (The output is different on the nodes) It's very important that all global cluster nodes have installed the same set of ha-cluster packages!

  • Boot the zone cluster
    # clzc boot zc1

  • Login into non-global-zones of zone cluster zc1 on all nodes and finish Solaris installation.
    # zlogin -C zc1

  • Check status of zone cluster
    # clzc status zc1

  • Login into non-global-zones of zone cluster zc1 and configure the shell environment for root (for PATH: /usr/cluster/bin, for MANPATH: /usr/cluster/man)
    # zlogin -C zc1

  • If using additional name service configure /etc/nsswitch.conf of zone cluster non-global zones.
    hosts: cluster files
    netmasks: cluster files

  • Configure /etc/inet/hosts of the zone cluster zones
    Enter all the logical hosts of non-global zones



B. Add resource groups and resources to zone cluster
  • Create a resource group in zone cluster
    # clrg create -n <zone-hostname-node1>,<zone-hostname-node2> app-rg

    Note1: Use command # cluster status for zone cluster resource group overview.
    Note2: You can also run all commands for zone cluster in global cluster by adding the option -Z to the command. e.g:
    # clrg create -Z zc1 -n <zone-hostname-node1>,<zone-hostname-node2> app-rg

  • Set up the logical host resource for zone cluster
    In the global zone do:
    # clzc configure zc1
    clzc:zc1> add net
    clzc:zc1:net> set address=<zone-logicalhost-ip>
    clzc:zc1:net> end
    clzc:zc1> commit
    clzc:zc1> exit
    Note: Check that logical host is in /etc/hosts file
    In zone cluster do:
    # clrslh create -g app-rg -h <zone-logicalhost> <zone-logicalhost>-rs

  • Set up storage resource for zone cluster
    Register HAStoragePlus
    # clrt register SUNW.HAStoragePlus

    Example1) ZFS storage pool
    In the global zone do:
    Configure zpool eg: # zpool create <zdata> mirror cXtXdX cXtXdX
    and
    # clzc configure zc1
    clzc:zc1> add dataset
    clzc:zc1:dataset> set name=zdata
    clzc:zc1:dataset> end
    clzc:zc1> verify
    clzc:zc1> commit
    clzc:zc1> exit
    Check setup with # clzc show -v zc1
    In the zone cluster do:
    # clrs create -g app-rg -t SUNW.HAStoragePlus -p zpools=zdata app-hasp-rs


    Example2) HA filesystem
    In the global zone do:
    Configure SVM diskset and SVM devices.
    and
    # clzc configure zc1
    clzc:zc1> add fs
    clzc:zc1:fs> set dir=/data
    clzc:zc1:fs> set special=/dev/md/datads/dsk/d0
    clzc:zc1:fs> set raw=/dev/md/datads/rdsk/d0
    clzc:zc1:fs> set type=ufs
    clzc:zc1:fs> add options [logging]
    clzc:zc1:fs> end
    clzc:zc1> verify
    clzc:zc1> commit
    clzc:zc1> exit
    Check setup with # clzc show -v zc1
    In the zone cluster do:
    # clrs create -g app-rg -t SUNW.HAStoragePlus -p FilesystemMountPoints=/data app-hasp-rs


    Example3) Global filesystem as loopback file system
    In the global zone configure global filesystem and it to /etc/vfstab on all global nodes e.g.:
    /dev/md/datads/dsk/d0 /dev/md/datads/dsk/d0 /global/fs ufs 2 yes global,logging
    and
    # clzc configure zc1
    clzc:zc1> add fs
    clzc:zc1:fs> set dir=/zone/fs (zc-lofs-mountpoint)
    clzc:zc1:fs> set special=/global/fs (globalcluster-mountpoint)
    clzc:zc1:fs> set type=lofs
    clzc:zc1:fs> end
    clzc:zc1> verify
    clzc:zc1> commit
    clzc:zc1> exit
    Check setup with # clzc show -v zc1
    In the zone cluster do: (Create scalable rg if not already done)
    # clrg create -p desired_primaries=2 -p maximum_primaries=2 app-scal-rg
    # clrs create -g app-scal-rg -t SUNW.HAStoragePlus -p FilesystemMountPoints=/zone/fs hasp-rs

    More details of adding storage available in the Installation Guide for zone cluster

  • Switch resource group and resources online in the zone cluster
    # clrg online -eM app-rg
    # clrg online -eM app-scal-rg

  • Test: Switch of the resource group in the zone cluster
    # clrg switch -n zonehost2 app-rg
    # clrg switch -n zonehost2 app-scal-rg

  • Add supported dataservice to zone cluster
    Documentation for SC4.0 is available here



  • Example output:



    Appendix: To delete a zone cluster do:
    # clrg delete -Z zc1 -F +

    Note: Zone cluster uninstall can only be done if all resource groups are removed in the zone cluster. The command 'clrg delete -F +' can be used in zone cluster to delete the resource groups recursively.
    # clzc halt zc1
    # clzc uninstall zc1

    Note: If clzc command is not successful to uninstall the zone, then run 'zoneadm -z zc1 uninstall -F' on the nodes where zc1 is configured
    # clzc delete zc1

Monday Dec 19, 2011

More robust control of zfs in Solaris Cluster 3.x

In some situations there is a possibility that a zpool will not be exported correctly if controlled by SUNW.HAStoragePlus resource. Please refer to the details in Document 1364018.1: Potential Data Integrity Issues After Switching Over a Solaris Cluster High Availability Resource Group With Zpools

I like to mention this because zfs is used more and more in Solaris Cluster environments. Therefore I highly recommend to install following patches to get a more reliable Solaris Cluster environment in combination with zpools on SC3.3 and SC3.2. So, if you already running such a setup, start planning NOW to install the following patch revision (or higher) for your environment...

Solaris Cluster 3.3:
145333-10 Oracle Solaris Cluster 3.3: Core Patch for Oracle Solaris 10
145334-10 Oracle Solaris Cluster 3.3_x86: Core Patch for Oracle Solaris 10_x86

Solaris Cluster 3.2
144221-07 Solaris Cluster 3.2: CORE patch for Solaris 10
144222-07 Solaris Cluster 3.2: CORE patch for Solaris 10_x86

Saturday Nov 19, 2011

The premier support for Sun Cluster 3.1 ended

In October 2011 the premier support for Sun Cluster 3.1 ended. See details in Oracle Lifetime Support Policy for Oracle and Sun System Software document. There no 'Extended Support' and the 'Sustaining Support Ends' is indefinite.

But for indefinite 'Sustaining Support' I like to point out from the mentioned document (version Sept. 2011) on page 5:

Sustaining Support does NOT include:
* New program updates, fixes, security alerts, general maintenance releases, selected functionality releases and documentation updates or upgrade tools
* Certification with most new third-party products/versions and most new Oracle products
* 24 hour commitment and response guidelines for Severity 1 service requests
*Previously released fixes or updates that Oracle no longer supports

This means Solaris 10 9/10 update9 is the last qualified release for Sun Cluster 3.1. So, Sun Cluster 3.1 is not qualified on Solaris 10 8/11 Update10. Furthermore there is an issue around with SVM patch 145899-06 or higher. This SVM patch is part of Solaris 10 8/11 Update10. The 145899-06 is the first released patch of this number, therefore the support for Sun Cluster 3.1 ends with the previous SVM patches 144622-01 and 139967-02. For details about the known problem with SVM patch 145899-06 please refer to doc 1378828.1.

Further this means you should freeze (no patching, no upgrade) your Sun Cluster 3.1 configuration not later than Solaris 10 9/10 update9. Or even better plan an upgrade to Solaris Cluster 3.3 now to get back to full support.

For general patch updates the 'Critical Patch Update July 2011 for Solaris 10 SPARC' is last working and supported for Sun Cluster 3.1.

Wednesday Aug 10, 2011

Oracle Solaris Cluster 3.3 05/11 Update1 patches

The Oracle Solaris Cluster 3.3 05/11 Update1 is released. Click here for further information.

The package version of the Oracle Solaris Cluster 3.3 05/11 Update1 are the same for the core framework and the agents as for Oracle Solaris Cluster 3.3. Therefore it's possible to patch up an existing Oracle Solaris Cluster 3.3.

The package version of the Oracle Solaris Cluster Geographic Edition 3.3 05/11 Update1 are NOT the same as Oracle Solaris Cluster Geographic Edition 3.3. But it's possible to upgrade the Geographic Edition 3.3 without interruption of the service. See documentation for details.

The following patches (with the mentioned revision) are included/updated in Oracle Solaris Cluster 3.3 05/11 Update1. If these patches are installed on Oracle Solaris Cluster 3.3 release, then the features for framework & agents are identical with Oracle Solaris Cluster 3.3 05/11 Update1. It's always necessary to read the "Special Install Instructions of the patch" but I made a note behind some patches where it's very important to read the "Special Install Instructions of the patch" (Using shortcut SIIOTP).

Included/updated patch revisions of Oracle Solaris Cluster 3.3 05/11 Update1 for Solaris 10 10/09 Update8 or higher
145333-08 Oracle Solaris Cluster 3.3 5/11: Core Patch for Oracle Solaris 10 Note: Please read SIIOTP
145335-07 Oracle Solaris Cluster 3.3 5/11: HA-Oracle Patch for Oracle Solaris 10 Note: Please read SIIOTP
145337-03 Oracle Solaris Cluster 3.3: HA-SunONE Message queue Patch for Oracle Solaris 10
145341-03 Oracle Solaris Cluster 3.3: HA-Tomcat Patch for Oracle Solaris 10 Note: Please read SIIOTP
145343-03 Oracle Solaris Cluster 3.3 5/11: HA-BEA WebLogic Patch for Oracle Solaris 10
145345-02 Oracle Solaris Cluster 3.3: HA-SunONE App Server Patch for Oracle Solaris 10 Note: Please read SIIOTP
145636-02 Oracle Solaris Cluster 3.3: HA-JavaWebServer Patch for Oracle Solaris 10 Note: Please read SIIOTP
145638-04 Oracle Solaris Cluster 3.3 5/11: SC Checks / sccheck Patch for Oracle Solaris 10 Note: Please read SIIOTP
145640-04 Oracle Solaris Cluster 3.3 5/11: SunPlex/Managability and Serviceability Patch Note: Please read SIIOTP
145644-02 Oracle Solaris Cluster 3.3: HA-PostgreSQL Patch for Oracle Solaris 10
145646-03 Oracle Solaris Cluster 3.3: HA-MySQL Patch for Oracle Solaris 10
146085-03 Oracle Solaris Cluster 3.3: SDS/SVM Mediator patch for Oracle Solaris 10
146087-02 Oracle Solaris Cluster 3.3: HA-Apache Patch for Oracle Solaris 10
146091-04 Oracle Solaris Cluster 3.3 5/11: HA-SAP WEB AS for Oracle Solaris 10 Note: Please read SIIOTP
146093-02 Oracle Solaris Cluster 3.3: HA-NFS for Oracle Solaris 10
146241-04 Oracle Solaris Cluster 3.3: SWIFTAlliance patch
146760-02 Oracle Solaris Cluster 3.3: HA-Oracle E-business suite
146762-03 Oracle Solaris Cluster 3.3: HA-xVM Patch
146764-04 Oracle Solaris Cluster 3.3: HA-Sybase Patch
147073-02 Oracle Solaris Cluster 3.3: Man Pages Patch
147077-01 Oracle Solaris Cluster 3.3: HA-Containers Patch for Oracle Solaris 10
147079-01 Oracle Solaris Cluster 3.3: HA-OracleBIE Patch
147080-01 Oracle Solaris Cluster 3.3: HA-Kerberos Patch
147082-01 Oracle Solaris Cluster 3.3: HA-DB patch
147084-01 Oracle Solaris Cluster 3.3: HA-DNS
147086-02 Oracle Solaris Cluster 3.3: HA-Siebel Patch
147087-01 Oracle Solaris Cluster 3.3: HA-LiveCache Patch
147089-01 Oracle Solaris Cluster 3.3: HA-SAP Patch
147091-03 Oracle Solaris Cluster 3.3 5/11: HA-SAPDB Patch
147095-01 Oracle Solaris Cluster 3.3: HA-SUNWscTimesTen Patch
147141-01 Oracle Solaris Cluster 3.3: Localization patch for Solaris 10 sparc
147358-01 Oracle Solaris Cluster 3.3 5/11: HA-Agfa Patch


Included/updated patch revisions of Solaris Cluster 3.3 05/11 Update1 for Solaris 10 x86 10/09 Update8 or higher
145334-08 Oracle Solaris Cluster 3.3_x86 5/11: Core Patch for Oracle Solaris 10_x86 Note: Please read SIIOTP
145336-07 Oracle Solaris Cluster 3.3_x86 5/11: HA-Oracle Patch for Oracle Solaris 10_x86 Note: Please read SIIOTP
145338-02 Oracle Solaris Cluster 3.3: HA-SunONE Message queue Patch for Oracle Solaris 10_x86
145342-03 Oracle Solaris Cluster 3.3: HA-Tomcat Patch for Oracle Solaris 10_x86 Note: Please read SIIOTP
145344-03 Oracle Solaris Cluster 3.3 5/11: HA-BEA WebLogic Patch for Oracle Solaris 10_x86
145346-02 Oracle Solaris Cluster 3.3: HA-SunONE App Server Patch for Solaris 10_x86 Note: Please read SIIOTP
145637-02 Oracle Solaris Cluster 3.3: HA-JavaWebServer Patch for Oracle Solaris 10_x86 Note: Please read SIIOTP
145639-04 Oracle Solaris Cluster 3.3 5/11: SC Checks / sccheck Patch for Solaris 10_x86 Note: Please read SIIOTP
145641-04 Oracle Solaris Cluster 3.3 5/11: SunPlex/Managability and Serviceability Patch Note: Please read SIIOTP
145645-02 Oracle Solaris Cluster 3.3: HA-PostgreSQL Patch for Oracle Solaris 10_x86
145647-03 Oracle Solaris Cluster 3.3: HA-MySQL Patch for Oracle Solaris 10_x86
146086-03 Oracle Solaris Cluster 3.3_x86: SDS/SVM Mediator patch for Oracle Solaris 10
146088-02 Oracle Solaris Cluster 3.3_x86: HA-Apache Patch for Oracle Solaris 10_x86
146092-04 Oracle Solaris Cluster 3.3_x86 5/11: HA-SAP WEB AS for Oracle Solaris 10_x86 Note: Please read SIIOTP
146094-02 Oracle Solaris Cluster 3.3_x86: HA-NFS for Oracle Solaris 10_x86 Note: Please read SIIOTP
146766-02 Oracle Solaris Cluster 3.3_x86: HA-Sybase patch
147074-02 Oracle Solaris Cluster 3.3_x86: Man Pages Patch
147078-01 Oracle Solaris Cluster 3.3_x86: HA-Containers Patch for Oracle Solaris 10_x86
147081-01 Oracle Solaris Cluster 3.3_x86: HA-Kerberos Patch
147083-01 Oracle Solaris Cluster 3.3_x86: HA-DB patch
147085-01 Oracle Solaris Cluster 3.3_x86: HA-DNS
147088-01 Oracle Solaris Cluster 3.3_x86: HA-LiveCache Patch
147090-01 Oracle Solaris Cluster 3.3_x86: HA-SAP Patch
147092-03 Oracle Solaris Cluster 3.3_x86 5/11: HA-SAPDB Patch
147142-01 Oracle Solaris Cluster 3.3: Localization patch for Solaris 10_x86


The quorum server is an alternative to the traditional quorum disk. The quorum server is outside of the Oracle Solaris Cluster and will be accessed through the public network. Therefore the quorum server can be a different architecture.
Note: The quorum server software is only required on the quorum server and NOT on the Solaris Cluster nodes which are using the quorum server.

Included/updated patch revisions in Oracle Solaris Cluster 3.3 05/11 Update1 for quorum server feature:
146089-03 Oracle Solaris Cluster 3.3: Quorum Server patch
146090-03 Oracle Solaris Cluster 3.3_x86: Quorum Server patch for Oracle Solaris 10


Beware of the "Install Requirements" of the Oracle Solaris Cluster Framework patches.
Further details refer to Overview of Patching Oracle Solaris Cluster in Oracle Solaris Cluster System Administration Guide

Last but not least, the whole actual lists can be found at:
Oracle Solaris Cluster 3.3/SPARC patches for Solaris 10
Oracle Solaris Cluster 3.3/x86 patches for Solaris 10

Tuesday Mar 15, 2011

Setup of local zpool on local or shared device with Solaris Cluster

Maybe there is a need (for whatever reason) to configure a local zpool in a Solaris Cluster environment. As local zpool I mean that this zpool should only be available on one Solaris Cluster node WITHOUT using SUNW.HAStoragePlus. Such a local zpool can be configured with local devices (only connected to one node) or shared devices (accessible from all nodes in the cluster via SAN). However in case of shared device it would be better to setup a zone in the SAN switch to make the device only available to one host.

The following procedure is necessary to use local devices in local zpool:

In this example I use the local device c1t3d0 to create a local zpool
a) Look for the did device of the device which should be used by the zpool
# scdidadm -l c1t3d0
49 node0:/dev/rdsk/c1t3d0 /dev/did/rdsk/d49
b) Check the settings of the used did device
# cldg show dsk/d49
Note: Only one node should be in the node list
c) Set localonly flag for the did device. Optional: set autogen flag
# cldg set -p localonly=true -p autogen=true dsk/d49
or disable fencing for the did device
# cldev set -p default_fencing=nofencing d49
d) Verify the settings
# cldg show dsk/d49
e) Create the zpool
# zpool create localpool c1t3d0
# zfs create localpool/data


The following procedure is necessary to use shared devices in local zpool:

In this example I use the shared device c6t600C0FF00000000007BA1F1023AE1711d0 to create a local zpool
a) Look for the did device of the device which should be used by the zpool
# scdidadm -L c6t600C0FF00000000007BA1F1023AE1711d0
11 node0:/dev/rdsk/c6t600C0FF00000000007BA1F1023AE1710d0 /dev/did/rdsk/d11
11 node1:/dev/rdsk/c6t600C0FF00000000007BA1F1023AE1710d0 /dev/did/rdsk/d11
b) Check the settings of the used did device
# cldg show dsk/d11
c) Remove the node which should not access the did device
# cldg remove-node -n node1 dsk/d11
d) Set localonly flag for the did device. Optional: set autogen flag
# cldg set -p localonly=true -p autogen=true dsk/d11
or disable fencing for the did device
# cldev set -p default_fencing=nofencing d11
e) Verify the settings
# cldg show dsk/d11
f) Create the zpool
# zpool create localpool c6t600C0FF00000000007BA1F1023AE1711d0
# zfs create localpool/data


If you forgot to do this for a local zpool then there is a possibility that the zpool will be FAULTED state after a boot.

Tuesday Aug 17, 2010

New numbers of Solaris Cluster 3.2 core patches

There was a rejuvenation of the Solaris Cluster 3.2 core patch. The new patches are

144220 Solaris Cluster 3.2: CORE patch for Solaris 9
144221 Solaris Cluster 3.2: CORE patch for Solaris 10
144222 Solaris Cluster 3.2: CORE patch for Solaris 10_x86
At this time these patches does NOT have the requirement to be installed in non-cluster-single-user-mode. They can be installed in order when cluster is running, but requires a reboot.

Beware the new patches requires the previous version -42 of the SC 3.2 core patch.
126105-42 Sun Cluster 3.2: CORE patch for Solaris 9
126106-42 Sun Cluster 3.2: CORE patch for Solaris 10
126107-42 Sun Cluster 3.2: CORE patch for Solaris 10_x86
And the -42 still have the requirement to be installed in non-cluster-single-user-mode. Furthermore carefully study the special install instructions and some entries of this blog.

The advantage is, when -42 is already applied then the patching of Solaris Cluster 3.2 becomes more easy.

Certainly, it's possible to apply the new SC core patch at the same time as the -42 core patch in non-cluster-single-user-mode.

Monday Jul 05, 2010

Example to update the boot archive via failsafe boot on sparc systems with configured SVM root mirror

Since Solaris 10 10/08 update6 the boot archive feature was implemented into Solaris 10 for sparc. When the system is configured with Solaris Volume Manager (SVM) root mirror then attention is necessary if using the failsafe boot (boot -F failsafe) on the 'ok prompt' environment.

In case of the following or similar message it could be necessary to use the Solaris failsafe boot:

Rebooting with command: boot
Boot device: disk:a File and args:
SunOS Release 5.10 Version Generic_141414-07 64-bit
Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hardware watchdog enabled
Hostname: scnode1

WARNING: The following files in / differ from the boot archive:

changed /kernel/drv/did.conf

The recommended action is to reboot to the failsafe archive to correct
the above inconsistency. To accomplish this, on a GRUB-based platform,
reboot and select the "Solaris failsafe" option from the boot menu.
On an OBP-based platform, reboot then type "boot -F failsafe". Then
follow the prompts to update the boot archive. Alternately, to continue
booting at your own risk, you may clear the service by running:
"svcadm clear system/boot-archive"

Jun 29 15:51:28 svc.startd[8]: svc:/system/boot-archive:default: Method "/lib/svc/method/boot-archive" failed with exit status 95.
Jun 29 15:51:28 svc.startd[8]: system/boot-archive:default failed fatally: transitioned to maintenance (see 'svcs -xv' for details)
Requesting System Maintenance Mode
(See /lib/svc/share/README for more information.)
Console login service(s) cannot run

Root password for system maintenance (control-d to bypass):

Normally after this message the issue can be solved by
a) login into system
b) run "svcadm clear system/boot-archive"
c) wait until system continuing the boot up
The system is already booted from the SVM root mirror and the svcadm clear command goes to all used devices within the root mirror.


But when this is not successful then it's necessary to do a Solaris failsafe boot. Now the issue is that the failsafe boot is not mounting the SVM root mirror and manual intervention is necessary to update the boot archive on all devices within the SVM root mirror. This procedure does NOT require to unmirror the SVM root mirror to update the boot archive.

The procedure in short:
1) Start failsafe boot:
ok boot -F failsafe
when system ask for boot drive selection answer with 'q'
...
Please select a device to be mounted (q for none) [?,??,q]: q
and a shell is starting

2) Mount the single root drive read-only! (use format if not know the root drive)
# mount -o ro /dev/dsk/c0t0d0s0 /a
3) Copy the md.conf file from the root drive to the active boot environment
# cp /a/kernel/drv/md.conf /kernel/drv/
4) Umount the single root drive
# umount /a
5) Update the md driver information
# update_drv -f md
devfsadm: mkdir failed for /dev 0x1ed: Read-only file system
#
6) Important: Wait a minute, it could take some time to configure the md devices!
7) Sync meta information (if not know the SVM root mirror then run metastat)
# metasync d0
8) Mount the SVM mirror
# mount /dev/md/dsk/d0 /a
9) Update the boot archive on the SVM mirror
# bootadm update-archive -v -R /a
some output will be shown
10) Umount SVM root mirror
# umount /a
11) reboot the system from repaired SVM mirror
# init 6





Monday May 17, 2010

Oracle Solaris Cluster Community


- Do you have a question around Oracle Solaris Cluster ?
- Do you have a login into My Oracle Support or SunSolve ?
- Do you like to collaborate and sharing knowledge with your peers ?


Then login into the moderated Oracle Solaris Cluster Community and feel free to ask and answer questions. In the Community it's possible to start discussions and knowledge documents can be created. For easy monitoring subscribe to discussions and documents. If somebody answers your question you can identify the thread with 'Correct Answer' or 'Helpful Answer' this helps to find solutions quickly. Additional in the 'Private Messages' tab you can talk to other community members…

At this time there are more than 140 online Oracle Support Communities where you can participate. So, please give it try and login into the 'My Oracle Support Community' , setup your profile and refer to 'Getting Started' region in the upper right corner of the main page.

See you there...

Friday Mar 26, 2010

Summary of install instructions for 126106-40 and 126107-40

My last blog describe some issues around these patches (please read)
126106-40 Sun Cluster 3.2: CORE patch for Solaris 10
126107-40 Sun Cluster 3.2: CORE patch for Solaris 10_x86
This is a follow up with a summary of best practices 'How to install?' these patches. There is a difference between new installations, 'normal' patching and live upgrade patching.
Important: The mentioned instructions are working if already Solaris Cluster 3.2 1/09 update2 (or Solaris Cluster 3.2 core patch revision -27(sparc) / -28(x86) ) or higher is installed. If running lower version of the Solaris Cluster 3.2 core patch then additional needs are necessary. Please refer to special install instructions of the patches for the additional needs.
Update: 28.Apr 2010
This also apply to the already released -41 and -42 SC core patches, when -40 is not already active

A) In case of new installations:

Install the SC core patch -40 immediately after the installation of the Solaris Cluster 3.2 software.
In brief:
    1.) Install Solaris Cluster 3.2 via JES installer
    2.) Install the SC core patch -40
    3.) Run scinstall
    4.) Do the reboot
Note: Do NOT do a reboot between 1.) and 2.). Follow the EIS Solaris Cluster 3.2 checklist which also has a note for this issue. If not available follow the standard installation process of Sun Cluster 3.2


B) In case of 'normal' patching

It is vital to use the following/right approach in case of patching. Because if you not use the following approach then the Solaris Cluster 3.2 can not boot anymore:
    0.) Only if using AVS 4.0
    # patchadd 12324[67]-05 (Follow Special Install Instructions)
    1.) # boot in non-cluster mode
    2.) # svcadm disable svc:/system/cluster/loaddid
    3.) # svccfg delete svc:/system/cluster/loaddid
    4.) # patchadd 12610[67]-40
    5.) # init 6


C) In case of LU (Live Upgrade feature) to install patches
    1.) Create ABE:
    For zfs root within the same root pool:
    # lucreate -n patchroot
    For ufs on different root drive:
    # prtvtoc /dev/rdsk/c1t3d0s2 | fmthard -s - /dev/rdsk/c1t2d0s2
    # lucreate -c "c1t3d0s0-root" -m /:/dev/dsk/c1t2d0s0:ufs -m /global/.devices/node@2:/dev/dsk/c1t2d0s6:ufs -n "c1t2d0s0-patchroot"
    2.) Install patch into ABE ( patch is already unpacked in /var/tmp )
    # luupgrade -t -n c1t2d0s0-patchroot -s /var/tmp 126106-40
    3.) Activate ABE
    # luactivate patchroot
    4.) # init 6
    # Some errors comes up at this point
    (dependency cycle & ORB error - Please look to example below)
    5.) # init 6 (the second reboot fix the problem) Bug 6938144





My personal recommendation to minimize the risk of the installation for the SC core patch -40 is:
Step 1) Upgrade the Solaris Cluster to
a) Solaris 10 10/09 update 8 and Solaris Cluster 3.2 11/09 update3.
or
b) EIS Baseline 26JAN10 which include the Solaris kernel update 14144[45]-09 and SC core patch -39. If EIS baseline not available use other patchset which include the mentioned patches.
Step 2) After the successful upgrade do a single patch install of the SC core patch -40 by using the installation instruction B) which is mentioned above. In this software state the -40 can be applied 'rolling' to the cluster.

Note: 'Rolling' means: Boot node1 in non-cluster-mode -> install -40 (see B) -> boot node1 back into cluster -> boot node2 in non-cluster-mode -> install -40 (see B) -> boot node2 back into cluster.

Wednesday Mar 03, 2010

Oracle Solaris Cluster core patch 126106-40 and 126107-40

This is a notify because there are some troubles around with the following Sun Cluster 3.2 -40 core patches:
126106-40 Sun Cluster 3.2: CORE patch for Solaris 10
126107-40 Sun Cluster 3.2: CORE patch for Solaris 10_x86
Before installing the patch carefully read the Special Install Instructions.
Update: 28.Apr 2010
This also apply to the already released -41 and -42 SC core patches, when -40 is not already active

Two new notes where added to these patches:


NOTE 16: Remove the loaddid SMF service by running the following
commands before installing this patch, if current patch level
(before installing this patch) is less than -40:
svcadm disable svc:/system/cluster/loaddid
svccfg delete svc:/system/cluster/loaddid


      So, the right approach is:
      # boot in non-cluster mode
      # svcadm disable svc:/system/cluster/loaddid
      # svccfg delete svc:/system/cluster/loaddid
      # patchadd 126106-40
      # init 6


NOTE 17:
Installing this patch on a machine with Availability Suite
software installed will cause the machine to fail to boot with
dependency errors due to BugId 6896134 (AVS does not wait for
did devices to startup in a cluster).
Please contact your Sun
Service Representative for relief before installing this patch.


The solution for Bug 6896134 is now available, please follow the right approach below for installation...
123246-05 Sun StorEdge Availability Suite 4.0: Patch for Solaris 10
123247-05 Sun StorEdge Availability Suite 4.0: Patch for Solaris 10_x86
       # patchadd 12324[67]-05 (Follow Special Install Instructions)
       # boot in non-cluster mode
       # svcadm disable svc:/system/cluster/loaddid
       # svccfg delete svc:/system/cluster/loaddid
       # patchadd 12610[67]-40
       # init 6



Important to know: These 2 issues only come up if using Solaris 10 10/09 Update8 or the Kernel patch 141444-09 or higher. There are changes in the startup of the iSCSI initiator (is now a SMF service) - please refer to 6888193 for details.

Hint: If using LU (live upgrade) for patching please refer to my blog Summary of installation instructions for 126106-40 and 126107-40

ATTENTION: the NOTE 16 is valid for removal of the patch - carefully read 'Special Removal Instructions'.


Additional information around NOTE 16
1) What happen if forgot to delete the loaddid service before the patch installation?

The following error (or similar) come up within patch installation on the console of the server:
Mar 2 12:01:46 svc.startd[7]: Transitioning svc:/system/cluster/loaddid:default to maintenance because it completes a dependency cycle (see svcs -xv for details):
svc:/network/iscsi/initiator:default
svc:/network/service
svc:/network/service:default
svc:/network/rpc/nisplus
svc:/network/rpc/nisplus:default
svc:/network/rpc/keyserv
svc:/network/rpc/keyserv:default
svc:/network/rpc/bind
svc:/network/rpc/bind:default
svc:/system/sysidtool:net
svc:/milestone/single-user:default
svc:/system/cluster/loaddid
svc:/system/cluster/loaddid:default
Mar 2 12:01:46 svc.startd[7]: system/cluster/loaddid:default transitioned to maintenance by request (see 'svcs -xv' for details)

But this should NOT be problem because the patch 126106-40 will be installed in non-cluster-mode. This means after the next boot into cluster mode the error should disappear. This is reported in Bug 6911030.

But to be sure that the system is booting correctly do:
- check the log file /var/svc/log/system-cluster-loaddid:default.log
- that one of the last lines is: [ Mar 16 10:31:15 Rereading configuration. ]
- if not go to 'Recovery procedure below'


2) What happen if doing the loaddid delete after the patch installation?

Maybe you the see the problem mentioned in 1). If disable and delete the 'svc:/system/cluster/loaddid' service after the patch installation then the system will no longer joining the cluster. The following error come up:
...
Mar 16 11:42:54 Cluster.Framework: Could not initialize the ORB. Exiting.
Mar 16 11:42:54 : problem waiting the deamon, read errno 0
Mar 16 11:42:54 Cluster.Framework: Could not initialize the ORB. Exiting.
Mar 16 11:42:54 svc.startd[8]: svc:/system/cluster/sc_failfast:default: Method "/usr/cluster/lib/svc/method/sc_failfast start" failed with exit status 1.
Mar 16 11:42:54 svc.startd[8]: svc:/system/cluster/cl_execd:default: Method "/usr/cluster/lib/svc/method/sc_cl_execd start" failed with exit status 1.
Mar 16 11:42:54 Cluster.Framework: Could not initialize the ORB. Exiting.
Mar 16 11:42:54 : problem waiting the deamon, read errno 0
Configuring devices.
Mar 16 11:42:54 svc.startd[8]: svc:/system/cluster/sc_failfast:default: Method "/usr/cluster/lib/svc/method/sc_failfast start" failed with exit status 1.
Mar 16 11:42:54 Cluster.Framework: Could not initialize the ORB. Exiting.
Mar 16 11:42:54 svc.startd[8]: svc:/system/cluster/cl_execd:default: Method "/usr/cluster/lib/svc/method/sc_cl_execd start" failed with exit status 1.
Mar 16 11:42:54 Cluster.Framework: Could not initialize the ORB. Exiting.
Mar 16 11:42:54 : problem waiting the deamon, read errno 0
Mar 16 11:42:54 svc.startd[8]: svc:/system/cluster/sc_failfast:default: Method "/usr/cluster/lib/svc/method/sc_failfast start" failed with exit status 1.
Mar 16 11:42:54 svc.startd[8]: system/cluster/sc_failfast:default failed: transitioned to maintenance (see 'svcs -xv' for details)
Mar 16 11:42:54 Cluster.Framework: Could not initialize the ORB. Exiting.
Mar 16 11:42:55 svc.startd[8]: svc:/system/cluster/cl_execd:default: Method "/usr/cluster/lib/svc/method/sc_cl_execd start" failed with exit status 1.
Mar 16 11:42:55 svc.startd[8]: system/cluster/cl_execd:default failed: transitioned to maintenance (see 'svcs -xv' for details)
…
If seeing this error then refer to Recovering procedure


Recovering procedure

1) boot in non-cluster-mode if not able to login
2) bring the files loaddid and loaddid.xml in place (normally using the files from SC core patch -40)
ONLY in case of trouble with the files from SC core patch -40 use the old files!
Note: If restore old file without the dependency to iSCSI initiator then there can be problems if trying to use iSCSI storage within Sun Cluster.
3) Repair loaddid service
# svcadm disable svc:/system/cluster/loaddid
# svccfg delete svc:/system/cluster/loaddid
# svccfg import /var/svc/manifest/system/cluster/loaddid.xml
# svcadm restart svc:/system/cluster/loaddid:default
4) check the log file /var/svc/log/system-cluster-loaddid:default.log
# tail /var/svc/log/system-cluster-loaddid:default.log
for the following line (which should be on the end of the log file)
[ Mar 16 11:43:06 Rereading configuration. ]
Note: Rereading configuration is necessary before booting!
5) reboot the system
# init 6


Additional information: Difference details of loaddid files after installation of SC core patch -40.

A) /var/svc/manifest/system/cluster/loaddid.xml
The SC core patch -40 delivers a new version with the following changes:
<       ident "@(#)loaddid.xml 1.3 06/05/12 SMI"
---
>       ident "@(#)loaddid.xml 1.5 09/11/04 SMI"
56,61c79,92
<       <dependent
<              name='loaddid_single-user'
<              grouping='optional_all'
<             restart_on='none'>
<             <service_fmri value='svc:/milestone/single-user:default' />
<       </dependent>
---
>       <!--
>              The following dependency is for did drivers to get loaded
>              properly for iSCSI based quorum and data devices. We want to
>              start loaddid service after the time when iSCSI connections
>              can be made.
>        -->
>       <dependency
>              name='cl_iscsi_initiator'
>              grouping='optional_all'
>              restart_on='none'
>              type='service'>
>              <service_fmri
>              value='svc:/network/iscsi/initiator:default' />
>       </dependency>


Before patch -40 is applied:
node1 # svcs -d loaddid:default
STATE STIME FMRI
online 11:29:45 svc:/system/cluster/cl_boot_check:default
online 11:29:49 svc:/system/coreadm:default
online 11:30:52 svc:/milestone/devices:default

node1 # svcs -D svc:/system/cluster/loaddid:default
STATE STIME FMRI
online 15:34:41 svc:/system/cluster/bootcluster:default
online 15:34:46 svc:/milestone/single-user:default


After patch -40 is applied:
node1 # svcs -d loaddid:default
STATE STIME FMRI
online 12:09:18 svc:/system/coreadm:default
online 12:09:20 svc:/system/cluster/cl_boot_check:default
online 12:09:21 svc:/network/iscsi/initiator:default
online 12:10:21 svc:/milestone/devices:default

node1 # svcs -D svc:/system/cluster/loaddid:default
STATE STIME FMRI
online 16:08:19 svc:/system/cluster/bootcluster:default


B) /usr/cluster/lib/svc/method/loaddid
The SC core patch -40 delivers a new version with the following changes:
< #ident "@(#)loaddid 1.7 06/08/07 SMI"
---
> #ident "@(#)loaddid 1.9 09/11/04 SMI"
15,16c36,44
<        svcprop -q -p system/reconfigure system/svc/restarter:default 2>/dev/null
<        if [ $? -eq 0 ] && [ `svcprop -p system/reconfigure system/svc/restarter:default` = "true" ]
---
>        # The property "reconfigure" is used to store whether the boot is
>        # a reconfiguration boot or not. The property "system/reconfigure"
>        # of the "system/svc/restarter" Solaris SMF service can be used
>        # for this purpose as well. However the system/reconfigure
>        # property is reset at the single-user milestone. SC requires this
>        # property for use by service after the single-user milestone as
>        # well.
>        svcprop -q -p clusterdata/reconfigure system/cluster/cl_boot_check 2>/dev/null
>        if [ $? -eq 0 ] && [ `svcprop -p clusterdata/reconfigure system/cluster/cl_boot_check` = "true" ]


Thursday Jan 21, 2010

Configuration of 3rd mediator host in campus cluster

One of the new features in Sun Cluster 3.2 11/09 update3 is the Solaris Volume Manager Three-Mediator support. This is very helpful for two-room campus cluster configurations. It's recommended to install the 3rd mediator host in a third-room. Please refer to the details about the Guidelines for Mediators.
In the Solaris Cluster 3.3 docs you can find Configuring Dual-String Mediator documentation.

Advantages:
-- The 3rd mediator host is used to get the majority of Solaris Volume Manager configuration when one room is lost due to an error.
-- The 3rd mediator host only need public network connection. (Must not be part of the cluster and need no connection to the shared storage).
Consider:
-- One more room necessary
-- One more host for administration, but if using a Sun Cluster quorum server then the 3rd mediator can be on the same host.


This example shows how to add a 3rd mediator host to an existing Solaris Volume Manager diskset

Configuration steps on the 3rd mediator host
  • A) Add root to sysadmin in /etc/group
    # vi /etc/group
    sysadmin::14:root

  • B) Create metadb and dummy diskset
    # metadb -afc 3 c0t0d0s7 c1t0d0s7
    # metaset -s <dummyds> -a -h <3rd_mediator_host>

  • Note: Maybe a good name for the dummyds can be a combination of the used setname on the campuscluster name e.g.:'setnameofcampuscluster_campusclustername'. If using more than one set it could be e.g: 'sets_of_campusclustername'. Or if using it for more than one cluster it's possible to create one set with a specific name for each cluster. This could be helpful for monitoring/configuration purposes. But keep in mind this is not required, one set is enough for all clusters which use this 3rd mediator host.

    Configuraiton steps on cluster nodes
  • A) Add 3rd mediator host to /etc/hosts
    # echo <ipaddress hostname> >> /etc/hosts

  • B) Add 3rd mediator host to existing diskset on one cluster node.
    # metaset -s <setname> -a -m <3rd_mediator_host>

  • ATTENTION: If using 3rd mediator host for more than one cluster. Each cluster node and diskset must have a unique name throughout all clusters, and a diskset cannot be named shared or admin.

    Example output:


    Hint1: On the cluster nodes and on the third mediator the configuration is stored in the file /etc/lvm/meddb

    Hint2: Use a script to monitor the mediator status on the cluster nodes. Similar to:

    Download the mediatorcheck.ksh script here!

    Wednesday Jan 20, 2010

    Solaris Volume Manager command overview

    Another quick reference guide. This time for Solaris Volume Manager. Please do not expect the whole command line in this one page. It should be a reminder for the most used commands for Solaris Volume Manager. I added a picture to this blog but also the pdf file is available for download.



    Saturday Dec 19, 2009

    Quorum server patches and Sun Cluster 3.2

    This is an early notify because there are some troubles around with the following Sun Cluster 3.2 quorum server patches:
    127404-03 Sun Cluster 3.2: Quorum Server Patch for Solaris 9
    127405-04 Sun Cluster 3.2: Quorum Server patch for Solaris 10
    127406-04 Sun Cluster 3.2: Quorum Server patch for Solaris 10_x86
    All these patches are part of Sun Cluster 3.2 11/09 Update3 release, but also available on My Oracle Support.

    These patches delivers new features which requires attention in case of upgrade or patching. The installation of the mentioned patches on a Sun Cluster 3.2 quorum server can lead to a panic of all Sun Cluster 3.2 nodes which use this quorum server. The panic of Sun Cluster 3.2 nodes are as follows:
    ...
    Dec 4 16:43:57 node1 \^Mpanic[cpu18]/thread=300041f0700:
    Dec 4 16:43:57 node1 unix: [ID 265925 kern.notice] CMM: Cluster lost operational quorum; aborting.
    ...

    Update 11.Jan.2010:
    More details available Alert 1021769.1: Sun Cluster 3.2 Quorum Server Patches Cause All Cluster Nodes to Panic with "Cluster lost operational quorum"

    General workaround before patching:
    If the Sun Cluster only use the quorum server as a quorum device, temporarily add a second quorum device.
    For example:
    1) Configure a second temporary quorum device on each cluster node which uses the quorum server (use a disk if available).
    # clq add d13 (or use clsetup)
    2) Un-configure the quorum server from each cluster node that uses the quorum server.
    # clq remove QuorumServer1 (or use clsetup)
    3) Verify that the quorum server no longer serves any cluster.
    on the Sun Cluster nodes # clq status
    on the Sun Cluster quorum server # clqs show +
    4) Install the Sun Cluster 3.2 Quorum Server patch
    # patchadd 12740x-0x
    5) Reboot and start the quorum server if not already started
    # init 6
    6) From a cluster node, configure the patched quorum server again as a quorum device.
    # clq add -t quorum_server -p qshost=129.152.200.5 -p port=9000 QuorumServer1
    7) Un-configure the temporary quorum device
    # clq remove d13


    Workaround if quorum server is already patched:
    If patch is already installed (and rebooted) on the Sun Cluster 3.2 quorum server but quorum server is not working correctly then the following messages on the Sun Cluster nodes are visible:
    ...
    Dec 3 17:24:24 node1 cl_runtime: [ID 868277 kern.warning] WARNING: CMM: Erstwhile online quorum device QuorumServer1 (qid 1) is inaccessible now.
    Dec 3 17:29:20 node1 cl_runtime: [ID 237999 kern.notice] NOTICE: CMM: Erstwhile inaccessible quorum device QuorumServer1 (qid 1) is online now.
    Dec 3 17:29:24 node1 cl_runtime: [ID 868277 kern.warning] WARNING: CMM: Erstwhile online quorum device QuorumServer1 (qid 1) is inaccessible now.
    Dec 3 17:32:58 node1 cl_runtime: [ID 237999 kern.notice] NOTICE: CMM: Erstwhile inaccessible quorum device QuorumServer1 (qid 1) is online now.
    ...
    DO NOT TRY to remove the quorum server on the Sun Cluster 3.2 nodes because this can end up in a panic loop.

    Do:
    1) Clear the configuration on the quorum server
    # clqs clear <clustername> <quorumname>
    or # clqs clear +

    2.) Re-add the quorum server to the Sun Cluster nodes
    # cluster set -p installmode=enabled
    # clq remove QuorumServer1
    # clq add -t quorum_server -p qshost=129.152.200.5 -p port=9000 QuorumServer1
    # cluster set -p installmode=disabled


    This is reported in Bug 6907934

    As stated in my last blog the following note of the Special Install Instructions of Sun Cluster 3.2 core patch -38 and higher is very important.
    NOTE 17: Quorum server patch 127406-04 (or greater) needs to be installed on quorum server host first, before installing 126107-37 (or greater) Core Patch on cluster nodes.
    This means if using a Sun Cluster 3.2 quorum server then it's necessary to upgrade the quorum server before upgrade the Sun Cluster 3.2 nodes to Sun Cluster 3.2 11/09 update3 which use the quorum server. AND furthermore the same apply in case of patching. If installing the Sun Cluster core patch -38 or higher (-38 is part of Sun Cluster 3.2 11/09 update3)
    126106-38 Sun Cluster 3.2: CORE patch for Solaris 10
    126107-38 Sun Cluster 3.2: CORE patch for Solaris 10_x86
    126105-38 Sun Cluster 3.2: CORE patch for Solaris 9
    then the same rule apply. First update the quorum server and then the Sun Cluster nodes. Please refer to details above how to do it...
    For upgrade also refer to the document: How to upgrade quorum server software

    Keep in mind: Fresh installations with Sun Cluster 3.2 11/09 update3 on the Sun Cluster nodes and on the quorum server are NOT affected!

    Sunday Dec 06, 2009

    Sun Cluster 3.2 11/09 Update3 Patches

    The Sun Cluster 3.2 11/09 Update3 is released. Click here for further information.

    The package version of the Sun Cluster 3.2 11/09 Update3 are the same for the core framework and the agents as for Sun Cluster 3.2, Sun Cluster 3.2 2/08 Update1 and Sun Cluster 3.2 1/09 Update2. Therefore it's possible to patch up an existing Sun Cluster 3.2, Sun Cluster 3.2 2/08 Update1 or Sun Cluster 3.2 1/09 Update2.

    The package version of the Sun Cluster Geographic Edition 3.2 11/09 Update3 are NOT the same as Sun Cluster Geographic Edition 3.2. But it's possible to upgrade the Geographic Edition 3.2 without interruption of the service. See documentation for details.

    The following patches (with the mentioned revision) are included/updated in Sun Cluster 3.2 11/09 Update3. If these patches are installed on Sun Cluster 3.2, Sun Cluster 3.2 2/08 Update1 or Sun Cluster 3.2 1/09 Update2 release, then the features for framework & agents are identical with Sun Cluster 3.2 11/09 Update3. It's always necessary to read the "Special Install Instructions of the patch" but I made a note behind some patches where it's very important to read the "Special Install Instructions of the patch" (Using shortcut SIIOTP).

    Included/updated patch revisions of Sun Cluster 3.2 11/09 Update3 for Solaris 10 05/09 Update7 or higher
    126106-38 Sun Cluster 3.2: CORE patch for Solaris 10 Note: Please read SIIOTP
    125992-05 Sun Cluster 3.2: SC Checks patch for Solaris 10
    126017-03 Sun Cluster 3.2: HA-DNS Patch for Solaris 10
    126032-09 Sun Cluster 3.2: Ha-MYSQL Patch for Solaris 10 Note: Please read SIIOTP
    126035-06 Sun Cluster 3.2: HA-NFS Patch for Solaris 10
    126044-06 Sun Cluster 3.2: HA-PostgreSQL Patch for Solaris 10 Note: Please read SIIOTP
    126047-12 Sun Cluster 3.2: Ha-Oracle patch for Solaris 10 Note: Please read SIIOTP
    126050-04 Sun Cluster 3.2: HA-Oracle E-business suite Patch for Solaris 10 (-04 not yet on SunSolve)
    126059-05 Sun Cluster 3.2: HA-SAPDB Patch for Solaris 10
    126071-02 Sun Cluster 3.2: HA-Tomcat Patch for Solaris 10
    126080-04 Sun Cluster 3.2: HA-Sun Java Systems App Server Patch for Solaris 10
    126083-04 Sun Cluster 3.2: HA-Sun Java Message Queue Patch for Solaris 10 Note: Please read SIIOTP
    126095-06 Sun Cluster 3.2: Localization patch for Solaris 9 sparc and Solaris 10 sparc
    128556-04 Sun Cluster 3.2: Man Pages Patch for Solaris 9 and Solaris 10, sparc
    137931-02 Sun Cluster 3.2: Sun Cluster 3.2: HA-Informix patch for Solaris 10


    Included/updated patch revisions of Sun Cluster 3.2 11/09 Update3 for Solaris 10 x86 05/09 Update7 or higher
    126107-38 Sun Cluster 3.2: CORE patch for Solaris 10_x86 Note: Please read SIIOTP
    125993-05 Sun Cluster 3.2: Sun Cluster 3.2: SC Checks patch for Solaris 10_x86
    126018-05 Sun Cluster 3.2: HA-DNS Patch for Solaris 10_x86
    126033-10 Sun Cluster 3.2: Ha-MYSQL Patch for Solaris 10_x86 Note: Please read SIIOTP
    126036-07 Sun Cluster 3.2: HA-NFS Patch for Solaris 10_x86
    126045-07 Sun Cluster 3.2: HA-PostgreSQL Patch for Solaris 10_x86 Note: Please read SIIOTP
    126048-12 Sun Cluster 3.2: Ha-Oracle patch for Solaris 10_x86 Note: Please read SIIOTP
    126060-06 Sun Cluster 3.2: HA-SAPDB Patch for Solaris 10_x86
    126072-02 Sun Cluster 3.2: HA-Tomcat Patch for Solaris 10_x86
    126081-05 Sun Cluster 3.2: HA-Sun Java Systems App Server Patch for Solaris 10_x86
    126084-06 Sun Cluster 3.2: HA-Sun Java Message Queue Patch for Solaris 10_x86 Note: Please read SIIOTP
    126096-06 Sun Cluster 3.2: Localization patch for Solaris 10 amd64
    128557-04 Sun Cluster 3.2: Man Pages Patch for Solaris 10_x86
    137932-02 Sun Cluster 3.2:Sun Cluster 3.2: HA-Informix patch for Solaris 10_x86


    Included/updated patch revisions of Sun Cluster 3.2 11/09 Update3 for Solaris 9 5/09 or higher
    126105-38 Sun Cluster 3.2: CORE patch for Solaris 9 Note: Please read SIIOTP
    125991-05 Sun Cluster 3.2: Sun Cluster 3.2: SC Checks patch for Solaris 9
    126016-03 Sun Cluster 3.2: HA-DNS Patch for Solaris 9
    126031-09 Sun Cluster 3.2: Ha-MYSQL Patch for Solaris 9 Note: Please read SIIOTP
    126034-06 Sun Cluster 3.2: HA-NFS Patch for Solaris 9
    126043-06 Sun Cluster 3.2: HA-PostgreSQL Patch for Solaris 9 Note: Please read SIIOTP
    126046-12 Sun Cluster 3.2: HA-Oracle patch for Solaris 9 Note: Please read SIIOTP
    126049-04 Sun Cluster 3.2: HA-Oracle E-business suite Patch for Solaris 9 (-04 not yet on SunSolve)
    126058-05 Sun Cluster 3.2: HA-SAPDB Patch for Solaris 9
    126070-02 Sun Cluster 3.2: HA-Tomcat Patch for Solaris 9
    126079-04 Sun Cluster 3.2: HA-Sun Java Systems App Server Patch for Solaris 9
    126082-04 Sun Cluster 3.2: HA-Sun Java Message Queue Patch for Solaris 9 Note: Please read SIIOTP
    126095-06 Sun Cluster 3.2: Localization patch for Solaris 9 sparc and Solaris 10 sparc
    128556-04 Sun Cluster 3.2: Man Pages Patch for Solaris 9 and Solaris 10, sparc


    The quorum server is an alternative to the traditional quorum disk. The quorum server is outside of the Sun Cluster and will be accessed through the public network. Therefore the quorum server can be a different architecture.

    Included/updated patch revisions in Sun Cluster 3.2 11/09 Update3 for quorum server feature:
    127404-03 Sun Cluster 3.2: Quorum Server Patch for Solaris 9
    127405-04 Sun Cluster 3.2: Quorum Server Patch for Solaris 10
    127406-04 Sun Cluster 3.2: Quorum Server Patch for Solaris 10_x86
    Please beware of the following note which in the Special Install Instructions of Sun Cluster 3.2 core patch -38 and higher:
    NOTE 17: Quorum server patch 127406-04 (or greater) needs to be installed on quorum server host first, before installing 126107-37 (or greater) Core Patch on cluster nodes.
    127408-02 Sun Cluster 3.2: Quorum Man Pages Patch for Solaris 9 and Solaris 10, sparc
    127409-02 Sun Cluster 3.2: Quorum Man Pages Patch for Solaris 10_x86


    If some patches must be applied when the node is in noncluster mode, you can apply them in a rolling fashion, one node at a time, unless a patch's instructions require that you shut down the entire cluster. Follow procedures in How to Apply a Rebooting Patch (Node) in Sun Cluster System Administration Guide for Solaris OS to prepare the node and boot it into noncluster mode. For ease of installation, consider applying all patches at once to a node that you place in noncluster mode.

    Friday Oct 30, 2009

    Kernel patch 141444-09 or 141445-09 with Sun Cluster 3.2

    As stated in my last blog the following kernel patches are included in Solaris 10 10/09 Update8.
    141444-09 SunOS 5.10: kernel patch or
    141445-09 SunOS 5.10_x86: kernel patch

    Update 10.Dec.2009:
    Support of Solaris 10 10/09 Update8 with Sun Cluster 3.2 1/09 Update2 is now announced. The recommendation is to use the 126106-39 (sparc) / 126107-39 (x86) with Solaris 10 10/09 Update8. Note: The -39 Sun Cluster core patch is a feature patch because the -38 Sun Cluster core patch is part of Sun Cluster 3.2 11/09 Update3 which is already released.
    For new installations/upgrades with Solaris 10 10/09 Update8 use:
    \* Sun Cluster 3.2 11/09 Update3 with Sun Cluster core patch -39 (fix problem 1)
    \* Use the patches 142900-02 / 142901-02 (fix problem 2)
    \* Add "set nautopush=64" to /etc/system (workaround for problem 3)

    For patch updates to 141444-09/141445-09 use:
    \* Sun Cluster core patch -39 (fix problem 1)
    \* Also use patches 142900-02 / 142901-02 (fix problem 2)
    \* Add "set nautopush=64" to /etc/system (workaround for problem 3)


    It's time to notify that there are some issues with these kernel patches in combination with Sun Cluster 3.2

    1.) The patch breaks the zpool cachefile feature if using SUNW.HAStoragePlus

    a.) If the kernel patch 141444-09 (sparc) / 141445-09 (x86) is installed on a Sun Cluster 3.2 system where the Sun Cluster core patch 126106-33 (sparc) / 126107-33 (x86) is already installed then hastorageplus_prenet_start will fail with the following error message:
    ...
    Oct 26 17:51:45 nodeA SC[,SUNW.HAStoragePlus:6,rg1,rs1,hastorageplus_prenet_start]: Started searching for devices in '/dev/dsk' to find the importable pools.
    Oct 26 17:51:53 nodeA SC[,SUNW.HAStoragePlus:6,rg1,rs1,hastorageplus_prenet_start]: Completed searching the devices in '/dev/dsk' to find the importable pools.
    Oct 26 17:51:54 nodeA zfs: [ID 427000 kern.warning] WARNING: pool 'zpool1' could not be loaded as it was last accessed by another system (host: nodeB hostid: 0x8516ced4). See: http://www.sun.com/msg/ZFS-8000-EY
    ...


    b.) If the kernel patch 141444-09 (sparc) / 141445-09 (x86) is installed on a Sun Cluster 3.2 system where the Sun Cluster core patch 126106-35 (sparc) / 126107-35 (x86) is already installed then hastorageplus_prenet_start will work but the zpool cachefile feature of SUNW.HAStoragePlus is disabled. Without the zpool cachefile feature the time of zpool import increases because the import will scan all available disks. The messages look like:
    ...
    Oct 30 15:37:45 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 148650 daemon.notice] Started searching for devices in '/dev/dsk' to find the importable pools.
    Oct 30 15:37:45 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 148650 daemon.notice] Started searching for devices in '/dev/dsk' to find the importable pools.
    Oct 30 15:37:49 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 547433 daemon.notice] Completed searching the devices in '/dev/dsk' to find the importable pools.
    Oct 30 15:37:49 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 547433 daemon.notice] Completed searching the devices in '/dev/dsk' to find the importable pools.
    Oct 30 15:37:49 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 792255 daemon.warning] Failed to update the cachefile contents in /var/cluster/run/HAStoragePlus/zfs/zpool1.cachefile to CCR table zpool1.cachefile for pool zpool1 : file /var/cluster/run/HAStoragePlus/zfs/zpool1.cachefile open failed: No such file or directory.
    Oct 30 15:37:49 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 792255 daemon.warning] Failed to update the cachefile contents in /var/cluster/run/HAStoragePlus/zfs/zpool1.cachefile to CCR table zpool1.cachefile for pool zpool1 : file /var/cluster/run/HAStoragePlus/zfs/zpool1.cachefile open failed: No such file or directory.
    Oct 30 15:37:49 nodeA SC[,SUNW.HAStoragePlus:8,nfs-rg,zpool1-rs,hastorageplus_validate]: [ID 205754 daemon.info] All specified device services validated successfully.
    ...


    If the ZFS cachefile feature is not required AND the above kernel patches are installed, problem a.) is resolved by installing Sun Cluster core patch 126106-35 (sparc) / 126107-35 (x86).
    Solution for a) and b):
    126106-39 Sun Cluster 3.2: CORE patch for Solaris 10
    126107-39 Sun Cluster 3.2: CORE patch for Solaris 10_x86

    Alert 1021629.1: A Solaris Kernel Change Stops Sun Cluster Using "zpool.cachefiles" to Import zpools Resulting in ZFS pool Import Performance Degradation or Failure to Import the zpools

    2.) The patch breaks probe-based IPMP if more than one interface is in the same IPMP group

    After installing the already mentioned kernel patch:
    141444-09 SunOS 5.10: kernel patch or
    141445-09 SunOS 5.10_x86: kernel patch
    then the probe-based IPMP group feature is broken if the system is using more than one interface in the same IPMP group. This means all Solaris 10 systems which are using more than one interface in the same probe-based IPMP group are affected!

    After installing this kernel patch the following errors will be sent to the system console after a reboot:
    ...
    nodeA console login: Oct 26 19:34:41 in.mpathd[210]: NIC failure detected on bge0 of group ipmp0
    Oct 26 19:34:41 in.mpathd[210]: Successfully failed over from NIC bge0 to NIC e1000g0
    ...

    Workarounds:
    a) Use link-based IPMP instead of probe-based IPMP
    b) Use only one interface in the same IPMP group if using probe-based IPMP
    See the blog "Tips to configure IPMP with Sun Cluster 3.x" for more details if you like to change the configuration.
    c) Do not install the listed kernel patch above. Note: Fix is already in progress and can be reached via a service request. I will update this blog when the general fix is available.

    Solution:
    142900-02 SunOS 5.10: kernel patch
    142901-02 SunOS 5.10_x86: kernel patch

    Alert 1021262.1 : Solaris 10 Kernel Patches 141444-09 and 141445-09 May Cause Interface Failure in IP Multipathing (IPMP)
    This is reported in Bug 6888928

    3.) When applying the patch Sun Cluster can hang on reboot

    After installing the already mentioned kernel patch:
    141444-09 SunOS 5.10: kernel patch or
    141511-05 SunOS 5.10_x86: ehci, ohci, uhci patch
    the Sun Cluster nodes can hang within boot because the Sun Cluster nodes has exhausted the default number of autopush structures. When clhbsndr module is loaded, it causes a lot more autopushes to occur than would otherwise happen on a non-clustered system. By default, we only allocate nautopush=32 of these structures.

    Workarounds:
    a) Do not use the mentioned kernel patch with Sun Cluster
    b) Boot in non-cluster-mode and add the following to /etc/system
    set nautopush=64

    Solution:
    126106-42 Sun Cluster 3.2: CORE patch for Solaris 10
    126107-42 Sun Cluster 3.2: CORE patch for Solaris 10_x86
    for Sun Cluster 3.1 the issue is fixed in:
    120500-26 Sun Cluster 3.1: Core Patch for Solaris 10

    Alert 1021684.1: Solaris autopush(1M) Changes (with patches 141444-09/141511-04) May Cause Sun Cluster 3.1 and 3.2 Nodes to Hang During Boot
    This is reported in Bug 6879232

    Monday Oct 19, 2009

    Solaris 10 kernel patches

    This is a short overview of Solaris 10 kernel patches. The table show which kernel patch revision is included in the Solaris 10 Update releases and there patch dependencies. Install the kernel patch of a Solaris 10 update release is not the same as do an upgrade to the Solaris 10 update release. Sometimes is advisable to do an upgrade from an Solaris 10 update release to a higher Solaris 10 update release. There is not a general rule but when it is necessary to jump over some Solaris 10 update releases then a upgrade is advisable. The list is sorted from newest to oldest...

    Last Update: 14.Apr.2014

    150400-11
    150401-11
    newest patchid sparc (but check for actual revision)
    newest patchid x86 (but check for actual revision)
     requires 148888-05 SunOS 5.10: kernel patch
     requires
    148889-05 SunOS 5.10_x86: kernel patch
    148888-05
    148889-05
    highest release sparc
    highest release x86
     requires 147147-26 SunOS 5.10: kernel patch
     requires
    147148-26 SunOS 5.10_x86: kernel patch
    147147-26
    147148-26
    Solaris 10 01/13 Update11 sparc
    Solaris 10 01/13 Update11 x86
     requires 144500-19
     requires 144501-19
    147440-27
    147441-27
    highest release sparc
    highest release x86
     Obsoleted by: 147147-26 SunOS 5.10: kernel patch
     Obsoleted by:
    147148-26 SunOS 5.10_x86: kernel patch
    144500-19

    144501-19
    Solaris 10 08/11 Update10 sparc
    post U10 release have 147440-01
    Solaris 10 08/11 Update10 x86
    post U10 release have 147441-01
     requires 142909-17

     requires 142910-17
    144488-17
    144489-17
    highest release sparc
    highest release x86
    Obsoleted by: 144500-19 SunOS 5.10: kernel patch
    Obsoleted by:
    144501-19 SunOS 5.10_x86: kernel patch
    142909-17
    142910-17
    Solaris 10 09/10 Update9 sparc
    Solaris 10 09/10 Update9 x86
     requires 141444-09
     requires 141445-09
    142900-15
    142901-15
    highest release sparc
    highest release x86
    Obsoleted by: 142909-17 SunOS 5.10: kernel patch
    Obsoleted by:
    142910-17 SunOS 5.10_x86: kernel patch
    141444-09
    141445-09
    Solaris 10 10/09 Update8 sparc
    Solaris 10 10/09 Update8 x86
     requires 139555-08
     requires 139556-08
    141414-10
    141415-10
    highest release sparc
    highest release x86
    Obsoleted by: 141444-09 SunOS 5.10: kernel patch
    Obsoleted by:
    141445-09 SunOS 5.10_x86: kernel patch
    139555-08
    139556-08
    Solaris 10 05/09 Update7 sparc
    Solaris 10 05/09 Update7 x86
     requires 137137-09
     requires 137138-09
    138888-08
    138889-08
    highest release sparc
    highest release x86
    Obsoleted by: 139555-08 SunOS 5.10: Kernel Patch
    Obsoleted by:
    139556-08 SunOS 5.10_x86: Kernel Patch
    137137-09
    137138-09
    Solaris 10 10/08 Update6 sparc
    Solaris 10 10/08 Update6 x86
     requires 127127-11
     requires 127128-11
    137111-08
    137112-08
    highest release sparc
    highest release x86
    Obsoleted by: 137137-09 SunOS 5.10: kernel patch
    Obsoleted by:
    137138-09 SunOS 5.10_x86: kernel patch
    127127-11
    127128-11
    Solaris 10 05/08 Update5 sparc
    Solaris 10 05/08 Update5 x86
     requires 120011-14
     requires 120012-14
    127111-11
    127112-11
    highest release sparc
    highest release x86
    Obsoleted by: 127127-11 SunOS 5.10: kernel patch
    Obsoleted by:
    127128-11 SunOS 5.10_x86: kernel patch
    120011-14
    120012-14
    Solaris 10 08/07 Update4 sparc
    Solaris 10 08/07 Update4 x86
     requires 118833-36
     requires 118855-36
    125100-10
    125101-10
    highest release sparc
    highest release x86
    Obsoleted by: 120011-14 SunOS 5.10: Kernel Update patch
    Obsoleted by:
    120012-14 SunOS 5.10_x86: Kernel Update patch
    118833-36
    118855-36
    highest release sparc
    highest release x86
     this is a must have for Solaris 10 11/06 Update3 sparc
     this is a must have for Solaris 10 11/06 Update3 x86
    118833-33
    118855-33
    Solaris 10 11/06 Update3 sparc
    Solaris 10 11/06 Update3 x86

    118833-17
    118855-14
    Solaris 10 06/06 Update2 sparc
    Solaris 10 06/06 Update2 x86

    118822-30
    118844-30
    highest release sparc
    highest release x86
    Obsoleted by: 118833-36 SunOS 5.10: kernel Patch
    Obsoleted by:
    118855-36 SunOS 5.10_x86: kernel Patch
    118822-25
    118844-26
    Solaris 10 01/06 Update1 sparc
    Solaris 10 01/06 Update1 x86

    118822-10
    118844-11
    Solaris 10 03/05 HW1 sparc
    Solaris 10 03/05 HW1 x86


    Solaris 10 sparc
    Solaris 10 x86


    Thursday Sep 24, 2009

    Configuration steps to create a zone cluster on Solaris Cluster 3.2/3.3

    This is a short overview on how to configure a zone cluster. It is highly recommended to use Solaris 10 5/09 update7 with patch baseline July 2009 (or higher) and Sun Cluster 3.2 1/09 with Sun Cluster 3.2 core patch revision -33 or higher. The name of the zone cluster must be unique throughout the global Sun Cluster and must be configured on a global Sun Cluster. Please read the requirements for zone cluster in Sun Cluster Software Installation Guide.
    For Solaris Cluster 4.0 please refer to blog How to configure a zone cluster on Solaris Cluster 4.0


    A. Configure the zone cluster into the global cluster
    • Check if zone cluster can be created
      # cluster show-netprops
      to change number of zone clusters use
      # cluster set-netprops -p num_zoneclusters=12
      Note: 12 zone clusters is the default, values can be customized!

    • Configure filesystem for zonepath on all physical nodes
      # mkdir -p /zones/zc1
      # chmod 0700 /zones/zc1

    • Create config file (zc1config) for zone cluster setup e.g:

    • Configure zone cluster
      # clzc configure -f zc1config zc1
      Note: If not using the config file the configuration can also be done manually # clzc configure zc1

    • Check zone configuration
      # clzc export zc1

    • Verify zone cluster
      # clzc verify zc1
      Note: The following message is a notice and comes up on several clzc commands
      Waiting for zone verify commands to complete on all the nodes of the zone cluster "zc1"...

    • Install the zone cluster
      # clzc install zc1
      Note: Monitor the console of the global zone to see how the install proceed!

    • Boot the zone cluster
      # clzc boot zc1

    • Check status of zone cluster
      # clzc status zc1

    • Login into non-global-zones of zone cluster zc1 and configure the shell environment for root (for PATH: /usr/cluster/bin, for MANPATH: /usr/cluster/man)
      # zlogin -C zc1

    B. Add resource groups and resources to zone cluster
    • Create a resource group in zone cluster
      # clrg create -n zone-hostname-node1,zone-hostname-node2 app-rg

      Note: Use command # cluster status for zone cluster resource group overview

    • Set up the logical host resource for zone cluster
      In the global zone do:
      # clzc configure zc1
      clzc:zc1> add net
      clzc:zc1:net> set address=<zone-logicalhost-ip>
      clzc:zc1:net> end
      clzc:zc1> commit
      clzc:zc1> exit
      Note: Check that logical host is in /etc/hosts file
      In zone cluster do:
      # clrslh create -g app-rg -h <zone-logicalhost> <zone-logicalhost>-rs

    • Set up storage resource for zone cluster
      Register HAStoragePlus
      # clrt register SUNW.HAStoragePlus

      Example1) ZFS storage pool
      In the global zone do:
      Configure zpool eg: # zpool create <zdata> mirror cXtXdX cXtXdX
      and
      # clzc configure zc1
      clzc:zc1> add dataset
      clzc:zc1:dataset> set name=zdata
      clzc:zc1:dataset> end
      clzc:zc1> exit
      Check setup with # clzc show -v zc1
      In the zone cluster do:
      # clrs create -g app-rg -t SUNW.HAStoragePlus -p zpools=zdata app-hasp-rs


      Example2) HA filesystem
      In the global zone do:
      Configure SVM diskset and SVM devices.
      and
      # clzc configure zc1
      clzc:zc1> add fs
      clzc:zc1:fs> set dir=/data
      clzc:zc1:fs> set special=/dev/md/datads/dsk/d0
      clzc:zc1:fs> set raw=/dev/md/datads/rdsk/d0
      clzc:zc1:fs> set type=ufs
      clzc:zc1:fs> end
      clzc:zc1> exit
      Check setup with # clzc show -v zc1
      In the zone cluster do:
      # clrs create -g app-rg -t SUNW.HAStoragePlus -p FilesystemMountPoints=/data app-hasp-rs

      More details of adding storage

    • Switch resource group and resources online in the zone cluster
      # clrg online -eM app-rg

    • Test: Switch of the resource group in the zone cluster
      # clrg switch -n zonehost2 app-rg

    • Add supported dataservice to zone cluster
      Further details about supported dataservices available at:
      - Zone Clusters - How to Deploy Virtual Clusters and Why
      - Running Oracle Real Application Clusters on Solaris Zone Cluster

    Example output:



    Appendix: To delete a zone cluster do:
    # clrg delete -Z zc1 -F +

    Note: Zone cluster uninstall can only be done if all resource groups are removed in the zone cluster. The command 'clrg delete -F +' can be used in zone cluster to delete the resource groups recursively.
    # clzc halt zc1
    # clzc uninstall zc1

    Note: If clzc command is not successful to uninstall the zone, then run 'zoneadm -z zc1 uninstall -F' on the nodes where zc1 is configured
    # clzc delete zc1

    Monday Sep 07, 2009

    Entries in infrastructure file if using tagged VLAN for cluster interconnect

    In some cases it's necessary to add a tagged VLAN id to the cluster interconnect. This example show the difference of the cluster interconnect configuration if using tagged VLAN id or not. The interface e1000g2 have a "normal" setup (no VLAN id) and the interface e1000g1 got a VLAN id of 2. The used ethernet switch must be configured first with tagged VLAN id before the cluster interconnect can be configured. Use "clsetup" to assign a VLAN id to cluster interconnect.

    Entries for "normal" cluster interconnect interface in /etc/cluster/ccr/global/infrastructure - no tagged VLAN:
    cluster.nodes.1.adapters.1.name e1000g2
    cluster.nodes.1.adapters.1.properties.device_name e1000g
    cluster.nodes.1.adapters.1.properties.device_instance 2
    cluster.nodes.1.adapters.1.properties.transport_type dlpi
    cluster.nodes.1.adapters.1.properties.lazy_free 1
    cluster.nodes.1.adapters.1.properties.dlpi_heartbeat_timeout 10000
    cluster.nodes.1.adapters.1.properties.dlpi_heartbeat_quantum 1000
    cluster.nodes.1.adapters.1.properties.nw_bandwidth 80
    cluster.nodes.1.adapters.1.properties.bandwidth 70
    cluster.nodes.1.adapters.1.properties.ip_address 172.16.1.129
    cluster.nodes.1.adapters.1.properties.netmask 255.255.255.128
    cluster.nodes.1.adapters.1.state enabled
    cluster.nodes.1.adapters.1.ports.1.name 0
    cluster.nodes.1.adapters.1.ports.1.state enabled


    Entries for cluster interconnect interface in /etc/cluster/ccr/global/infrastructure - with tagged VLAN:
    cluster.nodes.1.adapters.2.name e1000g2001

    cluster.nodes.1.adapters.2.properties.device_name e1000g
    cluster.nodes.1.adapters.2.properties.device_instance 1
    cluster.nodes.1.adapters.2.properties.vlan_id 2
    cluster.nodes.1.adapters.2.properties.transport_type dlpi
    cluster.nodes.1.adapters.2.properties.lazy_free 1
    cluster.nodes.1.adapters.2.properties.dlpi_heartbeat_timeout 10000
    cluster.nodes.1.adapters.2.properties.dlpi_heartbeat_quantum 1000
    cluster.nodes.1.adapters.2.properties.nw_bandwidth 80
    cluster.nodes.1.adapters.2.properties.bandwidth 70
    cluster.nodes.1.adapters.2.properties.ip_address 172.16.2.1
    cluster.nodes.1.adapters.2.properties.netmask 255.255.255.128
    cluster.nodes.1.adapters.2.state enabled
    cluster.nodes.1.adapters.2.ports.1.name 0
    cluster.nodes.1.adapters.2.ports.1.state enabled

    The tagged VLAN interface is a combination of the VLAN id and the used network interface. In this example e1000g2001, the 2 after the e1000g is the VLAN id and the 1 at the end is the instance of the e1000g driver. Normally this would be the e1000g1 interface but with the VLAN id it becomes the interface e1000g2001.

    The ifconfig -a of the above configuration is:
    # ifconfig -a

    lo0: flags=20010008c9 mtu 8232 index 1
           inet 127.0.0.1 netmask ff000000
    e1000g0: flags=9000843 mtu 1500 index 2
          inet 10.16.65.63 netmask fffff800 broadcast 10.16.55.255
          groupname sc_ipmp0
          ether 0:14:4f:20:6a:18
    e1000g2: flags=201008843 mtu 1500 index 4
          inet 172.16.1.129 netmask ffffff80 broadcast 172.16.1.255
          ether 0:14:4f:20:6a:1a
    e1000g2001: flags=201008843 mtu 1500 index 3
          inet 172.16.2.1 netmask ffffff80 broadcast 172.16.2.127
          ether 0:14:4f:20:6a:19

    clprivnet0: flags=1009843 mtu 1500 index 5
          inet 172.16.4.1 netmask fffffe00 broadcast 172.16.5.255
          ether 0:0:0:0:0:1

    Thursday Jul 23, 2009

    Sun Cluster 3.x command line overview

    I wrote together a quick reference guide for Sun Cluster 3.x. The guide includes the "old" command line which is used for Sun Cluster 3.0, 3.1, 3.2 and the already known Sun Cluster 3.2 new object based command line. Please do not expect the whole command line in this two pages. It should be a reminder for the most used commands within Sun Cluster 3.x. I added the pictures to this blog but also the pdf file is available for download.





    Further reference guide are available:
    Sun Cluster 3.2 Quick Reference Guide
    German Sun Cluster 3.2 Quick Reference Guide
    Sun Cluster 3.1 command line cheat sheet

    About

    I'm still mostly blogging around Solaris Cluster and support. Independently if for Sun Microsystems or Oracle. :-)

    Search

    Archives
    « April 2014
    SunMonTueWedThuFriSat
      
    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
       
           
    Today