By Juergens-Oracle on Oct 20, 2016
The support of mpgroup feature from Oracle VM Server software is coming into discussion with Solaris Cluster from time to time. Therefore I try to clarify some facts around mpgroup support and Solaris Cluster.
Generally Solaris Cluster has documented
SPARC: Guidelines for Oracle VM Server for SPARC Logical Domains as Cluster Nodes
when running Solaris Cluster in LDom.
Extract of the guidelines:
Virtual disk multipathing - Do not configure the virtual disk multipathing feature of Oracle VM Server for SPARC on a logical domain that is configured as a cluster node.
Additional details in the
Oracle Solaris Cluster 4 Compatibility Guide - See section: Oracle VM Server for SPARC Mixed Configurations
However, same statement is included :
Virtual Disk Multipathing - Setting multipathing groups via mpgroup is not supported.
In other words this statement means that mpgroup can not be used for shared Solaris Cluster devices. Because Solaris Cluster is doing scsi reservations on shared devices. The technical reason for this is that mpgroup and scsi reservations are not implemented and therefore simply not working and not supported.
If you like to reach redundancy of disk paths, coming from different I/O domains, in Solaris Cluster, then you can use SR-IOV feature of Oracle VM Server software. For details refer to the already mentioned Oracle Solaris Cluster 4 Compatibility Guide .
Furthermore look out the possibilities for vSAN/vHBA with Solaris Cluster in Compatibility Guide and
Oracle Solaris Cluster 4.x Requirements when using Virtual HBA in Oracle VM Server (Doc ID 2094724.1)
Now let’s come to "BUT…" mentioned in the headline and the question:
What about mpgroup and Solaris Cluster when using mpgroup feature only for boot disks?
It is best practices to use mpgroup for boot disks but how this can be achieved with Solaris Cluster? The answer is quite simple, because boot disks are local (non-shared) to the cluster node therefore fencing (scsi reservations) can/should be disabled for the boot disks.
Do the following to disable fencing (scsi reservations) on boot disks:
a) Look for the DID device of the device which is used in rpool
# scdidadm -l c1d0
17 node0:/dev/rdsk/c1d0 /dev/did/rdsk/d17
b) Check the settings of the used DID device
# cldg show dsk/d17
Note: Only one node should be in the node list
c) Set localonly and autogen flag for the DID device.
# cldg set -p localonly=true -p autogen=true dsk/d17
Alternative disable fencing for the DID device with
# cldev set -p default_fencing=nofencing d17
d) Verify the settings
# cldg show dsk/d17
or for alternative
# cldev show d17
Also ensure that there are no SCSI keys on these disks:
# /usr/cluster/lib/sc/scsi -c inkeys -d /dev/did/rdsk/d17
This output indicate that there are no keys on the disk.
There is already a request for enhancement (Bug 23561043 - mpgroups support for local disks in Solaris Cluster need further clarification) that this will be described in the official Solaris Cluster documentation.