Tips to configure IPMP with Sun Cluster 3.x

Configure IPMP (probe based or link based):
Setup IPMP (IP network multipathing) groups on all nodes for all public network interfaces which are used for a HA dataservice. This article describe a summary of possibilities and known issues. An overview about IPMP can be found in System Administration Guide: IP Services.


Example probe-based IPMP group active-active with interfaces qfe0 and qfe4 with one production IP:

Entry of /etc/hostname.qfe0:
<production_IP_host> netmask + broadcast + group ipmp1 up \\
addif <test_IP_host> netmask + broadcast + deprecated -failover up

Entry of /etc/hostname.qfe4:
<test_IP_host> netmask + broadcast + group ipmp1 deprecated -failover up
The IPMP group name ipmp1 is freely chosen in this example!

If the defaultrouter is NOT 100% available please read
Technical Instruction 1010640.1: Summary of typical IPMP Configurations
and
Technical Instruction 1001790.1: The differences between Network Adapter Failover (Sun Cluster 3.0) and IP Multipathing (Sun Cluster 3.1)

Notes:
\* Do not use test IP for normal applications.
\* When using Solaris9 12/02 or later & Sun Cluster 3.1 Update1 or later there is no need for a IPMP test address if you have only 1 IP address in the IPMP group. (RFE 4511634, 4741473)
  e.g: of /etc/hostname.qfe0 entry:
    <production_IP_host> netmask + broadcast + group ipmp1 up
\* Test IP for all adapters in the same IPMP group must belong to a single IP subnet.


Example link-based IPMP group active-active with interfaces qfe0 and qfe4 with one production IP:

Entry of /etc/hostname.qfe0:
<production_IP_host> netmask + broadcast + group ipmp1 up
Entry of /etc/hostname.qfe4:
<dummy_IP_host> netmask + broadcast + deprecated group ipmp1 up

Notes:
\* Do NOT use the 0.0.0.0 IP address as dummy_IP_host for link based due to bug 6457375.
\* This time the recommendation is to use valid IP address but it can also be a dummy IP address.

The bug 6457375 is fixed in kernel update patch 138888-01 (sparc) or 138889-01 (x86). These kernel patches are based on Solaris 10 10/08 Update6. Now it's possible to use the 0.0.0.0 IP address as described in the following example:
Entry of /etc/hostname.qfe0:
<production_IP_host> netmask + broadcast + group ipmp1 up
Entry of /etc/hostname.qfe4:
group ipmp1 up

Further Details:
Technical Instruction 1008064.1: IPMP Link-based Only Failure Detection with Solaris 10 Operating System (OS)


Hints / Checkpoints for all configurations:
  • You need an additional IP for each logical host.

  • If there is a firewall being used between clients and a HA service running on this cluster, and if this HA service is using UDP and does not bind to a specific address, the IP stack choses the source address for all outgoing packages from the routing table. So, as there is no guarantee that the same source address is chosen for all packages - the routing table might change - it is necessary to configure all addresses available on a network interface as valid source addresses in the firewall. More details can be found in the blog Why a logical IP is marked as DEPRECATED?

  • IPMP groups as active-standby configuration is also possible.

  • In the /etc/default/mpathd file, the value of TRACK_INTERFACES_ONLY_WITH_GROUPS must be yes (default).

  • In case of Sun Cluster 3.1 The FAILBACK in /etc/default/mpathd file must be yes (default). Bug 6429808. Fixed in Sun Cluster 3.2.

  • Use only one IPMP group in the same subnet. It's not supported to use more IPMP groups in the same subnet.

  • The SC installer adds an IPMP group to all public network adapters. If desired remove the IPMP configuration for network adapters that will NOT be used for HA dataservices.

  • Remove IPMP groups from dman interfaces (SunFire 12/15/20/25K) if exists. (Bug 6309869)

Comments:

Excellent quick walk through on IPMP configuration.

Posted by Vijay on November 28, 2011 at 12:20 AM CET #

Post a Comment:
  • HTML Syntax: NOT allowed
About

I'm still mostly blogging around Solaris Cluster and support. Independently if for Sun Microsystems or Oracle. :-)

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
23
24
25
26
27
28
29
30
   
       
Today