Wednesday Jul 10, 2013

Solaris 11 IPoIB + IPMP

I recently needed to create a two port active:standby IPMP group to be served over Infiniband on Solaris 11. Wow that's a mouthful of terminology! Here's how I did it:

List available IB links

[root@adrenaline ~]# dladm show-ib
LINK         HCAGUID         PORTGUID        PORT STATE  PKEYS
net5         21280001CF4C96  21280001CF4C97  1    up     FFFF
net6         21280001CF4C96  21280001CF4C98  2    up     FFFF
Partition the IB links. My pkey will be 8001.
[root@adrenaline ~]# dladm create-part -l net5 -P 0x8001 p8001.net5
[root@adrenaline ~]# dladm create-part -l net6 -P 0x8001 p8001.net6
[root@adrenaline ~]# dladm show-part
LINK         PKEY  OVER         STATE    FLAGS
p8001.net5   8001  net5         unknown  ----
p8001.net6   8001  net6         unknown  ----
Create test addresses for the newly created datalinks
[root@adrenaline ~]# ipadm create-ip p8001.net5
[root@adrenaline ~]# ipadm create-addr -T static -a 192.168.1.101 p8001.net5/ipv4
[root@adrenaline ~]# ipadm create-ip p8001.net6
[root@adrenaline ~]# ipadm create-addr -T static -a 192.168.1.102 p8001.net6/ipv4
[root@adrenaline ~]# ipadm show-addr
ADDROBJ           TYPE     STATE        ADDR
p8001.net5/ipv4   static   ok           192.168.1.101/24
p8001.net6/ipv4   static   ok           192.168.1.102/24
Create an IPMP group and add the IB datalinks
[root@adrenaline ~]# ipadm create-ipmp ipmp0
[root@adrenaline ~]# ipadm add-ipmp -i p8001.net5 -i p8001.net6 ipmp0
Set one IB datalink to standby
[root@adrenaline ~]# ipadm set-ifprop -p standby=on -m ip p8001.net6
Assign an IP address to the IPMP group
[root@adrenaline ~]# ipadm create-addr -T static -a 192.168.1.100/24 ipmp0/v4
That's it! Final checks:
[root@adrenaline ~]# ipadm
NAME              CLASS/TYPE STATE        UNDER      ADDR
ipmp0             ipmp       ok           --         --
   ipmp0/v4       static     ok           --         192.168.1.100/24
p8001.net5        ip         ok           ipmp0      --
   p8001.net5/ipv4 static    ok           --         192.168.1.101/24
p8001.net6        ip         ok           ipmp0      --
   p8001.net6/ipv4 static    ok           --         192.168.1.102/24

[root@adrenaline ~]# ping 192.168.1.100
192.168.1.100 is alive
About

Hiya, my name is Paul Johnson and I'm a software engineer working on the ZFS storage appliance .

Search

Categories
  • Oracle
Archives
« July 2013 »
SunMonTueWedThuFriSat
 
1
2
3
4
5
6
7
8
9
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
   
       
Today