X

Paulie's world in a blog

  • July 10, 2013

Solaris 11 IPoIB + IPMP

Guest Author
I recently needed to create a two port active:standby IPMP group to be served over Infiniband on Solaris 11. Wow that's a mouthful of terminology! Here's how I did it:

List available IB links

[root@adrenaline ~]# dladm show-ib
LINK HCAGUID PORTGUID PORT STATE PKEYS
net5 21280001CF4C96 21280001CF4C97 1 up FFFF
net6 21280001CF4C96 21280001CF4C98 2 up FFFF

Partition the IB links. My pkey will be 8001.
[root@adrenaline ~]# dladm create-part -l net5 -P 0x8001 p8001.net5
[root@adrenaline ~]# dladm create-part -l net6 -P 0x8001 p8001.net6
[root@adrenaline ~]# dladm show-part
LINK PKEY OVER STATE FLAGS
p8001.net5 8001 net5 unknown ----
p8001.net6 8001 net6 unknown ----

Create test addresses for the newly created datalinks
[root@adrenaline ~]# ipadm create-ip p8001.net5
[root@adrenaline ~]# ipadm create-addr -T static -a 192.168.1.101 p8001.net5/ipv4
[root@adrenaline ~]# ipadm create-ip p8001.net6
[root@adrenaline ~]# ipadm create-addr -T static -a 192.168.1.102 p8001.net6/ipv4
[root@adrenaline ~]# ipadm show-addr
ADDROBJ TYPE STATE ADDR
p8001.net5/ipv4 static ok 192.168.1.101/24
p8001.net6/ipv4 static ok 192.168.1.102/24

Create an IPMP group and add the IB datalinks
[root@adrenaline ~]# ipadm create-ipmp ipmp0
[root@adrenaline ~]# ipadm add-ipmp -i p8001.net5 -i p8001.net6 ipmp0

Set one IB datalink to standby
[root@adrenaline ~]# ipadm set-ifprop -p standby=on -m ip p8001.net6

Assign an IP address to the IPMP group
[root@adrenaline ~]# ipadm create-addr -T static -a 192.168.1.100/24 ipmp0/v4

That's it! Final checks:
[root@adrenaline ~]# ipadm
NAME CLASS/TYPE STATE UNDER ADDR
ipmp0 ipmp ok -- --
ipmp0/v4 static ok -- 192.168.1.100/24
p8001.net5 ip ok ipmp0 --
p8001.net5/ipv4 static ok -- 192.168.1.101/24
p8001.net6 ip ok ipmp0 --
p8001.net6/ipv4 static ok -- 192.168.1.102/24
[root@adrenaline ~]# ping 192.168.1.100
192.168.1.100 is alive

Join the discussion

Comments ( 4 )
  • Eli Kleinman Wednesday, July 10, 2013

    Dose IB support LACP or only ipmp?


  • Carlos Azevedo Thursday, July 11, 2013

    Thank you very much for help dis-mystifying IB.

    Where can we find good IB primers?

    By the way, I believe there's a slight copy/paste mistake on:

    [root@adrenaline ~]# ipadm >>>>create-addr<<<< set-ifprop -p standyby=on -m ip p8001.net6

    Regards.


  • Apostolos Giannakidis Monday, January 13, 2014

    Thank you very much for this informative article Paul.

    A small typo that I noticed:

    "standyby" should be "standby".

    Best regards


  • Paul Monday, January 13, 2014

    Thanks for the correction!


Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.