Configure IPoIB on Solaris 10 branded zone

Compute nodes in Exalogic communicate with each other and mount their shares from the ZFS storage appliance over the bond0 interface. Bond0 is a highly available network interface over the InfiniBand fabric using a portability layer called IPoIB that allows compute nodes and the storage appliance to communicate using TCP/IP protocol over InfiniBand.

In previous entry, we have created a Solaris 10 branded zone, obviously we would also like the branded zone to be able to mount shares from the ZFS storage appliance over IPoIB.

In this entry, I’ll show you the steps.

1. Create new IB partition links

Login to the global zone as root user and run the following commands

#dladm create-part -P ffff -l ibp0 s10zone0_bond0_0
#dladm create-part -P ffff -l ibp1 s10zone0_bond0_1


Check if the IB partition links are successfully created using the following command:

root@el01cn01:/# dladm show-part
LINK PKEY OVER STATE FLAGS
bond0_0 FFFF ibp0 up ----
bond0_1 FFFF ibp1 up ----
s10zone0_bond0_0 FFFF ibp0 unknown ----
s10zone0_bond0_1 FFFF ibp1 unknown ----


2. Add new IB partition links to zone

Use the following commands to add the IB partition links to our branded zone.

root@el01cn01:/# zonecfg -z s10-zone
zonecfg:s10-zone> add net
zonecfg:s10-zone:net> set physical=s10zone0_bond0_0
zonecfg:s10-zone:net> end
zonecfg:s10-zone> add net
zonecfg:s10-zone:net> set physical=s10zone0_bond0_1
zonecfg:s10-zone:net> end
zonecfg:s10-zone> verify
zonecfg:s10-zone> commit
zonecfg:s10-zone> exit


3. Configure bond0 on Solaris 10 branded zone

Boot up the branded zone and login to the console as root and create the following files in /etc directory.

# cat hostname.bond0

ipmp group bond0 IP_ADDR netmask NETMASK up

# cat hostname.s10zone0_bond0_0

group bond0

# cat hostname.s10zone0_bond0_1

group bond0 standby

Where IP_ADDR and NETMASK are the IP address and netmask that you want to assign, e.g. 192.168.10.1 and 255.255.255.0
Reboot the zone and you should be able to see bond0 up and running.

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

The primary contributors to this blog are comprised of the Exalogic and Cloud Application Foundation contingent of Oracle's Fusion Middleware Architecture Team, fondly known as the A-Team. As part of the Oracle development organization, The A-Team supports some of Oracle's largest and most strategic customers worldwide. Our mission is to provide deep technical expertise to support various Oracle field organizations and customers deploying Oracle Fusion Middleware related products. And to collect real world feedback to continuously improve the products we support. In this blog, our experts and guest experts will focus on Exalogic, WebLogic, Coherence, Tuxedo/mainframe migration, Enterprise Manager and JDK/JRockIT performance tuning. It is our way to share some of our experiences with Oracle community. We hope our followers took away something of value from our experiences. Thank you for visiting and please come back soon.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today