WAIT-VHUB ? Whats Going On ?

I know many of you have been working on Oracle's Exalogic and other Engineered Systems. With partitions enabled now, things have gone multi dimension. But its fun. Isn't it ?

While you have some EoIB configurations together with InfiniBand partitions, the VNICs are not coming up and staying in WAIT-VHUB state ? 

Chances are that you have forgot to add InfiniBand Gateway switches' Bridge-X port GUIDs to your partition. These must be added as FULL members for EoIB to work properly. VHUB means a virtual hub in EoIB. Bridge-x is the access point for hosts to work over EoIB so thats why it must be a full member in partition.

Step 1: Find out the port GUIDs of your bridge-x devices in IB Gateway switch.

# showgwports

INTERNAL PORTS:
---------------
Device   Port Portname  PeerPort PortGUID           LID    IBState  GWState
---------------------------------------------------------------------------
Bridge-0  1   Bridge-0-1    4    0x0010e00c1b60c001 0x0002 Active   Up
Bridge-0  2   Bridge-0-2    3    0x0010e00c1b60c002 0x0006 Active   Up
Bridge-1  1   Bridge-1-1    2    0x0010e00c1b60c041 0x0026 Active   Up
Bridge-1  2   Bridge-1-2    1    0x0010e00c1b60c042 0x002a Active   Up

Step 2: Add these port GUIDs to the IB partition associated with EoIB. Login to master SM switch for this task.

# smpartition start
# smpartition add -pkey <PKey> -port <port GUID> -m full
# smpartition commit

Enjoy ! 

Comments:

as far as I recollect from previous IB work with Mellanox bridge Z, the traffic between servers on same VLAN stays within the IB fabric only if the servers VNICs come off from the same vHUB. is this true?

in other words, If i have 2 servers in same VLAN but their vNICs are mapped to 2 different gateway ports (ie 2 different vHUBs) , eventhough they are on same VLAN, the traffic have to go through the external ethernet switch . Correct?

Posted by martin francis kallukalam on September 08, 2012 at 04:41 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

You have connected here over internet and already using the technologies under the hood for Networking and may be wondering how things work ?

This blog space will present you with various topics related to Oracle's Products and their close association with Networking. My goal is not to overwhelm you and I will try my best to present information in simple way.
Stay tuned !


About Author: Hi, I am Neeraj Gupta at Oracle. I worked at Sun Microsystems for 11 years specializing in InfiniBand, Ethernet, Security, HA and Telecom Computing Platforms. Prior to joining Sun, I spent 5 years in Telecom industry focusing on Internet Services and GSM Cellular Networks.
At present, I am part of Oracle's Engineered Systems team focused on Networking and Maximum Availability Architectures.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today