• September 7, 2012

WAIT-VHUB ? Whats Going On ?

I know many of you have been working on Oracle's Exalogic and other Engineered Systems. With partitions enabled now, things have gone multi dimension. But its fun. Isn't it ?

While you have some EoIB configurations together with InfiniBand partitions, the VNICs are not coming up and staying in WAIT-VHUB state ? 

Chances are that you have forgot to add InfiniBand Gateway switches' Bridge-X port GUIDs to your partition. These must be added as FULL members for EoIB to work properly. VHUB means a virtual hub in EoIB. Bridge-x is the access point for hosts to work over EoIB so thats why it must be a full member in partition.

Step 1: Find out the port GUIDs of your bridge-x devices in IB Gateway switch.

# showgwports

Device   Port Portname  PeerPort PortGUID           LID    IBState  GWState
Bridge-0  1   Bridge-0-1    4    0x0010e00c1b60c001 0x0002 Active   Up
Bridge-0  2   Bridge-0-2    3    0x0010e00c1b60c002 0x0006 Active   Up
Bridge-1  1   Bridge-1-1    2    0x0010e00c1b60c041 0x0026 Active   Up
Bridge-1  2   Bridge-1-2    1    0x0010e00c1b60c042 0x002a Active   Up

Step 2: Add these port GUIDs to the IB partition associated with EoIB. Login to master SM switch for this task.

# smpartition start
# smpartition add -pkey <PKey> -port <port GUID> -m full
# smpartition commit

Enjoy ! 

Join the discussion

Comments ( 1 )
  • martin francis kallukalam Saturday, September 8, 2012

    as far as I recollect from previous IB work with Mellanox bridge Z, the traffic between servers on same VLAN stays within the IB fabric only if the servers VNICs come off from the same vHUB. is this true?

    in other words, If i have 2 servers in same VLAN but their vNICs are mapped to 2 different gateway ports (ie 2 different vHUBs) , eventhough they are on same VLAN, the traffic have to go through the external ethernet switch . Correct?

Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.