Thursday Apr 19, 2012

Using NPIV with Oracle VM Server for SPARC

Oracle Solaris 11 provides native support for NPIV. Although NPIV is not directly integrated with Oracle VM Server SPARC (that is LDoms), you can still take advantage of NPIV with LDoms.

NPIV Overview

NPIV means N_Port ID Virtualization, and is a feature provided with Fiber Channel (FC) Storage Area Network (SAN). The exact definition, from the Technical Committee T11, is that "NPIV provides a Fiber Channel facility for sharing a single physical N_Port among multiple N_Port IDs, thereby allowing multiple initiators, each with its own N_Port ID, to share the N_Port."

More simply put, in a FC SAN environment, a Fabric, such as a SAN switch, is connected to some nodes; for example, a node can be a Host Bus Adapter (HBA), or a storage array. Each node has a N_Port (Node Port) that provides an attachment point to connect with the fabric. Each N_Port has a fabric unique identifier, the N_Port ID, by which an N_Port is uniquely known. NPIV allows a HBA to have multiple N_Port ID associated with a physical port of the HBA. That way, a HBA can have multiple virtual ports instead of a single physical port.

The benefit of NPIV is that you can then configure your SAN environment (such as zoning, LUN masking or QoS) using virtual ports instead of physical ports.This provides a lot of flexibility as you can then easily transfer a configuration from one physical HBA to another by just configuring a new HBA with the appropriate N_Port ID.

In a virtual environment, it can be convenient to associate a virtual port with a virtual machine, and to have all LUNs visible through this virtual port assigned to the virtual machine. By doing so, you can then manage the association of LUNs and virtual machines through the configuration of your SAN.

NPIV Requirements

As mentioned before, NPIV is a feature provided with FC SAN. In order to use NPIV, you need to have the appropriate hardware and software that supports NPIV; in particular you will need:

  • A FC SAN switch that supports NPIV — The switch has to be configured with a switched-fabric topology (FC-SW). NPIV is not available with arbitrary-loop topology (FC-AL).
  • A FC HBA that supports NPIV — The HBA has to be connected to a SAN switch that supports NPIV. You can not use NPIV if the HBA is directly connected to a FC storage array.
  • Oracle Solaris 11 — To use NPIV with LDoms, Solaris 11 has to be installed on the service domain with the FC HBA. Other domains can run Solaris 10 or Solaris 11.

Using NPIV with LDoms

Here are the different steps to use NPIV with LDoms, and to how to assign LUNs visible through a NPIV port to a LDoms guest domain.

Identify Existing Physical FC HBA Ports

The first step is to identify the different HBA ports available on the LDoms service domain, and select the physical port you want to create a virtual port on. Use the fcinfo command to get information about FC HBA ports. The command will provide information about physical and virtual ports.

# fcinfo hba-port
HBA Port WWN: 210000e08b899a79
        Port Mode: Initiator
        Port ID: 10400
        OS Device Name: /dev/cfg/c4
        Manufacturer: QLogic Corp.
        Model: 375-3356-01
        Firmware Version: 05.04.03
        FCode/BIOS Version:  BIOS: 1.04; fcode: 1.11; EFI: 1.00;
        Serial Number: RFC0612G20954
        Driver Name: qlc
        Driver Version: 20110321-3.05
        Type: N-port
        State: online
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: 4Gb
        Node WWN: 200000e08b899a79
        Max NPIV Ports: 127
        NPIV port list:
HBA Port WWN: 210100e08ba99a79
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c5
        Manufacturer: QLogic Corp.
        Model: 375-3356-01
        Firmware Version: 05.04.03
        FCode/BIOS Version:  BIOS: 1.04; fcode: 1.11; EFI: 1.00;
        Serial Number: RFC0612G20954
        Driver Name: qlc
        Driver Version: 20110321-3.05
        Type: unknown
        State: offline
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: not established
        Node WWN: 200100e08ba99a79
        Max NPIV Ports: 127
        NPIV port list:

In this example, we have two physical ports (the system is configured with a dual channel HBA card), and each port has no virtual port ("NPIV port list" is empty). We are going to create a virtual port on the physical port identified with the World-Wide Number (WWN) 210000e08b899a79.

Create a NPIV Port

Once you have identified the physical port where you want to create a virtual port on, use the fcadm command to create a new virtual port with NPIV.

# fcadm create-npiv-port 210000e08b899a79

Then use fcinfo to look at the port that was created:

# fcinfo hba-port 210000e08b899a79
HBA Port WWN: 210000e08b899a79
        Port Mode: Initiator
        Port ID: 10400
        OS Device Name: /dev/cfg/c4
        Manufacturer: QLogic Corp.
        Model: 375-3356-01
        Firmware Version: 05.04.03
        FCode/BIOS Version:  BIOS: 1.04; fcode: 1.11; EFI: 1.00;
        Serial Number: RFC0612G20954
        Driver Name: qlc
        Driver Version: 20110321-3.05
        Type: N-port
        State: online
        Supported Speeds: 1Gb 2Gb 4Gb
        Current Speed: 4Gb
        Node WWN: 200000e08b899a79
        Max NPIV Ports: 127
        NPIV port list:
          Virtual Port1:
                Node WWN: c0007d39993d1d61
                Port WWN: c0007d39993d1d60

As you can see, we now have one virtual port.The virtual port has a node and port WWN. By default, these WWN are randomly generated. However if you want to use a specific WWN, you can specify the node WWN with the -n option, and the port WWN with the -p option. The general syntax is:

 fcadm create-npiv-port [-p Virtual_Port_WWN] [-n Virtual_Node_WWN] Physical_Port_WWN

Identify LUNs Visible through the NPIV Port

After the virtual port has been created, you have to make sure that your SAN is properly configured (with zoning or LUN masking) so that the new virtual port can have access to some target devices (LUNs, storage array...).  Then you can identify which LUNs after visible through the new virtual port using the fcinfo command:

# fcinfo remote-port -s -p c0007d39993d1d60
Remote Port WWN: 256000c0ffc089d5
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name:
        Node WWN: 206000c0ff0089d5
        LUN: 0
          Vendor: SUN
          Product: StorEdge 3511
          OS Device Name: /dev/rdsk/c1t256000C0FFC089D5d0s2
        LUN: 1
          Vendor: SUN
          Product: StorEdge 3511
          OS Device Name: /dev/rdsk/c1t256000C0FFC089D5d1s2

In our example, two LUNs are visible through the virtual port: c1t256000C0FFC089D5d0s2 and c1t256000C0FFC089D5d1s2.

Virtualize LUNs Visible through the NPIV Port

Once you have identified which LUNs are visible through the NPIV port, you can virtualize and assign them to a LDoms guest domain using regular LDoms commands (ldm add-vdsdev and ldm add-vdisk). For example:

# ldm add-vdsdev /dev/rdsk/c1t256000C0FFC089D5d0s2 lun0@primary-vds0
# ldm add-vdsdev /dev/rdsk/c1t256000C0FFC089D5d1s2 lun1@primary-vds0

# ldm add-vdisk vdisk0 lun0@primary-vds0 myguest
# ldm add-vdisk vdisk1 lun1@primary-vds0 myguest

References

About

Alexandre Chartre is a senior principal engineer in the Oracle Virtualization Engineering organization. He is co-architect for Oracle VM Server for SPARC (LDoms).

Search

Top Tags
Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today