Introduction to iSCSI in Solaris 10 8/07 (U4)

After consulting with Jeff Victor a colleague of mine this week on the subject of iSCSI, I decide to do some reading and playing around myself with iSCSI. Jeff was actually testing iSCSI with LDOMS etc.. Look forward to seeing Jeff's work.


The Solaris 10 8/07 (U4) release provides support for iSCSI target devices, which can be disk or tape devices. Releases prior to Solaris 10 8/07 provided support for iSCSI initiators. 


So what does this mean?  Well, It means that Solaris can act as both a “server” (target) and a “client” (initiator) for iSCSI.


The advantage of setting up Solaris iSCSI targets is you might have existing fibre-channel devices that can be connected to clients without the cost of fibre-channel HBAs. In addition, systems with dedicated arrays can now export replicated storage with ZFS or UFS file systems.


While experimenting with iSCSI you'll need to remember some basic commands.


You will use the iscsitadm(1M) command to set up and manage your iSCSI target devices. For the disk device that you select as your iSCSI target, you'll need to provide an equivalently sized ZFS or UFS file system as the backing store for the iSCSI daemon.


After the target device is set up, you'll use the iscsiadm(1M) command to identify your iSCSI targets, which will discover and use the iSCSI target device.


A good reference for you to start looking at when configuring iSCSI would be the Configuring Solaris iSCSI Targets and Initiators (Tasks), in the Systems Administration Guide: Devices and File Systems.


To assist you with iSCSI, I’ve jotted down some steps which will allow you to export some targets, play around with them and then destroy them afterwards.


The following hosts will be used in this exercise:




















Hostname IP address Description
sunrise1 192.168.1.25 This machine has the zpool, called zpoolA for this exercise
spirit2 192.168.1.30 This machine will be used to mount our iSCSI targets

Our zpool, zpoolA, will be used for storage.

The goal of this exercise is to couple ZFS with the iSCSI target in Solaris. There are two new ZFS properties added to support this feature. For this exercise we will be only using the shareiscsi property. Note the other property is the iscsioptions.


    shareiscsi

Like the 'sharenfs' property, 'shareiscsi' indicates if a ZVOL should
be exported as an iSCSI target. The acceptable values for this property
are 'on', 'off', and 'direct'. In the future, we may support other
target types (for example, 'tape'). The default is 'off'. This property
may be set on filesystems, but has no direct effect; this is to allow
ZVOLs created under the ZFS hierarchy to inherit a default. For
example, an administrator may want ZVOLs to be shared by default, and
so set 'shareiscsi=on' for the pool.

    iscsioptions
	This read-only property, which is hidden, is used by the iSCSI target
daemon to store persistent information, such as the IQN. It cannot be
viewed or modified using the zfs command. The contents are not intended
for external consumers. 

First, we need to create some volumes to use. Since I already started with my zpoolA, we’ll use that. We’ll create three volumes, each 1Gbyte in
size using the zfs(1M) command.


sunrise1# zfs create -V 1G zpoolA/iscsi_luns/vol001
sunrise1# zfs create -V 1G zpoolA/iscsi_luns/vol002
sunrise1# zfs create -V 1G zpoolA/iscsi_luns/vol003

Now, you'll need to configure them as iSCSI targets


sunrise1# zfs set shareiscsi=on zpoolA/iscsi_luns/vol001
sunrise1# zfs set shareiscsi=on zpoolA/iscsi_luns/vol002
sunrise1# zfs set shareiscsi=on zpoolA/iscsi_luns/vol003

Walla! You've created three thinly provisioned 1GB volumes and then turned "shareiscsi" on.


It's that simple!  Seeing is believe! Here's how you check!


sunrise1# svcs -a | grep -i iscsi
disabled 7:02:49 svc:/network/iscsi_initiator:default
online 19:21:01 svc:/system/iscsitgt:default

sunrise1# iscsitadm list target -v
Target: zpoolA/iscsi_luns/vol001
iSCSI Name: iqn.1986-03.com.sun:02:7536c2517-91e6-cf6c-d566-d48fb182e9f7
Alias: zpoolA/iscsi_luns/vol001
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0x0
VID: SUN
PID: SOLARIS
Type: disk
Size: 1GB
Backing store: /dev/zvol/rdsk/zpoolA/iscsi_luns/vol001
Status: online
Target: zpoolA/iscsi_luns/vol002    
iSCSI Name: iqn.1986-03.com.sun:02:78692333-1fb4-ee45-b547-ea49922ee538
Alias: zpoolA/iscsi_luns/vol002
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0x0
VID: SUN
PID: SOLARIS
Type: disk
Size: 1GB
Backing store: /dev/zvol/rdsk/zpoolA/iscsi_luns/vol002
Status: online
Target: zpoolA/iscsi_luns/vol003    
iSCSI Name: iqn.1986-03.com.sun:02:792405e0-a3fc-4ccf-f86d-b79f7b1ee006
Alias: zpoolA/iscsi_luns/vol003
Connections: 0
ACL list:
TPGT list:
LUN information:
LUN: 0
GUID: 0x0
VID: SUN
PID: SOLARIS
Type: disk
Size: 1GB
Backing store: /dev/zvol/rdsk/zpoolA/iscsi_luns/vol003
Status: online 

Again make sure they’re visible by running the following command


sunrise1# iscsitadm list target

Target: zpoolA/iscsi_luns/vol001
iSCSI Name: iqn.1986-03.com.sun:02:7536c2517-91e6-cf6c-d566-d48fb182e9f7
Connections: 0
Target: zpoolA/iscsi_luns/vol002
iSCSI Name: iqn.1986-03.com.sun:02:78692333-1fb4-ee45-b547-ea49922ee538
Connections: 0
Target: zpoolA/iscsi_luns/vol003
iSCSI Name: iqn.1986-03.com.sun:02:792405e0-a3fc-4ccf-f86d-b79f7b1ee006
Connections: 0

Now, to start using the targets! We'll first need to tell our initiator where to look for the storage. Note: Keep in mind, that the iSCSI connection is not initiated until the discovery method is enabled. For the purpose here, we will be configuring the device for dynamical discovery (SendTargets). There are two new ZFS properties added to support this feature. For this exercise we will be only using the shareiscsi property. The other property is the iscsioptions.


    shareiscsi

Like the 'sharenfs' property, 'shareiscsi' indicates if a ZVOL should
be exported as an iSCSI target. The acceptable values for this property
are 'on', 'off', and 'direct'. In the future, we may support other
target types (for example, 'tape'). The default is 'off'. This property
may be set on filesystems, but has no direct effect; this is to allow
ZVOLs created under the ZFS hierarchy to inherit a default. For
example, an administrator may want ZVOLs to be shared by default, and
so set 'shareiscsi=on' for the pool.

    iscsioptions
	This read-only property, which is hidden, is used by the iSCSI target
daemon to store persistent information, such as the IQN. It cannot be
viewed or modified using the zfs command. The contents are not intended
for external consumers.

spirit2# iscsiadm add discovery-address 192.168.1.25:3260

We will now need to tell our initiator how to look for our targets. Note the -t, this is the flag to enable the sendtargets. You can use --sendtargets flag if you prefer.


spirit2# iscsiadm modify discovery -t enable

It is now time to use devfsadm(1M) to find our new storage. Note the -C flag, This flag is the Cleanup mode. Prompts devfsadm to cleanup dangling /dev links that are not normally removed. This -i flag is for configuring only the devices for the named driver, driver_name.


spirit2# devfsadm -C -i iscsi

I've found if you run iostat -En you'll see that we have new storage, like this


spirit2# iostat -En
...
c10t2d0 Soft Errors: 2 Hard Errors: 0 Transport Errors: 0
Vendor: SUN Product: SOLARIS Revision: 1 Serial No:
Size: 1.07GB <1073741824 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
...

If you look at the syslog, you’ll probably notice messages complaining about a bad magic number. This is because our newly visible LUNs aren’t labelled yet. So all you'll need to do is run the format(1M) command and label them like any other disk.


Note: If you want to make the iSCSI drive available on reboot, create the file system, and add an entry to the /etc/vfstab file as you would with a UFS file system on a SCSI device.


Now once, that the LUN's are formatted we can create filesystems and mount them, for example:


spirit2# newfs /dev/rdsk/c10t2d0s0
spirit2# mkdir /mnt/iscsi_mytest
spirit2# mount /dev/dsk/c10t2d0s0 /mnt/iscsi_mytest

Once we’ve finished playing around with our volumes, you might want to return your hosts to a clean state so simply unmount any filesystems and remove the directory where it was mounted.


spirit2# umount /mnt/iscsi_mytest
spirit2# rmdir /mnt/iscsi_mytest

Now, If you created any zpools or SVM metadevices, clear them up as well. Once everything is clear: These optional procedure, assumes that you are logged in to the local system where access to an iSCSI target device has already been configured.


Note: After removing a discovery address, iSNS server, static-config, or disabling a discovery method, the associated targets are logged out. If these associated targets are still in use, for example, have mounted file systems, and you did not execute the previous commands, (umount, rmdir), the logout of these devices will fail and they will remain on the active target list.


spirit2# iscsiadm modify discovery -t disable
spirit2# iscsiadm remove discovery-address 192.168.1.25:3260
spirit2# devfsadm -C -i iscsi

and on the target host do the following


sunrise1# zfs destroy zpoolA/iscsi_luns/vol001
sunrise1# zfs destroy zpoolA/iscsi_luns/vol002
sunrise1# zfs destroy zpoolA/iscsi_luns/vol003


There you go!  A simple exercise configuring using iSCSI 


In conclusion, please note that because iSCSI target LUN's are shared network accessible block storage devices they are built on top of the zvols. The advantage of these zvols is that they come with all that ZFS has to offer filesystems including snapshots, replication, compression, etc...


Special Thanks to Jeff Victor & Scott Dickson for their help on the subject. 


Well have fun with iSCSI... Until next time!! 

Technorati Profile

Technocrati Tags:

Comments:

For eval, can you share the targets over the loopback address? I for the life of me can not get this to work on Sol 10 8/07 on an ultra 10. I get as far as being able to view the targets with iscsiadm, but can not see them in format:

# iscsiadm list discovery-address -v
Discovery Address: 127.0.0.1:3260
Target name: iqn.1986-03.com.sun:02:ec421b1f-04a8-eb7e-caed-a8061bb85676
Target address: 127.0.0.1:3260, 1

in /var/adm/messages I get:
NOTICE: iscsi session(12) unable to enumerate logical unit - inquiry failed lun 0

Posted by Spencer Harvey on February 24, 2008 at 02:03 PM EST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

mhuff

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today