-
Sun
September 17, 2009
iSCSI enhancements in 2009.Q3
The
software update provides a new iSCSI software stack via the
COMSTAR is a Solaris framework which centralizes and simplifies the deployment
of SCSI targets. COMSTAR supports many different SCSI protocols, including
iSCSI, fibre channel, iSER, with support for more protocols under development.
In 2009.Q3, the COMSTAR iSCSI port provider has replaced the older iSCSI target
daemon. This new COMSTAR iSCSI provider replaces the entire iSCSI stack, from
the management of iSCSI targets and initiators down to the interactions with the
underlying ZFS volume. COMSTAR provides
as well as a more flexible management model. In this blog entry, I'll explain
the new management
model as well as talk about the advantages of using this new framework.
With the iSCSI target daemon in the 2009.Q2 update, every LUN was advertised
within its own iSCSI target. As a result, initiators could hit built-in limits
which prevent discovery of all necessary targets. Moreover, managing these
targets did not scale well as the number of LUNs on the system increased.
Access to a LUN was controlled by that LUN's ACL, a list of initiators which
were able to access that LUN.
The administrative model used with the COMSTAR framework has some key
differences and advantages. This model breaks this one-to-one mapping of LUN to
target, and instead allows an administrator to advertise multiple LUNs behind a
single target. Each LUN participates in a target group, a set of iSCSI targets
which all advertise that LUN. In addition, each LUN has its own initiator
group, a list of initiators not unlike the previous software's initiator ACL.
This group defines which initiators may access LUNs bound to that initiator
group, as well as the CHAP parameters to expect if CHAP authentication is
enabled.
Target and initiator groups define sets of targets and initiators
which are associated with LUNs. Besides any user-created target and initiator
groups, LUNs may be associated with the default target and initiator group.
These default groups contain all targets and all initiators, respectively.
While using the default target and initiator group can be useful for evaluation
purposes, their use is discouraged since their use may expose the LUN to
unwanted hosts.
In addition to the flexible mapping of LUNs to targets, the COMSTAR iSCSI stack
also allows targets to be bound to network interfaces:
This binding allows fine separation of iSCSI traffic1 from other data
protocols. In addition, iSCSI traffic can be routed over the fastest
interfaces, leaving other slow interfaces available management traffic.
Best Practices
When upgrading to 2009.Q3, you'll need to rebuild your iSCSI configuration in
this new model. With that reconfiguration in mind, here are some best practices
for deploying COMSTAR iSCSI. Keep in mind that IP and SCSI have similar
abstractions, and you may find yourself able to solve a problem at the IP level
(e.g., creating an LACP aggregation and binding a target to that aggregation's
interface) or the SCSI level (e.g., creating a target for each of several
interfaces and adding those targets to the same target group).
Use a single iSCSI target per target group. As the example above indicates, you
may also combine targets into a single target group, but I've found it simplest
to create a LACP aggregation for a single target.
Don't use the default initiator group -- that group grants every initiator
access to a LUN. A renegade or misconfigured initiator may discover an
incorrect LUN; explicitly using an initiator group prevents any damage.
Avoid using the default target group beyond evaluation.
If you know a LUN must have LU number 0, create that LUN first. LUNs which are
given automatically-assigned LU numbers can be given number zero.
Don't enable the write cache setting on a LUN unless you completely understand
the ramifications of doing so. Consult your application's documentation for
information about using this setting.
Upgrading to 2009.Q3
Once you've applied the 2009.Q3 software update, any iSCSI configuration from
2009.Q2 and earlier is set aside and the SAN environment must be rebuilt. All
LUNs are still intact and available; however, you'll need to define new iSCSI
targets and (optionally) initiators to access those LUNs. Here's a quick and
dirty set of steps to get up and running with 2009.Q3:
Create one or more iSCSI targets bound to the proper interfaces. This
configuration is available in the new Configuration > SAN section.
Assign the iSCSI targets you created in the first step to a target group.
For each LUN, change its target group to the group created above.
(Optional) Create initiator definitions, assign those initiators to initiators
groups, then bind LUNs to those initiator groups.
(Optional) Configure any CHAP or RADIUS authentication.
Once you've validated with the configuration, apply the COMSTAR deferred update
to allow complete PGR support.
Additional Information
Roch explains the
advantages of the COMSTAR stack.
COMSTAR project on
OpenSolaris.
Consult the documenation bundled with the 2009.Q3 update for details on
deploying iSCSI LUNs with COMSTAR.
[1] In some circumstances, traffic to or from and iSCSI target on one interface
may travel on a different interface. With certain network configurations (like
multiple default routes on the system), the routing configuration will dictate
that traffic must use certain interfaces which conflict with the iSCSI
configuration.
This change to the iSCSI layer has dramatically reduced the number of target IQNs you can present from a system. With the old model there was a target IQN for each LUN created. With COMSTAR it seems you can only add 32 targets to a system.
@Iain That's correct; the new model only allows for 32 iSCSI targets. Is there some configuration which is problematic using this new model?
For Sun VDI we have systems with 1000s of LUNs (each representing a virtual disk of a VM) on AR 2009.Q2. How do I do that with Q3?
And more on the practical side of things: I tried your "quick and dirty set of steps" and while the IQN can be seen from the outside after step 1 already, I cannot connect to it after I executed the steps 1,2,3,6 with either the Windows or the sol10u7 initiator. Do I miss something?
I created more iSCSI LUNs in a previous Software Release then the amount of iSCSI targets allowed in 2009.Q3.
After applying the new Software, such a iSCSI LUN is no longer exported as an iSCSI Target!
Note, that iSCSI LUNs are continued to be managed on the "Shares" screen.
To export a iSCSI LUN, you'll need to create an iSCSI Target Group on "Configuration - SAN" screen and associate this Group with one or more iSCSI LUNs.
For this purpose, choose an iSCSI LUN under "Shares" screen and select the iSCSI Target Group on the "Sharing Options" screen that you'll find under "Protocols".