Today, data centers host large servers with various SCSI devices, which are more complex than their earlier versions. To identify what kind of devices are installed on a server, an Administrator needs expertise on various vendor-specific utilities and System commands to perform a specific task. Some of the tasks include multiple steps over a large number of devices. An administrator generally has a handy bash script to improve the workflow for such a task. Such a script extracts needed information using a cat command. Once the cat command output is obtained, the administrator will perform the required action by sending the correct value using the echo command to a SysFS hook.
scsi-cli is a userspace utility that uses SysFS hooks provided by the SCSI devices and transports to display information in a human-readable format.
One of the most significant constraints that this utility aims to solve is the use of vendor-specific tools and utilities to extract user information. scsi-cli is available in the oracle-samples repository.
Most devices attached to a SCSI host in the data center are via Fibre Channel or iSCSI transport. For a non-expert data center user, the following are some of the everyday tasks:
- List all the SCSI devices installed on the system.
- Get a detailed description of a specific device of interest.
Based on the task’s need and requirements, users will need to issue more than a few commands to a specific SysFS hook exported by a device using the system command cat. The user will then use the output from the cat command and issue another system command, echo, with an appropriate value to the same SysFS hook to perform the action. This sequence needs multiple commands and a SysFS file system know-how for a user to perform an action on the device.
The scsi-cli tool helps users performing simple activities such as setging a device as ‘offline’ (remove from the host) or ‘online’ for a system to discover the removed device.
For Fibre Channel adapters, the user needs to download a vendor-provided utility to identify adapter ports and perform various additional vendor-specific tasks.
However, for a user who is not performing specialized tasks such as firmware updates, there is no need to download a vendor-specific tool to identify the adapter and information related to the adapter. scsi-cli will help users list out adapters installed on the server and their status. In addition, it would allow users to discover other details, such as the number of Rports discovered by the port.
For iSCSI transport, users typically use the iscsiadm utility provided by the open-iscsi package to perform various system administration tasks, such as the discovery of an iSCSI target and performing a log-in operation to that iSCSI target. Once the iSCSI initiator successfully logs into the target, an iSCSI session is established. For a non-expert user of iscsiadm, finding out details of iSCSI sessions/connections and the number of LUNs exported by an iSCSI target becomes a cumbersome task. scsi-cli will help the user list out details related to iSCSI devices.
Note: scsi-cli is not a replacement for iscsiadm, which has specialized usage in the form of discovery and establishing a session via the login option with the iscsi target.
list sub-command for scsi-cli
Listing a device installed on a computer host is one of the most important and the first tasks performed by a new user. scsi-cli helps users by listing devices installed on a system and, in addition, groups them into different types. When the scsi-cli list command is issued, the tool will list all the sub-commands supported by the list sub-command and display all devices in the system. The list sub-command is a more convenient way to list all devices. Sub-commands provide a more specific listing of devices per group.
Here’s an example of list sub-command output:
# ./scsi-cli list Following Sub-Commands are supported for 'list' ---------------------------------------------------------------- cd-rom List CD-ROM devices from the host tape List Tape devices from the host medium-changer List Medium Changer devices from the host enclosure List Enclosure devices from the host fc_hba List Fiber Channel HBA instlled iscsi List iSCSI Sessions disk List disks from the host controller List disk controllers from the host generic List generic disk from the host multipath List multipath disk from the host Displaying All Disk Controller devices on the system. ---------------------------------------------------------------- BUS ID Vendor Model Revision Major Minor Disk Type Disk Name Device Path ------------ ---------------- ---------------- -------- ----- ----- ---------------- -------- ------------------------ [0:0:0:0] IET Controller 0001 21 6 Storage Array sg6 /sys/class/scsi_generic/sg6 [0:0:0:0] IET Controller 0001 21 4 Storage Array sg4 /sys/class/scsi_generic/sg4 [0:0:0:0] IET Controller 0001 21 2 Storage Array sg2 /sys/class/scsi_generic/sg2 [0:0:0:0] IET Controller 0001 21 8 Storage Array sg8 /sys/class/scsi_generic/sg8 Displaying All block devices on the system. ---------------------------------------------------------------- BUS ID Vendor Model Revision Major Minor Disk Type Disk Name Device Path ------------ ---------------- ---------------- -------- ----- ----- ---------------- -------- ------------------------ [6:0:0:1] ORACLE BlockVolume 1.0 8 80 disk sdf /sys/block/sdf [4:0:0:1] ORACLE BlockVolume 1.0 8 48 disk sdd /sys/block/sdd [2:0:1:1] ORACLE BlockVolume 1.0 8 16 disk sdb /sys/block/sdb [5:0:0:1] ORACLE BlockVolume 1.0 8 64 disk sde /sys/block/sde [3:0:0:1] ORACLE BlockVolume 1.0 8 32 disk sdc /sys/block/sdc [2:0:0:1] ORACLE BlockVolume 1.0 8 0 disk sda /sys/block/sda Displaying All nvme-block devices on the system. ---------------------------------------------------------------- BUS ID Model Revision Major Minor Disk Type Disk Name Device Path ------------ -------------------------------- ------------ ----- ----- ---------------- -------- ------------------------ Displaying All Multipath Devices devices on the system. ---------------------------------------------------------------- BUS ID Major Minor Disk Name Device Path ------------ ----- ----- -------- ------------------------ [0:0:0:0] 252 1 dm-1 /sys/block/dm-1 [0:0:0:0] 252 0 dm-0 /sys/block/dm-0 Displaying All iSCSI devices on the system. ---------------------------------------------------------------- Host Name Transport IP Address Port Connection Session Target Name ------------ ---------- ---------------- ------ ---------------- ------------ ---------------------------------------------------------------- host5 tcp 169.254.2.7 3260 connection3:0 session3 iqn.2015-12.com.oracleiaas:886afa56-8c9f-4501-8a9a-565d0b4b1c3e host3 tcp 169.254.2.5 3260 connection1:0 session1 iqn.2015-12.com.oracleiaas:86161095-2195-477e-bc62-613e4f526301 host6 tcp 169.254.2.8 3260 connection4:0 session4 iqn.2015-12.com.oracleiaas:bd6ee62a-f309-4122-8d1b-b7e5eba5dde1 host4 tcp 169.254.2.6 3260 connection2:0 session2 iqn.2015-12.com.oracleiaas:2031e68f-73fc-40e4-8ecd-d325c3feb34b
show sub-command for scsi-cli
The show sub-command of scsi-cli is a valuable option for users. This option provides detailed output hen used with options fc_port, disk, and iSCSI port. The user will then be presented with all the devices discovered by the tool, similar to the list sub-command. The output below shows details for each option used with the show sub-command.
Example output using the disk option:
# ./scsi-cli show disk sdb
------------------------------------------------
Show Details for sdb
------------------------------------------------
Vendor : ORACLE
Model Name : BlockVolume
Size : 2147483648 Sectors, 1 TiB
Disk Type : disk
Revision : 1.0
Disk Path : /sys/block/sdb
Max Sectors : 0
Range : 16
Extent Range : 256
Capability : 0
Queue Depth : 128
Queue Type : simple
CDL Enabled : 0
CDL Supported : 0
EH Timeout : 10
TimeOut : 30
DH State : detached
IO Done cnt : 0x4d5ae
IO error cnt : 0x3
IO Request cnt : 0x4d5ae
IO Counter bits: 32
IO Timeout : 0
WWID : naa.60ab31b77e1849919f0bc3e3c41dcba6
Alignment Offset : 0
Discard Alignment: 0
Scsi Device Event Notification
Capacity Change Reported : 0
Inquiry Change Reported : 0
LUN Change Reported : 0
Media Change : 0
Mode Parameter Change Reported : 0
Soft Threshold Reached : 0
------------------------------------------------
Queue Data
------------------------------------------------
Scheduler:
none [mq-deadline] kyber bfq
Physical Block Size : 4 KiB
Logical Block Size : 512 B
Minimum IO Size : 4 KiB
Optimal IO Size : 1 MiB
Zoned Append Max Bytes : 0 B
Zone Write Granularity : 0 B
Discard Max Bytes : 2 GiB
Discard max HW bytes : 2 GiB
Discard zeroes data : 0 B
Discard Granularity : 0 B
IO Poll : 0
IO Poll Delay : -1
stable_writes : 0
wbt_lat_usec : 75000
rq_affinity : 1
DAX : 0 add_random : 1
FUA : 0 nomerges : 0
Segments : 126 Segment Size : 64 KiB
Zoned : 0 nr_zones : 0
Readahead : 2048 Write Cache : write through
WriteSame : 0 B Write Zeroes : 0 B
Example output using the fc_port option to display information bout FC HBA ports:
# ./scsi-cli show fc_port host10
------------------------------------------------
Show Details for FC host10
------------------------------------------------
Adapter Name : QLE2742
Model Desc : QLogic 32Gb 2-port FC to PCIe Gen3 x8 Adapter
Host Name : host10
Node Name : 0x2000f4e9d45a442b
Port Name : 0x2100f4e9d45a442b
Fabric Name : 0xffffffffffffffff
Serial Number : RFD1917D59426
Port ID : 0x000002
Port State : Online
Port Type : Point-To-Point (direct nport connection)
Driver Attached : qla2xxx
Driver Version : 10.02.09.100-k
Firmware Version : 9.12.00 (d0d5)
Supported Speed : 8 Gbit, 16 Gbit, 32 Gbit
Supported Class : Class 3
Symbolic Name : QLE2742 FW:v9.12.00 DVR:v10.02.09.100-k
Dev Loss Timeout : 30
Link State : Link Up - N_Port to N_Port
Active Mode : Initiator
Max NPIV VPorts : 254
In Use NPIV Port : 0
R-Ports Found : 1
Rport Rport_State Roles Rport_Name Node_Name Rport_ID
---------------- ---------------- -------------------------------- -------------------------------- -------------------------------- ----------------
rport-10:0-2 Online FCP Initiator 0x2100f4e9d45a442f 0x2000f4e9d45a442f 0x000001
Example output using the iscsi option:
# ./scsi-cli show iscsi host4
----------------------------------------------------------------
Displaying Session Information for: session2
----------------------------------------------------------------
Target Name : iqn.2015-12.com.oracleiaas:2031e68f-73fc-40e4-8ecd-d325c3feb34b
Target State : SCANNED
Session State : LOGGED_IN
Abort Timeout : 15
Creator ID : 2539
Data PDU In Order : 1
Data SEQ In Order : 1
Error Recovery Level : 0
Fast Abort : 1
First Burst Len : 65536
ifacename : default
immediate Data : 1
Initial R2T : 1
Lun Reset Timeout : 30
Max Burst Len : 262144
Max Outstanding R2T : 1
Recovery Timeout : 6000
----------------------------------------------------------------
Displaying HOST Information for: host4
----------------------------------------------------------------
Scsi04 channel: 00 Id: 00 Lun: 00
Attached Scsi Disk: State:
Scsi04 channel: 00 Id: 00 Lun: 01
Attached Scsi Disk: sdd State: running
----------------------------------------------------------------
Displaying Connection Information for: connection2:0
----------------------------------------------------------------
State : up
Port : 3260
Ping Timeout : 0
Recv Timeout : 0
Persistent Addr : 169.254.2.6i
Persistent Port : 3260
Header Digest : 0
Data Digest : 0
Expected StatSN : 163
Max Recv datalen : 262144
Max Xmit datalen : 8192
Address : 169.254.2.6
stats sub-command for scsi-cli
Statistics collection for various SCSI devices is a very cumbersome task for a novice user, especially when the user does not know which SysFS hook to use for the echo command. Also, when the resulting output is presented to the user, it needs to be explained by an expert. For example, a fiber channel device has a SysFS hook in the statistics display that helps users determine if a port has been dropping packets.
When the stats sub-command is issued without any other option, the tool will display sub-commands supported by stats sub-command.
# ./scsi-cli stats Following Sub-Commands are supported for 'stats' ---------------------------------------------------------------- disk Show statistics for block device fc_port Show statistics for Fiber Channel port
The User will then choose an appropriate fiber channel port/iSCSI session or a block disk to get statistics for.
Here’s an example of FC port statistics:
# ./scsi-cli stats fc_port host10
------------------------------------------------
FCP Statistics: host10
------------------------------------------------
TX Frames : 0x12
TX Words : 0x1da
RX Frames : 0x12
RX Words : 0x1c8
LIP Count : 0
NOS Count : 0x5
Error Frames : 0
Dumped Frames : 0
Invald TX Word : 0
Loss of Sync : 0x2
Loss of Signal : 0x2
Link Failure : 0x2
Invalid CRC : 0
----- FPIN Diagnostic Statistics -----
fpin_dn : 0
fpin_dn_device_specific : 0
fpin_dn_timeout : 0
fpin_dn_unable_to_route : 0
fpin_dn_unknown : 0
----- FPIN Link Integrity Statistics -----
fpin_li : 0
fpin_li_device_specific : 0
fpin_li_failure_unknown : 0
fpin_li_invalid_crc_count : 0
fpin_li_invalid_tx_word_count : 0
fpin_li_link_failure_count : 0
fpin_li_loss_of_signals_count : 0
fpin_li_loss_of_sync_count : 0
fpin_li_prim_seq_err_count : 0
----- FPIN Congestion Statistics -----
cn_sig_alarm : 0x7fffffffffffffff
cn_sig_warn : 0x7fffffffffffffff
fpin_cn : 0
fpin_cn_device_specific : 0
fpin_cn_credit_stall : 0
fpin_cn_lost_credits : 0
fpin_cn_oversubscription : 0
From the output above, a value of 0 in Dropped Frame indicates that there were no dropped frames on this port.
Conclusion
Over the years, SCSI device management has become very custom to a specific environment and depends mainly on custom scripts that users replicate. It also needs intimate knowledge of the SCSI device’s working and SYSFS file systems. Various hardware manufacturers have custom utilities that only work with their hardware. For a customer with a mixed hardware setup, the user must download different utilities for all the hardware vendor devices installed their environment. These utilities also present the challenge of having to learn about each different command. scsi-cli aims to simplify this aspect of device management with a simple human language-friendly command.