As NVMe over Fabrics implementations are increasingly being deployed in cloud environments, security implications are becoming very important. The following are some security capabilities that are defined by the NVMe Specification:

  • Fabric Secure Channel (encryption of traffic between host and controller) can be used with NVMe-tcp/TLS (Transport Later Security). Currently, NVMe-tcp/TLS isn’t supported by the upstream Linux kernel or Oracle Linux UEK, but is expected to be a future capability. Until then, IPsec can be setup that will encrypt host-controller traffic. IPsec, which is independent and transparent to NVMe over Fabrics, must be set up before configuring the NVMe over Fabrics configuration.
  • NVMe in-band Authentication has been added to the NVMe connect protocol which provides a challenge-response identify authentication protocol that uses a “shared secret” and doesn’t require the transmission of a password between the host and controller. Authentication doesn’t require a secure channel to be safely completed.

This blog discusses the NVMe In-band Authentication capability that will be available with Oracle Linux 9 with UEK7U2 and how to use it.

Authentication Protocol

As part of the normal NVMe over Fabrics initialization, an NVMe Connect is sent from the host to the controller. In the case of NVMe-tcp, the NVMe Connect happens after the TCP session is established. If the controller requires authentication, the Connect response includes a non-zero AUTHRQ field. The host is then required to initiate the authentication protocol using Authentication Send and Authentication Receive NVMe commands.

Authentication Example

NVMe Specification defines DH-HMAC-CHAP as the authentication protocol which is an enhanced version of CHAP that iSCSI uses.

  • CHAP (Challenge Handshake Authentication Protocol) is the base challenge-response identity authentication protocol that uses a “shared secret” between the requestor (host) and the authenticator (controller).

  • DH (Diffie-Hellman) key exchange is built on top of CHAP and provides for a more secure key exchange than the base CHAP and can provide a second exchange where the host can authenticate the controller (Bidirectional Authentication). The size of the key (in bits) is defined by the DH Group and is set by the target. Using a larger key size results in a more secure key. The DH part of the protocol is optional for NVMe. See RFC 7919 for more information.

     

    DH Group ID
    DH Group Size
    dhchap_dhgroup
    Linux target attribute
    00h
    NULL
    0 (default)
    01h
    2048-bit
    “ffdhe2048”
    02h
    3072-bit
    “ffdhe3072”
    03h
    4096-bit
    “ffdhe4096”
    04h
    6144-bit
    “ffdhe6144”
    05h
    8192-bit
    “ffdhe8192”

     

  • HMAC (Hashed Message Authentication Code) is required and provides more robust hash functions than the base CHAP by executing a hash function on the data that’s to be authenticated. The Hash Function is setup by the target. See RFC 6234 for more information.

    Hash Function ID
    Hash Function
    dhchap_hash
    Linux target attribute
    01h
    SHA-256
    “hmac(sha256)”
    02h
    SHA-384
    “hmac(sha384)”
    03h
    SHA-512
    “hmac(sha512)”

     

After authentication is successful, the connect is complete and queues can be set up.

Linux Components

The NVMe In-Band Authentication capability is made up of three components, 1) the userland nvme-cli package, 2) the kernel nvme host driver changes that supports the client/host portion of the authentication and 3) the kernel nvme target driver changes that allows the setting up of a NVMe Fabrics target with authentication.

The NVMe In-Band Authentication feature is made up of three components:

  1. The linux kernel NVMe Host driver changes that include the client/host part of the authentication.
  2. The linux kernel NVMe Target driver changes that include the code to set up an NVMe Fabrics target with authentication.
  3. A version of the nvme-cli user space utility that includes the code to handle NVMe In-Band Authentication.

nvme-cli Package

NVMe In-Band Authentication functionality was added to nvme-cli for version 2.0. The nvme-cli package shipped with Oracle Linux 9 is at version 2.2.3 and can be used with UEK7U2 for this functionality.

nvme-cli In-Band Authentication commands:

  • nvme gen-dhchap-key Generates a DH-HMAC-CHAP host key usable for NVMe In-Band Authentication. Options are described in the table following.

    Option
    Description
    –secret=
    Optional secret value (in hexadecimal) to be used for the key.
    If not specified, the key will be randomly created.
    –key-length=
    Length of the resulting key. Possible values are 32, 48, or 64.
    –nqn=
    Host-NQN to be used for the transformation. This parameter is only valid
    if a non-zero HMAC function has been specified.
    –hmac=
    Select a HMAC algorithm to use. Possible values are: 0 – No HMAC algorithm,
    1 – SHA-256, 2 – SHA-384, 3 – SHA-512

     

  • nvme check-dhchap-key Validates an NVMeoF DH-HMAC-CHAP host key.

    Options are described in the table following.

    Option
    Description
    –key=
    Key to validate.

     

nvme-cli In-Band Authentication connect options:

The following table describes the In-Band Authentication options when using the nvme connect command.

Option
Description
–dhchap-secret=
user-defined dhchap key 
–dhchap-ctrl-secret=
user-defined dhchap controller key (for bi-directional authentication)

 

NVMe Target Driver

The NVMe Target driver is used for creating a NVMe over Fabrics target on a Linux host. As part of initializing the target, DHCHAP keys, DH Group, and HMAC function can be set up for the target controller.

By default, the NVMe In-Band Authentication functionality is compiled into the UEK7U2 NVMe Target Driver, but if this functionality is required in an upstream kernel the CONFIG_NVME_TARGET_AUTH parameter must be enabled when compiling the upstream kernel.

The following table describes the authentication related sysfs attributes for the target.

sysfs Attribute
Description
/sys/kernel/config/nvmet/hosts/${hostnqn}/dhchap_key
Host key used for authentication.
/sys/kernel/config/nvmet/hosts/${hostnqn}/dhchap_ctrl_key
Controller key used for authentication
/sys/kernel/config/nvmet/hosts/${hostnqn}/dhchap_hash
HMAC hash used. Valid values:

* hmac(sha256) (default)
* hmac(sha384)
* hmac(sha512)
/sys/kernel/config/nvmet/hosts/${hostnqn}/dhchap_dhgroup
DH Group used. Valid values:

* null (default)
* ffdhe2048
* ffdhe3072
* ffdhe4096
* ffdhe6144
* ffdhe8192

 

NVMe Host Driver

The NVMe Host Driver is responsible for driving the host/client side of the DH-HMAC-CHAP protocol when the initial connect returns a non-zero AUTHRQ. The host and ctrl keys used for authentication are passed to the driver from the nvme connect command. By default, the NVMe In-Band Authentication functionality is compiled into the UEK7U2 NVMe Host Driver, but if this functionality is required in an upstream kernel the CONFIG_NVME_AUTH parameter must be enabled when compiling the upstream kernel.

After successful connect, the following sysfs attributes are made available on the host/client:

  • /sys/class/nvme/nvme1/dhchap_secret
  • /sys/class/nvme/nvme1/dhchap_ctrl_secret

Examples

Setting up a NVMe-tcp Target with Uni-Directional Authentication

  • hostid is the uuid of the connecting host
  • subsystem is the NVMe subsystem name
  • hostkey is set to the host shared secret, note that it’s generated using the nvme gen-dhchap-key command in the example following.
# hostid=
# subsystem=
# hostkey=`nvme gen-dhchap-key -n ${subsystem}`
# modprobe nvmet
# modprobe nvmet-tcp
# mkdir /sys/kernel/config/nvmet/subsystems/${subsystem}
# cd /sys/kernel/config/nvmet/subsystems/${subsystem}
# echo 0 > attr_allow_any_host
# mkdir namespaces/1
# cd namespaces/1/
# echo /dev/nvme0n1 > device_path  # null_blk device can be used for test purposes
# echo 1 > enable
# mkdir /sys/kernel/config/nvmet/ports/1
# cd /sys/kernel/config/nvmet/ports/1
# echo 10.129.136.200 > addr_traddr
# echo tcp > addr_trtype
# echo 4420 > addr_trsvcid
# echo ipv4 > addr_adrfam
# hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}"
# cd /sys/kernel/config/nvmet/hosts
# mkdir $hostnqn
# cd $hostnqn
# echo $hostkey > dhchap_key
# ln -s "/sys/kernel/config/nvmet/hosts/${hostnqn}" \
"/sys/kernel/config/nvmet/subsystems/${subsystem}/allowed_hosts/${hostnqn}"
# ln -s "/sys/kernel/config/nvmet/subsystems/${subsystem}"  \
"/sys/kernel/config/nvmet/ports/1/subsystems/${subsystem}"
#

Connecting to the Target (Uni-Directional Authentication)

  • hostkey is set to the host shared secret, this must match the key generated in the previous example.
  • subsystem is the NVMe subsystem name
# hostkey=
# subsystem=
# hostid=`cat /etc/nvme/hostid`
# modprobe nvme
# modprobe nvme-tcp
# hostid=`cat /etc/nvme/hostid`
# nvme discover -t tcp -a 10.129.136.200 -s 4420
Discovery Log Number of Records 1, Generation counter 2
=====Discovery Log Entry 0======
trtype: tcp
adrfam: ipv4
subtype: current discovery subsystem
treq: not specified, sq flow control disable supported
portid: 1 trsvcid: 4420
subnqn: nqn.2014-08.org.nvmexpress.discovery
traddr: 10.129.136.200
eflags: not specified
sectype: none
# nvme connect -t tcp -n ${subsystem} -a 10.129.136.200 -s 4420 \
--hostnqn=nqn.2014-08.org.nvmexpress:uuid:$hostid --hostid=$hostid --dhchap-secret=$hostkey
#

Connecting to the Target with no key

  • The connect fails if you try to connect to a target without a key or with an improper key if the target has been set up with authentication. The failure is illustrated in the example.
  • subsystem is the NVMe subsystem name
# subsystem=
# hostid=`cat /etc/nvme/hostid`
# nvme connect -t tcp -n ${subsystem} -a 10.129.136.200 -s 4420 \
--hostnqn=nqn.2014-08.org.nvmexpress:uuid:$hostid --hostid=$hostid

no controller found: failed to write to nvme-fabrics device
#

Setting up a NVMe-tcp Target with Bi-Directional Authentication

  • First, set up target with Unidirectional Authentication (see first example).
  • hostid is the uuid of the connecting host.
  • subsystem is the NVMe subsystem name.
  • ctrlkey is set to the controller shared secret, note that it’s generated using the nvme gen-dhchap-key command in the example following.
# hostid=
# subsystem=
# ctrlkey=`nvme gen-dhchap-key -n ${subsystem}`
# hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}"
# cd /sys/kernel/config/nvmet/hosts
# cd $hostnqn
# echo $ctrlkey > dhchap_ctrl_key
#

Setting up a NVMe-tcp Target to use a HMAC Hash Function and DH Group

  • First, setup target with Unidirectional Authentication, then Bidirectional Authentication (see first example).
  • hostid is the uuid of the connecting host.
  • hmac is set to a hash function: “hmac(sha256)” or “hmac(sha384)” or “hmac(sha512)”.
  • dhgroup is set to: “ffdhe2048” or “ffdhe3072” or “ffdhe4096” or “ffdhe6144” or “ffdhe8192”.
# hmac=
# dhgroup=
# hostnqn="nqn.2014-08.org.nvmexpress:uuid:${hostid}"
# cd /sys/kernel/config/nvmet/hosts/${hostnqn}
# echo ${hmac} > dhchap_hash
# echo ${dhgroup} > dhchap_dhgroup
#

Connecting to the Target (Bi-Directional Authentication)

  • subsystem is the NVMe subsystem name.
  • hostkey is set to the host shared secret.
  • ctrlkey is set to the controller shared secret.
# subsystem=
# hostid=`cat /etc/nvme/hostid`
# nvme connect -t tcp -n $`{subsystem} -a 10.129.136.200 -s 4420 \
--hostnqn=nqn.2014-08.org.nvmexpress:uuid:$hostid --hostid=$hostid \
--dhchap-secret=$hostkey --dhchap-ctrl-secret=$ctrlkey
#

References

NVM Express Base Specification, Revision 2.0c, October 4, 2022. Available from https://nvmexpress.org/specifications/.

RFC 7919, D. Gillmor, “Negotiated Finite Field Diffie-Hellman Ephemeral Parameters for Transport Layer Security (TLS)”, August 2016. Available from https://www.ietf.org/rfc.html.

RFC 6234, D. Eastlake 3rd, and T. Hansen, “US Secure Hash Algorithms (SHA and SHA-based HMAC and HKDF)”, May 2011. Available from https://www.ietf.org/rfc.html.