LIO (SCSI target)

Last updated
LIO Target
Original author(s) Nicholas Bellinger
Jerome Martin
Developer(s) Datera, Inc.
Initial releaseJanuary 14, 2011 (2011-01-14)
Repository github.com/open-iscsi
Written in C, Python
Operating system Linux
Type Block storage
License GNU General Public License
Website linux-iscsi.org

The Linux-IOTarget (LIO) is an open-source Small Computer System Interface (SCSI) target implementation included with the Linux kernel. [1]

Contents

Unlike initiators, which begin sessions, LIO functions as a target, presenting one or more Logical Unit Numbers (LUNs) to a SCSI initiator, receiving SCSI commands, and managing the input/output data transfers. [2]

LIO supports a wide range of storage protocols and transport fabrics, including but not limited to Fibre Channel over Ethernet (FCoE), Fibre Channel, IEEE 1394 and iSCSI. [3]

It is utilized in several Linux distributions and is a popular choice for cloud environments due to its integration with tools like QEMU/KVM, libvirt, and OpenStack. [4]

The LIO project is maintained by Datera, Inc. ,[ dubious – discuss ] a Silicon Valley-based storage solutions provider. On January 15, 2011, LIO was merged into the Linux kernel mainline with version 2.6.38, which was officially released on March 14, 2011. [5] [6] Subsequent versions of the Linux kernel have introduced additional fabric modules to expand its compatibility.[ citation needed ]

LIO competes with other SCSI target modules in the Linux ecosystem. The SCSI Target Framework (SCST) [7] is a prominent alternative for general SCSI target functionality, while for iSCSI-specific targets, the older iSCSI Enterprise Target (IET) and SCSI Target Framework (STGT) also have industry adoption. [8] [9]

Background

The SCSI standard provides an extensible semantic abstraction for computer data storage devices, and is used with data storage systems. The SCSI T10 standards [10] define the commands [11] and protocols of the SCSI command processor (sent in SCSI CDBs), and the electrical and optical interfaces for various implementations.

A SCSI initiator is an endpoint that initiates a SCSI session. A SCSI target is the endpoint that waits for initiator commands and executes the required I/O data transfers. The SCSI target usually exports one or more LUNs for initiators to operate on.

The LIO Linux SCSI Target implements a generic SCSI target that provides remote access to most data storage device types over all prevalent storage fabrics and protocols. LIO neither directly accesses data nor does it directly communicate with applications.

Architecture

LIO Architecture LIO-Architecture.png
LIO Architecture

LIO implements a modular and extensible architecture around a parallelised SCSI command processing engine. [12]

The LIO SCSI target engine is independent of specific fabric modules or backstore types. Thus, LIO supports mixing and matching any number of fabrics and backstores at the same time. The LIO SCSI target engine implements a comprehensive SPC-3/SPC-4 [13] feature set with support for high-end features, including SCSI-3/SCSI-4 Persistent Reservations (PRs), SCSI-4 Asymmetric Logical Unit Assignment (ALUA), VMware vSphere APIs for Array Integration (VAAI), [14] T10 DIF, etc.

LIO is configurable via a configfs-based [15] kernel API, and can be managed via a command-line interface and API ( targetcli ).

SCSI target

The concept of a SCSI target isn't restricted to physical devices on a SCSI bus, but instead provides a generalised model for all receivers on a logical SCSI fabric. This includes SCSI sessions across interconnects with no physical SCSI bus at all. Conceptually, the SCSI target provides a generic block storage service or server in this scenario.

Back-stores

Back-stores provide the SCSI target with generalised access to data storage devices by importing them via corresponding device drivers. Back-stores do not need to be physical SCSI devices.

The most important back-store media types are:

As a result, LIO provides a generalised model to export block storage.

Fabric modules

Fabric modules implement the front-end of the SCSI target by encapsulating and abstracting the properties of the various supported interconnect. The following fabric modules are available.

FCoE

Combined storage and local area network Storage FCoE.png
Combined storage and local area network

The Fibre Channel over Ethernet (FCoE) fabric module allows the transport of Fibre Channel protocol (FCP) traffic across lossless Ethernet networks. The specification, supported by a large number of network and storage vendors, is part of the Technical Committee T11 FC-BB-5 standard. [16]

LIO supports all standard Ethernet NICs.

The FCoE fabric module was contributed by Cisco and Intel, and released with Linux 3.0 on July 21, 2011. [17]

Fibre Channel

Fibre Channel is a high-speed network technology primarily used for storage networking. It is standardized in the Technical Committee T11 [18] of the InterNational Committee for Information Technology Standards (INCITS).

The QLogic Fibre Channel fabric module supports 4- and 8-gigabit speeds with the following HBAs:

  • QLogic 2400 Series (QLx246x), 4GFC
  • QLogic 2500 Series (QLE256x), 8GFC (fully qual'd)

The Fibre Channel fabric module [19] and low-level driver [20] (LLD) were released with Linux 3.5 on July 21, 2012. [21]

With Linux 3.9, the following QLogic HBAs and CNAs are also supported:

  • QLogic 2600 Series (QLE266x), 16GFC, SR-IOV
  • QLogic 8300 Series (QLE834x), 16GFS/10 GbE, PCIe Gen3 SR-IOV
  • QLogic 8100 Series (QLE81xx), 8GFC/10 GbE, PCIe Gen2

This makes LIO the first open source target to support 16-gigabit Fibre Channel.

IEEE 1394

LIO Firewire Target for Mac OS X LIO-Firewire-Target.png
LIO Firewire Target for Mac OS X

The FireWire SBP-2 fabric module enables Linux to export local storage devices via IEEE 1394, so that other systems can mount them as an ordinary IEEE 1394 storage device.

IEEE 1394 is a serial-bus interface standard for high-speed communications and isochronous real-time data transfer. It was developed by Apple as "FireWire" in the late 1980s and early 1990s, and Macintosh computers have supported "FireWire target disk mode" since 1999. [22]

The FireWire SBP-2 fabric module was released with Linux 3.5 on July 21, 2012. [21] [23]

iSCSI

The Internet Small Computer System Interface (iSCSI) fabric module allows the transport of SCSI traffic across standard IP networks.

By carrying SCSI sessions across IP networks, iSCSI is used to facilitate data transfers over intranets and manage storage over long distances. iSCSI can be used to transmit data over local area networks (LANs), wide area networks (WANs), or the Internet, and can enable location-independent and location-transparent data storage and retrieval.

The LIO iSCSI fabric module also implements a number of advanced iSCSI features that increase performance and resiliency, such as Multiple Connections per Session (MC/S) and Error Recovery Levels 0-2 (ERL=0,1,2).

LIO supports all standard Ethernet NICs.

The iSCSI fabric module was released with Linux 3.1 on October 24, 2011. [24]

iSER

Networks supporting remote direct memory access (RDMA) can use the iSCSI Extensions for RDMA (iSER) fabric module to transport iSCSI traffic. iSER permits data to be transferred directly into and out of remote SCSI computer memory buffers without intermediate data copies (direct data placement or DDP) by using RDMA. [25] RDMA is supported on InfiniBand networks, on Ethernet with data center bridging (DCB) networks via RDMA over Converged Ethernet (RoCE), and on standard Ethernet networks with iWARP enhanced TCP offload engine controllers.

The iSER fabric module was developed together by Datera and Mellanox Technologies, and first released with Linux 3.10 on June 30, 2013. [26]

SRP

The SCSI RDMA Protocol (SRP) fabric module allows the transport of SCSI traffic across RDMA (see above) networks. As of 2013, SRP was more widely used than iSER, although it is more limited, as SCSI is only a peer-to-peer protocol, whereas iSCSI is fully routable. The SRP fabric module supports the following Mellanox host channel adapters (HCAs):

  • Mellanox ConnectX-2 VPI PCIe Gen2 HCAs (x8 lanes), single/dual-port QDR 40 Gbit/s
  • Mellanox ConnectX-3 VPI PCIe Gen3 HCAs (x8 lanes), single/dual-port FDR 56 Gbit/s
  • Mellanox ConnectX-IB PCIe Gen3 HCAs (x16 lanes), single/dual-port FDR 56 Gbit/s

The SRP fabric module was released with Linux 3.3 on March 18, 2012. [27]

In 2012, c't magazine measured almost 5000 MB/s throughput with LIO SRP Target over one Mellanox ConnectX-3 port in 56 Gbit/s FDR mode on a Sandy Bridge PCI Express 3.0 system with four Fusion-IO ioDrive PCI Express flash memory cards.

USB

The USB Gadget fabric module enables Linux to export local storage devices via the Universal Serial Bus (USB), so that other systems can mount them as an ordinary storage device.

USB was designed in the mid-1990s to standardize the connection of computer peripherals, and has also become common for data storage devices.

The USB Gadget fabric module was released with Linux 3.5 on July 21, 2012. [28]

Targetcli

targetcli is a user space single-node management command line interface (CLI) for LIO. [29] It supports all fabric modules and is based on a modular, extensible architecture, with plug-in modules for additional fabric modules or functionality.

targetcli provides a CLI that uses an underlying generic target library through a well-defined API. Thus the CLI can easily be replaced or complemented by a UI with other metaphors, such as a GUI.

targetcli is implemented in Python and consists of three main modules:

Detailed instructions on how to set up LIO targets can be found on the LIO wiki. [29]

Linux distributions

targetcli and LIO are included in most Linux distributions per default. Here is an overview of the most popular ones, together with the initial inclusion dates:

DistributionVersion [a] ReleaseArchiveInstallationSource gitDocumentation
Alpine Linux 2.52011-11-07 Alpine Linux mirror apk add targetcli-fb targetcli-fb.git How-to
CentOS 6.22011-12-20 CentOS mirror su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
Debian 7.0 ("wheezy")2013-05-04 Debian pool su -c 'apt-get install targetcli' targetcli.git LIO Wiki
Fedora 16 2011-11-08 Fedora Rawhide su -c 'yum install targetcli' targetcli-fb.git Target Wiki
openSUSE 12.12011-11-08Requires manual installation from Datera targetcli.git repos.
RHEL [b] 6.22011-11-16 Fedora Rawhide su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
Scientific Linux 6.2 2012-02-16 SL Mirror su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
SLES 11 SP3 MR2013-12-su -c 'zypper in targetcli' targetcli.git LIO Wiki
Ubuntu 12.04 LTS (precise) 2012-04-26 Ubuntu universe sudo apt-get install targetcli targetcli.git LIO Wiki

See also

Notes

  1. ↑ The distribution release where LIO was included first.
  2. ↑ RHEL 6 included LIO, but it was the default only for FCoE targets, while STGT was used for iSCSI. In RHEL 7 beta, LIO is the default for FCoE, iSCSI, and for Mellanox InfiniBand iSER/SRP. [31]

Related Research Articles

<span class="mw-page-title-main">SCSI</span> Set of computer and peripheral connection standards

Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices, best known for its use with storage devices such as hard disk drives. SCSI was introduced in the 1980s and has seen widespread use on servers and high-end workstations, with new SCSI standards being published as recently as SAS-4 in 2017.

Internet Small Computer Systems Interface or iSCSI is an Internet Protocol-based storage networking standard for linking data storage facilities. iSCSI provides block-level access to storage devices by carrying SCSI commands over a TCP/IP network. iSCSI facilitates data transfers over intranets and to manage storage over long distances. It can be used to transmit data over local area networks (LANs), wide area networks (WANs), or the Internet and can enable location-independent data storage and retrieval.

<span class="mw-page-title-main">InfiniBand</span> Network standard

InfiniBand (IB) is a computer networking communications standard used in high-performance computing that features very high throughput and very low latency. It is used for data interconnect both among and within computers. InfiniBand is also used as either a direct or switched interconnect between servers and storage systems, as well as an interconnect between storage systems. It is designed to be scalable and uses a switched fabric network topology. Between 2014 and June 2016, it was the most commonly used interconnect in the TOP500 list of supercomputers.

Fibre Channel (FC) is a high-speed data transfer protocol providing in-order, lossless delivery of raw block data. Fibre Channel is primarily used to connect computer data storage to servers in storage area networks (SAN) in commercial data centers.

<span class="mw-page-title-main">Host adapter</span> Computer hardware device

In computer hardware a host controller, host adapter or host bus adapter (HBA) connects a computer system bus which acts as the host system to other network and storage devices. The terms are primarily used to refer to devices for connecting SCSI, SAS, NVMe, Fibre Channel and SATA devices. Devices for connecting to FireWire, USB and other devices may also be called host controllers or host adapters.

sysfs is a pseudo file system provided by the Linux kernel that exports information about various kernel subsystems, hardware devices, and associated device drivers from the kernel's device model to user space through virtual files. In addition to providing information about various devices and kernel subsystems, exported virtual files are also used for their configuration.

<span class="mw-page-title-main">QLogic</span> American network device manufacturer

QLogic Corporation was an American manufacturer of networking server and storage networking connectivity and application acceleration products, based in Aliso Viejo, California through 2016. QLogic's products include Fibre Channel adapters, converged network adapters for Fibre Channel over Ethernet (FCoE), Ethernet network interface controllers, iSCSI adapters, and application-specific integrated circuits (ASICs). It was a public company from 1992 to 2016.

<span class="mw-page-title-main">IBM BladeCenter</span> Blade server architecture by IBM

The IBM BladeCenter was IBM's blade server architecture, until it was replaced by Flex System in 2012. The x86 division was later sold to Lenovo in 2014.

The iSCSI Extensions for RDMA (iSER) is a computer network protocol that extends the Internet Small Computer System Interface (iSCSI) protocol to use Remote Direct Memory Access (RDMA). RDMA can be provided by the Transmission Control Protocol (TCP) with RDMA services (iWARP), which uses an existing Ethernet setup and therefore has lower hardware costs, RoCE, which does not need the TCP layer and therefore provides lower latency, or InfiniBand. iSER permits data to be transferred directly into and out of SCSI computer memory buffers without intermediate data copies and with minimal CPU involvement.

<span class="mw-page-title-main">OpenFabrics Alliance</span> Organization

The OpenFabrics Alliance is a non-profit organization that promotes remote direct memory access (RDMA) switched fabric technologies for server and storage connectivity. These high-speed data-transport technologies are used in high-performance computing facilities, in research and various industries.

<span class="mw-page-title-main">Network block device</span> Network storage protocol

On Linux, network block device (NBD) is a network protocol that can be used to forward a block device from one machine to a second machine. As an example, a local machine can access a hard disk drive that is attached to another computer.

<span class="mw-page-title-main">Fibre Channel over Ethernet</span> Computer network technology

Fibre Channel over Ethernet (FCoE) is a computer network technology that encapsulates Fibre Channel frames over Ethernet networks. This allows Fibre Channel to use 10 Gigabit Ethernet networks while preserving the Fibre Channel protocol. The specification was part of the International Committee for Information Technology Standards T11 FC-BB-5 standard published in 2009. FCoE did not see widespread adoption.

In computing the SCSI RDMA Protocol (SRP) is a protocol that allows one computer to access SCSI devices attached to another computer via remote direct memory access (RDMA). The SRP protocol is also known as the SCSI Remote Protocol. The use of RDMA makes higher throughput and lower latency possible than what is generally available through e.g. the TCP/IP communication protocol.

<span class="mw-page-title-main">Storage area network</span> Network which provides access to consolidated, block-level data storage

A storage area network (SAN) or storage network is a computer network which provides access to consolidated, block-level data storage. SANs are primarily used to access data storage devices, such as disk arrays and tape libraries from servers so that the devices appear to the operating system as direct-attached storage. A SAN typically is a dedicated network of storage devices not accessible through the local area network (LAN).

IBM Storwize systems were virtualizing RAID computer data storage systems with raw storage capacities up to 32 PB. Storwize is based on the same software as IBM SAN Volume Controller (SVC).

<span class="mw-page-title-main">Dell M1000e</span> Server computer

The Dell blade server products are built around their M1000e enclosure that can hold their server blades, an embedded EqualLogic iSCSI storage area network and I/O modules including Ethernet, Fibre Channel and InfiniBand switches.

<span class="mw-page-title-main">Converged network adapter</span> Computer input/output device

A converged network adapter (CNA), also called a converged network interface controller (C-NIC), is a computer input/output device that combines the functionality of a host bus adapter (HBA) with a network interface controller (NIC). In other words, it "converges" access to, respectively, a storage area network and a general-purpose computer network.

RDMA over Converged Ethernet (RoCE) is a network protocol which allows remote direct memory access (RDMA) over an Ethernet network. There are multiple RoCE versions. RoCE v1 is an Ethernet link layer protocol and hence allows communication between any two hosts in the same Ethernet broadcast domain. RoCE v2 is an internet layer protocol which means that RoCE v2 packets can be routed. Although the RoCE protocol benefits from the characteristics of a converged Ethernet network, the protocol can also be used on a traditional or non-converged Ethernet network.

<span class="mw-page-title-main">SCST</span>

SCST is a GPL licensed SCSI target software stack. The design goals of this software stack are high performance, high reliability, strict conformance to existing SCSI standards, being easy to extend and easy to use. SCST does not only support multiple SCSI protocols but also supports multiple local storage interfaces and also storage drivers implemented in user-space via the scst_user driver.

Enterprise Storage OS, also known as ESOS, is a Linux distribution that serves as a block-level storage server in a storage area network (SAN). ESOS is composed of open-source software projects that are required for a Linux distribution and several proprietary build and install time options. The SCST project is the core component of ESOS; it provides the back-end storage functionality.

References

  1. ↑ Target. 323328-001. linux-iscsi.org. 2012-10-23. Retrieved 2012-12-25.
  2. ↑ RFC 7144 - Internet Small Computer System Interface (iSCSI) Requirements and Implications for the Data Center (April 2014)
  3. ↑ RFC 7144 - Internet Small Computer System Interface (iSCSI) Requirements and Implications for the Data Center (April 2014)
  4. ↑ "A tale of two SCSI targets". Lwn.net. Archived from the original on 2014-02-02. Retrieved 2014-01-20.
  5. ↑ Linus Torvalds (2011-01-14). "Trivial merge". Kernel.org . Retrieved 2019-09-28.
  6. ↑ Thorsten Leemhuis (2011-03-02). "Kernel Log: Coming in 2.6.38 (Part 4) - Storage". Heise Online.
  7. ↑ "A tale of two SCSI targets". Lwn.net. Archived from the original on 2014-02-02. Retrieved 2014-01-20.
  8. ↑ Haas, Florian (May 2012). "Replicate Everything! Highly Available iSCSI Storage with DRBD and Pacemaker". Linux Journal. Archived from the original on 2014-01-20. Retrieved 2019-09-28.
  9. ↑ Bolkhovitin, Vladislav (2018-04-11). "SCST vs STGT". Generic SCSI Target Subsystem for Linux. Retrieved 2019-04-01.
  10. ↑ Technical Committee T10. "SCSI Storage Interfaces". t10.org. Retrieved 2012-12-24.{{cite web}}: CS1 maint: numeric names: authors list (link)
  11. ↑ SCSI Commands Reference Manual (PDF). 100293068, Rev. C. Scotts Valley: Seagate Technology. April 2010. Archived from the original (PDF) on 2012-07-31. Retrieved 2012-12-25.
  12. ↑ Bellinger, Nicholas (2009). Current Status and Future of iSCSI on the Linux platform (PDF). Linux Plumbers Conference.
  13. ↑ Ralph Weber (2011-01-17). "SCSI Primary Commands - 4 (SPC-4)". t10.org. Retrieved 2011-03-07.
  14. ↑ LIO Linux SCSI Target (2012-12-23). "VAAI". linux-iscsi.org. Retrieved 2012-12-25.
  15. ↑ Jonathan Corbet (2005-08-24). "Configfs - an introduction". lwn.net. Retrieved 2011-03-07.
  16. ↑ "Fibre Channel: Backbone - 5 revision 2.00" (PDF). American National Standard for Information Technology International Committee for Information Technology Standards Technical Group T11. June 4, 2009. Retrieved 2011-05-05.
  17. ↑ Linus Torvalds (2011-04-18). "[SCSI] tcm_fc: Adding FC_FC4 provider (tcm_fc) for FCoE target (TCM - target core) support". Kernel.org. Retrieved 2019-09-28.
  18. ↑ "T11 Home Page". t11.org. Retrieved 2012-12-25.
  19. ↑ Linus Torvalds (2012-05-15). "[SCSI] tcm_qla2xxx: Add >= 24xx series fabric module for target-core". Kernel.org. Retrieved 2019-09-28.
  20. ↑ Linus Torvalds (2012-05-15). "[SCSI] qla2xxx: Add LLD target-mode infrastructure for >= 24xx series". Kernel.org. Retrieved 2019-09-28.
  21. 1 2 Thorsten Leemhuis (2012-07-03). "Kernel Log: Coming in 3.5 (Part 2) - Filesystems and storage". Heise Online. Retrieved 2013-01-14.
  22. ↑ "How to use and troubleshoot FireWire target disk mode". apple.com. Retrieved 2012-12-24.
  23. ↑ Linus Torvalds (2012-04-15). "sbp-target: Initial merge of firewire/ieee-1394 target mode support". Kernel.org. Retrieved 2019-09-28.
  24. ↑ Linus Torvalds (2011-07-27). "iSCSI merge". Kernel.org. Retrieved 2019-09-28.
  25. ↑ RFC   5041
  26. ↑ Linus Torvalds (2013-04-30). "Merge branch 'for-next-merge'". Kernel.org. Retrieved 2019-09-28.
  27. ↑ Linus Torvalds (2012-01-18). "InfiniBand/SRP merge". Kernel.org. Retrieved 2019-09-28.
  28. ↑ "Merge branch 'usb-target-merge'". Kernel.org. Retrieved 2019-09-28.
  29. 1 2 LIO Linux SCSI Target (2012-12-09). "Targetcli". linux-iscsi.org. Archived from the original on 2013-03-02. Retrieved 2012-12-25.
  30. ↑ Jerome Martin (2011-08-03). "Package rtslib". daterainc.com. Retrieved 2012-12-25.
  31. ↑ "Chapter 6. Storage". Access.redhat.com. Retrieved 2014-01-20.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.