Fibre Channel networks form a switched fabric because the switches in a network operate in unison as one big switch. Fibre Channel typically runs on optical fiber cables within and between data centers, but can also run on copper cabling.[3][4] Supported data rates include 1, 2, 4, 8, 16, 32, 64, and 128 gigabit per second resulting from improvements in successive technology generations. The industry now notates this as Gigabit Fibre Channel (GFC).
There are various upper-level protocols for Fibre Channel, including two for block storage. Fibre Channel Protocol (FCP) is a protocol that transports SCSI commands over Fibre Channel networks.[3][4]FICON is a protocol that transports ESCON commands, used by IBM mainframe computers, over Fibre Channel. Fibre Channel can be used to transport data from storage systems that use solid-state flash memory storage medium by transporting NVMe protocol commands.
Etymology
When the technology was originally devised, it ran over optical fiber cables only and, as such, was called "Fiber Channel". Later, the ability to run over copper cabling was added to the specification. In order to avoid confusion and to create a unique name, the industry decided to change the spelling and use the British Englishfibre for the name of the standard.[5]
History
Fibre Channel is standardized in the T11 Technical Committee of the International Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI)-accredited standards committee. Fibre Channel started in 1988, with ANSI standard approval in 1994, to merge the benefits of multiple physical layer implementations including SCSI, HIPPI and ESCON.
Fibre Channel was designed as a serial interface to overcome limitations of the SCSI and HIPPI physical-layer parallel-signal copper wire interfaces. Such interfaces face the challenge of, among other things, maintaining signal timing coherence across all the data-signal wires (8, 16 and finally 32 for SCSI, 50 for HIPPI) so that a receiver can determine when all the electrical signal values are "good" (stable and valid for simultaneous reception sampling). This challenge becomes evermore difficult in a mass-manufactured technology as data signal frequencies increase, with part of the technical compensation being ever reducing the supported connecting copper-parallel cable length. See Parallel SCSI. FC was developed with leading-edge multi-mode optical fiber technologies that overcame the speed limitations of the ESCON protocol. By appealing to the large base of SCSI disk drives and leveraging mainframe technologies, Fibre Channel developed economies of scale for advanced technologies and deployments became economical and widespread.
Commercial products were released while the standard was still in draft.[6] By the time the standard was ratified lower speed versions were already growing out of use.[7] Fibre Channel was the first serial storage transport to achieve gigabit speeds[8] where it saw wide adoption, and its success grew with each successive speed. Fibre Channel has doubled in speed every few years since 1996.
In addition to a modern physical layer, Fibre Channel also added support for any number of "upper layer" protocols, including ATM, IP (IPFC) and FICON, with SCSI (FCP) being the predominant usage.
Fibre Channel has seen active development since its inception, with numerous speed improvements on a variety of underlying transport media. The following tables shows the progression of native Fibre Channel speeds:[9]
FC used throughout all applications for Fibre Channel infrastructure and devices, including edge and ISL interconnects. Each speed maintains backward compatibility at least two previous generations (I.e., 32GFC backward compatible to 16GFC and 8GFC)
Inter-Switch Links, ISLs, are usually multi-lane interconnects used for non-edge, core connections, and other high speed applications demanding maximum bandwidth. ISL’s utilize high bit-rates to accommodate the funneling of edge connections. Some ISL solutions are vendor-proprietary.
Characteristics
Two major characteristics of Fibre Channel networks are in-order delivery and lossless delivery of raw block data. Lossless delivery of raw data block is achieved based on a credit mechanism.[1]
Topologies
There are three major Fibre Channel topologies, describing how a number of ports are connected together. A port in Fibre Channel terminology is any entity that actively communicates over the network, not necessarily a hardware port. This port is usually implemented in a device such as disk storage, a Host Bus Adapter (HBA) network connection on a server or a Fibre Channel switch.[3]
Topology diagram of a Fibre Channel point-to-point connection
Point-to-point (see FC-FS-3). Two devices are connected directly to each other using N_ports. This is the simplest topology, with limited connectivity.[3] The bandwidth is dedicated.
Arbitrated loop (see FC-AL-2). In this design, all devices are in a loop or ring, similar to Token Ring networking. Adding or removing a device from the loop causes all activity on the loop to be interrupted. The failure of one device causes a break in the ring. Fibre Channel hubs exist to connect multiple devices together and may bypass failed ports. A loop may also be made by cabling each port to the next in a ring.
A minimal loop containing only two ports, while appearing to be similar to point-to-point, differs considerably in terms of the protocol.
Only one pair of ports can communicate concurrently on a loop.
Maximum speed of 8GFC.
Arbitrated Loop has been rarely used after 2010 and its support is being discontinued for new gen switches.
Switched Fabric (see FC-SW-6). In this design, all devices are connected to Fibre Channel switches, similar conceptually to modern Ethernet implementations. Advantages of this topology over point-to-point or Arbitrated Loop include:
The Fabric can scale to tens of thousands of ports.
The switches manage the state of the Fabric, providing optimized paths via Fabric Shortest Path First (FSPF) data routing protocol.
The traffic between two ports flows through the switches and not through any other ports like in Arbitrated Loop.
Failure of a port is isolated to a link and should not affect operation of other ports.
Multiple pairs of ports may communicate simultaneously in a Fabric.
Fibre Channel does not follow the OSI model layering, and is split into five layers:
Fibre Channel is a layered technology that starts at the physical layer and progresses through the protocols to the upper level protocols like SCSI and SBCCS.
FC-4 – Protocol-mapping layer, in which upper level protocols such as NVM Express (NVMe), SCSI, IP, and FICON are encapsulated into Information Units (IUs) for delivery to FC-2. Current FC-4s include FCP-4, FC-SB-5, and FC-NVMe.
FC-3 – Common services layer, a thin layer that could eventually implement functions like encryption or RAID redundancy algorithms; multiport connections;
FC-2 – Signaling Protocol, defined by the Fibre Channel Framing and Signaling 4 (FC-FS-5) standard, consists of the low level Fibre Channel network protocols; port to port connections;
FC-1 – Transmission Protocol, which implements line coding of signals;
Layers FC-0 are defined in Fibre Channel Physical Interfaces (FC-PI-6), the physical layers of Fibre Channel.
Fibre Channel products are available at 1, 2, 4, 8, 10, 16 and 32 and 128 Gbit/s; these protocol flavors are called accordingly 1GFC, 2GFC, 4GFC, 8GFC, 10GFC, 16GFC, 32GFC or 128GFC. The 32GFC standard was approved by the INCITS T11 committee in 2013, and those products became available in 2016. The 1GFC, 2GFC, 4GFC, 8GFC designs all use 8b/10b encoding, while the 10GFC and 16GFC standard uses 64b/66b encoding. Unlike the 10GFC standards, 16GFC provides backward compatibility with 4GFC and 8GFC since it provides exactly twice the throughput of 8GFC or four times that of 4GFC.
Ports
FC topologies and port types: This diagram shows how N_Ports can be connected to a fabric or to another N_Port. A Loop Port (L_Port) communicates through a shared loop and is rarely used anymore.
Fibre Channel ports come in a variety of logical configurations. The most common types of ports are:
N_Port (Node port) An N_Port is typically an HBA port that connects to a switch's F_Port or another N_Port. Nx_Port communicating through a PN_Port that is not operating a Loop Port State Machine.[13]
F_Port (Fabric port) An F_Port is a switch port that is connected to an N_Port.[14]
E_Port (Expansion port) Switch port that attaches to another E_Port to create an Inter-Switch Link.[14]
Fibre Channel Loop protocols create multiple types of Loop Ports:
L_Port (Loop port) FC_Port that contains Arbitrated Loop functions associated with the Arbitrated Loop topology.[14]
FL_Port (Fabric Loop port) L_Port that is able to perform the function of an F_Port, attached via a link to one or more NL_Ports in an Arbitrated Loop topology.[14]
NL_Port (Node Loop port) PN_Port that is operating a Loop port state machine.[14]
If a port can support loop and non-loop functionality, the port is known as:
Fx_Port switch port capable of operating as an F_Port or FL_Port.[13]
Nx_Port end point for Fibre Channel frame communication, having a distinct address identifier and Name_Identifier,providing an independent set of FC-2V functions to higher levels, and having the ability to act as an Originator, a Responder, or both.[13]
A Port has a physical structure as well as logical or virtual structure. This diagram shows how a virtual port may have multiple physical ports and vice versa.
Ports have virtual components and physical components and are described as:
PN_Port entity that includes a Link_Control_Facility and one or more Nx_Ports.[14]
VF_Port (Virtual F_Port) instance of the FC-2V sublevel that connects to one or more VN_Ports.[14]
VN_Port (Virtual N_Port) instance of the FC-2V sublevel. VN_Port is used when it is desired to emphasize support for multiple Nx_Ports on a single Multiplexer (e.g., via a single PN_Port).[13]
VE_Port (Virtual E_Port) instance of the FC-2V sublevel that connects to another VE_Port or to a B_Port to create an Inter-Switch Link.[14]
The following types of ports are also used in Fibre Channel:
A_Port (Adjacent port) combination of one PA_Port and one VA_Port operating together.[14]
B_Port (Bridge Port) Fabric inter-element port used to connect bridge devices with E_Ports on a Switch.[13]
D_Port (Diagnostic Port) A configured port used to perform diagnostic tests on a link with another D_Port.[15]
EX_Port A type of E_Port used to connect to an FC router fabric.[15]
G_Port (Generic Fabric port) Switch port that may function either as an E_Port, A_Port, or as an F_Port.[14]
GL_Port (Generic Fabric Loop port) Switch port that may function either as an E_Port, A_Port, or as an Fx_Port.[14]
PE_Port LCF within the Fabric that attaches to another PE_Port or to a B_Port through a link.[13]
PF_Port LCF within a Fabric that attaches to a PN_Port through a link.[13]
TE_Port (Trunking E_Port) A trunking expansion port that expands the functionality of E ports to support VSAN trunking, Transport quality of service (QoS) parameters, and Fibre Channel trace (fctrace) feature.[16]
U_Port(Universal port) A port waiting to become another port type[15]
VA_Port (Virtual A_Port) instance of the FC-2V sublevel of Fibre Channel that connects to another VA_Port.[14]
VEX_Port VEX_Ports are no different from EX_Ports, except underlying transport is IP rather than FC.[15]
Media and modules
This section is missing information about legacy copper interfaces. Please expand the section to include this information. Further details may exist on the talk page.(November 2023)
This section appears to contradict itself. Please see the talk page for more information.(November 2023)
Fibre Channel predominantly uses SFP/SFP+ modules with LC connector and duplex cabling, but 128GFC uses QSFP28 modules with MPO connectors and ribbon cabling.
The Fibre Channel physical layer is based on serial connections that use fiber optics to copper between corresponding pluggable modules. The modules may have a single lane, dual lanes or quad lanes that correspond to the SFP, SFP-DD and QSFP form factors. Fibre Channel does not use 8- or 16-lane modules (like CFP8, QSFP-DD, or COBO used in 400GbE) and there are no plans to use these expensive and complex modules.
The small form-factor pluggable transceiver (SFP) module and its enhanced version SFP+, SFP28 and SFP56 are common form factors for Fibre Channel ports. SFP modules support a variety of distances via multi-mode and single-mode optical fiber as shown in the table below. SFP modules use duplex fiber cabling with LC connectors.
SFP-DD modules are used in high-density applications that need to double the throughput of traditional SFP ports.
SFP-DD modules are used for high-density applications that need to double the throughput of an SFP Port. SFP-DD is defined by the SFP-DD MSA and enables breakout to two SFP ports. Two rows of electrical contacts enable doubling the throughput of SFP modules in a similar fashion as QSFP-DD.
The quad small form-factor pluggable (QSFP) module began being used for switch inter-connectivity and was later adopted for use in 4-lane implementations of Gen-6 Fibre Channel supporting 128GFC. QSFP uses either LC connectors for 128GFC-CWDM4 or MPO connectors for 128GFC-SW4 or 128GFC-PSM4. MPO cabling uses 8- or 12-fiber cabling infrastructure that connects to another 128GFC port or may be broken out into four duplex LC connections to 32GFC SFP+ ports. Fibre Channel switches use either SFP or QSFP modules.
Modern Fibre Channel devices support SFP+ transceiver, mainly with LC (Lucent Connector) fiber connector. Older 1GFC devices used GBIC transceiver, mainly with SC (Subscriber Connector) fiber connector.
Storage area networks
The Fibre Channel SAN connects servers to storage via Fibre Channel switches.
The goal of Fibre Channel is to create a storage area network (SAN) to connect servers to storage.
The SAN is a dedicated network that enables multiple servers to access data from one or more storage devices. Enterprise storage uses the SAN to backup to secondary storage devices including disk arrays, tape libraries, and other backup while the storage is still accessible to the server. Servers may access storage from multiple storage devices over the network as well.
SANs are often designed with dual fabrics to increase fault tolerance. Two completely separate fabrics are operational and if the primary fabric fails, then the second fabric becomes the primary.
Switches
Fibre Channel director with SFP+ modules and LC optical fiber connectors with Optical Multimode 3 (OM3) fiber (aqua)
Fibre Channel switches can be divided into two classes. These classes are not part of the standard, and the classification of every switch is a marketing decision of the manufacturer:
Directors offer a high port-count in a modular (slot-based) chassis with no single point of failure (high availability).
Switches are typically smaller, fixed-configuration (sometimes semi-modular), less redundant devices.
A fabric consisting entirely of one vendors products is considered to be homogeneous. This is often referred to as operating in its "native mode" and allows the vendor to add proprietary features which may not be compliant with the Fibre Channel standard.
If multiple switch vendors are used within the same fabric it is heterogeneous, the switches may only achieve adjacency if all switches are placed into their interoperability modes. This is called the "open fabric" mode as each vendor's switch may have to disable its proprietary features to comply with the Fibre Channel standard.
Some switch manufacturers offer a variety of interoperability modes above and beyond the "native" and "open fabric" states. These "native interoperability" modes allow switches to operate in the native mode of another vendor and still maintain some of the proprietary behaviors of both. However, running in native interoperability mode may still disable some proprietary features and can produce fabrics of questionable stability.
Host bus adapters
Dual port 8Gb FC host bus adapter cardDual port 16Gb FC host bus adapter card
Fibre Channel HBAs, as well as CNAs, are available for all major open systems, computer architectures, and buses, including PCI and SBus. HBAs connect servers to the Fibre Channel network and are part of a class of devices known as translation devices. Some are OS dependent. Each HBA has a unique World Wide Name (WWN), which is similar to an Ethernet MAC address in that it uses an Organizationally Unique Identifier (OUI) assigned by the IEEE. However, WWNs are longer (8 bytes). There are two types of WWNs on an HBA; a World Wide Node Name (WWNN), which can be shared by some or all ports of a device, and a World Wide Port Name (WWPN), which is necessarily unique to each port. Adapters or routers can connect Fibre Channel networks to IP or Ethernet networks.[18]
Small Computer System Interface is a set of standards for physically connecting and transferring data between computers and peripheral devices, best known for its use with storage devices such as hard disk drives. SCSI was introduced in the 1980s and has seen widespread use on servers and high-end workstations, with new SCSI standards being published as recently as SAS-4 in 2017.
A network switch is networking hardware that connects devices on a computer network by using packet switching to receive and forward data to the destination device.
Internet Small Computer Systems Interface or iSCSI is an Internet Protocol-based storage networking standard for linking data storage facilities. iSCSI provides block-level access to storage devices by carrying SCSI commands over a TCP/IP network. iSCSI facilitates data transfers over intranets and to manage storage over long distances. It can be used to transmit data over local area networks (LANs), wide area networks (WANs), or the Internet and can enable location-independent data storage and retrieval.
InfiniBand (IB) is a computer networking communications standard used in high-performance computing that features very high throughput and very low latency. It is used for data interconnect both among and within computers. InfiniBand is also used as either a direct or switched interconnect between servers and storage systems, as well as an interconnect between storage systems. It is designed to be scalable and uses a switched fabric network topology. Between 2014 and June 2016, it was the most commonly used interconnect in the TOP500 list of supercomputers.
In computer hardware, a host controller, host adapter, or host bus adapter (HBA), connects a computer system bus, which acts as the host system, to other network and storage devices. The terms are primarily used to refer to devices for connecting SCSI, SAS, NVMe, Fibre Channel and SATA devices. Devices for connecting to FireWire, USB and other devices may also be called host controllers or host adapters.
Small Form-factor Pluggable (SFP) is a compact, hot-pluggable network interface module format used for both telecommunication and data communications applications. An SFP interface on networking hardware is a modular slot for a media-specific transceiver, such as for a fiber-optic cable or a copper cable. The advantage of using SFPs compared to fixed interfaces is that individual ports can be equipped with different types of transceivers as required, with the majority including optical line terminals, network cards, switches and routers.
In the computer storage field, a Fibre Channel switch is a network switch compatible with the Fibre Channel (FC) protocol. It allows the creation of a Fibre Channel fabric, that is the core component of a storage area network (SAN). The fabric is a network of Fibre Channel devices which allows many-to-many communication, device name lookup, security, and redundancy. FC switches implement zoning, a mechanism that disables unwanted traffic between certain fabric nodes.
In computing, Serial Attached SCSI (SAS) is a point-to-point serial protocol that moves data to and from computer-storage devices such as hard disk drives and tape drives. SAS replaces the older Parallel SCSI bus technology that first appeared in the mid-1980s. SAS, like its predecessor, uses the standard SCSI command set. SAS offers optional compatibility with Serial ATA (SATA), versions 2 and later. This allows the connection of SATA drives to most SAS backplanes or controllers. The reverse, connecting SAS drives to SATA backplanes, is not possible.
The arbitrated loop, also known as FC-AL, is a Fibre Channel topology in which devices are connected in a one-way loop fashion in a ring topology. Historically it was a lower-cost alternative to a fabric topology. It allowed connection of many servers and computer storage devices without using then very costly Fibre Channel switches. The cost of the switches dropped considerably, so by 2007, FC-AL had become rare in server-to-storage communication. It is however still common within storage systems.
Switched fabric or switching fabric is a network topology in which network nodes interconnect via one or more network switches. Because a switched fabric network spreads network traffic across multiple physical links, it yields higher total throughput than broadcast networks, such as the early 10BASE5 version of Ethernet and most wireless networks such as Wi-Fi.
In Fibre Channel protocol, a registered state change notification (RSCN) is a Fibre Channel fabric's notification sent to all specified nodes in case of any major fabric changes. This allows nodes to immediately gain knowledge about the fabric and react accordingly.
Fibre Channel Protocol (FCP) is the SCSI interface protocol utilising an underlying Fibre Channel connection. The Fibre Channel standards define a high-speed data transfer mechanism that can be used to connect workstations, mainframes, supercomputers, storage devices and displays. FCP addresses the need for very fast transfers of large volumes of information and could relieve system manufacturers from the burden of supporting a variety of channels and networks, as it provides one standard for networking, storage and data transfer. Some Fibre Channel characteristics are:
The current portfolio of PowerConnect switches are now being offered as part of the Dell Networking brand: information on this page is an overview of all current and past PowerConnect switches as per August 2013, but any updates on current portfolio will be detailed on the Dell Networking page.
Fibre Channel over Ethernet (FCoE) is a computer network technology that encapsulates Fibre Channel frames over Ethernet networks. This allows Fibre Channel to use 10 Gigabit Ethernet networks while preserving the Fibre Channel protocol. The specification was part of the International Committee for Information Technology Standards T11 FC-BB-5 standard published in 2009. FCoE did not see widespread adoption.
The Cisco Nexus series switches are modular and fixed port network switches designed for the data center. Cisco Systems introduced the Nexus Series of switches on January 28, 2008. The first chassis in the Nexus 7000 family is a 10-slot chassis with two supervisor engine slots and eight I/O module slots at the front, as well as five crossbar switch fabric modules at the rear. Beside the Nexus 7000 there are also other models in the Nexus range.
A storage area network (SAN) or storage network is a computer network which provides access to consolidated, block-level data storage. SANs are primarily used to access data storage devices, such as disk arrays and tape libraries from servers so that the devices appear to the operating system as direct-attached storage. A SAN typically is a dedicated network of storage devices not accessible through the local area network (LAN).
In computing, Linux-IO (LIO) Target is an open-source implementation of the SCSI target that has become the standard one included in the Linux kernel. Internally, LIO does not initiate sessions, but instead provides one or more Logical Unit Numbers (LUNs), waits for SCSI commands from a SCSI initiator, and performs required input/output data transfers. LIO supports common storage fabrics, including FCoE, Fibre Channel, IEEE 1394, iSCSI, iSCSI Extensions for RDMA (iSER), SCSI RDMA Protocol (SRP) and USB. It is included in most Linux distributions; native support for LIO in QEMU/KVM, libvirt, and OpenStack makes LIO also a storage option for cloud deployments.
The Dell blade server products are built around their M1000e enclosure that can hold their server blades, an embedded EqualLogic iSCSI storage area network and I/O modules including Ethernet, Fibre Channel and InfiniBand switches.
Dell Networking is the name for the networking portfolio of Dell. In the first half of 2013, Dell started to rebrand their different existing networking product brands to Dell Networking. Dell Networking is the name for the networking equipment that was known as Dell PowerConnect, as well as the Force10 portfolio.
↑ Transmitter values listed are the currently specified values for the variant listed. Some older versions of the FC standards listed slightly different values (however, the values listed here fall within the +/− variance allowed). Individual variations for each specification are listed in the references associated with those entries in this table. FC-PH = X3T11 Project 755D; FC-PH-2 = X3T11 Project 901D; FC-PI-4 = INCITS Project 1647-D; FC-PI-5 = INCITS Project 2118D. Copies are available from INCITSArchived 2010-09-15 at the Wayback Machine .
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
