Dell M1000e

Last updated

The Dell blade server products are built around their M1000e enclosure that can hold their server blades, an embedded EqualLogic iSCSI storage area network and I/O modules including Ethernet, Fibre Channel and InfiniBand switches.

Contents

M1000e enclosure with selection of G12 server blades M1000e-front-G12.jpg
M1000e enclosure with selection of G12 server blades

Enclosure

The M1000e fits in a 19-inch rack and is 10 rack units high (44 cm), 17.6" (44.7 cm) wide and 29.7" (75.4 cm) deep. The empty blade enclosure weighs 44.5 kg while a fully loaded system can weigh up to 178.8 kg. [1]

On the front the servers are inserted while at the backside the power-supplies, fans and I/O modules are inserted together with the management modules(s) (CMC or chassis management controller) and the KVM switch. A blade enclosure offers centralized management for the servers and I/O systems of the blade-system. Most servers used in the blade-system offer an iDRAC card and one can connect to each servers iDRAC via the M1000e management system. It is also possible to connect a virtual KVM switch to have access to the main-console of each installed server.

In June 2013, Dell introduced the PowerEdge VRTX, which is a smaller blade system that shares modules with the M1000e. The blade servers, although following the traditional naming strategy e.g. M520, M620 (only blades supported) are not interchangeable between the VRTX and the M1000e. The blades differ in firmware and mezzanine connectors. [ citation needed ]

In 2018, Dell introduced the Dell PE MX7000, a new MX enclosure model, next generation of Dell enclosures.

The M1000e enclosure has a front-side and a back-side and thus all communication between the inserted blades and modules goes via the midplane, which has the same function as a backplane but has connectors at both sides where the front side is dedicated for server-blades and the back for I/O modules.

Midplane

Indication on back of chassis to see which midplane was installed in factory M1000e midplane versions.png
Indication on back of chassis to see which midplane was installed in factory

The midplane is completely passive. The server-blades are inserted in the front side of the enclosure while all other components can be reached via the back. [2]
The original midplane 1.0 capabilities are Fabric A - Ethernet 1 Gb; Fabrics B&C - Ethernet 1 Gb, 10 Gb, 40 Gb - Fibre Channel 4 Gb, 8 Gb - IfiniBand DDR, QDR, FDR10. The enhanced midplane 1.1 capabilities are Fabric A - Ethernet 1 Gb, 10 Gb; Fabrics B&C - Ethernet 1 Gb, 10 Gb, 40 Gb - Fibre Channel 4 Gb, 8 Gb, 16 Gb - IfiniBand DDR, QDR, FDR10, FDR. The original M1000e enclosures came with midplane version 1.0 but that midplane did not support the 10GBASE-KR standard on fabric A (10GBASE-KR standard is supported on fabrics B&C). To have 10 Gb Ethernet on fabric A or 16 Gb Fibre Channel or InfiniBand FDR (and faster) on fabrics B&C, midplane 1.1 is required. Current versions of the enclosure come with midplane 1.1 and it is possible to upgrade the midplane. Via the markings on the back-side of the enclosure, just above the I/O modules: if an "arrow down" can be seen above the 6 I/O slots the 1.0 midplane was installed in the factory; if there are 3 or 4 horizontal bars, midplane 1.1 was installed. As it is possible to upgrade the midplane the outside markings are not decisive: via the CMC management interface actual installed version of the midplane is visible [3]

Front:Blade servers

Each M1000e enclosure can hold up to 32 quarter-height, 16 half-height blades or 8 full-height or combinations (e.g. 1 full-height + 14 half-height). The slots are numbered 1-16 where 1-8 are the upper blades and 9-16 are directly beneath 1-8. When using full-height blades one use slot n (where n=1 to 8) and slot n+8 Integrated at the bottom of the front-side is a connection-option for 2 x USB, meant for a mouse and keyboard, as well as a standard VGA monitor connection (15 pin). Next to this is a power-button with power-indication.

Next to this is a small LCD screen with navigation buttons which allows one to get system-information without the need to access the CMC/management system of the enclosure. Basic status and configuration information is available via this display. To operate the display one can pull it towards one and tilt it for optimal view and access to the navigation button. For quick status checks, an indicator light sits alongside the LCD display and is always visible, with a blue LED indicating normal operation and an orange LED indicating a problem of some kind.

This LCD display can also be used for the initial configuration wizard in a newly delivered (unconfigured) system, allowing the operator to configure the CMC IP address. [2]

Back:power, management and I/O

All other parts and modules are placed at the rear of the M1000e. The rear-side is divided in 3 sections: top: here one insert the 3 management-modules: one or two CMC modules and an optional iKVM module. At the bottom of the enclosure there are 6 bays for power-supply units. A standard M1000e operates with three PSU's The area in between offers 3 x 3 bays for cooling-fans (left - middle - right) and up to 6 I/O modules: three modules to the left of the middle fans and three to the right. The I/O modules on the left are the I/O modules numbered A1, B1 and C1 while the right hand side has places for A2, B2 and C2. The A fabric I/O modules connect to the on-board I/O controllers which in most cases will be a dual 1 Gb or 10 Gb Ethernet NIC. When the blade has a dual port on-board 1 Gb NIC the first NIC will connect to the I/O module in fabric A1 and the 2nd NIC will connect to fabric A2 (and the blade-slot corresponds with the internal Ethernet interface: e.g. the on-board NIC in slot 5 will connect to interface 5 of fabric A1 and the 2nd on-board NIC goes to interface 5 of fabric A2)

I/O modules in fabric B1/B2 will connect to the (optional) Mezzanine card B or 2 in the server and fabric C to Mezzanine C or 3.

All modules can be inserted or removed on a running enclosure (Hot swapping) [2]

Available server-blades

An M1000e holds up to 32 quarter-height, 16 half-height blades or 8 full-height blades or a mix of them (e.g. 2 full height + 12 half-height). The 1/4 height blades require a full-size sleeve to install. The current list are the currently available 11G blades and the latest generation 12 models. There are also older blades like the M605, M805 and M905 series.

Power Edge M420

Released in 2012, [4] PE M420 is a "quarter-size" blade: where most servers are 'half-size', allowing 16 blades per M1000e enclosure, with the new M420 up to 32 blade servers can be installed in a single chassis. Implementing the M420 has some consequences for the system: many people have reserved 16 IP addresses per chassis to support the "automatic IP address assignment" for the iDRAC management card in a blade, but as it is now possible to run 32 blades per chassis people might need to change their management IP assignment for the iDRAC. To support the M420 server one needs to run CMC firmware 4.1 or later [5] and one needs a full-size "sleeve" that holds up to four M420 blades. It also has consequences for the "normal" I/O NIC assignment: most (half-size) blades have two LOMs (LAN On Motherboard): one connecting to the switch in the A1 fabric, the other to the A2 fabric. And the same applies to the Mezzanine cards B and C. All available I/O modules (except for the PCM6348, MXL an MIOA) have 16 internal ports: one for each half-size blade. As an M420 has two 10 Gb LOM NICs, a fully loaded chassis would require 2 × 32 internal switch ports for LOM and the same for Mezzanine. An M420 server only supports a single Mezzanine card (Mezzanine B OR Mezzanine C depending on their location) whereas all half-height and full-height systems support two Mezzanine cards. To support all on-board NICs one would need to deploy a 32-slot Ethernet switch such as the MXL or Force10 I/O Aggregator. But for the Mezzanine card it is different: the connections from Mezzanine B on the PE M420 are "load-balanced" between the B and C-fabric of the M1000e: the Mezzanine card in "slot A" (top slot in the sleeve) connects to Fabric C while "slot B" (the second slot from the top) connects to fabric B, and that is then repeated for C and D slots in the sleeve. [4]

Power Edge M520

A half-height server with up to 2x 8 core Intel Xeon E5-2400 CPU, running the Intel C600 chipset and offering up to 384 Gb RAM memory via 12 DIMM slots. Two on-blade disks (2.5-inch PCIe SSD, SATA HDD or SAS HDD) are installable for local storage and a choice of Intel or Broadcom LOM + 2 Mezzanine slots for I/O. [6] The M520 can also be used in the PowerEdge VRTX system.

Power Edge M600

A half-height server with a Quad-Core Intel Xeon and 8 DIMM slots for up to 64 GB RAM

Power Edge M610

A half-height server with a quad-core or six-core Intel 5500 or 5600 Xeon CPU and Intel 5520 chipset. RAM memory options via 12 DIMM slots for up to 192 Gb RAM DDR3. A maximum of two on-blade hot-pluggable 2.5-inch hard-disks or SSDs and a choice of built-in NICs for Ethernet or converged network adapter (CNA), Fibre Channel or InfiniBand. The server has the Intel 5520 chipset and a Matrox G200 video card [7]

Power Edge M610x

A full-height blade server that has the same capabilities as the half-height M610 but offering an expansion module containing x16 PCI Express (PCIe) 2.0 expansion slots that can support up to two standard full-length/full-height PCIe cards. [8]

Power Edge M620

A half-height server with up to 2x 12 core Intel Xeon E5-2600 or Xeon E5-2600 v2 CPUs, running the Intel C600 chipset and offering up to 768 GB RAM memory via 24 DIMM slots. Two on-blade disks (2,5" PCIe SSD, SATA HDD or SAS HDD) are installable for local storage with a range of RAID controller options. Two external and one internal USB ports and two SD card slots. The blades can come pre-installed with Windows 2008 R2 SP1, Windows 2012 R2, SuSE Linux Enterprise or RHEL. It can also be ordered with Citrix XenServer or VMWare vSphere ESXi or using Hyper-V which comes with W2K8 R2. [9] According to the vendor all Generation 12 servers are optimized to run as virtualisation platform. [10] Out-of-band management is done via iDRAC 7 via the CMC.

Power Edge M630

A half-height server with up to 2x 22-core Intel Xeon E5-2600 v3/v4 CPUs, running the Intel C610 chipset and offering up to 768 GB RAM memory via 24 DIMM slots, or 640 GB RAM memory via 20 DIMM slots when using 145w CPUs. Two on-blade disks (2,5" PCIe SSD, SATA HDD or SAS HDD) are installable for local storage and a choice of Intel or Broadcom LOM + 2 Mezzanine slots for I/O. [6] The M630 can also be used in the PowerEdge VRTX system. Amulet HotKey offers a modified M630 server that can be fitted with a GPU or Teradici PCoIP Mezzanine module.

Power Edge M640

A half-height server with up to 2x 28-core Xeon Scalable CPU. Supported on both the M1000e and PowerEdge VRTX chassis. The server can support up to 16 DDR4 RDIMM memory slots for up to 1024 GB RAM and 2 drive bays supporting SAS / SATA or NVMe drives (with an adapter). The server uses iDRAC 9.

Power Edge M710

A full-height server with a quad-core or six-core Intel 5500 or 5600 Xeon CPU and up to 192 Gb RAM. A maximum of four on-blade hot-pluggable 2.5" hard-disks or SSD's and a choice of built-in NICs for Ethernet or converged network adapter, Fibre Channel or InfiniBand. The video card is a Matrox G200.The server has the Intel 5520 chipset [11]

Power Edge M710HD

A two-socket version of the M710 but now in a half-height blade. CPU can be two quad-core or 6-core Xeon 5500 or 5600 with the Intel 5520 chipset. Via 18 DIMM slots up to 288 Gb DDR3 RAM can put on this blade and the standard choice of on-board Ethernet NICs based on Broadcom or Intel and one or two Mezzanine cards for Ethernet, Fibre Channel or InfiniBand. [12]

Power Edge M820

A full-height server with 4x 8 core Intel Xeon E5-4600 CPU, running the Intel C600 chipset and offering up to 1.5 TB RAM memory via 48 DIMM slots. Up to four on-blade 2,5" SAS HDD/SSD or two PCIe flash SSD are installable for local storage. The M820 offers a choice of 3 different on-board converged Ethernet adaptors for 10 Gbit/s Fibre Channel over Ethernet (FCoE) from Broadcom, Brocade or QLogic and up to two additional Mezzanine for Ethernet, Fibre Channel or InfiniBand I/O [13]

Power Edge M910

A full-height server of the 11th generation with up to 4x 10-core Intel XEON E7 CPU or 4 x 8 core XEON 7500 series or 2 x 8 core XEON 6500 series, 512 Gb or 1 Tb DDR3 RAM and two hot-swappable 2,5" hard-drives (spinning or SSD). It uses the Intel E 7510 chipset. A choice of built-in NICs for Ethernet, Fibre Channel or InfiniBand [14]

Power Edge M915

Also a full-height 11G server using the AMD Opteron 6100 or 6200 series CPU with the AMD SR5670 and SP5100 chipset. Memory via 32 DDR3 DIMM slots offering up to 512 Gb RAM. On-board up to two 2,5 inch HDD or SSD's. The blade comes with a choice of on-board NICs and up to two mezzanine cards for dual-port 10 Gb Ethernet, dual-port FCoE, dual-port 8 Gb fibre-channel or dual port Mellanox Infiniband. Video is via the on-board Matrox G200eW with 8 MB memory [15]

Mezzanine cards

Each server comes with Ethernet NICs on the motherboard. These 'on board' NICs connect to a switch or pass-through module inserted in the A1 or the A2 bay at the back of the switch. To allow more NICs or non-Ethernet I/O each blade [16] has two so-called mezzanine slots: slot B connecting to the switches/modules in bay B1 and B2 and slot C connecting to C1/C2: An M1000e chassis holds up to 6 switches or pass-through modules. For redundancy one would normally install switches in pairs: the switch in bay A2 is normally the same as the A1 switch and connects the blades on-motherboard NICs to connect to the data or storage network.

(Converged) Ethernet Mezzanine cards

Standard blade-servers have one or more built-in NICs that connect to the 'default' switch-slot (the A-fabric) in the enclosure (often blade-servers also offer one or more external NIC interfaces at the front of the blade) but if one want the server to have more physical (internal) interfaces or connect to different switch-blades in the enclosure one can place extra mezzanine cards on the blade. The same applies to adding a Fibre Channel host bus adapter or a Fibre Channel over Ethernet (FCoE) converged network adapter interface. Dell offers the following (converged) Ethernet mezzanine cards for their PowerEdge blades: [17]

Non-Ethernet cards

Apart from the above the following mezzanine cards are available: [17]

Blade storage

In most setups the server-blades will use external storage (NAS using iSCSI, FCoE or Fibre Channel) in combination with local server-storage on each blade via hard disk drives or SSDs on the blades (or even only a SD-card with boot-OS like VMware ESX [18] ). It is also possible to use completely diskless blades that boot via PXE or external storage. But regardless of the local and boot-storage: the majority of the data used by blades will be stored on SAN or NAS external from the blade-enclosure.

EqualLogic Blade-SAN

Dell has put the EqualLogic PS M4110 models of iSCSI storage arrays [19] that are physically installed in the M1000e chassis: this SAN will take the same space in the enclosure as two half-height blades next to each other. Apart from the form factor (the physical size, getting power from the enclosure system etc.) it is a "normal" iSCSI SAN: the blades in the (same) chassis communicate via Ethernet and the system does require an accepted Ethernet blade-switch in the back (or a pass-through module + rack-switch): there is no option for direct communication of the server-blades in the chassis and the M4110: it only allows a user to pack a complete mini-datacentre in a single enclosure (19" rack, 10 RU)

Depending on the model and used disk driver the PS M4110 offers a system (raw) storage capacity between 4.5 TB (M4110XV with 14 × 146 Gb, 15K SAS HDD) and 14 TB (M4110E with 14 x 1 TB, 7,2K SAS HDD). The M4110XS offer 7.4 TB using 9 HDD's and 5 SSD's. [20]

Each M4110 comes with one or two controllers and two 10-gigabit Ethernet interfaces for iSCSI. The management of the SAN goes via the chassis-management interface (CMC). Because the iSCSI uses 10 Gb interfaces the SAN should be used in combination with one of the 10G blade switches: the PCM 8024-k or the Force10 MXL switch. [20] The enclosure's mid-plane hardware version should be at least version 1.1 to support 10 Gb KR connectivity [21] [22]

PowerConnect switches

Drawing of M1000e enclosure with 2 x FTOS MXL, 2 x M8024-k and 2x FibreChannel 5424 M1000e-back.jpg
Drawing of M1000e enclosure with 2 x FTOS MXL, 2 x M8024-k and 2x FibreChannel 5424

At the rear side of the enclosure one will find the power-supplies, fan-trays, one or two chassis-management modules (the CMC's) and a virtual KVM switch. And the rear offers 6 bays for I/O modules numbered in 3 pairs: A1/A2, B1/B2 and C1/C2. The A bays connect the on-motherboard NICs to external systems (and/or allowing communication between the different blades within one enclosure).

The Dell PowerConnect switches are modular switches for use in the Dell blade server enclosure M1000e. The M6220, M6348, M8024 and M8024K are all switches in the same family, based on the same fabrics (Broadcom) and running the same firmware-version. [23]

All the M-series switches are OSI layer 3 capable: so one can also say that these devices are layer 2 Ethernet switches with built-in router or layer3 functionality.

The most important difference between the M-series switches and the Dell PowerConnect classic switches (e.g. the 8024 model) is the fact that most interfaces are internal interfaces that connect to the blade-servers via the midplane of the enclosure. Also the M-series can't be running outside the enclosure: it will only work when inserted in the enclosure.

PowerConnect M6220

This is a 20-port switch: 16 internal and 4 external Gigabit Ethernet interfaces and the option to extend it with up to four 10 Gb external interfaces for uplinks or two 10 Gb uplinks and two stacking ports to stack several PCM6220's into one large logical switch.

PowerConnect M6348

This is a 48 port switch: 32 internal 1 Gb interfaces (two per serverblade) and 16 external copper (RJ45) gigabit interfaces. There are also two SFP+ slots for 10 Gb uplinks and two CX4 slots that can either be used for two extra 10 Gb uplinks or to stack several M6348's blades in one logical switch. The M6348 offers four 1 Gb interfaces to each blade which means that one can only utilize the switch to full capacity when using blades that offer 4 internal NICs on the A fabric (=the internal/on motherboard NIC). The M6348 can be stacked with other M6348 but also with the PCT7000 series rack-switches.

PowerConnect M8024 and M8024k

The M8024 and M8024-k offer 16 internal autosensing 1 or 10 Gb interfaces and up to 8 external ports via one or two I/O modules each of which can offer: 4 × 10 Gb SFP+ slots, 3 x CX4 10 Gb (only) copper or 2 x 10G BaseT 1/10 Gb RJ-45 interfaces. The PCM8024 is 'end of sales' since November 2011 and replaced with the PCM8024-k. [24] Since firmware update 4.2 the PCM8024-k supports partially FCoE via FIP (FCoE Initialisation Protocol) and thus Converged network adapters but unlike the PCM8428-k it has no native fibre channel interfaces.

Also since firmware 4.2 the PCM8024-k can be stacked using external 10 Gb Ethernet interfaces by assigning them as stacking ports. Although this new stacking-option is also introduced in the same firmware release for the PCT8024 and PCT8024-f one can't stack blade (PCM) and rack (PCT)-versions in a single stack. The new features are not available on the 'original' PCM8024. Firmware 4.2.x for the PCM8024 only corrected bugs: no new features or new functionality are added to 'end of sale' models. [25] [26]

To use the PCM8024-k switches one will need the backplane that supports the KR or IEEE 802.3ap standards [21] [22]

Powerconnect capabilities

All PowerConnect M-series ("PCM") switches are multi-layer switches thus offering both layer 2 (Ethernet) options as well as layer 3 or IP routing options.
Depending on the model the switches offer internally 1 Gbit/s or 10 Gbit/s interfaces towards the blades in the chassis. The PowerConnect M series with "-k" in the model-name offer 10 Gb internal connections using the 10GBASE-KR standard. The external interfaces are mainly meant to be used as uplinks or stacking-interfaces but can also be used to connect non-blade servers to the network.
On the link-level PCM switches support link aggregation: both static LAG's as well as LACP. As all PowerConnect switches the switches are running RSTP as Spanning Tree Protocol, but it is also possible to run MSTP or Multiple Spanning Tree. The internal ports towards the blades are by default set as edge or "portfast" ports. Another feature is to use link-dependency. One can, for example, configure the switch that all internal ports to the blades are shut down when the switch gets isolated because it loses its uplink to the rest of the network.
All PCM switches can be configured as pure layer-2 switches or they can be configured to do all routing: both routing between the configured VLAN's as external routing. Besides static routes the switches also support OSPF and RIP routing. When using the switch as routing switch one need to configure vlan interfaces and assign an IP address to that vlan interface: it is not possible to assign an IP address directly to a physical interface.
[23]

Stacking

All PowerConnect blade switches, except for the original PC-M8024, can be stacked. To stack the new PC-M8024-k switch the switches need to run firmware version 4.2 or higher. [27] In principle one can only stack switches of the same family; thus stacking multiple PCM6220's together or several PCM8024-k. The only exception is the capability to stack the blade PCM6348 together with the rack-switch PCT7024 or PCT7048. Stacks can contain multiple switches within one M1000e chassis but one can also stack switches from different chassis to form one logical switch. [28]

Force10 switches

MXL 10/40 Gb switch

At the Dell Interop 2012 in Las Vegas Dell announced the first FTOS based blade-switch: the Force10 MXL 10/40Gpbs blade switch, and later a 10/40 Gbit/s concentrator. The FTOS MXL 40 Gb was introduced on 19 July 2012. [29] The MXL provides 32 internal 10 Gbit/s links (2 ports per blade in the chassis), two QSFP+ 40 Gbit/s ports and two empty expansion slots allowing a maximum of 4 additional QSFP+ 40 Gbit/s ports or 8 10 Gbit/s ports. Each QSFP+ port can be used for a 40 Gbit/s switch to switch (stack) uplink or, with a break-out cable, 4 x 10 Gbit/s links. Dell offers direct attach cables with on one side the QSFP+ interface and 4 x SFP+ on the other end or a QSFP+ transceiver on one end and 4 fibre-optic pairs to be connected to SFP+ transceivers on the other side. Up to six MXL blade-switch can be stacked into one logical switch.

Besides the above 2x40 QSFP module the MXL also supports a 4x10 Gb SFP+ and a 4x10GbaseT module. All Ethernet extension modules for the MXL can also be used for the rack based N4000 series (fka Power connector 8100).

The MXL switches also support Fibre Channel over Ethernet so that server-blades with a converged network adapter Mezzanine card can be used for both data as storage using a Fibre Channel storage system. The MXL 10/40 Gbit/s blade switch will run FTOS [30] and because of this will be the first M1000e I/O product without a Web graphical user interface. The MXL can either forward the FCoE traffic to an upstream switch or, using a 4 port 8 Gb FC module, perform the FCF function, connecting the MXL to a full FC switch or directly to a FC SAN.

I/O Aggregator

In October 2012 Dell also launched the I/O Aggregator for the M1000e chassis running on FTOS. The I/O Aggregator offers 32 internal 10 Gb ports towards the blades and standard two 40 Gbit/s QSFP+ uplinks and offers two extension slots. Depending on one's requirements one can get extension modules for 40 Gb QSFP+ ports, 10 Gb SFP+ or 1-10 GBaseT copper interfaces. One can assign up to 16 x 10 Gb uplinks to one's distribution or core layer. The I/O aggregator supports FCoE and DCB (Data center bridging) features [31]

Cisco switches

Dell also offered some Cisco Catalyst switches for this blade enclosure. Cisco offers a range of switches for blade-systems from the main vendors. Besides the Dell M1000e enclosure Cisco offers similar switches also for HP, FSC and IBM blade-enclosures. [32]

For the Dell M1000e there are two model-ranges for Ethernet switching: (note: Cisco also offers the Catalyst 3030, but this switch is for the old Generation 8 or Gen 9 blade system, not for the current M1000e enclosure [33] )

As per 2017 the only available Cisco I/O device for the M1000e chassis is the Nexus FEX [34]

Catalyst 3032

The Catalyst 3032: a layer 2 switch with 16 internal and 4 external 1 Gb Ethernet interfaces with an option to extend to 8 external 1 Gb interfaces. The built-in external ports are 10/100/1000 BaseT copper interfaces with an RJ45 connector and up to 4 additional 1Gb ports can be added using the extension module slots that each offer 2 SFP slots for fiber-optic or Twinax 1 Gb links. The Catalyst 3032 doesn't offer stacking (virtual blade switching) [35]

Catalyst 3130

The 3130 series switches offer 16 internal 1 Gb interfaces towards the blade-servers. For the uplink or external connections there are two options: the 3130G offering 4 built-in 10/100/1000BaseT RJ-45 slots and two module-bays allowing for up to 4 SFP 1 Gb slots using SFP transceivers or SFP Twinax cables. [36]

The 3130X also offers the 4 external 10/100/1000BaseT connections and two modules for X2 10 Gb uplinks. [37]

Both 3130 switches offer 'stacking' or 'virtual blade switch'. One can stack up to 8 Catalyst 3130 switches to behave like one single switch. This can simplify the management of the switches and simplify the (spanning tree) topology as the combined switches are just one switch for spanning tree considerations. It also allows the network manager to aggregate uplinks from physically different switch-units into one logical link. [35] The 3130 switches come standard with IP Base IOS offering all layer 2 and the basic layer 3 or routing-capabilities. Users can upgrade this basic license to IP Services or IP Advanced services adding additional routing capabilities such as EIGRP, OSPF or BGP4 routing protocols, IPv6 routing and hardware based unicast and multicast routing. These advances features are built into the IOS on the switch, but a user has to upgrade to the IP (Advanced) Services license to unlock these options [38]

Nexus Fabric Extender

Since January 2013 Cisco and Dell offer a Nexus Fabric Extender for the M1000e chassis: Nexus B22Dell. Such FEX's were already available for HP and Fujitsu blade systems, and now there is also a FEX for the M1000e blade system. The release of the B22Dell is approx. 2,5 years after the initially planned and announced date: a disagreement between Dell and Cisco resulted in Cisco stopping the development of the FEX for the M1000e in 2010. [39] Customers manage a FEX from a core Nexus 5500 series switch. [40]

Other I/O cards

An M1000e enclosure can hold up to 6 switches or other I/O cards. Besides the Ethernet switches as the Powerconnect M-series, Force10 MXL and Cisco Catalyst 3100 switches mentioned above the following I/O modules are available or usable in a Dell M1000e enclosure: [1] [41]

PCM 8428-k Brocade FCoE

Although the PCM8024-k and MXL switch do support Fibre Channel over Ethernet, it is not a 'native' FCoE switch: it has no Fibre Channel interfaces. These switches would need to be connected to a "native" FCoE switch such as the Powerconnect B-series 8000e (same as a Brocade 8000 switch) or a Cisco Nexus 5000 series switch with fibre channel interfaces (and licenses). The PCM8428 is the only full Fibre Channel over Ethernet capable switch for the M1000e enclosure that offers 16 x enhanced Ethernet 10 Gb internal interfaces, 8 x 10 Gb (enhanced) Ethernet external ports and also up to four 8 Gb Fibre Channel interfaces to connect directly to a FC SAN controller or central Fibre Channel switch.
The switch runs Brocade FC firmware for the fabric and fibre-channel switch and Foundry OS for the Ethernet switch configuration. [46] In capabilities it is very comparable to the Powerconnect-B8000, only the formfactor and number of Ethernet and FC interfaces are different. [1] [47]

PowerConnect M5424 / Brocade 5424

This is a Brocade full Fibre Channel switch. It uses either the B or C fabrics to connect the Fibre Channel mezzanine card in the blades to the FC based storage infrastructure. The M5424 offers 16 internal ports connecting to the FC Mezzanine cards in the blade-servers and 8 external ports. From factory only the first two external ports (17 and 18) are licensed: additional connections require extra Dynamic Ports On Demand (DPOD) licenses. The switch runs on a PowerPC 440EPX processor at 667 MHz and 512 MB DDR2 RAM system memory. Further it has 4 Mb boot flash and 512 Mb compact flash memory on board. [48]

Brocade M6505

Similar capabilities as above, but offering 16 X 16 Gb FC towards server mezzanine and 8 external. Standard license offers 12 connections which can be increased by 12 to support all 24 ports. auto-sensing speed 2,4,8 and 16 Gb. Total aggregate bandwidth 384 GB [49]

Brocade 4424

As the 5424, the 4424 is also a Brocade SAN I/O offering 16 internal and 8 external ports. The switch supports speeds up to 4 Gbit/s. When delivered 12 of the ports are licensed to be operation and with additional licenses one can enable all 24 ports. The 4424 runs on a PowerPC 440GP processor at 333 MHz with 256 SDRAM system memory, 4 Mb boot flash and 256 Mb compact flash memory. [50]

Infiniband

There are several modules available offering Infiniband connectivity on the M1000e chassis. Infiniband offers high bandwidth/low-latency intra-computer connectivity such as required in Academic HPC clusters, large enterprise datacenters and cloud applications. [51] There is the SFS M7000e InfiniBand switch from Cisco. The Cisco SFS offers 16 internal 'autosensing' interfaces for single (10) (SDR) or double (20 Gbit/s) data rate (DDR) and 8 DDR external/uplink ports. The total switching capacity is 960 Gbit/s [52]

Other options are the Mellanox SwitchX M4001F and M4001Q [53] and the Mellanox M2401G 20 Gb Infiniband switch for the M1000e enclosure [54]

The M4001 switches offer either 40 GBit/s (M4001Q) or the 56 Gbit/s (M4001F) connectivity and has 16 external interfaces using QSFP ports and 16 internal connections to the Infiniband Mezzanine card on the blades. As with all other non-Ethernet based switches it can only be installed in the B or C fabric of the M1000e enclosure as the A fabric connects to the "on motherboard" NICs of the blades and they only come as Ethernet NICs or converged Ethernet.

The 2401G offers 24 ports: 16 internal and 8 external ports. Unlike the M4001 switches where the external ports are using QSFP ports for fiber transceivers, the 2401 has CX4 copper cable interfaces. The switching capacity of the M2401 is 960 Gbit/s [54]

The 4001, with 16 internal and 16 external ports at either 40 or 56 Gbit/s offers a switching capacity of 2.56 Tbit/s

Passthrough modules

In some setups one don't want or need switching capabilities in one's enclosure. For example: if only a few of the blade-servers do use fibre-channel storage one don't need a fully manageble FC switch: one just want to be able to connect the 'internal' FC interface of the blade directly to one's (existing) FC infrastructure. A pass-through module has only very limited management capabilities. Other reasons to choose for pass-through instead of 'enclosure switches' could be the wish to have all switching done on a 'one vendor' infrastructure; and if that isn't available as an M1000e module (thus not one of the switches from Dell Powerconnect, Dell Force10 or Cisco) one could go for pass-through modules:

Managing enclosure

An M1000e enclosure offers several ways for management. The M1000e offers 'out of band' management: a dedicated VLAN (or even physical LAN) for management. The CMC modules in the enclosure offer management Ethernet interfaces and do not rely on network-connections made via I/O switches in the blade. One would normally connect the Ethernet links on the CMC avoiding a switch in the enclosure. Often a physically isolated LAN is created for management allowing management access to all enclosures even when the entire infrastructure is down. Each M1000e chassis can hold two CMC modules.

Each enclosure can have either one or two CMC controllers and by default one can access the CMC Webgui via https and SSH for command-line access. It is also possible to access the enclosure management via a serial port for CLI access or using a local keyboard, mouse and monitor via the iKVM switch. It is possible to daisy-chain several M1000e enclosures.

Management interface

Main page of the CMC Webgui Mainpage of the M1000e CMC Webgui.png
Main page of the CMC Webgui

Below information assumes the use of the Webgui of the M1000e CMC, although all functions are also available via the text-based CLI access. To access the management system one must open the CMC Webgui via https using the out of band management IP address of the CMC. When the enclosure is in 'stand alone' mode one will get a general overview of the entire system: the webgui gives one an overview how the system looks in reality, including the status-leds etc. By default the Ethernet interface of a CMC card will get an address from a DHCP server but it is also possible to configure an IPv4 or IPv6 address via the LED display at the front of the chassis. Once the IP address is set or known the operator can access the webgui using the default root-account that is built in from factory.

Via the CMC management one can configure chassis-related features: management IP addresses, authentication features (local user-list, using RADIUS or Tacacs server), access-options (webgui, cli, serial link, KVM etc.), error-logging (syslog server), etc. Via the CMC interface one can configure blades in the system and configuring iDRAC access to those servers. Once enabled one can access the iDRAC (and with that the console of the server) via this webgui or directly opening the webgui of the iDRAC.

The same applies to the I/O modules in the rear of the system: via the CMC one can assign an IP address to the I/O module in one of the 6 slots and then surf to the webgui of that module (if there is a web-based gui: unmanaged pass-through modules won't offer a webgui as there is nothing to configure.

LCD screen

On the front-side of the chassis there is a small hidden LCD screen with 3 buttons: one 4 way directional button allowing one to navigate through the menus on the screen and two "on/off" push buttons which work as an "OK" or "Escape" button. The screen can be used to check the status of the enclosure and the modules in it: one can for example check active alarms on the system, get the IP address of the CMC of KVM, check the system-names etc. Especially for an environment where there are more enclosures in one datacenter it can be useful to check if one are working on the correct enclosure. Unlike the rack or tower-servers there are only a very limited set of indicators on individual servers: a blade server has a power-led and (local) disc-activity led's but no LCD display offering one any alarms, hostnames etc. Nor are there LED's for I/O activity: this is all combined in this little screen giving one information on both the enclosure as well as information over the inserted servers, switches, fans, power-supplies etc. The LCD screen can also be used for the initial configuration of an unconfigured chassis. One can use the LCD screen to set the interface-language and to set the IP address of the CMC for further CLI or web-based configuration. [2] During normal operation the display can be "pushed" into the chassis and is mainly hidden. To use it one would need to pull it out and tilt it to read the screen and have access to the buttons.

Blade 17: Local management I/O

A blade-system is not really designed for local (on-site) management and nearly all communication with the modules in the enclosure and the enclosure itself are done via the "CMC" card(s) at the back of the enclosure. At the front-side of the chassis, directly adjacent to the power-button, one can connect a local terminal: a standard VGA monitor connector and two USB connectors. This connection is referred to inside the system as 'blade 17' and allows one a local interface to the CMC management cards. [2]

iDRAC remote access

Apart from normal operation access to one's blade servers (e.g. SSH sessions to a Linux-based OS, RDP to a Windows-based OS etc.) there are roughly two ways to manage one's server blades: via the iDRAC function or via the iKVM switch. Each blade in the enclosure comes with a built-in iDRAC that allows one to access the console over an IP connection. The iDRAC on a blade-server works in the same way as an iDRAC card on a rack or tower-server: there is a special iDRAC network to get access to the iDRAC function. In rack or tower-servers a dedicated iDRAC Ethernet interface connects to a management LAN. On blade-servers it works the same: via the CMC one configure the setup of iDRAC and access to the iDRAC of a blade is NOT linked to any of the on-board NICs: if all one's server NICs would be down (thus all the on-motherboard NICs and also the Mezzanine B and C) one can still access the iDRAC.

iKVM: Remote console access

Apart from that, one can also connect a keyboard, mouse and monitor directly to the server: on a rack or tower switch one would either connect the I/O devices when needed or one have all the servers connected to a KVM switch. The same is possible with servers in a blade-enclosure: via the optional iKVM module in an enclosure one can access each of one's 16 blades directly. It is possible to include the iKVM switch in an existing network of digital or analog KVM switches. The iKVM switch in the Dell enclosure is an Avocent switch and one can connect (tier) the iKVM module to other digital KVM switches such as the Dell 2161 and 4161 or Avocent DSR digital switches. Also tiering the iKVM to analog KVM switches as the Dell 2160AS or 180AS or other Avocent (compatible) KVM switches is possible. [2] Unlike the CMC, the iKVM switch is not redundant but as one can always access a server (also) via its iDRAC any outage of the KVM switch doesn't stop one from accessing the server-console.

Flex addresses

The M1000e enclosure offers the option of flex-addresses. This feature allows the system administrators to use dedicated or fixed MAC addresses and World Wide Names (WWN) that are linked to the chassis, the position of the blade and location of the I/O interface. It allows administrators to physically replace a server-blade and/or a Mezzanine card while the system will continue to use the same MAC addresses and/or WWN for that blade without the need to manually change any MAC or WWN addresses and avoid the risk of introducing duplicate addresses: with flex-addresses the system will assign a globally unique MAC/WWN based on the location of that interface in the chassis. The flex-addresses are stored on a SD-card that is inserted in the CMC module of a chassis and when used it overwrites the address burned in into the interfaces of the blades in the system. [2]

Power and cooling

The M1000e enclosure is, as most blade systems, for IT infrastructures demanding high availability. (Nearly) everything in the enclosure supports redundant operation: each of the 3 I/O fabrics (A, B and C) support two switches or pass-through cards and it supports two CMC controllers, even though one can run the chassis with only one CMC. Also power and cooling is redundant: the chassis supports up to six power-supplies and nine fan units. All power supplies and fan-units are inserted from the back and are all hot-swappable. [2] The power supplies are located at the bottom of the enclosure while the fan-units are located next to and in between the switch or I/O modules. Each power-supply is a 2700-watt power-supply and uses 208–240 V AC as input voltage. A chassis can run with at least two power-supplies (2+0 non-redundant configuration). Depending on the required redundancy one can use a 2+2 or 3+3 setup (input redundancy where one would connect each group of supplies to two different power sources) or a 3+1, 4+2 or 5+1 setup, which gives protection if one power-supply unit would fail - but not for losing an entire AC power group [1]

Related Research Articles

<span class="mw-page-title-main">Backplane</span> Group of electrical connectors specifically aligned

A backplane or backplane system is a group of electrical connectors in parallel with each other, so that each pin of each connector is linked to the same relative pin of all the other connectors, forming a computer bus. It is used to connect several printed circuit boards together to make up a complete computer system. Backplanes commonly use a printed circuit board, but wire-wrapped backplanes have also been used in minicomputers and high-reliability applications.

<span class="mw-page-title-main">Small Form-factor Pluggable</span> Modular communications interface

Small Form-factor Pluggable (SFP) is a compact, hot-pluggable network interface module format used for both telecommunication and data communications applications. An SFP interface on networking hardware is a modular slot for a media-specific transceiver, such as for a fiber-optic cable or a copper cable. The advantage of using SFPs compared to fixed interfaces is that individual ports can be equipped with different types of transceivers as required, with the majority including optical line terminals, network cards, switches and routers.

<span class="mw-page-title-main">Blade server</span> Server computer that uses less energy and space than a conventional server

A blade server is a stripped-down server computer with a modular design optimized to minimize the use of physical space and energy. Blade servers have many components removed to save space, minimize power consumption and other considerations, while still having all the functional components to be considered a computer. Unlike a rack-mount server, a blade server fits inside a blade enclosure, which can hold multiple blade servers, providing services such as power, cooling, networking, various interconnects and management. Together, blades and the blade enclosure form a blade system, which may itself be rack-mounted. Different blade providers have differing principles regarding what to include in the blade itself, and in the blade system as a whole.

Brocade was an American technology company specializing in storage networking products, now a subsidiary of Broadcom Inc. The company is known for its Fibre Channel storage networking products and technology. Prior to the acquisition, the company expanded into adjacent markets including a wide range of IP/Ethernet hardware and software products. Offerings included routers and network switches for data center, campus and carrier environments, IP storage network fabrics; Network Functions Virtualization (NFV) and software-defined networking (SDN) markets such as a commercial edition of the OpenDaylight Project controller; and network management software that spans physical and virtual devices.

Advanced Telecommunications Computing Architecture is the largest specification effort in the history of the PCI Industrial Computer Manufacturers Group (PICMG), with more than 100 companies participating. Known as AdvancedTCA, the official specification designation PICMG 3.x was ratified by the PICMG organization in December 2002. AdvancedTCA is targeted primarily to requirements for "carrier grade" communications equipment, but has recently expanded its reach into more ruggedized applications geared toward the military/aerospace industries as well. This series of specifications incorporates the latest trends in high speed interconnect technologies, next-generation processors, and improved Reliability, Availability and Serviceability (RAS).

The Dell Remote Access Controller (DRAC) is an out-of-band management platform on certain Dell servers. The platform may be provided on a separate expansion card, or integrated into the main board; when integrated, the platform is referred to as iDRAC.

<span class="mw-page-title-main">IBM BladeCenter</span> Blade server architecture by IBM

The IBM BladeCenter was IBM's blade server architecture, until it was replaced by Flex System in 2012. The x86 division was later sold to Lenovo in 2014.

<span class="mw-page-title-main">Out-of-band management</span> Management of networking equipment

In systems management, out-of-band management is a process for accessing and managing devices and infrastructure at remote locations through a separate management plane from the production network. OOB allows a system administrator to monitor and manage servers and other network-attached equipment by remote control regardless of whether the machine is powered on or whether an OS is installed or functional. It is contrasted to in-band management which requires the managed systems to be powered on and available over their operating system's networking facilities.

The current portfolio of PowerConnect switches are now being offered as part of the Dell Networking brand: information on this page is an overview of all current and past PowerConnect switches as per August 2013, but any updates on current portfolio will be detailed on the Dell Networking page.

Dell Force10, was a United States company that developed and marketed 10 Gigabit and 40 Gigabit Ethernet switches for computer networking to corporate, educational, and governmental customers. It had offices in North America, Europe, and the Asia Pacific region.

The Cisco Nexus series switches are modular and fixed port network switches designed for the data center. Cisco Systems introduced the Nexus Series of switches on January 28, 2008. The first chassis in the Nexus 7000 family is a 10-slot chassis with two supervisor engine slots and eight I/O module slots at the front, as well as five crossbar switch fabric modules at the rear. Beside the Nexus 7000 there are also other models in the Nexus range.

<span class="mw-page-title-main">HPE BladeSystem</span> Line of blade server machines by Hewlett Packard Enterprise

BladeSystem is a line of blade server machines from Hewlett Packard Enterprise that was introduced in June 2006.

EqualLogic, Inc. was an American computer data storage company based in Nashua, New Hampshire, active from 2001 to 2007. In 2008, the company was merged into Dell Inc. Dell-branded EqualLogic products are iSCSI-based storage area network (SAN) systems. Dell has 3 different lines of SAN products: EqualLogic, Compellent and Dell PowerVault.

The Intel Modular Server System is a blade system manufactured by Intel using their own motherboards and processors. The Intel Modular Server System consists of an Intel Modular Server Chassis, up to six diskless Compute Blades, an integrated storage area network (SAN), and three to five Service Modules. The system was formally announced in January 2008. The server is aimed at small to medium businesses with "50 to 300 employees".

<span class="mw-page-title-main">PAROLI</span> Internet routing protocol

PAROLI is a proprietary protocol used inside a multi-shelf Carrier Routing System from Cisco and stands for "parallel optical link."

<span class="mw-page-title-main">Converged network adapter</span> Computer input/output device

A converged network adapter (CNA), also called a converged network interface controller (C-NIC), is a computer input/output device that combines the functionality of a host bus adapter (HBA) with a network interface controller (NIC). In other words, it "converges" access to, respectively, a storage area network and a general-purpose computer network.

HP Virtual Connect is a virtualization technology created by Hewlett-Packard (HP) that de-couples fixed blade server adapter network addresses from the associated external networks so that changes in the blade server infrastructure and the LAN and SAN environments don’t require choreography among server, LAN, and SAN teams for every task. It brings virtualization to the blade server edge. It extends virtual machine technology. Virtual machine technology moves workloads across virtual machines on a single server. It becomes a challenge when moving virtual machines from one physical machine to another or between data center locations because changes to the LAN and SAN environments require manual intervention by network and storage administrators. By pooling and sharing multiple network connections across multiple servers and virtual machines, Virtual Connect extends Data Center capability by allowing physical setup and movement of Virtual Machine workloads between servers and Virtual Machines, transparently from the LAN and SAN infrastructure. Another name for Virtual Connect is PowerConnect Switches.

Dell Networking is the name for the networking portfolio of Dell. In the first half of 2013, Dell started to rebrand their different existing networking product brands to Dell Networking. Dell Networking is the name for the networking equipment that was known as Dell PowerConnect, as well as the Force10 portfolio.

Dell PowerEdge VRTX is a computer hardware product line from Dell. It is a mini-blade chassis with built-in storage system. The VRTX comes in two models: a 19" rack version that is 5 rack units high or as a stand-alone tower system.

References

  1. 1 2 3 4 5 Dell website Tech specs for the M1000e, visited 10 March 2013
  2. 1 2 3 4 5 6 7 8 Dell support website M1000e owners manual, retrieved 26 October 2012
  3. ↑ PowerEdge M1000e Installation Guide, Revision A05, page 47-51. Date: March 2011. Retrieved: 25 January 2013
  4. 1 2 "Details on the Dell PowerEdge M420 Blade Server". BladesMadeSimple.com. May 22, 2012. Retrieved January 29, 2017.
  5. ↑ "Dell PowerEdge M420 Blade Server - Dell". Dell.com. Retrieved January 29, 2017.
  6. 1 2 Dell website: Poweredge M630 Technical specifications, visited 29 August 2016.
  7. ↑ Tech Specs brochure PowerEdge M610, updated 20 December 2011
  8. ↑ Technical specs of the Dell PowerEdge M610x, retrieved 20 December 2011
  9. ↑ Overview of technical specifications of the Poweredge M620, visited 12 June 2012
  10. ↑ Dell website announcing G12 servers with details on virtualisation Archived 2012-06-14 at the Wayback Machine , visited 12 June 2012
  11. ↑ Tech Specs brochure PowerEdge M710, retrieved 27 June 2011
  12. ↑ Tech Specs for Power Edge PowerEdge M710HD, retrieved 20 December 2011
  13. ↑ Dell website: Poweredge M820 Technical specifications, visited 28 July 2012
  14. ↑ Technical specs on the:M910, retrieved 20 December 2011
  15. ↑ Dell website with technical specification of the M915 blade, retrieved 20 December 2011
  16. ↑ Footnote:Except the PE M420 which only supports one Mezzanine card: The PE M420 quarter height blade server only has a Mezzanine B slot
  17. 1 2 Dell support site with an Overview manuals for the M1000e chassis, visited 27 June 2011
  18. ↑ Whitepaper on redundant SD card installation of Hypervisors, visited 19 February 2013
  19. ↑ Technical specifications of the Equallogic PS M4110 blade array, visited 27 September 2012
  20. 1 2 Dell datasheet for the PS-M4110, downloaded: 2 March 2013
  21. 1 2 Using M1000e System with an AMCC QT2025 Backplane PHY in a 10GBASE-KR Application, retrieved 12 June 2012
  22. 1 2 How to find midplane revision of M1000e, visited 19 September 2012
  23. 1 2 PowerConnect M-series User Guide, firmware 4.x, March 2011, retrieved 26 June 2011
  24. ↑ Dell website: available blade-switches PCM8024 not listed as available, 29 December 2011
  25. ↑ Dell website PCM8024-k, visited 29 December 2012
  26. ↑ Release notes page 6 and further included in firmware package PC 4.2.1.3, release-date 2 February 2012, downloaded: 16 February 2012
  27. ↑ Stacking the PowerConnect 10G switches, December 2011. Visited 10 March 2013
  28. ↑ PCM6348 User Configuration Guide [ permanent dead link ], downloaded 10 March 2013
  29. ↑ Dell community website: Dell announces F10 MXL switch, 24 April 2012. Visited 18 May 2012
  30. ↑ EWeek: Dell unveils 40GbE Enabled networking switch [ permanent dead link ], 24 April 2012. Visited 18 May 2012
  31. ↑ Dell website: PowerEdge M I/O Aggregator, August, 2012. Visited: 26 October 2012
  32. ↑ Cisco website: Comprehensive Blade Server I/O Solutions, visited: 14 April 2012
  33. ↑ Catalyst 3032 for Dell, visited: 14 April 2012
  34. ↑ Nexus FEX for M1000e, visited 2 July, 2017
  35. 1 2 Catalyst for Dell at a glance, retrieved: 14 April 2012
  36. ↑ Dell website Catalyst 3130G Archived 2011-06-21 at the Wayback Machine , visited 14 April 2012
  37. ↑ Dell website on Catalyst 3130X Archived 2011-06-21 at the Wayback Machine , visited 14 April 2012
  38. ↑ Cisco datasheet on the Catalyst 3130, section: 3130 software. Visited: 14 April 2012
  39. ↑ TheRegister website: Cisco cuts Nexus 4001d blade switch, 16 February 2010. Visited: 10 March 2013
  40. ↑ Cisco datasheet: Cisco Nexus B22 Blade Fabric Extender Data Sheet, 2013. Downloaded: 10 March 2013
  41. ↑ Manuals and Documents for PowerEdge M1000E [ permanent dead link ], visited 9 March 2013
  42. ↑ Usermanual for the 10GbE XAUI passthrough module [ permanent dead link ], 2010, visited: 10 March 2013
  43. ↑ Usermanual for the 10 Gb passthrough -k for M1000e [ permanent dead link ], 2011. Visited: 10 March 2013
  44. ↑ Brocade M6505 for M1000e chassis, visited 2 July, 2017
  45. ↑ Userguide for the Infiniscal IV [ permanent dead link ], 2009. Downloaded: 10 March 2013
  46. ↑ Dell website Specifications of the M8424 Converged 10Gbe switch, visited 12 October 2012
  47. ↑ Details on the PC-B-8000 switch, visited 18 March 2012
  48. ↑ "Brocade M5424 Blade Server SAN I/O Module Hardware Reference Manual, September 2008" (PDF). Support.Euro.Dell.com. Retrieved 12 October 2012.
  49. ↑ M6505 technical overview, visited 2 July, 2017
  50. ↑ Dell manual: Brocade 4424 Blade Server SAN I/O Module Hardware Reference, November 2007. Downloaded: 12 October 2012
  51. ↑ News, NO: IDG, archived from the original on 2017-01-10, retrieved 2012-10-13
  52. ↑ Cisco datasheet on the SFS M7000e Infiniband switch, March 2008. Visited: 12 October 2012
  53. ↑ Melanox Userguide for the SwitcX M4001 M4001 Infiniband switches, November, 2011. Retrieved: 12 October 2012
  54. 1 2 Melanox userguide for the M2401 Infiniband switch, June, 2008. Visited: 12 October 2012
  55. ↑ Dell website Gigabit passthrough module for M-series Archived 2010-12-18 at the Wayback Machine , visited 26 June 2011
  56. ↑ 10Gb Pass Through Specifications, PDF, retrieved 27 June 2011
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.