Disaggregated storage

Disaggregated storage is a type of data storage within computer data centers. It allows compute resources within a computer server to be separated from storage resources without modifying any physical connections. ^[1]

A form of composable disaggregated infrastructure, disaggregated storage allows resources to be connected via a network fabric providing flexibility when upgrading, replacing, or adding individual resources. It also allows servers to be built for future growth, offering greater storage efficiency, scale and performance than traditional data storage without compromising throughput and latency. ^[2]

Background

In the past, data center storage existed in two forms.

• Direct-attached storage – disks or drives attached to a single server. Disk capacity and performance were available to that server, and only that server. Capacity expansion was limited to the number of drive bays in the server or the limits of expansion chassis. Capacity and performance can scale-up (adding drives to a server) or out (by adding servers). ^[3]
• Storage area networks – disks or drives in a storage array which could be provisioned to one or many servers on the network. Capacity expansion is limited to the number of supported expansion chassis. ^[4]

Direct-attached storage has one critical advantage—it offers high-performance for any workloads running on that server. However, it comes with two critical disadvantages: Overall performance across the network is low, as storage can't be shared over the network without performance impact. Capacity utilization is low because disk capacity can't be directly used by other servers. ^[5]

Storage area networks are used to allocate storage to dozens or possibly hundreds of servers, which increases capacity utilization, but storage area networks use specialized network hardware and/or protocols that can come with disadvantages. ^[6] Conventional storage networking does not provide sufficient throughput or latency minimization needed by many applications, and fails to provide enough bandwidth to utilize the full performance of new flash technologies. ^[7]

Disaggregated storage overview

Disaggregated storage is a form of scale-out storage, built with some number of storage devices that function as a logical pool of storage that can be allocated to any server on the network over a very high performance network fabric. Disaggregated storage solves the limitations of storage area networks or direct-attached storage. Disaggregated storage is dynamically reconfigurable and optimally reconfigures physical resources to maximize performance and limit latency. ^[8] Disaggregated storage provides the performance of local storage with the flexibility of storage area networks.

A number of technology improvements are combining to make storage disaggregation a reality. These include:

• Modern server performance: due to the PCIe Gen 4 serial bus, many servers can deliver more than 8 GB/sec of throughput, which far exceeds traditional storage networking performance capabilities.
• The shift toward NVMe: The shift from disk to SAS/SATA flash, and now NVMe flash, puts pressure on servers and networks alike. A single NVMe drive delivers millions of IOPS, far beyond the usual capabilities of conventional storage networking. ^[9]
• 10Gb/25Gb/40Gb/100Gb Ethernet. More and more data centers are replacing slower network connections with faster Ethernet, removing bandwidth limitations and bottlenecks. ^[10]

Protocols like NVMe-oF on these very high bandwidth connections take full advantage of network improvements, removing bottlenecks, boosting performance and reducing latency.

Different levels of storage disaggregation functionality exist, with the most flexible, ^[11] full disaggregation, enabling storage capacity and/or performance to be provisioned from any storage device to any server on the network, then expanded, shrunk, or reprovisioned as new requirements emerge.

References

1. "Storage Disaggregation in the Data Center". Data Center Knowledge. 2013-10-18. Retrieved 2019-11-30.
2. Tomsho, Greg (2017-07-26). MCSA Guide to Installation, Storage, and Compute with Microsoft Windows Server2016, Exam 70-740. Cengage Learning. ISBN   978-1-337-40066-4.
3. "What is direct-attached storage (DAS)? - Definition from WhatIs.com". SearchStorage. Retrieved 2019-11-30.
4. "What Is a Storage Area Network (SAN)? | SNIA". www.snia.org. Retrieved 2019-11-30.
5. "Direct Attached Storage". www.enterprisestorageforum.com. 15 May 2019. Retrieved 2019-11-30.
6. "Fibre Channel Really Is Dead". IT Infrastructure Advice, Discussion, Community - Network Computing. 2015-04-23. Retrieved 2019-11-30.
7. "Advantages and disadvantages of storage area network (SAN)". IT Release. 2019-05-02. Retrieved 2019-11-30.
8. Zervas, G.; Yuan, H.; Saljoghei, A.; Chen, Q.; Mishra, V. (February 2018). "Optically disaggregated data centers with minimal remote memory latency: Technologies, architectures, and resource allocation [Invited]" . IEEE/OSA Journal of Optical Communications and Networking. 10 (2): A270 –A285. doi:10.1364/JOCN.10.00A270. ISSN   1943-0639. S2CID   3397054.
9. "Industry Outlook: NVMe and NVMe-oF For Storage". InterOperability Laboratory. 2018-02-08. Retrieved 2019-11-30.
10. John, Kim, Chair SNIA Ethernet Storage Forum. "Servers and storage rapidly adopting 25GbE and 100 GbE networking" (PDF). snia.org. Retrieved 30 November 2019.
11. Understanding Rack-Scale Disaggregated StorageSergey Legtchenko, Hugh Williams, Kaveh Razavi†, Austin Donnelly, Richard Black, Andrew Douglas, Nathanael Cheriere†, Daniel Fryer†, Kai Mast†, Angela Demke Brown‡, Ana Klimovic†, Andy Slowey and Antony Rowstron. "Understanding Rack-Scale Disaggregated Storage" (PDF). usenix.org. Retrieved 30 November 2019.