Shingled magnetic recording

Last updated

Partial updating of data is difficult with SMR. Data will be written to adjacent tracks that do not need to be rewritten. SMR HDD Rewrite.png
Partial updating of data is difficult with SMR. Data will be written to adjacent tracks that do not need to be rewritten.

Shingled magnetic recording (SMR) is a magnetic storage data recording technology used in hard disk drives (HDDs) to increase storage density and overall per-drive storage capacity. [1] Conventional hard disk drives record data by writing non-overlapping concentric magnetic tracks (conventional magnetic recording, CMR), while shingled recording writes new tracks that overlap part of the previously written magnetic track, leaving the previous track narrower and allowing higher track density. Thus, the tracks partially overlap similar to roof shingles. This approach was selected because, if the writing head is made too narrow, it cannot provide the very high fields required in the recording layer of the disk. [2] [3] [4] [5] : 7–9 

Contents

The overlapping-tracks architecture complicates the writing process, since writing to one track also overwrites an adjacent track. If adjacent tracks contain valid data, they must be rewritten as well. As a result, SMR drives are divided into many append-only (sequential) zones of overlapping tracks that need to be rewritten entirely when full, resembling flash blocks in solid-state drives. Device-managed SMR devices hide this complexity by managing it in the firmware, presenting an interface like any other hard disk. Other SMR devices are host-managed and depend on the operating system to know how to handle the drive, and only write sequentially to certain regions of the drive. [5] : 11 ff.  [6] While SMR drives can use DRAM, flash memory, and even a portion of their own platter reserved for use with CMR instead of SMR, [7] as a cache to improve writing performance, continuous writing of large amount of data is noticeably slower than with CMR drives. [8] [9] [10]

History

Seagate started shipping device-managed SMR hard drives in September 2013, stating an increase in overall capacity of about 25% compared to non-shingled storage. [1] [11] In September 2014, HGST announced a 10 TB drive filled with helium that uses host-managed shingled magnetic recording, [12] although in December 2015 it followed this with a 10 TB helium-filled drive that uses conventional non-shingled perpendicular recording. [13]

Western Digital (WD), Toshiba, and Seagate have sold SMR drives without labeling them as such, generating a large controversy, as SMR drives behave much slower under some circumstances (such as random writes) than CMR drives. [9] Some have even claimed that these may cause data loss. [14] These mislabeling practices were used in both consumer-centric and dedicated data storage HDDs for servers, NASes, RAIDs, and cold storage. A United States class-action suit against Western Digital alleging that the technology is inferior was settled on or before August 27, 2021. [15] Due to the controversy, as of June 2020, Western Digital labels their DM-SMR NAS drives as "WD Red" and brands their CMR NAS drives as "WD Red Plus" and "WD Red Pro". [16] [14]

Heavily overlapped (shingled) tracks also appeared earlier in the consumer helical scan video cassette recorders (VCRs) that were popular in the 1980s and 1990s. In Extended Play (EP or SLP) mode, both VHS and Betamax reduced the track pitch by a factor of three. The severe interference from the adjacent tracks was partially mitigated by the use of slant azimuth recording.

Data management

There are three different ways that data can be managed on an SMR drive: device-managed, host-managed and host-aware. [17] [18]

Device-managed

A device-managed or drive-managed drive appears to the host identically to a non-shingled drive. It is not necessary for the host to follow any special protocols. All handling of data, as it relates to the shingled nature of the storage, is managed by the device. Sequential writes are more efficient. In addition, the host is unaware that the storage is shingled. [5]

The disk controller in a device-managed drive internally handles any re-writing required by the special characteristics of a shingled drive, similar to the way a flash memory controller in a solid-state drive internally handles re-writing required by the special characteristics of flash media. [19] SMR drives have append-only zones which operate like SSD blocks, where a sector in a zone cannot be modified without re-writing the entire contents of the zone, so writes are first sent to a CMR cache and the disk moves these data to SMR parts when idle. [20]

Until backlash against Western Digital in 2020, this type of SMR drive was often not labelled by the manufacturer, except in disks labelled as "archival". [9] [10]

RAID resilvering tends to overload the cache,[ why? ] sending SMR drives into minutes-long pauses.[ citation needed ] Faulty firmware (such as revision 82.00A82 on the WD40EFAX) may also cause the drive to return IDNF S.M.A.R.T. errors under intensive workloads. [7] [19] Both behaviors tend to be interpreted as drive failure by the RAID controller. [20]

The zoned nature of SMR also means that the disk suffers from write amplification when garbage collecting, [21] although for hard drives the main problem with writes is speed instead of longevity. Some SMR hard drives support the TRIM command for this reason. [22]

Host-managed

A host-managed device requires strict adherence to a special protocol by the host. Since the host manages the shingled nature of the storage, it is required to write sequentially so as to not destroy existing data. The drive will refuse to execute commands which violate this protocol. [5]

Host-aware

Host-aware is a combination of drive-managed and host-managed. The drive is capable of managing the shingled nature of the storage and will execute any command the host gives it, regardless of if it is sequential or not. However, the host is aware that the drive is shingled, and able to query the drive for fill levels. This allows the host to optimize writes for the shingled nature, while also allowing the drive to be flexible and backwards-compatible. [5]

Protocol

SMR devices are considered zoned devices, as the storage is divided into zones of usually 256 MiB size. [23] Two sets of specialized commands, ZBC (Zoned Block Commands, ANSI INCITS 536) for SCSI and ZAC (Zoned ATA Commands, ANSI INCITS 537) for SATA are available for SMR devices. They tell the host about whether each zone is CMR or SMR and allow them to address these zones directly. [24] Unless specifically mentioned, the commands are only available on host-aware/-managed devices. The specific commands are: [25]

Each zone has a range of LBA addresses associated with it, and all LBA-based commands can be used as long as the sequential requirement is followed on host-managed drives.

SMR devices identify themselves per the following: [26] [27] [5] : 14 

A newer version of the sibling standards, ZAC-2/ZBC-2 is under development. The new version introduces a new type of "domains and realms zoned block devices" that allow for non-contiguous LBAs, and it can benefits for SATA SSDs. [28] The ZONED field has been retired following a proposal from Western Digital. [29]

The zoned interface is also useful for flash storage. ZNS spec has been released by the NVM Express organization. [30]

Software and application

The higher density of SMR drives, combined with its random-read nature, fills a niche between the sequential-access tape storage and the random-access conventional hard drive storage. They are suited to storing data that are unlikely to be modified, but need to be read from any point efficiently. One example of the use case is Dropbox's Magic Storage system, which runs the on-disk extents in an append-only way. [31] Device-managed SMR disks have also been marketed as "Archive HDDs" due to this property. [32]

A number of file systems in Linux are or can be tuned for SMR drives: [33]

In addition to Linux, FreeBSD has protocol-level support for host-managed SMR drives. [23] [38] As of April 2024, neither Windows nor MacOS supports the HM-SMR drives. [39] However, Windows 11 and Windows 10 22H2 have support of ZAC / ZBC command sets for HA-SMR HDDs and SSDs, as they support non-sequential LBAs. [40]

Dynamic hybrid SMR

While for traditional SMR models each zone is assigned a type at manufacture time, dynamic hybrid SMR drives allow to reconfigure the zone type from shingled to conventional and back by the customer. [41] [42] Adjusting the SMR/CMR setting helps suit the drive to the current workload of "hot" and "cold" data. [21]

Criticism

SMR drives have received criticism online for slow read / write speeds and low stability. [43] [44] [45] [46] [47] Drive manufacturers have been further criticised for attempting to conceal the use of SMR technology in their products. [48] [49]

See also

Related Research Articles

<span class="mw-page-title-main">Disk storage</span> General category of storage mechanisms

Disk storage is a data storage mechanism based on a rotating disk. The recording employs various electronic, magnetic, optical, or mechanical changes to the disk's surface layer. A disk drive is a device implementing such a storage mechanism. Notable types are hard disk drives (HDD), containing one or more non-removable rigid platters; the floppy disk drive (FDD) and its removable floppy disk; and various optical disc drives (ODD) and associated optical disc media.

<span class="mw-page-title-main">Hard disk drive</span> Electro-mechanical data storage device

A hard disk drive (HDD), hard disk, hard drive, or fixed disk is an electro-mechanical data storage device that stores and retrieves digital data using magnetic storage with one or more rigid rapidly rotating platters coated with magnetic material. The platters are paired with magnetic heads, usually arranged on a moving actuator arm, which read and write data to the platter surfaces. Data is accessed in a random-access manner, meaning that individual blocks of data can be stored and retrieved in any order. HDDs are a type of non-volatile storage, retaining stored data when powered off. Modern HDDs are typically in the form of a small rectangular box.

<span class="mw-page-title-main">Western Digital</span> American digital storage company

Western Digital Corporation is an American computer drive manufacturer and data storage company, headquartered in San Jose, California. It designs, manufactures and sells data technology products, including data storage devices, data center systems and cloud storage services.

<span class="mw-page-title-main">Seagate Technology</span> American data storage company

Seagate Technology Holdings plc is an American data storage company. It was incorporated in 1978 as Shugart Technology and commenced business in 1979. Since 2010, the company has been incorporated in Dublin, Ireland, with operational headquarters in Fremont, California, United States.

A log-structured filesystem is a file system in which data and metadata are written sequentially to a circular buffer, called a log. The design was first proposed in 1988 by John K. Ousterhout and Fred Douglis and first implemented in 1992 by Ousterhout and Mendel Rosenblum for the Unix-like Sprite distributed operating system.

<span class="mw-page-title-main">Self-Monitoring, Analysis and Reporting Technology</span> Monitoring system in computer drives

Self-Monitoring, Analysis, and Reporting Technology is a monitoring system included in computer hard disk drives (HDDs) and solid-state drives (SSDs). Its a primary function is to detect and report various indicators of drive reliability, or how long a drive can function while anticipating imminent hardware failures.

<span class="mw-page-title-main">Magnetic storage</span> Recording of data on a magnetizable medium

Magnetic storage or magnetic recording is the storage of data on a magnetized medium. Magnetic storage uses different patterns of magnetisation in a magnetizable material to store data and is a form of non-volatile memory. The information is accessed using one or more read/write heads.

Density is a measure of the quantity of information bits that can be stored on a given physical space of a computer storage medium. There are three types of density: length of track, area of the surface, or in a given volume.

<span class="mw-page-title-main">Microdrive</span> One-inch hard disk format by IBM and Hitachi

The Microdrive is a type of miniature, 1-inch hard disk produced by IBM and Hitachi. These rotational media storage devices were designed to fit in CompactFlash (CF) Type II slots.

Perpendicular recording, also known as conventional magnetic recording (CMR), is a technology for data recording on magnetic media, particularly hard disks. It was first proven advantageous in 1976 by Shun-ichi Iwasaki, then professor of the Tohoku University in Japan, and first commercially implemented in 2005. The first industry-standard demonstration showing unprecedented advantage of PMR over longitudinal magnetic recording (LMR) at nanoscale dimensions was made in 1998 at IBM Almaden Research Center in collaboration with researchers of Data Storage Systems Center (DSSC) – a National Science Foundation (NSF) Engineering Research Center (ERCs) at Carnegie Mellon University (CMU).

A hybrid drive is a logical or physical computer storage device that combines a faster storage medium such as solid-state drive (SSD) with a higher-capacity hard disk drive (HDD). The intent is adding some of the speed of SSDs to the cost-effective storage capacity of traditional HDDs. The purpose of the SSD in a hybrid drive is to act as a cache for the data stored on the HDD, improving the overall performance by keeping copies of the most frequently used data on the faster SSD drive.

Heat-assisted magnetic recording (HAMR) is a magnetic storage technology for greatly increasing the amount of data that can be stored on a magnetic device such as a hard disk drive by temporarily heating the disk material during writing, which makes it much more receptive to magnetic effects and allows writing to much smaller regions.

<span class="mw-page-title-main">History of hard disk drives</span> Development of computer data storage

In 1953, IBM recognized the immediate application for what it termed a "Random Access File" having high capacity and rapid random access at a relatively low cost. After considering technologies such as wire matrices, rod arrays, drums, drum arrays, etc., the engineers at IBM's San Jose California laboratory invented the hard disk drive. The disk drive created a new level in the computer data hierarchy, then termed Random Access Storage but today known as secondary storage, less expensive and slower than main memory but faster and more expensive than tape drives.

<span class="mw-page-title-main">Solid-state drive</span> Computer storage device with no moving parts

A solid-state drive (SSD) is a type of solid-state storage device that uses integrated circuits to store data persistently. It is sometimes called semiconductor storage device, solid-state device, and solid-state disk.

<span class="mw-page-title-main">Seagate Barracuda</span> Series of hard disk drives produced by Seagate Technology

The Seagate Barracuda is a series of hard disk drives and later solid state drives produced by Seagate Technology that was first introduced in 1993.

<span class="mw-page-title-main">Advanced Format</span> Disk format and access using sector sizes larger than 512 bytes

Advanced Format (AF) is any disk sector format used to store data on magnetic disks in hard disk drives (HDDs) that exceeds 528 bytes per sector, frequently 4096, 4112, 4160, or 4224-byte (4 KB) sectors. Larger sectors of an Advanced Format Drive (AFD) enable the integration of stronger error correction algorithms to maintain data integrity at higher storage densities.

Higher performance in hard disk drives comes from devices which have better performance characteristics. These performance characteristics can be grouped into two categories: access time and data transfer time .

bcache is a cache mechanism in the Linux kernel's block layer, which is used for accessing secondary storage devices. It allows one or more fast storage devices, such as flash-based solid-state drives (SSDs), to act as a cache for one or more slower storage devices, such as hard disk drives (HDDs); this effectively creates hybrid volumes and provides performance improvements.

<span class="mw-page-title-main">Two-dimensional magnetic recording</span>

Two-dimensional magnetic recording (TDMR) is a technology introduced in 2017 in hard disk drives (HDD) used for computer data storage. Most of the world's data is recorded on HDDs, and there is continuous pressure on manufacturers to create greater data storage capacity in a given HDD form-factor and for a given cost. In an HDD, data is stored using magnetic recording on a rotating magnetic disk and is accessed through a write-head and read-head. TDMR allows greater storage capacity by advantageously combining signals simultaneously from multiple read-back heads to enhance the recovery of one or more data-tracks. In this manner, data can be stored with higher areal-density on the disks thus providing higher capacity in each HDD. TDMR is a read-back technology and thus applies equally well to future recording (writing) technologies such as heat-assisted magnetic recording (HAMR) and microwave-assisted magnetic recording (MAMR).

Append-only is a property of computer data storage such that new data can be appended to the storage, but where existing data is immutable.

References

  1. 1 2 Anand Lal Shimpi (September 9, 2013). "Seagate to Ship 5TB HDD in 2014 using Shingled Magnetic Recording". AnandTech . Retrieved February 9, 2015.
  2. ↑ Roger Wood (October 19, 2010). "Shingled Magnetic Recording and Two-Dimensional Magnetic Recording" (PDF). IEEE . Retrieved December 14, 2014.
  3. ↑ "What is Shingled Magnetic Recording (SMR)?". storagereview.com. January 30, 2015. Retrieved February 9, 2015.
  4. ↑ K. Shimomura, "Large-Capacity HDDs Applying SMR Technology for Data Centers", Toshiba Technology Review, Vol. 24, No. 6., pp. 12-16, Nov 2019
  5. 1 2 3 4 5 6 Mary Dunn; Timothy Feldman (September 22, 2014). "Shingled Magnetic Recording: Models, Standardization, and Applications" (PDF). Storage Networking Industry Association . Retrieved February 9, 2015.
  6. ↑ Jake Edge (March 26, 2014). "Support for shingled magnetic recording devices". LWN.net . Retrieved December 14, 2014.
  7. 1 2 Taillac, Will (May 28, 2020). "WD Red SMR vs CMR Tested Avoid Red SMR". ServeTheHome. Retrieved February 13, 2023.
  8. ↑ Shilov, Anton. "The Evolution of HDDs in the Near Future: Speaking with Seagate CTO, Mark Re". www.anandtech.com. Retrieved May 30, 2020.
  9. 1 2 3 Salter, Jim. "Sneaky Marketing Redux: Toshiba, Seagate Shipping Slower SMR Drives Without Disclosure, Too". Ars Technica. Retrieved June 17, 2020.
  10. 1 2 Alcorn, Paul. "Sneaky Marketing Redux: Toshiba, Seagate Shipping Slower SMR Drives Without Disclosure, Too". Tom’s Hardware. Retrieved April 17, 2020.
  11. ↑ "Seagate Delivers On Technology Milestone: First To Ship Hard Drives Using Next-Generation Shingled Magnetic Recording". www.seagate.com. September 9, 2013. Retrieved October 28, 2021.
  12. ↑ Geoff Gasior (September 9, 2020). "Shingled platters breathe helium inside HGST's 10TB hard drive". The Tech Report. Retrieved February 9, 2020.
  13. ↑ Sebastian Anthony (December 3, 2020). "HGST releases helium-filled 10TB hard drive; Seagate twiddles shingled fingers". Ars Technica. Retrieved December 3, 2015.
  14. 1 2 Team, iX (June 29, 2020). "WD Red Plus drives are "Coke Classic"". TrueNAS - Welcome to the Open Storage Era. Retrieved September 24, 2023.
  15. ↑ "Western Digital Hard Drive $2.7M Class Action Settlement". Top Class Actions. Archived from the original on November 24, 2021. Retrieved November 24, 2021.
  16. ↑ S, Ganesh T. "Western Digital Announces Red Plus HDDs, Cleans Up Red SMR Mess with Plus Branding". www.anandtech.com. Retrieved September 24, 2023.
  17. ↑ "Zoned Block Commands (ZBC)" (PDF). t10.org. ANSI T10 Committee. Retrieved January 22, 2018.
  18. ↑ Campello, Jorge (September 24, 2015). "SMR: The Next Generation of Storage Technology" (PDF). Retrieved January 22, 2018.
  19. 1 2 "WD Red SMR Drive Compatibility with ZFS". www.truenas.com. Retrieved February 15, 2024.
  20. 1 2 Mellor, Chris (April 15, 2020). "Shingled hard drives have non-shingled zones for caching writes". Blocks and Files.
  21. 1 2 Brewer, Eric; Ying, Lawrence; Greenfield, Lawrence; Cypher, Robert; T'so, Theodore (2016). "Disks for Data Centers". Proceedings of USENIX FAST 2016.
  22. ↑ "TRIM Command Support for WD External Drives". WD support.
  23. 1 2 zonectl(8)  –  FreeBSD System Manager's Manual
  24. ↑ "SMR (Shingled Magnetic Recording) 101". Tom's IT Pro. Archived from the original on June 11, 2017. Retrieved March 3, 2018.
  25. ↑ "Introduction to Shingled Magnetic Recording". ZonedStorage.io.
  26. ↑ "Information technology - ATA Command Set - 4 (ACS-4), Draft revision 18" (PDF).
  27. ↑ Seagate. "SCSI Commands Reference Manual, Rev. J" (PDF). p. 472.
  28. ↑ T10, 2020.
  29. ↑ Weber, Ralph O (April 23, 2020). "SBC-5, ZBC-2: Obsolete the ZONED field" (PDF). www.t10.org.
  30. ↑ "NVMe Command Set Specifications". January 10, 2020.
  31. ↑ Magic Pocket Hardware Engineering Teams. "Extending Magic Pocket Innovation with the first petabyte scale SMR drive deployment". dropbox.tech.
  32. ↑ "Archive HDD" (PDF). Seagate. Retrieved March 3, 2019.
  33. ↑ "File Systems". ZonedStorage.io.
  34. ↑ "Zoned mode — BTRFS documentation". btrfs.readthedocs.io. Retrieved March 26, 2024.
  35. ↑ "Seagate/SMR_FS-EXT4: an addition to the popular EXT4 to enable support for devices that use the ZBC or ZAC standards". Seagate Technology. December 10, 2019.
  36. ↑ "Device Mapper". ZonedStorage.io.
  37. ↑ Le Moal, Damien. "fs: New zonefs file system". lwn.net.
  38. ↑ Merry, Kenneth (May 19, 2015). "FreeBSD Revision 300207: Add support for managing Shingled Magnetic Recording (SMR) drives".
  39. ↑ "Seagate Exos X26z Review - 25TB Host-Managed SMR HDD". StorageReview.com. Retrieved July 31, 2024.
  40. ↑ ATA/ATAPI Command Set 4 (ACS-4)
  41. ↑ Collins, Brendan (November 13, 2017). "Dynamic Hybrid SMR". Western Digital. Retrieved August 25, 2018.
  42. ↑ "Dynamic Hybrid-SMR: an OCP proposal to improve data center disk drives". blog.google. November 13, 2017. Retrieved January 22, 2018.
  43. ↑ https://www.tenforums.com/performance-maintenance/106725-very-slow-boot-time-100-disk-active-time-post2019066.html#post2019066.{{cite web}}: Missing or empty |title= (help)
  44. ↑ Illeazar (August 3, 2020). "I knew SMR had slow write speeds, but 1MB/second?". r/DataHoarder. Retrieved January 29, 2024.
  45. ↑ NeaZerros (June 22, 2021). "Slow read speeds on SMR drives". r/DataHoarder. Retrieved January 29, 2024.
  46. ↑ vanderworp (January 17, 2023). "SMR hard disks: Avoid them!". vanderworp.org. Retrieved January 29, 2024.
  47. ↑ Taillac, Will (May 28, 2020). "WD Red SMR vs CMR Tested Avoid Red SMR". ServeTheHome. Retrieved January 30, 2024.
  48. ↑ Mellor, Chris (April 15, 2020). "Seagate 'submarines' SMR into 3 Barracuda drives and a Desktop HDD". Blocks and Files. Retrieved January 30, 2024.
  49. ↑ updated, Paul Alcorn last (April 14, 2020). "Western Digital Fesses Up: Some Red HDDs Use Slow SMR Tech Without Disclosure". Tom's Hardware. Retrieved January 30, 2024.

Specifications

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.