Advanced Format

Last updated
Advanced Format (AF)
Advanced format logo.png
Advanced Format 512e logo
Generation-one standard
4096 (4  KiB) bytes per sector
Generation-one categories
512 emulation (512e)4K physical sectors on the drive media with 512 byte logical configuration
4K native (4Kn)4K physical sectors on the drive media and 4K configuration reported to the host
4K-ready host [1] A host system which works equally well with legacy 512 as well as 512e hard disk drives
Year standard completed
March 2010
Created by
IDEMA Long Data Sector Committee, composed of Dell, Fujitsu (now Toshiba Storage Device Corporation), Hewlett-Packard, Hitachi Global Storage Technologies, IDEMA, LSI Corporation, Maxtor (now Seagate), Microsoft, Phoenix Technologies, Samsung, Seagate Technology, Western Digital

Advanced Format (AF) is any disk sector format used to store data on magnetic disks in hard disk drives (HDDs) that exceeds 528 bytes per sector, frequently 4096, 4112, 4160, or 4224-byte sectors. Larger sectors of an Advanced Format Drive (AFD) enable the integration of stronger error correction algorithms to maintain data integrity at higher storage densities.

Contents

History

The use of long data sectors was suggested in 1998 in a technical paper issued by the National Storage Industry Consortium (NSIC) [2] calling attention to the conflict between continuing increases in areal density and the traditional 512-byte-per-sector format used in hard disk drives. [3] Without revolutionary breakthroughs in magnetic recording system technologies, areal densities, and with them the storage capacities, hard disk drives were projected to stagnate.

The storage industry trade organization, International Disk Drive Equipment and Materials Association (IDEMA), responded by organizing the IDEMA Long Data Sector Committee in 2000, where IDEMA and leading hardware and software suppliers collaborated on the definition and development of standards governing long data sectors, including methods by which compatibility with legacy computing components would be supported. [3] In August 2005, Seagate shipped test drives with 1K physical sectors to industry partners for testing. [4] : Figure 3  In 2010, industry standards for the first official generation of long data sectors using a configuration of 4096 bytes per sector, or 4K, were completed. All hard drive manufacturers committed to shipping new hard drive platforms for desktop and notebook products with the Advanced Format sector formatting by January 2011. [4] [5]

Advanced Format was coined to cover what was expected to become several generations of long-data-sector technologies, and its logo was created to distinguish long-data-sector–based hard disk drives from those using legacy 512-byte sector. Enterprise disks can be formatted with additional 8-byte Data Integrity Fields, resulting in a 520 or 528-byte physical sectors. [6]

Overview

Comparison of 512- and 4096-byte sector formats [7]
Description512-byte sector4096-byte sector
Gap, sync, address mark15 bytes
User data512 bytes4096 bytes
Error-correcting code50 bytes100 bytes
Total577 bytes4211 bytes
Efficiency88.7%97.3%
512-byte emulated device sector size
0123456789101112131415
Physical sector 1Physical sector 2

Generation-one Advanced Format, 4K sector technology, uses the storage surface media more efficiently by combining data that would have been stored in eight 512-byte sectors into one single sector that is 4096 bytes in length. Key design elements of the traditional 512-byte sector architecture are maintained, specifically, the identification and synchronization marks at the beginning and the error correction coding (ECC) area at the end of the sector. Between the sector header and ECC areas, eight 512-byte sectors are combined, eliminating the need for redundant header areas between each individual chunk of 512-byte data. The Long Data Sector Committee selected the 4K block length for the first generation AF standard for several reasons, including its correspondence to the paging size used by processors and some operating systems as well as its correlation to the size of standard transactions in relational database systems. [8]

Format efficiency gains resulting from the 4K sector structure range from 7 to 11 percent in physical platter space. [9] The 4K format provides enough space to expand the ECC field from 50 to 100 bytes to accommodate new ECC algorithms. The enhanced ECC coverage improves the ability to detect and correct processed data errors beyond the 50-byte defect length associated with the 512-byte sector legacy format. [10] The Advanced Format standard employs the same gap, sync and address mark configuration as the traditional 512-byte sector layout, but combines eight 512-byte sectors into one data field. [11]

Hard disk drive format efficiency with Advanced Format 4K technology and distributed ECC Advanced format (4Kib) HDD sector.svg
Hard disk drive format efficiency with Advanced Format 4K technology and distributed ECC

Having a huge number of legacy 512-byte-sector–based hard disk drives shipped up to the middle of 2010, many systems, programs and applications accessing the hard disk drive are designed around the 512-byte-per-sector convention. Early engagement with the Long Data Sector Committee provided the opportunity for component and software suppliers to prepare for the transition to Advanced Format.

For example, Windows Vista, Windows 7, Windows Server 2008, and Windows Server 2008 R2 (with certain hotfixes installed) support 512e format drives (but not 4Kn), [12] as do contemporary versions of FreeBSD [13] [14] [15] and Linux. [16] [17] Mac OS X Tiger and onwards can use Advanced Format drives [18] and OS X Mountain Lion 10.8.2 additionally supports encrypting those. Windows 8 and Windows Server 2012 also support 4Kn Advanced Format. [12] Oracle Solaris 10 and 11 support 4Kn and 512e hard disk drives for non-root ZFS file systems, while version 11.1 provides installation and boot support for 512e devices. [19] Prior to Windows Vista, Windows 2000 and Windows XP use 4096 bytes as default allocation unit size when use NTFS to format local hard disks, but do not align to 4096-byte boundaries.

Categories

Among the Advanced Format initiatives undertaken by the Long Data Sector Committee, methods to maintain backward compatibility with legacy computing solutions were also addressed. For this purpose, several categories of Advanced Format devices were created.

512 emulation (512e)

Many host computer hardware and software components assume the hard drive is configured around 512-byte sector boundaries. This includes a broad range of items including chipsets, operating systems, database engines, hard drive partitioning and imaging tools, backup and file system utilities as well as a small fraction of other software applications. In order to maintain compatibility with legacy computing components, many hard disk drive suppliers support Advanced Format technologies on the recording media coupled with 512-byte conversion firmware. Hard drives configured with 4096-byte physical sectors with 512-byte firmware are referred to as Advanced Format 512e, or 512 emulation drives. On 512e drives, one LBA is equal to 512 bytes.

Potential areas using 512-byte-based code Afdiag3.jpg
Potential areas using 512-byte-based code

The translation of the native 4096, 4112, 4160, or 4224-byte physical format (with 0, 8, 64, or 128-byte Data Integrity Fields) to a virtual 512, 520 or 528-byte increment is transparent to the entity accessing the hard disk drive. Read and write commands are issued to Advanced Format drives in the same format as legacy drives. However, during the read process, the Advanced Format hard drive loads the entire 4096-byte sector containing the requested 512-byte data into memory located on the drive. The emulation firmware extracts and re-formats the specific data into a 512-byte chunk before sending the data to the host. The entire process typically occurs with little or no degradation in performance.

The translation process is more complicated when writing data that is not a multiple of 4K or not aligned to a 4K boundary. In these instances, the hard drive must read the entire 4096-byte sector containing the targeted data into internal memory, integrate the new data into the previously existing data and then rewrite the entire 4096-byte sector onto the disk media. This operation, known as read-modify-write (RMW), can require additional revolution of the magnetic disks, resulting in a perceptible performance impact to the system user. Performance analysis conducted by IDEMA and the hard drive vendors indicates that approximately five to ten percent of all write operations in a typical business PC user environment may be misaligned and a RMW performance penalty incurred. [20] [21]

When using Advanced Format drives with legacy operating systems, it is important to realign the disk drive using software provided by the hard disk manufacturer. Disk realignment is necessary to avoid a performance degrading condition known as cluster straddling where a shifted partition causes filesystem clusters to span partial physical disk sectors. Since cluster-to-sector alignment is determined when creating hard drive partitions, the realignment software is used after partitioning the disk. This can help reduce the number of unaligned writes generated by the computing ecosystem. Further activities to make applications ready for the transition to Advanced Format technologies were spearheaded by the Advanced Format Technology Committee (formerly Long Data Sector Committee) [22] [23] and by the hard disk drive manufacturers. [24] [25] [26]

4K native (4Kn)

Advanced Format 4K native logo Advanced Format 4Kn logo.png
Advanced Format 4K native logo

For hard disk drives working in the 4K native mode, there is no emulation layer in place, and the disk media directly exposes its 4096, 4112, 4160, or 4224-byte physical sector size to the system firmware and operating system. That way, the externally visible logical sectors organization of the 4K native drives is directly mapped to their internal physical sectors organization. Since April 2014, enterprise-class 4K native hard disk drives have been available on the market. [27] [28]

Readiness of the support for 4096-byte logical sectors within operating systems differs among their types, vendors and versions. [12] For example, Microsoft Windows supports 4K native drives since Windows 8 and Windows Server 2012 (both released in 2012) in UEFI. [29] 4K native drives may work on older operating systems such as Windows 7, but cannot be used as boot drive. [30]

Linux supports 4K native drives since the Linux kernel version 2.6.31 and util-linux-ng version 2.17 (released in 2009 and 2010, respectively). [31] [32] [33]

The color version of the logo indicating a 4K native drive is somewhat different from the 512e logo, featuring four rounded corners, a blue background, and text "4Kn" at the center of the logo. [34]

See also

Related Research Articles

A binary prefix is a unit prefix that indicates a multiple of a unit of measurement by an integer power of two. The most commonly used binary prefixes are kibi (symbol Ki, meaning 210 = 1024), mebi (Mi, 220 = 1048576), and gibi (Gi, 230 = 1073741824). They are most often used in information technology as multipliers of bit and byte, when expressing the capacity of storage devices or the size of computer files.

<span class="mw-page-title-main">Hard disk drive</span> Electro-mechanical data storage device

A hard disk drive (HDD), hard disk, hard drive, or fixed disk is an electro-mechanical data storage device that stores and retrieves digital data using magnetic storage with one or more rigid rapidly rotating platters coated with magnetic material. The platters are paired with magnetic heads, usually arranged on a moving actuator arm, which read and write data to the platter surfaces. Data is accessed in a random-access manner, meaning that individual blocks of data can be stored and retrieved in any order. HDDs are a type of non-volatile storage, retaining stored data when powered off. Modern HDDs are typically in the form of a small rectangular box.

File Allocation Table (FAT) is a file system developed for personal computers and was the default filesystem for the MS-DOS and Windows 9x operating systems. Originally developed in 1977 for use on floppy disks, it was adapted for use on hard disks and other devices. The increase in disk drive capacity over time drove modifications to the design that resulted in versions: FAT12, FAT16, FAT32, and exFAT. FAT was replaced with NTFS as the default file system on Microsoft operating systems starting with Windows XP. Nevertheless, FAT continues to be commonly used on relatively small capacity solid-state storage technologies such as SD card, MultiMediaCard (MMC) and eMMC because of its compatibility and ease of implementation.

RAID is a data storage virtualization technology that combines multiple physical disk drive components into one or more logical units for the purposes of data redundancy, performance improvement, or both. This is in contrast to the previous concept of highly reliable mainframe disk drives referred to as "single large expensive disk" (SLED).

<span class="mw-page-title-main">GNU GRUB</span> Boot loader package

GNU GRUB is a boot loader package from the GNU Project. GRUB is the reference implementation of the Free Software Foundation's Multiboot Specification, which provides a user the choice to boot one of multiple operating systems installed on a computer or select a specific kernel configuration available on a particular operating system's partitions.

Disk formatting is the process of preparing a data storage device such as a hard disk drive, solid-state drive, floppy disk, memory card or USB flash drive for initial use. In some cases, the formatting operation may also create one or more new file systems. The first part of the formatting process that performs basic medium preparation is often referred to as "low-level formatting". Partitioning is the common term for the second part of the process, dividing the device into several sub-devices and, in some cases, writing information to the device allowing an operating system to be booted from it. The third part of the process, usually termed "high-level formatting" most often refers to the process of generating a new file system. In some operating systems all or parts of these three processes can be combined or repeated at different levels and the term "format" is understood to mean an operation in which a new disk medium is fully prepared to store files. Some formatting utilities allow distinguishing between a quick format, which does not erase all existing data and a long option that does erase all existing data.

<span class="mw-page-title-main">Self-Monitoring, Analysis and Reporting Technology</span> Monitoring system in computer drives

Self-Monitoring, Analysis, and Reporting Technology is a monitoring system included in computer hard disk drives (HDDs) and solid-state drives (SSDs). Its primary function is to detect and report various indicators of drive reliability, or how long a drive can function while anticipating imminent hardware failures.

<span class="mw-page-title-main">Cylinder-head-sector</span> Historical method for giving addresses to physical data blocks on hard disk drives

Cylinder-head-sector (CHS) is an early method for giving addresses to each physical block of data on a hard disk drive.

A hybrid drive is a logical or physical computer storage device that combines a faster storage medium such as solid-state drive (SSD) with a higher-capacity hard disk drive (HDD). The intent is adding some of the speed of SSDs to the cost-effective storage capacity of traditional HDDs. The purpose of the SSD in a hybrid drive is to act as a cache for the data stored on the HDD, improving the overall performance by keeping copies of the most frequently used data on the faster SSD drive.

<span class="mw-page-title-main">GUID Partition Table</span> Computer disk partitioning standard

The GUID Partition Table (GPT) is a standard for the layout of partition tables of a physical computer storage device, such as a hard disk drive or solid-state drive, using universally unique identifiers (UUIDs), which are also known as globally unique identifiers (GUIDs). Forming a part of the Unified Extensible Firmware Interface (UEFI) standard, it is nevertheless also used for some BIOSs, because of the limitations of master boot record (MBR) partition tables, which use 32 bits for logical block addressing (LBA) of traditional 512-byte disk sectors.

INT 13h is shorthand for BIOS interrupt call 13hex, the 20th interrupt vector in an x86-based computer system. The BIOS typically sets up a real mode interrupt handler at this vector that provides sector-based hard disk and floppy disk read and write services using cylinder-head-sector (CHS) addressing. Modern PC BIOSes also include INT 13h extension functions, originated by IBM and Microsoft in 1992, that provide those same disk access services using 64-bit LBA addressing; with minor additions, these were quasi-standardized by Phoenix Technologies and others as the EDD BIOS extensions.

The Logical Disk Manager (LDM) is an implementation of a logical volume manager for Microsoft Windows NT, developed by Microsoft and Veritas Software. It was introduced with the Windows 2000 operating system, and is supported in Windows XP, Windows Server 2003, Windows Vista, Windows 7, Windows 8, Windows 10 and Windows 11. The MMC-based Disk Management snap-in hosts the Logical Disk Manager. On Windows 8 and Windows Server 2012, Microsoft deprecated LDM in favor of Storage Spaces.

<span class="mw-page-title-main">Solid-state drive</span> Computer storage device with no moving parts

A solid-state drive (SSD) is a type of solid-state storage device that uses integrated circuits to store data persistently. It is sometimes called semiconductor storage device, solid-state device, and solid-state disk.

<span class="mw-page-title-main">Disk sector</span> Logical or physical division of storage media

In computer disk storage, a sector is a subdivision of a track on a magnetic disk or optical disc. For most disks, each sector stores a fixed amount of user-accessible data, traditionally 512 bytes for hard disk drives (HDDs), and 2048 bytes for CD-ROMs, DVD-ROMs and BD-ROMs. Newer HDDs and SSDs use 4096 byte (4 KiB) sectors, which are known as the Advanced Format (AF).

The BIOS boot partition is a partition on a data storage device that GNU GRUB uses on legacy BIOS-based personal computers in order to boot an operating system, when the actual boot device contains a GUID Partition Table (GPT). Such a layout is sometimes referred to as BIOS/GPT boot.

This timeline of binary prefixes lists events in the history of the evolution, development, and use of units of measure that are germane to the definition of the binary prefixes by the International Electrotechnical Commission (IEC) in 1998, used primarily with units of information such as the bit and the byte.

Fixed-block architecture (FBA) is an IBM term for the hard disk drive (HDD) layout in which each addressable block on the disk has the same size, utilizing 4 byte block numbers and a new set of command codes. FBA as a term was created and used by IBM for its 3310 and 3370 HDDs beginning in 1979 to distinguish such drives as IBM transitioned away from their variable record size format used on IBM's mainframe hard disk drives beginning in 1964 with its System/360.

A master boot record (MBR) is a type of boot sector in the first block of partitioned computer mass storage devices like fixed disks or removable drives intended for use with IBM PC-compatible systems and beyond. The concept of MBRs was publicly introduced in 1983 with PC DOS 2.0.

Data Integrity Field (DIF) is an approach to protect data integrity in computer data storage from data corruption. It was proposed in 2003 by the T10 subcommittee of the International Committee for Information Technology Standards. A similar approach for data integrity was added in 2016 to the NVMe 1.2.1 specification.

<span class="mw-page-title-main">Floppy disk variants</span> Types of floppy disk formats

The floppy disk is a data storage and transfer medium that was ubiquitous from the mid-1970s well into the 2000s. Besides the 3½-inch and 5¼-inch formats used in IBM PC compatible systems, or the 8-inch format that preceded them, many proprietary floppy disk formats were developed, either using a different disk design or special layout and encoding methods for the data held on the disk.

References

  1. ↑ "Advanced Format Definitions, Abbreviations, and Conventions". IDEMA. Archived from the original on 2012-03-16. Retrieved March 13, 2012.
  2. ↑ "Home- INSIC | Information Storage Industry Consortium". INSIC. Archived from the original on 2014-03-12. Retrieved 2014-03-12.
  3. 1 2 "The Advent of Advanced Format". IDEMA. Archived from the original on 2012-05-10. Retrieved 2013-11-18.
  4. 1 2 "Transition to Advanced Format 4K Sector Hard Drives". Seagate. Archived from the original on 2014-12-20. Retrieved 2014-12-15.
  5. ↑ "Advanced Format – The Migration to 4K Sectors". Seagate Technology. Archived from the original on 19 May 2011. Retrieved June 27, 2011.
  6. ↑ Martin K. Petersen (30 August 2008). "Linux Data Integrity" (PDF). Oracle Corporation. p. 7. Archived (PDF) from the original on 9 January 2015. Retrieved 14 December 2020. Most disk drives use 512-byte sectors. [...] Enterprise drives (Parallel SCSI/SAS/FC) support 520/528 byte 'fat' sectors.
  7. ↑ Mueller, Scott (2013). Upgrading and Repairing PCs (21st ed.). Que Publishing. pp. 472–473. ISBN   978-0789750006.
  8. ↑ Smith, Ryan (18 December 2009). "Western Digital's Advanced Format: The 4K Sector Transition Begins". www.anandtech.com . Archived from the original on 28 December 2020. Retrieved 14 December 2020.{{cite web}}: CS1 maint: bot: original URL status unknown (link)
  9. ↑ Swinburne, Richard (April 1, 2010). "The Facts: 4K Advanced Format Hard Disks". bit-tech.net. Archived from the original on 2012-03-06. Retrieved March 13, 2012.
  10. ↑ Hassner, Martin; Grochowski, Ed (May 31, 2005). 4K Byte-Sector HDD-Data Format Standard. Windows Hardware Engineering Conference. Archived from the original on March 28, 2012. Retrieved March 13, 2012.
  11. ↑ Curtis E. Stevens (2011). "Advanced Format in Legacy Infrastructures: More Transparent than Disruptive" (PDF). idema.org. Archived (PDF) from the original on 2013-11-05. Retrieved 2013-11-05.
  12. 1 2 3 "Advanced format (4K) disk compatibility update (Windows)". November 28, 2012. Archived from the original on 2013-01-11. Retrieved January 3, 2013.
  13. ↑ "The arrow of time – FreeBSD on 4K sector drives". Ivoras.net. Archived from the original on 2014-03-16. Retrieved 2014-03-12.
  14. ↑ "2.7. Allocating Disk Space". Freebsd.org. Archived from the original on 2014-03-20. Retrieved 2014-03-12.
  15. ↑ "Disk Setup On FreeBSD". Wonkity.com. 2013-06-24. Archived from the original on 2014-07-12. Retrieved 2014-03-12.
  16. ↑ Jonathan Corbet (2010-03-09). "4K-sector drives and Linux". LWN.net. Archived from the original on 2013-10-04. Retrieved 2013-10-04.
  17. ↑ Martin K. Petersen (2009-11-24). "Linux Storage Topology and Advanced Features" (PDF). Oracle Corporation. Archived (PDF) from the original on 2013-10-05. Retrieved 2013-10-04.
  18. ↑ "How to install a WD Advanced Format Drive on a non-Windows Operating System". January 19, 2012. Archived from the original on May 27, 2014. Retrieved April 12, 2013.
  19. ↑ "Oracle Solaris 11.1 Administration: Devices and File Systems". Oracle Corporation. Archived from the original on 2014-03-06. Retrieved 2014-03-06.
  20. ↑ Michael E. Fitzpatrick. "4K Sector Disk Drives: Transitioning to the Future with Advanced Format Technologies" (PDF). Toshiba. Archived (PDF) from the original on 2014-10-06. Retrieved 2013-10-24.
  21. ↑ Goldwyn Rodrigues (2009-03-11). "Linux and 4K disk sectors". LWN.net. Archived from the original on 2013-10-05. Retrieved 2013-10-24.
  22. ↑ "About the Advanced Format Technology Committee (formerly LDS Committee)". www.idema.org . Archived from the original on 30 March 2016. Retrieved 14 December 2020.
  23. ↑ "4kB Data Sector Update - IDEMA 4kB Technical Committee" (PDF). www.snia.org . September 2008. Archived (PDF) from the original on 11 August 2016. Retrieved 14 December 2020.
  24. ↑ "SmartAlign Technology for Advanced Format Hard Drives" (PDF). www.seagate.com . 2010. Archived (PDF) from the original on 29 November 2010. Retrieved 14 December 2020.
  25. ↑ "Download the Hitachi Align Tool". www.hitachigst.com . 2010. Archived from the original on 23 June 2011. Retrieved 14 December 2020.
  26. ↑ "Advanced Format Software". www.wdc.com . 2011. Archived from the original on 29 December 2011. Retrieved 14 December 2020.
  27. ↑ "Enterprise Capacity 3.5 HDD Data Sheet" (PDF). Seagate Technology. April 23, 2014. p. 2. Archived (PDF) from the original on 2014-08-12. Retrieved August 10, 2014.
  28. ↑ "WD Re Datacenter Distribution Specification Sheet" (PDF). Western Digital. January 21, 2016. p. 2. Archived (PDF) from the original on 2015-09-06. Retrieved February 14, 2016.
  29. ↑ "Microsoft support policy for 4K sector hard drives in Windows". Microsoft. Archived from the original on 2011-08-19. Retrieved October 24, 2013.
  30. ↑ "The brave new world of 4Kn hard disks: A test with Windows (XP x64), Truecrypt, HDTune and others (Update: Now with Linux, XP 32-Bit) – The GAT at XIN.at".
  31. ↑ "Linux kernel 2.6.31, Section 11. Block". kernelnewbies.org. September 9, 2009. Archived from the original on 2015-11-05. Retrieved October 10, 2015.
  32. ↑ "util-linux-ng 2.17 Release Notes". kernel.org. January 8, 2010. Archived from the original on 2016-03-04. Retrieved October 10, 2015.
  33. ↑ "Linux_2_6_37-DriversArch - Linux Kernel Newbies, Section 2.3. STORAGE". kernelnewbies.org. Retrieved 2024-03-23.
  34. ↑ "Advanced Format Logo Overview". IDEMA. Archived from the original on 2013-10-19. Retrieved 2014-01-10.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.