IOPS

Last updated

Input/output operations per second (IOPS, pronounced eye-ops) is an input/output performance measurement used to characterize computer storage devices like hard disk drives (HDD), solid state drives (SSD), and storage area networks (SAN). Like benchmarks, IOPS numbers published by storage device manufacturers do not directly relate to real-world application performance. [1] [2]

Contents

Background

To meaningfully describe the performance characteristics of any storage device, it is necessary to specify a minimum of three metrics simultaneously: IOPS, response time, and (application) workload. Absent simultaneous specifications of response-time and workload, IOPS are essentially meaningless. In isolation, IOPS can be considered analogous to "revolutions per minute" of an automobile engine i.e. an engine capable of spinning at 10,000 RPMs with its transmission in neutral does not convey anything of value, however an engine capable of developing specified torque and horsepower at a given number of RPMs fully describes the capabilities of the engine.

The specific number of IOPS possible in any system configuration will vary greatly, depending upon the variables the tester enters into the program, including the balance of read and write operations, the mix of sequential and random access patterns, the number of worker threads and queue depth, as well as the data block sizes. [1] There are other factors which can also affect the IOPS results including the system setup, storage drivers, OS background operations etc. Also, when testing SSDs in particular, there are preconditioning considerations that must be taken into account. [3]

Performance characteristics

Random access compared to sequential access. Random vs sequential access.svg
Random access compared to sequential access.

The most common performance characteristics measured are sequential and random operations. Sequential operations access locations on the storage device in a contiguous manner and are generally associated with large data transfer sizes, e.g. ≥ 128  kB. Random operations access locations on the storage device in a non-contiguous manner and are generally associated with small data transfer sizes, e.g. 4kB.

The most common performance characteristics are as follows:

MeasurementDescription
Total IOPSTotal number of I/O operations per second (when performing a mix of read and write tests)
Random Read IOPSAverage number of random read I/O operations per second
Random Write IOPSAverage number of random write I/O operations per second
Sequential Read IOPSAverage number of sequential read I/O operations per second
Sequential Write IOPSAverage number of sequential write I/O operations per second

For HDDs and similar electromechanical storage devices, the random IOPS numbers are primarily dependent upon the storage device's random seek time, whereas, for SSDs and similar solid state storage devices, the random IOPS numbers are primarily dependent upon the storage device's internal controller and memory interface speeds. On both types of storage devices, the sequential IOPS numbers (especially when using a large block size) typically indicate the maximum sustained bandwidth that the storage device can handle. [1] Often sequential IOPS are reported as a simple Megabytes per second number as follows:

(then converted to MB/s)

Some HDDs will improve in performance as the number of outstanding IOs (i.e. queue depth) increases. This is usually the result of more advanced controller logic on the drive performing command queuing and reordering commonly called either Tagged Command Queuing (TCQ) or Native Command Queuing (NCQ). Most commodity SATA drives either cannot do this, or their implementation is so poor that no performance benefit can be seen.[ citation needed ] Enterprise class SATA drives, such as the Western Digital Raptor and Seagate Barracuda NL will improve by nearly 100% with deep queues. [4] High-end SCSI drives more commonly found in servers, generally show much greater improvement, with the Seagate Savvio exceeding 400 IOPS—more than doubling its performance.[ citation needed ]

While traditional HDDs have about the same IOPS for read and write operations, many NAND flash-based SSDs and USB sticks are much slower writing than reading due to the inability to rewrite directly into a previously written location forcing a procedure called garbage collection. [5] [6] [7] This has caused hardware test sites to start to provide independently measured results when testing IOPS performance.

Flash SSDs, such as the Intel X25-E (released 2010), have much higher IOPS than traditional HDD. In a test done by Xssist, using Iometer, 4 KB random transfers, 70/30 read/write ratio, queue depth 4, the IOPS delivered by the Intel X25-E 64GB G1 started around 10000 IOPs, and dropped sharply after 8 minutes to 4000 IOPS, and continued to decrease gradually for the next 42 minutes. IOPS vary between 3000 and 4000 from approximately 50 minutes and onwards, for the rest of the 8+ hours the test ran. [8] Even with the drop in random IOPS after the 50th minute, the X25-E still has much higher IOPS compared to traditional hard disk drives. Some SSDs, including the OCZ RevoDrive 3 x2 PCIe using the SandForce controller, have shown much higher sustained write performance that more closely matches the read speed. [9]

Examples

Mechanical hard drives

Block size used when testing significantly affects the number of IOPS performed by a given drive. See below for some typical performance figures: [10]

Drive (Type / RPM)IOPS

(4KB block, random)

IOPS

(64KB block, random)

MB/s (64KB block, random)IOPS

(512KB block, random)

MB/s (512KB block, random)MB/s (large block, sequential)
SAS / 15K188 - 203175 - 19211.2 – 12.3115 – 13558.9 – 68.991.5 – 126.3
FC / 15K163 - 178151 - 1699.7 – 10.897 – 12349.7 – 63.173.5 – 127.5
FC / 10K142 - 151130 – 1438.3 – 9.280 – 10440.9 – 53.158.1 – 107.2
SAS / 10K142 - 151130 – 1438.3 – 9.280 – 10440.9 – 53.158.1 – 107.2
SATA / 720073 - 7969 - 764.4 – 4.947 – 6324.3 – 32.143.4 – 97.8
SATA / 540057553.54422.6

Solid-state devices

DeviceTypeIOPSInterfaceNotes
Intel X25-M G2 (MLC) SSD ~8,600 IOPS [11] SATA 3 Gbit/sIntel's data sheet [12] claims 6,600/8,600 IOPS (80 GB/160 GB version) and 35,000 IOPS for random 4 KB writes and reads, respectively.
Intel X25-E (SLC)SSD~5,000 IOPS [13] SATA 3 Gbit/sIntel's data sheet [14] claims 3,300 IOPS and 35,000 IOPS for writes and reads, respectively. 5,000 IOPS are measured for a mix. Intel X25-E G1 has around 3 times higher IOPS compared to the Intel X25-M G2. [15]
G.Skill Phoenix ProSSD~20,000 IOPS [16] SATA 3 Gbit/s SandForce-1200 based SSD drives with enhanced firmware, states up to 50,000 IOPS, but benchmarking shows for this particular drive ~25,000 IOPS for random read and ~15,000 IOPS for random write. [16]
Samsung SSD 850 PROSSDUp to[ neutrality is disputed ] 100,000 read IOPS
Up to[ neutrality is disputed ] 90,000 write IOPS [17] [ non-primary source needed ]
SATA 6 Gbit/s4 KB aligned random I/O at QD32
10,000 read IOPS, 36,000 write IOPS at QD1
550 MB/s sequential read, 520 MB/s sequential write on 256 GB and larger models
550 MB/s sequential read, 470 MB/s sequential write on 128 GB model [17]
Virident Systems tachIOnSSDUp to[ neutrality is disputed ] 320,000 sustained READ IOPS using 4kB blocks, and up to 200,000 sustained WRITE IOPS using 4kB blocks [18] PCIe
OCZ RevoDrive 3 X2SSDUp to[ neutrality is disputed ] 200,000 Random Write 4k IOPS [19] PCIe
DDRdrive X1SSDUp to[ neutrality is disputed ] 300,000+ (512B Random Read IOPS) and 200,000+ (512B Random Write IOPS) [20] [21] [22] [ non-primary source needed ] [23] PCIe
Samsung SSD 960 EVOSSDUp to[ neutrality is disputed ] 380,000 read IOPS
Up to[ neutrality is disputed ] 360,000 write IOPS [24] [ non-primary source needed ]
NVMe over PCIe 3.0 x4, M.2 4 kB aligned random I/O with four workers at QD4 (effectively QD16), [25] 1 TB model
14,000 read IOPS, 50,000 write IOPS at QD1
330,000 read IOPS, 330,000 write IOPS on 500 GB model
300,000 read IOPS, 330,000 write IOPS on 250 GB model
Up to[ neutrality is disputed ] 3.2 GB/s sequential read, 1.9 GB/s sequential write [24]
Samsung SSD 960 PROSSDUp to[ neutrality is disputed ] 440,000 read IOPS
Up to[ neutrality is disputed ] 360,000 write IOPS [24] [ non-primary source needed ]
NVMe over PCIe 3.0 x4, M.2 4kB aligned random I/O with four workers at QD4 (effectively QD16), [25] 1 TB and 2 TB models
14,000 read IOPS, 50,000 write IOPS at QD1
330,000 read IOPS, 330,000 write IOPS on 512 GB model
Up to[ neutrality is disputed ] 3.5 GB/s sequential read, 2.1 GB/s sequential write [24]
Kaminario K2SSDUp to[ neutrality is disputed ] 2,000,000 IOPS. [26]
1,200,000 IOPS in SPC-1 benchmark simulating business applications [27] [28]
FC MLC Flash
NetApp FAS6240 clusterFlash/Disk1,261,145 SPECsfs2008 nfsv3 IOPs using 1,440 15k disks, across 60 shelves, with virtual storage tiering. [29] [ unreliable source? ]NFS, SMB, FC, FCoE, iSCSISPECsfs2008 is the latest version of the Standard Performance Evaluation Corporation benchmark suite measuring file server throughput and response time, providing a standardized method for comparing performance across different vendor platforms.
EMC DSSD D5FlashUp to[ neutrality is disputed ] 10 million IOPS [30] [ non-primary source needed ]PCIe Out of Box, up to 48 clients with high availability.PCIe Rack Scale Flash Appliance. Product discontinued. [31]

See also

Related Research Articles

<span class="mw-page-title-main">Self-Monitoring, Analysis and Reporting Technology</span> Monitoring system in computer drives

Self-Monitoring, Analysis, and Reporting Technology is a monitoring system included in computer hard disk drives (HDDs) and solid-state drives (SSDs). Its primary function is to detect and report various indicators of drive reliability with the intent of anticipating imminent hardware failures.

<span class="mw-page-title-main">Native Command Queuing</span>

In computing, Native Command Queuing (NCQ) is an extension of the Serial ATA protocol allowing hard disk drives to internally optimize the order in which received read and write commands are executed. This can reduce the amount of unnecessary drive head movement, resulting in increased performance for workloads where multiple simultaneous read/write requests are outstanding, most often occurring in server-type applications.

In computing, a hybrid drive is a logical or physical storage device that combines a faster storage medium such as solid-state drive (SSD) with a higher-capacity hard disk drive (HDD). The intent is adding some of the speed of SSDs to the cost-effective storage capacity of traditional HDDs. The purpose of the SSD in a hybrid drive is to act as a cache for the data stored on the HDD, improving the overall performance by keeping copies of the most frequently used data on the faster SSD drive.

<span class="mw-page-title-main">Solid-state drive</span> Data storage device

A solid-state drive (SSD) is a solid-state storage device that uses integrated circuit assemblies to store data persistently, typically using flash memory, and functions as secondary storage in the hierarchy of computer storage. It is also sometimes called a semiconductor storage device, a solid-state device, or a solid-state disk, even though SSDs lack the physical spinning disks and movable read-write heads used in hard disk drives (HDDs) and floppy disks. SSD also has rich internal parallelism for data processing.

<span class="mw-page-title-main">JMicron</span>

JMicron Technology Corporation is a Taiwan based fabless technology design house based in Hsinchu, Taiwan. As a manufacturer of integrated circuits, they produce controller chips for bridge devices.

<span class="mw-page-title-main">Multi-level cell</span> Memory cell capable of storing more than a single bit of information

In electronics, a multi-level cell (MLC) is a memory cell capable of storing more than a single bit of information, compared to a single-level cell (SLC), which can store only one bit per memory cell. A memory cell typically consists of a single floating-gate MOSFET, thus multi-level cells reduce the number of MOSFETs required to store the same amount of data as single-level cells.

Indilinx, Inc. was a formerly South Korean-based solid-state drive (SSD) controller manufacturer with business headquarters in San Jose, California. It was a wholly owned subsidiary of the OCZ Technology Group until the latter sold the former's assets to Toshiba in OCZ's bankruptcy sale, and those assets were fully integrated into the newly founded OCZ Storage Solutions division of Toshiba.

vRPM, or virtual Revolutions Per Minute, was a term for a synthetic measurement of performance introduced by SanDisk for solid state drive (SSD) storage devices inside client PCs. vRPM was created to give users a metric to compare SSD performance to the hard disk drive (HDD) and other SSDs.

A trim command allows an operating system to inform a solid-state drive (SSD) which blocks of data are no longer considered to be "in use" and therefore can be erased internally.

<span class="mw-page-title-main">Write amplification</span> Phenomenon associated with solid state storage

Write amplification (WA) is an undesirable phenomenon associated with flash memory and solid-state drives (SSDs) where the actual amount of information physically written to the storage media is a multiple of the logical amount intended to be written.

<span class="mw-page-title-main">X25-M</span> Line of SSDs by Intel

The Intel X25-M was a line of Serial ATA interface solid-state drives developed by Intel for personal computers, announced in late 2008. The SSD was a multi-level-cell solid-state drive available in a 2.5" form factor, came in 80 GB and 160 GB capacities and utilized NAND flash memory on a 50 nm process. The second-generation SSD which was called the "X25-M G2". The X25-M G2 was also available in a 2.5" form factor and 80 GB and 160 GB capacities, but with NAND flash memory on a more efficient 34 nm process.

NVM Express (NVMe) or Non-Volatile Memory Host Controller Interface Specification (NVMHCIS) is an open, logical-device interface specification for accessing a computer's non-volatile storage media usually attached via the PCI Express bus. The initial NVM stands for non-volatile memory, which is often NAND flash memory that comes in several physical form factors, including solid-state drives (SSDs), PCIe add-in cards, and M.2 cards, the successor to mSATA cards. NVM Express, as a logical-device interface, has been designed to capitalize on the low latency and internal parallelism of solid-state storage devices.

Higher performance in hard disk drives comes from devices which have better performance characteristics. These performance characteristics can be grouped into two categories: access time and data transfer time.

<span class="mw-page-title-main">SATA Express</span> Computer device interface

SATA Express is a computer bus interface that supports both Serial ATA (SATA) and PCI Express (PCIe) storage devices, initially standardized in the SATA 3.2 specification. The SATA Express connector used on the host side is backward compatible with the standard SATA data connector, while it also provides two PCI Express lanes as a pure PCI Express connection to the storage device.

bcache is a cache in the Linux kernel's block layer, which is used for accessing secondary storage devices. It allows one or more fast storage devices, such as flash-based solid-state drives (SSDs), to act as a cache for one or more slower storage devices, such as hard disk drives (HDDs); this effectively creates hybrid volumes and provides performance improvements.

dm-cache is a component of the Linux kernel's device mapper, which is a framework for mapping block devices onto higher-level virtual block devices. It allows one or more fast storage devices, such as flash-based solid-state drives (SSDs), to act as a cache for one or more slower storage devices such as hard disk drives (HDDs); this effectively creates hybrid volumes and provides secondary storage performance improvements.

ExpressCache is a Windows-based SSD caching technology developed by Condusiv Technologies and licensed to a number of laptop manufacturers including Acer, ASUS, Samsung, Sony, Lenovo, and Fujitsu. ExpressCache is also bundled with some SanDisk products such as ReadyCache; SanDisk currently holds an exclusive ExpressCache license for stand-alone storage products.

Solid-state storage (SSS) is a type of non-volatile computer storage that stores and retrieves digital information using only electronic circuits, without any involvement of moving mechanical parts. This differs fundamentally from the traditional electromechanical storage, which records data using rotating or linearly moving media coated with magnetic material.

<span class="mw-page-title-main">3D XPoint</span> Discontinued computer memory type

3D XPoint is a discontinued non-volatile memory (NVM) technology developed jointly by Intel and Micron Technology. It was announced in July 2015 and was available on the open market under the brand name Optane (Intel) from April 2017 to July 2022. Bit storage is based on a change of bulk resistance, in conjunction with a stackable cross-grid data access array. Initial prices are less than dynamic random-access memory (DRAM) but more than flash memory.

References

  1. 1 2 3 Lowe, Scott (2010-02-12). "Calculate IOPS in a storage array". techrepublic.com. Retrieved 2011-07-03.
  2. "Getting The Hang Of IOPS v1.3". 2012-08-03. Retrieved 2013-08-15.
  3. Smith, Kent (2009-08-11). "Benchmarking SSDs: The Devil is in the Preconditioning Details" (PDF). SandForce.com. Retrieved 2015-05-05.
  4. "SATA in the Enterprise - A 500 GB Drive Roundup | StorageReview.com - Storage Reviews". StorageReview.com. 2006-07-13. Archived from the original on 2013-05-05. Retrieved 2013-05-13.
  5. Xiao-yu Hu; Eleftheriou, Evangelos; Haas, Robert; Iliadis, Ilias; Pletka, Roman (2009). "Write Amplification Analysis in Flash-Based Solid State Drives". IBM. CiteSeerX   10.1.1.154.8668 .{{cite journal}}: Cite journal requires |journal= (help)
  6. "SSDs - Write Amplification, TRIM and GC" (PDF). OCZ Technology. Archived from the original (PDF) on 2012-05-26. Retrieved 2010-05-31.
  7. "Intel Solid State Drives". Intel. Retrieved 2010-05-31.
  8. "Intel X25-E 64GB G1, 4KB Random IOPS, iometer benchmark". 2010-03-27. Retrieved 2010-04-01.
  9. "OCZ RevoDrive 3 x2 PCIe SSD Review – 1.5GB Read/1.25GB Write/200,000 IOPS As Little As $699". 2011-06-28. Retrieved 2011-06-30.
  10. "RAID Performance Calculator - WintelGuy.com". wintelguy.com. Retrieved 2019-04-01.
  11. Schmid, Patrick; Roos, Achim (2008-09-08). "Intel's X25-M Solid State Drive Reviewed" . Retrieved 2011-08-02.
  12. "Intel X18-M/X25-M SATA Solid State Drive — 34 nm Product Line" (PDF). Intel. January 2010. Archived from the original (PDF) on 2010-08-12. Retrieved 2010-07-20.
  13. Schmid, Patrick; Roos, Achim (27 February 2009). "Intel's X25-E SSD Walks All Over The Competition: They Did It Again: X25-E For Servers Takes Off". TomsHardware.com. Retrieved 2013-05-13.
  14. "Archived copy" (PDF). Archived from the original (PDF) on 2009-02-06. Retrieved 2009-03-18.{{cite web}}: CS1 maint: archived copy as title (link)
  15. "Intel X25-E G1 vs Intel X25-M G2 Random 4 KB IOPS, iometer". May 2010. Retrieved 2010-05-19.
  16. 1 2 "G.Skill Phoenix Pro 120 GB Test - SandForce SF-1200 SSD mit 50K IOPS - HD Tune Access Time IOPS (Diagramme) (5/12)". Tweakpc.de. Retrieved 2013-05-13.
  17. 1 2 "Samsung SSD 850 PRO Specifications". Samsung Electronics . Retrieved 7 June 2017.
  18. "Virident's tachIOn SSD flashes by". theregister.co.uk.
  19. "OCZ RevoDrive 3 X2 480GB Review | StorageReview.com - Storage Reviews". StorageReview.com. 2011-06-28. Retrieved 2013-05-13.
  20. http://www.ddrdrive.com/ddrdrive_press.pdf [ bare URL PDF ]
  21. "Archived copy" (PDF). Archived from the original (PDF) on 2009-05-20. Retrieved 2009-05-22.{{cite web}}: CS1 maint: archived copy as title (link)
  22. "Archived copy" (PDF). Archived from the original (PDF) on 2009-05-20. Retrieved 2009-05-22.{{cite web}}: CS1 maint: archived copy as title (link)
  23. Allyn Malventano (2009-05-04). "DDRdrive hits the ground running - PCI-E RAM-based SSD | PC Perspective". Pcper.com. Archived from the original on 2013-07-14. Retrieved 2013-05-13.
  24. 1 2 3 4 Samsung Electronics. "NVMe SSD 960 PRO/EVO" . Retrieved 7 June 2017.
  25. 1 2 Ramseyer, Chris (18 October 2016). "Samsung 960 Pro SSD Review". Tom's Hardware . Retrieved 9 June 2017. Samsung tests NVMe products with four workers at QD4
  26. Lyle Smith. "Kaminario Boasts Over 2 Million IOPS and 20 GB/s Throughput on a Single All-Flash K2 Storage System". Archived from the original on 2016-04-22. Retrieved 2015-12-19.
  27. Mellor, Chris (2012-07-30). "Chris Mellor, The Register, July 30, 2012: "Million-plus IOPS: Kaminario smashes IBM in DRAM decimation"". Theregister.co.uk. Retrieved 2013-11-14.
  28. Storage Performance Council. "Storage Performance Council: Active SPC-1 Results". storageperformance.org. Archived from the original on 2014-09-25. Retrieved 2012-09-25.
  29. "SpecSFS2008" . Retrieved 2014-02-07.
  30. "Rack-Scale Flash Appliance - DSSD D5 EMC". EMC. Retrieved 2016-03-23.
  31. "Dell kills off standalone DSSD D5, scatters remains into other gear". The Register .