Data erasure

Last updated

Data erasure (sometimes referred to as data clearing, data wiping, or data destruction) is a software-based method of data sanitization that aims to completely destroy all electronic data residing on a hard disk drive or other digital media by overwriting data onto all sectors of the device in an irreversible process. By overwriting the data on the storage device, the data is rendered irrecoverable.

Contents

Ideally, software designed for data erasure should:

  1. Allow for selection of a specific standard, based on unique needs, and
  2. Verify the overwriting method has been successful and removed data across the entire device.

Permanent data erasure goes beyond basic file deletion commands, which only remove direct pointers to the data disk sectors and make the data recovery possible with common software tools. Unlike degaussing and physical destruction, which render the storage media unusable, data erasure removes all information while leaving the disk operable. New flash memory-based media implementations, such as solid-state drives or USB flash drives, can cause data erasure techniques to fail allowing remnant data to be recoverable. [1]

Software-based overwriting uses a software application to write a stream of zeros, ones or meaningless pseudorandom data onto all sectors of a hard disk drive. There are key differentiators between data erasure and other overwriting methods, which can leave data intact and raise the risk of data breach, identity theft or failure to achieve regulatory compliance. Many data eradication programs also provide multiple overwrites so that they support recognized government and industry standards, though a single-pass overwrite is widely considered to be sufficient for modern hard disk drives. Good software should provide verification of data removal, which is necessary for meeting certain standards.

To protect the data on lost or stolen media, some data erasure applications remotely destroy the data if the password is incorrectly entered. Data erasure tools can also target specific data on a disk for routine erasure, providing a hacking protection method that is less time-consuming than software encryption. Hardware/firmware encryption built into the drive itself or integrated controllers is a popular solution with no degradation in performance at all.

Encryption

When encryption is in place, data erasure acts as a complement to crypto-shredding, or the practice of 'deleting' data by (only) deleting or overwriting the encryption keys. [2]

Presently, dedicated hardware/firmware encryption solutions can perform a 256-bit full AES encryption faster than the drive electronics can write the data. Drives with this capability are known as self-encrypting drives (SEDs); they are present on most modern enterprise-level laptops and are increasingly used in the enterprise to protect the data. Changing the encryption key renders inaccessible all data stored on a SED, which is an easy and very fast method for achieving a 100% data erasure. Theft of an SED results in a physical asset loss, but the stored data is inaccessible without the decryption key that is not stored on a SED, assuming there are no effective attacks against AES or its implementation in the drive hardware.[ citation needed ]

Importance

Information technology assets commonly hold large volumes of confidential data. Social security numbers, credit card numbers, bank details, medical history and classified information are often stored on computer hard drives or servers. These can inadvertently or intentionally make their way onto other media such as printers, USB, flash, Zip, Jaz, and REV drives.

Data breach

Increased storage of sensitive data, combined with rapid technological change and the shorter lifespan of IT assets, has driven the need for permanent data erasure of electronic devices as they are retired or refurbished. Also, compromised networks and laptop theft and loss, as well as that of other portable media, are increasingly common sources of data breaches.

If data erasure does not occur when a disk is retired or lost, an organization or user faces a possibility that the data will be stolen and compromised, leading to identity theft, loss of corporate reputation, threats to regulatory compliance and financial impacts. Companies spend large amounts of money to make sure their data is erased when they discard disks. [3] [ dubious – discuss ] High-profile incidents of data theft include:

Regulatory compliance

Strict industry standards and government regulations are in place that force organizations to mitigate the risk of unauthorized exposure of confidential corporate and government data. Regulations in the United States include HIPAA (Health Insurance Portability and Accountability Act); FACTA (The Fair and Accurate Credit Transactions Act of 2003); GLB (Gramm-Leach Bliley); Sarbanes-Oxley Act (SOx); and Payment Card Industry Data Security Standards (PCI DSS) and the Data Protection Act in the United Kingdom. Failure to comply can result in fines and damage to company reputation, as well as civil and criminal liability.[ citation needed ]

Preserving assets and the environment

Data erasure offers an alternative to physical destruction and degaussing for secure removal of all the disk data. Physical destruction and degaussing destroy the digital media, requiring disposal and contributing to electronic waste while negatively impacting the carbon footprint of individuals and companies. [10] Hard drives are nearly 100% recyclable and can be collected at no charge from a variety of hard drive recyclers after they have been sanitized. [11]

Limitations

Data erasure may not work completely on flash based media, such as Solid State Drives and USB Flash Drives, as these devices can store remnant data which is inaccessible to the erasure technique, and data can be retrieved from the individual flash memory chips inside the device. [1] Data erasure through overwriting only works on hard drives that are functioning and writing to all sectors. Bad sectors cannot usually be overwritten, but may contain recoverable information. Bad sectors, however, may be invisible to the host system and thus to the erasing software. Disk encryption before use prevents this problem. Software-driven data erasure could also be compromised by malicious code. [12]

Differentiators

Software-based data erasure uses a disk accessible application to write a combination of ones, zeroes and any other alpha numeric character also known as the "mask" onto each hard disk drive sector. The level of security when using software data destruction tools are increased dramatically by pre-testing hard drives for sector abnormalities and ensuring that the drive is 100% in working order. The number of wipes has become obsolete with the more recent inclusion of a "verify pass" which scans all sectors of the disk and checks against what character should be there i.e.; 1 Pass of AA has to fill every writable sector of the hard disk. This makes any more than 1 Pass an unnecessary and certainly a more damaging act especially as drives have passed the 1TB mark.

Full disk overwriting

While there are many overwriting programs, only those capable of complete data erasure offer full security by destroying the data on all areas of a hard drive. Disk overwriting programs that cannot access the entire hard drive, including hidden/locked areas like the host protected area (HPA), device configuration overlay (DCO), and remapped sectors, perform an incomplete erasure, leaving some of the data intact. By accessing the entire hard drive, data erasure eliminates the risk of data remanence.[ citation needed ]

Data erasure can also bypass the Operating System (OS). Overwriting programs that operate through the OS will not always perform a complete erasure because they cannot modify the contents of the hard drive that are actively in use by that OS. Because of this, many data erasure programs are provided in a bootable format, where you run off of a live CD that has all of the necessary software to erase the disk.[ citation needed ]

Hardware support

Data erasure can be deployed over a network to target multiple PCs rather than having to erase each one sequentially. In contrast with DOS-based overwriting programs that may not detect all network hardware, Linux-based data erasure software supports high-end server and storage area network (SAN) environments with hardware support for Serial ATA, Serial Attached SCSI (SAS) and Fibre Channel disks and remapped sectors. It operates directly with sector sizes such as 520, 524, and 528, removing the need to first reformat back to 512 sector size. WinPE has now overtaken Linux as the environment of choice since drivers can be added with little effort. This also helps with data destruction of tablets and other handheld devices that require pure UEFI environments without hardware NIC's installed and/or are lacking UEFI network stack support.

Standards

Many government and industry standards exist for software-based overwriting that removes the data. A key factor in meeting these standards is the number of times the data is overwritten. Also, some standards require a method to verify that all the data have been removed from the entire hard drive and to view the overwrite pattern. Complete data erasure should account for hidden areas, typically DCO, HPA and remapped sectors.

The 1995 edition of the National Industrial Security Program Operating Manual (DoD 5220.22-M) permitted the use of overwriting techniques to sanitize some types of media by writing all addressable locations with a character, its complement, and then a random character. This provision was removed in a 2001 change to the manual and was never permitted for Top Secret media, but it is still listed as a technique by many providers of the data erasure software. [13]

Data erasure software should provide the user with a validation certificate indicating that the overwriting procedure was completed properly. Data erasure software should[ citation needed ] also comply with requirements to erase hidden areas, provide a defects log list and list bad sectors that could not be overwritten.

Overwriting StandardDateOverwriting RoundsPatternNotes
U.S. Navy Staff Office Publication NAVSO P-5239-26 [14] 19933A character, its complement, randomVerification is mandatory
U.S. Air Force System Security Instruction 5020 [15] 19963All zeros, all ones, any characterVerification is mandatory
Peter Gutmann's Algorithm 19961 to 35Various, including all of the other listed methodsOriginally intended for MFM and RLL disks, which are now obsolete
Bruce Schneier's Algorithm [16] 19967All ones, all zeros, pseudo-random sequence five times
Standard VSITR of Germany Federal Office for Information Security 19997The disk is filling with sequences 0x00 and 0xFF, and on the last pass - 0xAA.
U.S. DoD Unclassified Computer Hard Drive Disposition [17] 20013A character, its complement, another pattern
German Federal Office for Information Security [18] 20042 to 3Non-uniform pattern, its complement
Communications Security Establishment Canada ITSG-06 [19] 20063All ones or zeros, its complement, a pseudo-random patternFor unclassified media
NIST SP-800-88 [20] 20061 ?
U.S. National Industrial Security Program Operating Manual (DoD 5220.22-M) [13] 20063 ?No longer specifies any method.
NSA/CSS Storage Device Declassification Manual (SDDM) [21] 20070 ?Degauss or destroy only
New Zealand Government Communications Security Bureau NZSIT 402 [22] 20081 ?For data up to Confidential
Australian Government ICT Security Manual 2014 – Controls [23] 20141Random pattern (only for disks larger than 15 GB)Degauss magnetic media or destroy Top Secret media
NIST SP-800-88 Rev. 1 [24] 20141All zerosOutlines solutions based on media type. [25]
British HMG Infosec Standard 5, Baseline Standard [26]  ?1Random PatternVerification is mandatory
British HMG Infosec Standard 5, Enhanced Standard ?3All ones, all zeros, randomVerification is mandatory

Data can sometimes be recovered from a broken hard drive. However, if the platters on a hard drive are damaged, such as by drilling a hole through the drive (and the platters inside), then the data can only theoretically be recovered by bit-by-bit analysis of each platter with advanced forensic technology.

Number of overwrites needed

Data on floppy disks can sometimes be recovered by forensic analysis even after the disks have been overwritten once with zeros (or random zeros and ones). [27]

This is not the case with modern hard drives:

Even the possibility of recovering floppy disk data after overwrite is disputed. Gutmann's famous article cites a non-existent source and sources that do not actually demonstrate recovery, only partially-successful observations. The definition of "random" is also quite different from the usual one used: Gutmann expects the use of pseudorandom data with sequences known to the recovering side, not an unpredictable one such as a cryptographically secure pseudorandom number generator. [31]

E-waste and information security

The e-waste centre of Agbogbloshie, Ghana. Agbogbloshie.JPG
The e-waste centre of Agbogbloshie, Ghana.

E-waste presents a potential security threat to individuals and exporting countries. Hard drives that are not properly erased before the computer is disposed of can be reopened, exposing sensitive information. Credit card numbers, private financial data, account information and records of online transactions can be accessed by most willing individuals. Organized criminals in Ghana commonly search the drives for information to use in local scams. [32]

Government contracts have been discovered on hard drives found in Agbogbloshie.

See also

Related Research Articles

In cryptography, plaintext usually means unencrypted information pending input into cryptographic algorithms, usually encryption algorithms. This usually refers to data that is transmitted or stored unencrypted.

Disk formatting is the process of preparing a data storage device such as a hard disk drive, solid-state drive, floppy disk, memory card or USB flash drive for initial use. In some cases, the formatting operation may also create one or more new file systems. The first part of the formatting process that performs basic medium preparation is often referred to as "low-level formatting". Partitioning is the common term for the second part of the process, dividing the device into several sub-devices and, in some cases, writing information to the device allowing an operating system to be booted from it. The third part of the process, usually termed "high-level formatting" most often refers to the process of generating a new file system. In some operating systems all or parts of these three processes can be combined or repeated at different levels and the term "format" is understood to mean an operation in which a new disk medium is fully prepared to store files. Some formatting utilities allow distinguishing between a quick format, which does not erase all existing data and a long option that does erase all existing data.

<span class="mw-page-title-main">USB flash drive</span> Data storage device

A flash drive is a data storage device that includes flash memory with an integrated USB interface. A typical USB drive is removable, rewritable, and smaller than an optical disc, and usually weighs less than 30 g (1 oz). Since first offered for sale in late 2000, the storage capacities of USB drives range from 8 to 256 gigabytes (GB), 512 GB and 1 terabyte (TB). As of 2023, 2 TB flash drives were the largest currently in production. Some allow up to 100,000 write/erase cycles, depending on the exact type of memory chip used, and are thought to physically last between 10 and 100 years under normal circumstances.

<span class="mw-page-title-main">Darik's Boot and Nuke</span> Data erasure software

Darik's Boot and Nuke, also known as DBAN, is a free and open-source project hosted on SourceForge. The program is designed to securely erase a hard disk until its data is permanently removed and no longer recoverable, which is achieved by overwriting the data with pseudorandom numbers generated by Mersenne Twister or ISAAC. The Gutmann method, Quick Erase, DoD Short, and DOD 5220.22-M are also included as options to handle data remanence. DBAN can be booted from a CD, DVD, USB flash drive or diskless using a Preboot Execution Environment. It is based on Linux and supports PATA (IDE), SCSI and SATA hard drives. DBAN can be configured to automatically wipe every hard disk that it sees on a system or entire network of systems, making it very useful for unattended data destruction scenarios. DBAN exists for x86 systems.

Data security means protecting digital data, such as those in a database, from destructive forces and from the unwanted actions of unauthorized users, such as a cyberattack or a data breach.

Data remanence is the residual representation of digital data that remains even after attempts have been made to remove or erase the data. This residue may result from data being left intact by a nominal file deletion operation, by reformatting of storage media that does not remove data previously written to the media, or through physical properties of the storage media that allow previously written data to be recovered. Data remanence may make inadvertent disclosure of sensitive information possible should the storage media be released into an uncontrolled environment.

File deletion is the removal of a file from a computer's file system.

The Gutmann method is an algorithm for securely erasing the contents of computer hard disk drives, such as files. Devised by Peter Gutmann and Colin Plumb and presented in the paper Secure Deletion of Data from Magnetic and Solid-State Memory in July 1996, it involved writing a series of 35 patterns over the region to be erased.

Disk encryption software is a computer security software that protects the confidentiality of data stored on computer media by using disk encryption.

In computing, data recovery is a process of retrieving deleted, inaccessible, lost, corrupted, damaged, or formatted data from secondary storage, removable media or files, when the data stored in them cannot be accessed in a usual way. The data is most often salvaged from storage media such as internal or external hard disk drives (HDDs), solid-state drives (SSDs), USB flash drives, magnetic tapes, CDs, DVDs, RAID subsystems, and other electronic devices. Recovery may be required due to physical damage to the storage devices or logical damage to the file system that prevents it from being mounted by the host operating system (OS).

Redaction or sanitization is the process of removing sensitive information from a document so that it may be distributed to a broader audience. It is intended to allow the selective disclosure of information. Typically, the result is a document that is suitable for publication or for dissemination to others rather than the intended audience of the original document.

Anti-computer forensics or counter-forensics are techniques used to obstruct forensic analysis.

In computer security, a cold boot attack is a type of side channel attack in which an attacker with physical access to a computer performs a memory dump of a computer's random-access memory (RAM) by performing a hard reset of the target machine. Typically, cold boot attacks are used for retrieving encryption keys from a running operating system for malicious or criminal investigative reasons. The attack relies on the data remanence property of DRAM and SRAM to retrieve memory contents that remain readable in the seconds to minutes following a power switch-off.

Hardware-based full disk encryption (FDE) is available from many hard disk drive (HDD/SSD) vendors, including: Hitachi, Integral Memory, iStorage Limited, Micron, Seagate Technology, Samsung, Toshiba, Viasat UK, Western Digital. The symmetric encryption key is maintained independently from the computer's CPU, thus allowing the complete data store to be encrypted and removing computer memory as a potential attack vector.

Secure USB flash drives protect the data stored on them from access by unauthorized users. USB flash drive products have been on the market since 2000, and their use is increasing exponentially. As both consumers and businesses have increased demand for these drives, manufacturers are producing faster devices with greater data storage capacities.

shred is a command on Unix-like operating systems that can be used to securely delete files and devices so that it is extremely difficult to recover them, even with specialized hardware and technology; assuming it's even possible to recover the file at all. It is a part of GNU Core Utilities. Being based on the Gutmann method paper, it suffers from the same criticisms and possible shortcomings.

A trim command allows an operating system to inform a solid-state drive (SSD) which blocks of data are no longer considered to be "in use" and therefore can be erased internally.

ISO/IEC 27040 is part of a growing family of International Standards published by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) in the area of security techniques; the standard is being developed by Subcommitee 27 (SC27) - IT Security techniques of the first Joint Technical Committee 1 of the ISO/IEC. A major element of SC27's program of work includes International Standards for information security management systems (ISMS), often referred to as the 'ISO/IEC 27000-series'.

Crypto-shredding is the practice of 'deleting' data by deliberately deleting or overwriting the encryption keys. This requires that the data have been encrypted. Data may be considered to exist in three states: data at rest, data in transit and data in use. General data security principles, such as in the CIA triad of confidentiality, integrity, and availability, require that all three states must be adequately protected.

Data sanitization involves the secure and permanent erasure of sensitive data from datasets and media to guarantee that no residual data can be recovered even through extensive forensic analysis. Data sanitization has a wide range of applications but is mainly used for clearing out end-of-life electronic devices or for the sharing and use of large datasets that contain sensitive information. The main strategies for erasing personal data from devices are physical destruction, cryptographic erasure, and data erasure. While the term data sanitization may lead some to believe that it only includes data on electronic media, the term also broadly covers physical media, such as paper copies. These data types are termed soft for electronic files and hard for physical media paper copies. Data sanitization methods are also applied for the cleaning of sensitive data, such as through heuristic-based methods, machine-learning based methods, and k-source anonymity.

References

  1. 1 2 Michael Wei; Laura M. Grupp; Frederick E. Spada; Steven Swanson. "Reliably Erasing Data From Flash-Based Solid State Drives" (PDF). FAST '11: 9th USENIX Conference on File and Storage Technologies. Retrieved 31 October 2013. For sanitizing entire disks, built-in sanitize commands are effective when implemented correctly, and software techniques work most, but not all, of the time. We found that none of the available software techniques for sanitizing individual files were effective.
  2. ↑ "Securely erase a solid-state drive". University Information Technology Services. Retrieved 7 February 2022. you may be able to quickly sanitize the device by deleting the encryption key, which renders the data on the drive irretrievable.
  3. ↑ Fontana, John (2 November 2006). "Average data breach costs companies $5 million". Network World . Archived from the original on 8 August 2011. Retrieved 20 July 2010.
  4. ↑ Evers, Joris (19 June 2005). "Credit card breach exposes 40 million accounts". ZDNET. CNET News. Archived from the original on 21 April 2010. Retrieved 20 July 2010.
  5. ↑ Powers, Mary (13 February 2008). "Laptops missing with IDs of donors". Memphis Commercial Appeal . Retrieved 20 July 2010.
  6. ↑ Sharp, David (17 March 2008). "Breach exposes 4.2 million credit, debit cards". NBC News. Associated Press . Retrieved 20 July 2010.
  7. ↑ Vijayan, Jaikumar (21 March 2008). "Programmer who stole drive containing 1 million bank records gets 42 months" Archived 2 March 2007 at the Wayback Machine . Computer World. Retrieved 2010-07-20.
  8. ↑ "UF warns patients of security breach". Jacksonville Business Journal. 2008-05-20. Retrieved 2010-07-20.
  9. ↑ "OKC buyer finds sensitive information on server". Tulsa World . Associated Press. 21 May 2008. Retrieved 20 July 2010.
  10. ↑ "Is America exporting a huge environmental problem?". 20/20 . ABC News. 6 January 2006. Retrieved 20 July 2010.
  11. ↑ "Hard Drive Recycling - Cohen" . Retrieved 4 September 2021.
  12. ↑ "NSA/CSS Storage Device Declassification Manual" (PDF). NSA. Archived from the original (PDF) on 20 March 2016. Retrieved 19 January 2009. This Manual 912 supersedes NSA/CSS Manual 1302, dated 10 November 2000.
  13. 1 2 "U.S. National Industrial Security Program Operating Manual (DoD 5220.22-M)". dtic.mil. United States Department of Defense National Industrial Security Program. 2006. Archived from the original (PDF) on 22 August 2008.
  14. ↑ "Navy Remanence Regulation, U.S. Navy Publication NAVSO P-5239-26". Fas.org. U.S. Navy Staff Office. 30 May 2008. Retrieved 20 July 2010.
  15. ↑ "Air Force System Security Instruction 5020 – Remanence Security". JYA.com. 1996. Archived from the original on 15 March 2010. Retrieved 20 July 2010.
  16. ↑ Schneier, Bruce (1996). Applied Cryptography. New York: Wiley. p.  229. ISBN   0-471-12845-7.
  17. ↑ "Unclassified Computer Hard Drive Disposition" (PDF). U.S. DoD. 2001. Retrieved 20 July 2010.[ permanent dead link ]
  18. ↑ . German Federal Office for Information Security, 2004. Archived 26 June 2008 at the Wayback Machine
  19. ↑ "Clearing and Declassifying Electronic Data Storage Devices ITSG-06" (PDF). Communications Security Establishment Canada. July 2006. Archived from the original (PDF) on 24 January 2016. Retrieved 26 November 2014.
  20. ↑ Kissel, Scholl; Skolochenko, Li (September 2006). "SP800-88 Guidelines for Media Sanitization" (PDF). Computer Security Division, Information Technology Laboratory. NIST. doi:10.6028/NIST.SP.800-88 . Retrieved 20 July 2010.
  21. ↑ "Storage Device Declassification Manual" (PDF). NSA. Archived from the original (PDF) on 20 March 2016. Retrieved 19 January 2009.
  22. ↑ "New Zealand Security of Information NZSIT 402". Government Communications Security Bureau. 2008. Archived from the original on 19 August 2010. Retrieved 20 July 2010.
  23. ↑ "Australian Government Information Security Manual (ISM)". Australian Signals Directorate. 2014. Retrieved 9 December 2014.
  24. 1 2 Kissel, Richard; Regenscheid, Andrew; Scholl, Matthew; Stine, Kevin (December 2014). "SP800-88 Rev. 1 Guidelines for Media Sanitization". Computer Security Division, Information Technology Laboratory. NIST. doi: 10.6028/NIST.SP.800-88r1 . Retrieved 18 January 2018.
  25. ↑ Kissel, Richard; Regenscheid, Andrew; Scholl, Matthew; Stine, Kevin (December 2014). "SP800-88 Rev. 1 Guidelines for Media Sanitization" (PDF). Computer Security Division, Information Technology Laboratory. NIST. pp. 27–40. Retrieved 18 January 2018.
  26. ↑ "How to Choose a Secure Data Destruction Method" (PDF). Archived from the original (PDF) on 12 June 2013. Retrieved 6 January 2016.
  27. ↑ Gutmann, Peter (1996). "Secure Deletion of Data from Magnetic and Solid-State Memory". Department of Computer Science, University of Auckland . Retrieved 20 July 2010.
  28. ↑ Hughes, Gordon; Coughlin, Tom (2007). "Tutorial on Disk Drive Data Sanitization" (PDF). University of California, San Diego Center for Magnetic Recording Research. Archived from the original (PDF) on 30 December 2017. Retrieved 10 June 2008.
  29. ↑ "Q & A on Secure Erase". University of California, San Diego Center for Magnetic Recording Research. Archived from the original (DOC) on 30 December 2017.
  30. ↑ Craig Wright; Kleiman, Dave; Sundhar R.S., Shyaam. R. Sekar, R.; Pujari, Arun K. (eds.). Overwriting Hard Drive Data: The Great Wiping Controversy. 4th International Conference, ICISS 2008, Hyderabad, India, December 16–20, 2008. Proceedings. Lecture Notes in Computer Science. Vol. 5352. Springer-Verlag. pp. 243–57. doi:10.1007/978-3-540-89862-7_21. ISBN   978-3-540-89861-0.{{cite conference}}: CS1 maint: multiple names: authors list (link)
  31. ↑ Daniel Feenberg (2003). "Can Intelligence Agencies Recover Overwritten Data?" . Retrieved 10 December 2007.
  32. ↑ "Africa's Agbogbloshie Market Is a Computer Graveyard" NewsBreakingOnline.com. Web. 20 February 2011.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.