Linear Tape File System

Last updated

The Linear Tape File System (LTFS) is a file system that allows files stored on magnetic tape to be accessed in a similar fashion to those on disk or removable flash drives. It requires both a specific format of data on the tape media and software to provide a file system interface to the data.

Contents

The technology, based around a self-describing tape format developed by IBM, was adopted by the LTO Consortium in 2010.

History

Magnetic tape data storage has been used for over 50 years, but typically did not hold file metadata in a form easy to access or modify independent of the file content data. Often external databases were used to maintain file metadata (file names, timestamps, directory hierarchy) to hold this data but these external databases were generally not designed for interoperability and tapes might or might not contain an index of their content. In Unix-like systems, there is the tar interoperable standard, but this is not well-suited to allow modification of file metadata independent of modifying file content data - and does not maintain a central index of files nor provide a filesystem interface or characteristics.

LTFS technology was first implemented by IBM as a prototype running on Linux and Mac OS X during 2008/2009. This prototype was demonstrated at the NAB show in 2009. Based on feedback from this initial demonstration and experience within IBM the filesystem was overhauled in preparation for release as a product. The LTFS development team worked with the vendors of LTO tape products (HP and Quantum) to build support and understanding of the LTFS format and filesystem implementation leading up to the public release.

The LTFS Format Specification and filesystem implementation were released on April 12, 2010 with the support of IBM, HP, Quantum, and the LTO Consortium. [1]

LTFS v2.0.0 was released in March 2011, improving the text to clarify and remove ambiguity. It also added support for sparse files; persistent file identifiers; virtual extended attributes for filesystem metadata and control - and defined minimum and recommended blocksize values for LTFS volumes, for compatibility across various HBA hardware implementations.

LTFS v2.2.0 was released in December 2013. It was the first version to become an ISO standard (20919:2016).

LTFS v2.5.1 was released in May 2019. It became the second version of the ISO standard (20919:2021). Version 2.5 contained fairly major updates, as it was the first version to define Incremental (sparse) Indexes.

Format specification

The ISO/IEC 20919:2021 standard defines the LTFS Format requirements for interchanged media that claims LTFS compliance. It defines the data format, independent of the physical storage media and the software commands format, to make data truly interchangeable. The ISO standard was prepared by SNIA. It is based on LTFS v2.5.1, and was adopted to ISO by a joint technical committee ISO/IEC JTC 1 Information Technology.

The SNIA workgroup continues to develop LTFS and release updates. Version 2.0.0 defines rules for how the version number may change in future, and how compatibility is maintained across varying implementations. All implementations must:

LTFS Format Specification History
VersionPublishedISO/IECConforming Software
1.0 [2] April 2010— IBM Long Term File System (LTFS) v1.0.0, v1.0.1
HP Linear Tape File System (LTFS) v1.0.0, v1.1.0
2.0.0 [3] March 2011— IBM Linear Tape File System - Single Drive Edition (LTFS-SDE) v1.2.0
Oracle StorageTek Linear Tape File System, Open Edition v1.0.0
2.1.0October 2012—?
2.2.0 [4] December 201320919:2016?
2.3.0 [5] March 2014—?
2.4.0 [6] December 2017—?
2.5.1 [7] May 201920919:2021?

SNIA Technical Work Group

In August 2012, SNIA announced [8] that it was forming a TWG (Technical Work Group) to continue technical development of the specification. LTFS Format Specification v 2.1 is the baseline for the technical work and standards accreditation process; SNIA LTFS TWG members include HP, IBM, Oracle and Quantum.

Nature

While LTFS can make a tape appear to behave like a disk, it does not change the fundamentally sequential nature of tape. Files are always appended to the end of the tape. If a file is modified and overwritten or removed from the volume, the associated tape blocks used are not freed up, they are simply marked as unavailable and the used volume capacity is not recovered. Data is only deleted and capacity recovered if the whole tape is reformatted.[ citation needed ]

In spite of these disadvantages, there are several uses cases where LTFS formatted tape is superior to disk and other data storage technologies. While LTO seek times can range from 10 to 100 seconds, the streaming data transfer rate can match or exceed spinning-disk data transfer rates. Additionally, LTO cartridges are easily transportable and hold far more data than any other removable data storage format. The ability to copy a large file or a large selection of files (up to 1.5TB uncompressed data for LTO-5, and 18TB for LTO-9) to an LTFS formatted tape, allows easy exchange of data to a collaborator, or the saving of an archival copy.

Since LTFS is an open standard, LTFS formatted tapes are usable by a wide variety of computing systems and operating systems, avoiding the incompatibilities caused by previous proprietary LTO filesystems. [9]

Implementations

Tape drives manufacturers often offer two different editions, one for Single Drives and one for Tape Libraries, based on the LTFS Reference Implementation.

IBM Linear Tape File System - Single Drive Edition

The IBM Linear Tape File System - Single Drive Edition, (initially released as "IBM Long Term File System"), allows tapes to be formatted as an LTFS volume, and for these volumes to be mounted - and users and applications access files and directories stored on the tape directly, including drag-and-drop of files.

IBM Linear Tape File System - Library Edition

The IBM Linear Tape File System - Library Edition (LTFS-LE) product allows LTFS volumes to be used in a tape library. Each LTFS-formatted tape cartridge in the library appears as a separate folder under the filesystem mount point and the user or application can navigate into each of these folders to access the files stored on each tape. The LTFS-LE software automatically controls the tape library robotics to load and unload the necessary LTFS volumes.

Oracle's StorageTek Linear Tape File System, Open Edition

Oracle's free open source StorageTek Linear Tape File System (LTFS), Open Edition software [10] is claimed to be the first to store 8.5TB (native capacity) on a single cartridge. It supports Oracle’s midrange StorageTek LTO 5 and LTO 6 tape drives from HP and IBM as well as Oracle’s StorageTek T10000C and T10000D tape drives. [11] [12]

Oracle's StorageTek Linear Tape File System, Library Edition

Oracle’s StorageTek LTFS-LE software offering supports the StorageTek SL8500 Modular Library System, the StorageTek SL3000 Modular Library System, and the StorageTek SL150 Modular Tape. [13]

HP Linear Tape File System

The HP Linear Tape File System (HP LTFS) is HP's implementation. It is a free open source software application.

Quantum Linear Tape File System

Quantum Corporation provided an LTFS product with Windows, Linux and Mac OS X support.

The Scalar LTFS Appliance was a file system that presented a Quantum tape library as an NAS share. This appliance made files viewable as if they resided on a local disk and allowed users to drag and drop files directly to and from a tape cartridge.

LTFS compatible products

DDS Tape Drives

Enterprise Tape Drives

LTO Tape Drives

Appliances and ISVs (Independent Software Vendors) supporting LTFS

A full set of vendors are listed at LTO website. [14]

LTFS projects

Industry recognition


Limitations

As of standard version 2.5.1, LTFS does not support hard links. [7]

When files are deleted, they become invisible to the user. However, the space occupied by a file is not freed. Because of this, it is possible to "roll back" the tape to an earlier state, in order to recover erroneously deleted (or incorrectly updated) files. [7] To free up space a tape needs to be re-formatted.

Related Research Articles

Universal Disk Format (UDF) is an open, vendor-neutral file system for computer data storage for a broad range of media. In practice, it has been most widely used for DVDs and newer optical disc formats, supplanting ISO 9660. Due to its design, it is very well suited to incremental updates on both write-once and re-writable optical media. UDF was developed and maintained by the Optical Storage Technology Association (OSTA).

RAID is a data storage virtualization technology that combines multiple physical disk drive components into one or more logical units for the purposes of data redundancy, performance improvement, or both. This is in contrast to the previous concept of highly reliable mainframe disk drives referred to as "single large expensive disk" (SLED).

Quantum Corporation is a data storage, management, and protection company that provides technology to store, manage, archive, and protect video and unstructured data throughout the data life cycle. Their products are used by enterprises, media and entertainment companies, government agencies, big data companies, and life science organizations. Quantum is headquartered in San Jose, California and has offices around the world, supporting customers globally in addition to working with a network of distributors, VARs, DMRs, OEMs and other suppliers.

<span class="mw-page-title-main">Digital Data Storage</span> Computer data storage technology based on magnetic tape

Digital Data Storage (DDS) is a computer data storage technology that is based upon the Digital Audio Tape (DAT) format that was developed during the 1980s. DDS is primarily intended for use as off-line storage, especially for generating backup copies of working data.

<span class="mw-page-title-main">Linear Tape-Open</span> Magnetic tape-based data storage technology

Linear Tape-Open (LTO), also known as LTO Ultrium, is a magnetic tape data storage technology used for backup, data archiving, and data transfer. It was originally developed in the late 1990s as an open standards alternative to the proprietary magnetic tape formats that were available at the time. Upon introduction, LTO rapidly defined the super tape market segment and has consistently been the best-selling super tape format. The latest generation as of 2021, LTO-9, can hold 18 TB in one cartridge.

In computer storage, a tape library is a physical area that holds magnetic data tapes. In an earlier era, tape libraries were maintained by people known as tape librarians and computer operators and the proper operation of the library was crucial to the running of batch processing jobs. Although tape libraries of this era were not automated, the use of tape management system software could assist in running them.

<span class="mw-page-title-main">File system</span> Format or program for storing files and directories

In computing, a file system or filesystem is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified. Taking its name from the way a paper-based data management system is named, each group of data is called a "file". The structure and logic rules used to manage the groups of data and their names is called a "file system."

The Storage Management Initiative Specification, commonly called SMI-S, is a computer data storage management standard developed and maintained by the Storage Networking Industry Association (SNIA). It has also been ratified as an ISO standard. SMI-S is based upon the Common Information Model and the Web-Based Enterprise Management standards defined by the Distributed Management Task Force, which define management functionality via HTTP. The most recent approved version of SMI-S is available on the SNIA website.

<span class="mw-page-title-main">Storage Technology Corporation</span> Data storage company

Storage Technology Corporation was a data storage technology company headquartered in Louisville, Colorado. New products include data retention systems, which it calls "information lifecycle management" (ILM).

In computing, an archive file is a computer file that is composed of one or more files along with metadata. Many archive formats also support compression of member files. Archive files are used to collect multiple data files together into a single file for easier portability and storage, or simply to compress files to use less storage space. Archive files often store directory structures, error detection and correction information, comments, and some use built-in encryption.

IBM Storage Protect is a data protection platform that gives enterprises a single point of control and administration for backup and recovery. It is the flagship product in the IBM Spectrum Protect family.

Storage Technology Corporation created several magnetic tape data storage formats. These are commonly used with large computer systems, typically in conjunction with a robotic tape library. The most recent format is the T10000. StorageTek primarily competed with IBM in this market, and continued to do so after its acquisition by Sun Microsystems in 2005 and as part of the Sun Microsystems acquisition by Oracle in 2009.

Magnetic-tape data storage is a system for storing digital information on magnetic tape using digital recording.

<span class="mw-page-title-main">IBM storage</span> Product portfolio of IBM

The IBM Storage product portfolio includes disk, flash, tape, NAS storage products, storage software and services. IBM's approach is to focus on data management.

StorNext File System (SNFS), colloquially referred to as StorNext is a shared disk file system made by Quantum Corporation. StorNext enables multiple Windows, Linux and Apple workstations to access shared block storage over a Fibre Channel network. With the StorNext file system installed, these computers can read and write to the same storage volume at the same time enabling what is known as a "file-locking SAN." StorNext is used in environments where large files must be shared, and accessed simultaneously by users without network delays, or where a file must be available for access by multiple readers starting at different times. Common use cases include multiple video editor environments in feature film, television and general video post production.

Tape labels are identifiers given to volumes of magnetic tape.

<span class="mw-page-title-main">Active Archive Alliance</span> Trade association

The Active Archive Alliance is a trade association that promotes a method of tiered storage. This method provides users access to data across a virtual file system that migrates data between multiple storage systems and media types including solid-state drive/flash, hard disk drives, magnetic tape, optical disk, and cloud. The result of an active archive implementation is that data can be stored on the most appropriate media type for the given retention and restoration requirements of that data. This allows less time sensitive or infrequently accessed data to be stored on less expensive media and eliminates the need for an administrator to manually migrate data between storage systems. Additionally, since storage systems such as tape libraries have low power consumption, the operational expense of storing data in an active archive is significantly reduced.

Resilient File System (ReFS), codenamed "Protogon", is a Microsoft proprietary file system introduced with Windows Server 2012 with the intent of becoming the "next generation" file system after NTFS.

ZFS is a file system with volume management capabilities. It began as part of the Sun Microsystems Solaris operating system in 2001. Large parts of Solaris – including ZFS – were published under an open source license as OpenSolaris for around 5 years from 2005 before being placed under a closed source license when Oracle Corporation acquired Sun in 2009–2010. During 2005 to 2010, the open source version of ZFS was ported to Linux, Mac OS X and FreeBSD. In 2010, the illumos project forked a recent version of OpenSolaris, including ZFS, to continue its development as an open source project. In 2013, OpenZFS was founded to coordinate the development of open source ZFS. OpenZFS maintains and manages the core ZFS code, while organizations using ZFS maintain the specific code and validation processes required for ZFS to integrate within their systems. OpenZFS is widely used in Unix-like systems.

References

  1. ↑ NAB-2010 Archived 2012-09-04 at archive.today
  2. ↑ Linear Tape File System (LTFS) Format v1.0 Specification
  3. ↑ Linear Tape File System (LTFS) Format v2.0.1 Specification
  4. ↑ Linear Tape File System (LTFS) Format v2.2.0 Specification
  5. ↑ Linear Tape File System (LTFS) Format v2.3.1 Specification
  6. ↑ Linear Tape File System (LTFS) Format v2.4.0 Specification
  7. 1 2 3 Linear Tape File System (LTFS) Format v2.5.1 Specification
  8. ↑ "SNIA Announces Linear Tape File System Technical Work Group"
  9. ↑ "Affordable Easy LTO Archiving". Backupworks.com.
  10. ↑ Oracle StorageTek Linear Tape File System, Open Edition Webpage
  11. ↑ Oracle StorageTek T10000C Tape Drive Press Release
  12. ↑ Oracle Introduces StorageTek T10000D Tape Drive
  13. ↑ Oracle StorageTek LTFS, Library Edition Press Release
  14. ↑ LTFS Implementers
  15. ↑ Thought Equity Motion - Homepage
  16. ↑ Thought Equity Motion - Press Release: Film Archive Digitization and Preservation Project for EYE Film Institute Netherlands
  17. ↑ NAB 2011 Pick Hit Announcement
  18. ↑ "IBM Almaden Blog". Archived from the original on 25 April 2012. Retrieved 26 October 2011.
  19. ↑ HPA Award Announcement
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.