QFS

Last updated
QFS
Full nameQuick File System
Limits
Max file size8 EiB [1]
Max no. of files4.1 billion per filesystem
Other
Supported
operating systems
OpenSolaris
Linux (client only) [2]

QFS (Quick File System) is a filesystem from Oracle. It is tightly integrated with SAM, the Storage and Archive Manager, and hence is often referred to as SAM-QFS. SAM provides the functionality of a hierarchical storage manager.

Contents

Features

QFS supports some volume management capabilities, allowing many disks to be grouped together into a file system. File system metadata can be kept on a separate set of disks, which is useful for streaming applications where long disk seeks cannot be tolerated.

SAM extends the QFS file system transparently to archival storage. A SAM-QFS file system may have a relatively small (gigabytes to terabytes) "disk cache" backed by petabytes of tape or other bulk storage. Files are copied to archival storage in the background, and transparently retrieved to disk when accessed. SAM-QFS supports up to four archival copies, each of which can be on disk, tape, optical media, or may be stored at a remote site also running SAM-QFS.

Shared QFS adds a multi-writer global filesystem, allowing multiple machines to read from & write to the same disks concurrently through the use of multi-ported disks or a storage area network. (QFS also has a single-writer/multi-reader mode which can be used to share disks between hosts without the need for a network connection.)

History

SAM-QFS was designed and implemented at Large Storage Systems (LSC). [3] The lead architect of SAM-QFS was Harriet Coverston, the founder and VP of Technology at LSC. [4] LSC and SAM-QFS were purchased by Sun in 2001. [5] Sun released the SAM-QFS source code to the OpenSolaris project in March 2008. [6] After Oracle acquired Sun, Oracle continued to develop the SAM-QFS product. Later Oracle renamed SAM-QFS to Oracle HSM (Oracle Hierarchical Storage Manager). [7] In 2014 Versity, a storage startup co-founded by SAM-QFS lead architect Coverston, released the Versity Storage Manager (VSM), a Linux-based HSM based on the SAM-QFS code. [3] Oracle announced 2019 the end of life to 2021, Versity shows an migration path. [8]

Related Research Articles

<span class="mw-page-title-main">Sun Microsystems</span> American computer company, 1982–2010

Sun Microsystems, Inc. was an American technology company that sold computers, computer components, software, and information technology services and created the Java programming language, the Solaris operating system, ZFS, the Network File System (NFS), and SPARC microprocessors. Sun contributed significantly to the evolution of several key computing technologies, among them Unix, RISC processors, thin client computing, and virtualized computing. Notable Sun acquisitions include Cray Business Systems Division, Storagetek, and Innotek GmbH, creators of VirtualBox. Sun was founded on February 24, 1982. At its height, the Sun headquarters were in Santa Clara, California, on the former west campus of the Agnews Developmental Center.

<span class="mw-page-title-main">Disk partitioning</span> Creation of separate accessible storage areas on a secondary computer storage device

Disk partitioning or disk slicing is the creation of one or more regions on secondary storage, so that each region can be managed separately. These regions are called partitions. It is typically the first step of preparing a newly installed disk after a partitioning scheme is chosen for the new disk before any file system is created. The disk stores the information about the partitions' locations and sizes in an area known as the partition table that the operating system reads before any other part of the disk. Each partition then appears to the operating system as a distinct "logical" disk that uses part of the actual disk. System administrators use a program called a partition editor to create, resize, delete, and manipulate the partitions. Partitioning allows the use of different filesystems to be installed for different kinds of files. Separating user data from system data can prevent the system partition from becoming full and rendering the system unusable. Partitioning can also make backing up easier. A disadvantage is that it can be difficult to properly size partitions, resulting in having one partition with too much free space and another nearly totally allocated.

<span class="mw-page-title-main">Virtual file system</span> Abstract layer on top of a more concrete file system

A virtual file system (VFS) or virtual filesystem switch is an abstract layer on top of a more concrete file system. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, classic Mac OS/macOS and Unix filesystems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing.

<span class="mw-page-title-main">File system</span> Computer filing system

In computing, a file system or filesystem governs file organization and access. A local file system is a capability of an operating system that services the applications running on the same computer. A distributed file system is a protocol that provides file access between networked computers.

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

In computing, an extent is a contiguous area of storage reserved for a file in a file system, represented as a range of block numbers, or tracks on count key data devices. A file can consist of zero or more extents; one file fragment requires one extent. The direct benefit is in storing each range compactly as two numbers, instead of canonically storing every block number in the range. Also, extent allocation results in less file fragmentation.

Lustre is a type of parallel distributed file system, generally used for large-scale cluster computing. The name Lustre is a portmanteau word derived from Linux and cluster. Lustre file system software is available under the GNU General Public License and provides high performance file systems for computer clusters ranging in size from small workgroup clusters to large-scale, multi-site systems. Since June 2005, Lustre has consistently been used by at least half of the top ten, and more than 60 of the top 100 fastest supercomputers in the world, including the world's No. 1 ranked TOP500 supercomputer in November 2022, Frontier, as well as previous top supercomputers such as Fugaku, Titan and Sequoia.

Hierarchical storage management (HSM), also known as tiered storage, is a data storage and data management technique that automatically moves data between high-cost and low-cost storage media. HSM systems exist because high-speed storage devices, such as solid-state drive arrays, are more expensive than slower devices, such as hard disk drives, optical discs and magnetic tape drives. While it would be ideal to have all data available on high-speed devices all the time, this is prohibitively expensive for many organizations. Instead, HSM systems store the bulk of the enterprise's data on slower devices, and then copy data to faster disk drives when needed. The HSM system monitors the way data is used and makes best guesses as to which data can safely be moved to slower devices and which data should stay on the fast devices.

<span class="mw-page-title-main">StorageTek</span> Data storage company

Storage Technology Corporation was a data storage technology company headquartered in Louisville, Colorado. New products include data retention systems, which it calls "information lifecycle management" (ILM).

IBM Storage Protect is a data protection platform that gives enterprises a single point of control and administration for backup and recovery. It is the flagship product in the IBM Spectrum Protect family.

GPFS is high-performance clustered file system software developed by IBM. It can be deployed in shared-disk or shared-nothing distributed parallel modes, or a combination of these. It is used by many of the world's largest commercial companies, as well as some of the supercomputers on the Top 500 List. For example, it is the filesystem of the Summit at Oak Ridge National Laboratory which was the #1 fastest supercomputer in the world in the November 2019 Top 500 List. Summit is a 200 Petaflops system composed of more than 9,000 POWER9 processors and 27,000 NVIDIA Volta GPUs. The storage filesystem is called Alpine.

CacheFS is a family of software technologies designed to speed up distributed file system file access for networked computers. They store copies (caches) of files on secondary memory, typically a local hard disk, so that if a file is accessed again, it can be fetched locally at much higher speeds than networks typically allow.

A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system. Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance.

The Sun Fire X4500 data server integrates server and storage technologies. It was announced in July, 2006 and is part of the Sun Fire server line from Sun Microsystems.

An automounter is any program or software facility which automatically mounts filesystems in response to access operations by user programs. An automounter system utility, when notified of file and directory access attempts under selectively monitored subdirectory trees, dynamically and transparently makes local or remote devices accessible.

StorNext File System (SNFS), colloquially referred to as StorNext is a shared disk file system made by Quantum Corporation. StorNext enables multiple Windows, Linux and Apple workstations to access shared block storage over a Fibre Channel network. With the StorNext file system installed, these computers can read and write to the same storage volume at the same time enabling what is known as a "file-locking SAN." StorNext is used in environments where large files must be shared, and accessed simultaneously by users without network delays, or where a file must be available for access by multiple readers starting at different times. Common use cases include multiple video editor environments in feature film, television and general video post production.

The Linear Tape File System (LTFS) is a file system that allows files stored on magnetic tape to be accessed in a similar fashion to those on disk or removable flash drives. It requires both a specific format of data on the tape media and software to provide a file system interface to the data.

ZFS is a file system with volume management capabilities. It began as part of the Sun Microsystems Solaris operating system in 2001. Large parts of Solaris, including ZFS, were published under an open source license as OpenSolaris for around 5 years from 2005 before being placed under a closed source license when Oracle Corporation acquired Sun in 2009–2010. During 2005 to 2010, the open source version of ZFS was ported to Linux, Mac OS X and FreeBSD. In 2010, the illumos project forked a recent version of OpenSolaris, including ZFS, to continue its development as an open source project. In 2013, OpenZFS was founded to coordinate the development of open source ZFS. OpenZFS maintains and manages the core ZFS code, while organizations using ZFS maintain the specific code and validation processes required for ZFS to integrate within their systems. OpenZFS is widely used in Unix-like systems.

Harriet Coverston is an American computer scientist and software developer focused on large-scale secondary storage environments, who has previously participated to various kernel developments in HPC systems. Harriet is an expert in large scale archiving systems, having participated to several large projects and product developments.

References

  1. "Sun QFS File System 5.3 Configuration and Administration Guide".
  2. "SAM-QFS 5.3 What Works With What".
  3. 1 2 Mellor, Chris (21 June 2016). "New storage upstart Versity offers S3 object storage interface". The Register . Retrieved 21 February 2020.
  4. "Harriet Coverston" . Retrieved 22 February 2020.
  5. Vance, Ashlee (14 May 2001). "Sun sharpens focus on one-stop storage". Computer World. Retrieved 22 February 2020.
  6. SAM/QFS at OpenSolaris.org
  7. Announcing Oracle Hierarchical Storage Manager
  8. Versity and Spectra Logic: The Path Forward for SAM-QFS and Oracle's HSM Replacement