QFS

Last updated
QFS
Full nameQuick File System
Limits
Max file size8 EiB [1]
Max no. of files4.1 billion per filesystem
Other
Supported
operating systems
OpenSolaris
Linux (client only) [2]

QFS (Quick File System) is a filesystem from Oracle. It is tightly integrated with SAM, the Storage and Archive Manager, and hence is often referred to as SAM-QFS. SAM provides the functionality of a hierarchical storage manager.

Contents

Features

QFS supports some volume management capabilities, allowing many disks to be grouped together into a file system. File system metadata can be kept on a separate set of disks, which is useful for streaming applications where long disk seeks cannot be tolerated.

SAM extends the QFS file system transparently to archival storage. A SAM-QFS file system may have a relatively small (gigabytes to terabytes) "disk cache" backed by petabytes of tape or other bulk storage. Files are copied to archival storage in the background, and transparently retrieved to disk when accessed. SAM-QFS supports up to four archival copies, each of which can be on disk, tape, optical media, or may be stored at a remote site also running SAM-QFS.

Shared QFS adds a multi-writer global filesystem, allowing multiple machines to read from & write to the same disks concurrently through the use of multi-ported disks or a storage area network. (QFS also has a single-writer/multi-reader mode which can be used to share disks between hosts without the need for a network connection.)

History

SAM-QFS was designed and implemented at Large Storage Systems (LSC). [3] The lead architect of SAM-QFS was Harriet Coverston, the founder and VP of Technology at LSC. [4] LSC and SAM-QFS were purchased by Sun in 2001. [5] Sun released the SAM-QFS source code to the OpenSolaris project in March 2008. [6] After Oracle acquired Sun, Oracle continued to develop the SAM-QFS product. Later Oracle renamed SAM-QFS to Oracle HSM (Oracle Hierarchical Storage Manager). [7] In 2014 Versity, a storage startup co-founded by SAM-QFS lead architect Coverston, released the Versity Storage Manager (VSM), a Linux-based HSM based on the SAM-QFS code. [3] oracle announced 2019 the end of life to 2021, Versity shows an migration path. [8]

Related Research Articles

<span class="mw-page-title-main">Sun Microsystems</span> American computer company, 1982–2010

Sun Microsystems, Inc., was an American technology company that sold computers, computer components, software, and information technology services and created the Java programming language, the Solaris operating system, ZFS, the Network File System (NFS), and SPARC microprocessors. Sun contributed significantly to the evolution of several key computing technologies, among them Unix, RISC processors, thin client computing, and virtualized computing. Notable Sun acquisitions include Cray Business Systems Division, Storagetek, and Innotek GmbH, creators of VirtualBox. Sun was founded on February 24, 1982. At its height, the Sun headquarters were in Santa Clara, California, on the former west campus of the Agnews Developmental Center.

<span class="mw-page-title-main">Disk partitioning</span> Creation of separate accessible storage areas on a secondary computer storage device

Disk partitioning or disk slicing is the creation of one or more regions on secondary storage, so that each region can be managed separately. These regions are called partitions. It is typically the first step of preparing a newly installed disk, before any file system is created. The disk stores the information about the partitions' locations and sizes in an area known as the partition table that the operating system reads before any other part of the disk. Each partition then appears to the operating system as a distinct "logical" disk that uses part of the actual disk. System administrators use a program called a partition editor to create, resize, delete, and manipulate the partitions. Partitioning allows the use of different filesystems to be installed for different kinds of files. Separating user data from system data can prevent the system partition from becoming full and rendering the system unusable. Partitioning can also make backing up easier. A disadvantage is that it can be difficult to properly size partitions, resulting in having one partition with too much free space and another nearly totally allocated.

<span class="mw-page-title-main">Virtual file system</span> Abstract layer on top of a more concrete file system

A virtual file system (VFS) or virtual filesystem switch is an abstract layer on top of a more concrete file system. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, classic Mac OS/macOS and Unix filesystems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing.

tmpfs is a temporary file storage paradigm implemented in many Unix-like operating systems. It is intended to appear as a mounted file system, but data is stored in volatile memory instead of a persistent storage device. A similar construction is a RAM disk, which appears as a virtual disk drive and hosts a disk file system.

<span class="mw-page-title-main">File system</span> Format or program for storing files and directories

In computing, a file system or filesystem is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified. Taking its name from the way a paper-based data management system is named, each group of data is called a "file". The structure and logic rules used to manage the groups of data and their names is called a "file system."

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

In computing, an extent is a contiguous area of storage reserved for a file in a file system, represented as a range of block numbers, or tracks on count key data devices. A file can consist of zero or more extents; one file fragment requires one extent. The direct benefit is in storing each range compactly as two numbers, instead of canonically storing every block number in the range. Also, extent allocation results in less file fragmentation.

Lustre is a type of parallel distributed file system, generally used for large-scale cluster computing. The name Lustre is a portmanteau word derived from Linux and cluster. Lustre file system software is available under the GNU General Public License and provides high performance file systems for computer clusters ranging in size from small workgroup clusters to large-scale, multi-site systems. Since June 2005, Lustre has consistently been used by at least half of the top ten, and more than 60 of the top 100 fastest supercomputers in the world, including the world's No. 1 ranked TOP500 supercomputer in November 2022, Frontier, as well as previous top supercomputers such as Fugaku, Titan and Sequoia.

Hierarchical storage management (HSM), also known as Tiered storage, is a data storage and Data management technique that automatically moves data between high-cost and low-cost storage media. HSM systems exist because high-speed storage devices, such as solid state drive arrays, are more expensive than slower devices, such as hard disk drives, optical discs and magnetic tape drives. While it would be ideal to have all data available on high-speed devices all the time, this is prohibitively expensive for many organizations. Instead, HSM systems store the bulk of the enterprise's data on slower devices, and then copy data to faster disk drives when needed. The HSM system monitors the way data is used and makes best guesses as to which data can safely be moved to slower devices and which data should stay on the fast devices.

IBM Spectrum Protect is a data protection platform that gives enterprises a single point of control and administration for backup and recovery. It is the flagship product in the IBM Spectrum Protect family.

The Oracle Cluster File System is a shared disk file system developed by Oracle Corporation and released under the GNU General Public License. The first version of OCFS was developed with the main focus to accommodate Oracle's database management system that used cluster computing. Because of that it was not a POSIX-compliant file system. With version 2 the POSIX features were included.

Content-addressable storage (CAS), also referred to as content-addressed storage or fixed-content storage, is a way to store information so it can be retrieved based on its content, not its name or location. It has been used for high-speed storage and retrieval of fixed content, such as documents stored for compliance with government regulations. Content-addressable storage is similar to content-addressable memory.

CacheFS is the name used for several similar software technologies designed to speed up distributed file system file access for networked computers. These technologies operate by storing (cached) copies of files on secondary memory, typically a local hard disk, so that if a file is accessed again, it can be done locally at much higher speeds than networks typically allow.

A clustered file system is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system. Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance.

StorNext File System (SNFS), colloquially referred to as StorNext is a shared disk file system made by Quantum Corporation. StorNext enables multiple Windows, Linux and Apple workstations to access shared block storage over a Fibre Channel network. With the StorNext file system installed, these computers can read and write to the same storage volume at the same time enabling what is known as a "file-locking SAN." StorNext is used in environments where large files must be shared, and accessed simultaneously by users without network delays, or where a file must be available for access by multiple readers starting at different times. Common use cases include multiple video editor environments in feature film, television and general video post production.

The Linear Tape File System (LTFS) is a file system that allows files stored on magnetic tape to be accessed in a similar fashion to those on disk or removable flash drives. It requires both a specific format of data on the tape media and software to provide a file system interface to the data.

<span class="mw-page-title-main">OpenZFS</span> Open-source implementation of the ZFS file system

OpenZFS is an open-source implementation of the ZFS file system and volume manager initially developed by Sun Microsystems for the Solaris operating system and now maintained by the OpenZFS Project. It supports features like data compression, data deduplication, copy-on-write clones, snapshots, and RAID-Z. It also supports the creation of virtual devices, which allows for the creation of file systems that span multiple disks.

ZFS is a file system with volume management capabilities. It began as part of the Sun Microsystems Solaris operating system in 2001. Large parts of Solaris – including ZFS – were published under an open source license as OpenSolaris for around 5 years from 2005, before being placed under a closed source license when Oracle Corporation acquired Sun in 2009–2010. During 2005 to 2010, the open source version of ZFS was ported to Linux, Mac OS X and FreeBSD. In 2010, the illumos project forked a recent version of OpenSolaris, to continue its development as an open source project, including ZFS. In 2013, OpenZFS was founded to coordinate the development of open source ZFS. OpenZFS maintains and manages the core ZFS code, while organizations using ZFS maintain the specific code and validation processes required for ZFS to integrate within their systems. OpenZFS is widely used in Unix-like systems.

Harriet Coverston is an American computer scientist and software developer focused on large-scale secondary storage environments, who has previously participated to various kernel developments in HPC systems. Harriet is an expert in large scale archiving systems, having participated to several large projects and product developments.

References

  1. "Sun QFS File System 5.3 Configuration and Administration Guide".
  2. "SAM-QFS 5.3 What Works With What".
  3. 1 2 Mellor, Chris (21 June 2016). "New storage upstart Versity offers S3 object storage interface". The Register . Retrieved 21 February 2020.
  4. "Harriet Coverston" . Retrieved 22 February 2020.
  5. Vance, Ashlee (14 May 2001). "Sun sharpens focus on one-stop storage". Computer World. Retrieved 22 February 2020.
  6. SAM/QFS at OpenSolaris.org
  7. Announcing Oracle Hierarchical Storage Manager
  8. Versity and Spectra Logic: The Path Forward for SAM-QFS and Oracle's HSM Replacement