List of file systems

Last updated

The following lists identify, characterize, and link to more thorough information on file systems. Many older operating systems support only their one "native" file system, which does not bear any name apart from the name of the operating system itself.

Contents

Disk file systems

Disk file systems are usually block-oriented. Files in a block-oriented file system are sequences of blocks, often featuring fully random-access read, write, and modify operations.

File systems with built-in fault-tolerance

These file systems have built-in checksumming and either mirroring or parity for extra redundancy on one or several block devices:

File systems optimized for flash memory, solid state media

Solid state media, such as flash memory, are similar to disks in their interfaces, but have different problems. At low level, they require special handling such as wear leveling and different error detection and correction algorithms. Typically a device such as a solid-state drive handles such operations internally and therefore a regular file system can be used. However, for certain specialized installations (embedded systems, industrial applications) a file system optimized for plain flash memory is advantageous.

Record-oriented file systems

In record-oriented file systems files are stored as a collection of records. They are typically associated with mainframe and minicomputer operating systems. Programs read and write whole records, rather than bytes or arbitrary byte ranges, and can seek to a record boundary but not within records. The more sophisticated record-oriented file systems have more in common with simple databases than with other file systems.

Shared-disk file systems

Shared-disk file systems (also called shared-storage file systems, SAN file system, Clustered file system or even cluster file systems) are primarily used in a storage area network where all nodes directly access the block storage where the file system is located. This makes it possible for nodes to fail without affecting access to the file system from the other nodes. Shared-disk file systems are normally used in a high-availability cluster together with storage on hardware RAID. Shared-disk file systems normally do not scale over 64 or 128 nodes.

Shared-disk file systems may be symmetric where metadata is distributed among the nodes or asymmetric with centralized metadata servers.

Distributed file systems

Distributed file systems are also called network file systems. Many implementations have been made, they are location dependent and they have access control lists (ACLs), unless otherwise stated below.

Distributed fault-tolerant file systems

Distributed fault-tolerant replication of data between nodes (between servers or servers/clients) for high availability and offline (disconnected) operation.

Distributed parallel file systems

Distributed parallel file systems stripe data over multiple servers for high performance. They are normally used in high-performance computing (HPC).

Some of the distributed parallel file systems use an object storage device (OSD) (in Lustre called OST) for chunks of data together with centralized metadata servers.

Distributed parallel fault-tolerant file systems

Distributed file systems, which also are parallel and fault tolerant, stripe and replicate data over multiple servers for high performance and to maintain data integrity. Even if a server fails no data is lost. The file systems are used in both high-performance computing (HPC) and high-availability clusters.

All file systems listed here focus on high availability, scalability and high performance unless otherwise stated below.

NameByLicenseOSDescription
Alluxio UC Berkeley, Alluxio Apache License cross-platform An open-source virtual distributed file system (VDFS).
BeeGFS (formerly FhGFS) Fraunhofer Society GNU GPL v2 for client, other components are proprietary Linux A free to use file system with optional professional support, designed for easy usage and high performance, used on some of the fastest computer clusters in the world. BeeGFS allows replication of storage volumes with automatic failover and self-healing.
CephFS Inktank Storage, a company acquired by Red Hat GNU LGPL Linux kernel, FreeBSD via FUSE [22] A massively scalable object store. CephFS was merged into the Linux kernel in 2010. Ceph's foundation is the reliable autonomic distributed object store (RADOS), which provides object storage via programmatic interface and S3 or Swift REST APIs, block storage to QEMU/KVM/Linux hosts, and POSIX filesystem storage which can be mounted by Linux kernel and FUSE clients.
Chiron FS GNU GPL v3 It's a FUSE-based, transparent replication file system, layering on an existing file system and implementing at the file system level what RAID 1 does at the device level. A notably convenient consequence is the possibility of picking single target directories, without the need of replicating entire partitions. (The project has no visible activity after 2008, a status request in Oct. 2009 in the chironfs-forum is unanswered.)
CloudStore Kosmix Apache License Google File System workalike. Replaced by Quantcast File System (QFS)
dCache DESY and others proprietary (Free for non-commercial usage) [23] A write once filesystem, accessible via various protocols.
General Parallel File System (GPFS) IBM Proprietary AIX, Linux and WindowsA POSIX-compliant, high-performance, parallel filesystem. Support synchronous replication between attached block storage, and asynchronous replication to remote filesystems. Also support erasure coding on dual homed SAS attached storage, and distributed over multiple storage nodes.
Gfarm file system NPO Tsukuba OSS Technical Support Center X11 License Linux, macOS, FreeBSD, NetBSD and Solaris Uses PostgreSQL for metadata and FUSE for mounting.
GlusterFS Gluster, a company acquired by Red HatGNU GPL v3 Linux, NetBSD, FreeBSD, OpenSolaris A general purpose distributed file system for scalable storage. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. GlusterFS is the main component in Red Hat Storage Server.
Google File System (GFS) Google internal softwareFocus on fault tolerance, high throughput and scalability.
Hadoop Distributed File System Apache Software Foundation Apache License Cross-platformOpen source GoogleFS clone.
IBRIX Fusion IBRIX Proprietary
JuiceFS Juicedata Apache License cross-platform An open-source POSIX-compliant file system built on top of Redis and object storage (e.g. Amazon S3), designed and optimized for cloud native environment.
LizardFS SkytechnologyGNU GPL v3 cross-platform An open source, highly available POSIX-compliant file system that supports Windows clients.
Lustre originally developed by Cluster File Systems and currently supported by OpenSFSGNU GPL v2 & LGPL Linux A POSIX-compliant, high-performance filesystem used on a majority of systems in the Top-500 list of HPC systems. Lustre has high availability via storage failover.
MapR FS MapR Proprietary LinuxHighly scalable, POSIX compliant, fault tolerant, read/write filesystem with a distributed, fault tolerant metadata service. It provides an HDFS and NFS interface to clients as well as a noSQL table interface and Apache Kafka compatible messaging system.
MooseFS Core Technology GNU GPL v2 and proprietary [24] cross-platform (Linux, NetBSD, FreeBSD, macOS, OpenSolaris)A fault tolerant, highly available and high performance scale-out network distributed file system. It spreads data over several physical commodity x86 servers, which are visible to the user as one namespace. For standard file operations MooseFS acts like any other Unix-like file systems.
ObjectiveFS Objective Security Corporation Proprietary Linux, macOS POSIX-compliant shared distributed filesystem. Uses object store as a backend. Runs on AWS S3, GCS and object store devices.
OneFS distributed file system Isilon Proprietary [25] FreeBSD BSD-based OS on dedicated Intel based hardware, serving NFS v3 and SMB/CIFS to Windows, macOS, Linux and other UNIX clients under a proprietary software.
OIO-FS OpenIO Proprietary Linux OIO-FS provides file-oriented access to OpenIO SDS object storage backend. It is based on FUSE technology and presents a POSIX file system to users. This access can be used locally, or over a network using NFS or SMB. [26]
PanFS Panasas Proprietary Linux, macOS, FreeBSD A POSIX-compliant, high-performance, parallel filesystem used by HPC clusters. It uses erasure coding and snapshots for data protection, is based upon a scale-out object store, and is focused on transparent failure recovery and ease of use.
Quobyte DCFS Quobyte Proprietary Linux, macOS, FreeBSD A fault-tolerant, parallel POSIX file system, with block (VMs) and object (S3) interfaces, and advanced enterprise features like multi-tenancy, strong authentication, encryption. Split-brain safe fault-tolerance is achieved through Paxos-based leader election and erasure coding.
RozoFS Rozo SystemsGNU GPL v2 Linux A POSIX DFS focused on fault-tolerance and high-performance, based on the Mojette erasure code to reduce significantly the amount of redundancy (compared to plain replication).
Scality Scality ring Proprietary Linux A POSIX file system[ citation needed ] focused on high availability and performance. Also provides S3/REST/NFS interfaces.
Tahoe-LAFS Tahoe-LAFS Software FoundationGNU GPL v2+ and other [27] Windows, Linux, macOS A secure, decentralized, fault-tolerant, peer-to-peer distributed data store and distributed file system.
XtreemFS Contrail E.U. project, the German MoSGrid project and the German project "First We Take Berlin" BSD 3-Clause [28] Linux, Solaris, macOS, WindowsA cross-platform file system for wide area networks. It replicates the data for fault tolerance and caches metadata and data to improve performance over high-latency links. SSL and X.509 certificates support makes XtreemFS usable over public networks. It also supports striping for usage in a cluster.

In development:

Peer-to-peer file systems

Some of these may be called cooperative storage cloud.

Special-purpose file systems

Pseudo file systems

Encrypted file systems

File system interfaces

These are not really file systems; they allow access to file systems from an operating system standpoint.

See also

Related Research Articles

ext2, or second extended file system, is a file system for the Linux kernel. It was initially designed by French software developer Rémy Card as a replacement for the extended file system (ext). Having been designed according to the same principles as the Berkeley Fast File System from BSD, it was the first commercial-grade filesystem for Linux.

<span class="mw-page-title-main">Disk partitioning</span> Creation of separate accessible storage areas on a secondary computer storage device

Disk partitioning or disk slicing is the creation of one or more regions on secondary storage, so that each region can be managed separately. These regions are called partitions. It is typically the first step of preparing a newly installed disk, before any file system is created. The disk stores the information about the partitions' locations and sizes in an area known as the partition table that the operating system reads before any other part of the disk. Each partition then appears to the operating system as a distinct "logical" disk that uses part of the actual disk. System administrators use a program called a partition editor to create, resize, delete, and manipulate the partitions. Partitioning allows the use of different filesystems to be installed for different kinds of files. Separating user data from system data can prevent the system partition from becoming full and rendering the system unusable. Partitioning can also make backing up easier. A disadvantage is that it can be difficult to properly size partitions, resulting in having one partition with too much free space and another nearly totally allocated.

The Unix file system (UFS) is a family of file systems supported by many Unix and Unix-like operating systems. It is a distant descendant of the original filesystem used by Version 7 Unix.

<span class="mw-page-title-main">Live CD</span> Complete, bootable computer installation that runs directly from a CD-ROM

A live CD is a complete bootable computer installation including operating system which runs directly from a CD-ROM or similar storage device into a computer's memory, rather than loading from a hard disk drive. A live CD allows users to run an operating system for any purpose without installing it or making any changes to the computer's configuration. Live CDs can run on a computer without secondary storage, such as a hard disk drive, or with a corrupted hard disk drive or file system, allowing data recovery.

<span class="mw-page-title-main">Defragmentation</span> Rearrangement of sectors on a hard disk into contiguous units

In the maintenance of file systems, defragmentation is a process that reduces the degree of fragmentation. It does this by physically organizing the contents of the mass storage device used to store files into the smallest number of contiguous regions. It also attempts to create larger regions of free space using compaction to impede the return of fragmentation. Some defragmentation utilities try to keep smaller files within a single directory together, as they are often accessed in sequence.

tmpfs is a temporary file storage paradigm implemented in many Unix-like operating systems. It is intended to appear as a mounted file system, but data is stored in volatile memory instead of a persistent storage device. A similar construction is a RAM disk, which appears as a virtual disk drive and hosts a disk file system.

<span class="mw-page-title-main">File system</span> Computer filing system

In computing, a file system or filesystem governs file organization and access. A local file system is a capability of an operating system that services the applications running on the same computer. A distributed file system is a protocol that provides file access between networked computers.

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

In computing, an extent is a contiguous area of storage reserved for a file in a file system, represented as a range of block numbers, or tracks on count key data devices. A file can consist of zero or more extents; one file fragment requires one extent. The direct benefit is in storing each range compactly as two numbers, instead of canonically storing every block number in the range. Also, extent allocation results in less file fragmentation.

GPFS is high-performance clustered file system software developed by IBM. It can be deployed in shared-disk or shared-nothing distributed parallel modes, or a combination of these. It is used by many of the world's largest commercial companies, as well as some of the supercomputers on the Top 500 List. For example, it is the filesystem of the Summit at Oak Ridge National Laboratory which was the #1 fastest supercomputer in the world in the November 2019 TOP500 list of supercomputers. Summit is a 200 Petaflops system composed of more than 9,000 POWER9 processors and 27,000 NVIDIA Volta GPUs. The storage filesystem called Alpine has 250 PB of storage using Spectrum Scale on IBM ESS storage hardware, capable of approximately 2.5TB/s of sequential I/O and 2.2TB/s of random I/O.

Extended file attributes are file system features that enable users to associate computer files with metadata not interpreted by the filesystem, whereas regular attributes have a purpose strictly defined by the filesystem. Unlike forks, which can usually be as large as the maximum file size, extended attributes are usually limited in size to a value significantly smaller than the maximum file size. Typical uses include storing the author of a document, the character encoding of a plain-text document, or a checksum, cryptographic hash or digital certificate, and discretionary access control information.

<span class="mw-page-title-main">TestDisk</span> Data recovery utility

TestDisk is a free and open-source data recovery utility that helps users recover lost partitions or repair corrupted filesystems. TestDisk can collect detailed information about a corrupted drive, which can then be sent to a technician for further analysis. TestDisk supports DOS, Microsoft Windows, Linux, FreeBSD, NetBSD, OpenBSD, SunOS, and MacOS. TestDisk handles non-partitioned and partitioned media. In particular, it recognizes the GUID Partition Table (GPT), Apple partition map, PC/Intel BIOS partition tables, Sun Solaris slice and Xbox fixed partitioning scheme. TestDisk uses a command line user interface. TestDisk can recover deleted files with 97% accuracy.

AdvFS, also known as Tru64 UNIX Advanced File System, is a file system developed in the late 1980s to mid-1990s by Digital Equipment Corporation for their OSF/1 version of the Unix operating system. In June 2008, it was released as free software under the GPL-2.0-only license. AdvFS has been used in high-availability systems where fast recovery from downtime is essential.

The following tables compare general and technical information for a number of file systems.

The following is a timeline of virtualization development. In computing, virtualization is the use of a computer to simulate another computer. Through virtualization, a host simulates a guest by exposing virtual hardware devices, which may be done through software or by allowing access to a physical device connected to the machine.

A clustered file system (CFS) is a file system which is shared by being simultaneously mounted on multiple servers. There are several approaches to clustering, most of which do not employ a clustered file system. Clustered file systems can provide features like location-independent addressing and redundancy which improve reliability or reduce the complexity of the other parts of the cluster. Parallel file systems are a type of clustered file system that spread data across multiple storage nodes, usually for redundancy or performance.

Ceph is a free and open-source software-defined storage platform that provides object storage, block storage, and file storage built on a common distributed cluster foundation. Ceph provides completely distributed operation without a single point of failure and scalability to the exabyte level, and is freely available. Since version 12 (Luminous), Ceph does not rely on any other conventional filesystem and directly manages HDDs and SSDs with its own storage backend BlueStore and can expose a POSIX filesystem.

<span class="mw-page-title-main">OpenZFS</span> Open-source implementation of the ZFS file system

OpenZFS is an open-source implementation of the ZFS file system and volume manager initially developed by Sun Microsystems for the Solaris operating system and now maintained by the OpenZFS Project. It supports features like data compression, data deduplication, copy-on-write clones, snapshots, and RAID-Z. It also supports the creation of virtual devices, which allows for the creation of file systems that span multiple disks.

References

  1. 1 2 3 Koen Vervloesem (2010-04-21). "DragonFly BSD 2.6: towards a free clustering operating system". LWN.net . Retrieved 2019-03-07.
  2. 1 2 Matt Dillon (2017-09-23). "hammer_disk.h". BSD Cross Reference. DragonFly BSD . Retrieved 2019-03-06.
  3. ↑ "DragonFly BSD 5.2". DragonFly BSD. 2018-06-18. Retrieved 2019-03-06. We can now recommend H2 as the default root filesystem in non-clustered mode.
  4. 1 2 Matt Dillon (2018-05-05). "hammer2_disk.h". BSD Cross Reference. DragonFly BSD . Retrieved 2019-03-06.
  5. 1 2 3 Matt Dillon (2018-12-09). "hammer2/DESIGN". BSD Cross Reference. DragonFly BSD . Retrieved 2019-03-06.
  6. ↑ Corbet, Jonathan. "The Next3 filesystem". LWN.
  7. ↑ "RTEMS File System" . Retrieved 20 April 2013.
  8. ↑ Overstreet, Kent. "Bcachefs main site".
  9. ↑ Edge, Jake. "LWN - An update on bcachefs".
  10. ↑ "Bcachefs merged in linux 6.7".
  11. ↑ Overstreet, Kent. "Bcachefs on Patreon".
  12. ↑ Michael Larabel (2011-10-05). "Samsung Introduces New Linux File-System: F2FS". phoronix.com. Retrieved 2012-12-07.
  13. ↑ "United States Patent: 5392427". Patft.uspto.gov. Retrieved 2012-06-15.
  14. ↑ "Linux Kernel Mailing List: logfs: remove from tree" . Retrieved 2017-03-31.
  15. ↑ "Jo's Embedded Serial File System (for Standard Serial NOR-Flash)". GitHub . 2019-06-18.
  16. ↑ Pirkola, G. C. (June 1975). "A file system for a general-purpose time-sharing environment". Proceedings of the IEEE. 63 (6): 918–924. doi:10.1109/PROC.1975.9856. ISSN   0018-9219. S2CID   12982770.
  17. ↑ IBM. 4690 OS Programming Guide Version 5.2, IBM document SC30-4137-01, 2007-12-06 ().
  18. ↑ Caldera (1997). Caldera OpenDOS Machine Readable Source Kit 7.01. The FDOS.EQU file in the machine readable source kit has equates for the corresponding directory entries.
  19. ↑ LizardFS.org
  20. ↑ IBM (2003). Information about 4690 OS unique file distribution attributes, IBM document R1001487, 2003-07-30. ( "IBM Information about 4690 OS unique file distribution attributes - United States". Archived from the original on 2014-05-21. Retrieved 2014-05-20.): "[...] file types are stored in the "Reserved bits" portion of the PC-DOS file directory structure [...] only 4690 respects and preserves these attributes. Various non-4690 operating systems take different actions if these bits are turned on [...] when copying from a diskette created on a 4690 system. [...] PC-DOS and Windows 2000 Professional will copy the file without error and zero the bits. OS/2 [...] 1.2 [...] will refuse to copy the file unless [...] first run CHKDSK /F on the file. After [...] CHKDSK, it will copy the file and zero the bits. [...] when [...] copy [...] back to the 4690 system, [...] file will copy as a local file."
  21. ↑ IBM. 4690 save and restore file distribution attributes. IBM document R1000622, 2010-08-31 ( "IBM 4690 save and restore file distribution attributes - United States". Archived from the original on 2014-05-21. Retrieved 2014-05-20.).
  22. ↑ "net/ceph14: Ceph delivers object, block, and file storage in a unified system". FreshPorts. Retrieved 2021-07-11.
  23. ↑ "dCache Software License".
  24. ↑ "MooseFS".
  25. ↑ "OneFS 8.1 eLicensing and remote support changes".
  26. ↑ "Features of OIO-FS — OpenIO SDS 18.10 Object Storage documentation". docs.openio.io. Retrieved 2018-12-20.
  27. ↑ "about.rst in trunk/docs – tahoe-lafs". Tahoe-lafs.org. Retrieved 2014-02-09.
  28. ↑ "XtreemFS - License".
  29. ↑ "mount_sysctlfs(8)". NetBSD Manual Pages. 2010-04-11. Retrieved 2021-07-10.
  30. ↑ Pawel Jakub Dawidek (2002-12-24). "sysctlfs.README". Archived from the original (text/plain) on 2005-02-23. Retrieved 2021-07-10.
  31. ↑ Pawel Jakub Dawidek (2002-12-24). "Re: Hmm, sysctlfs" . Retrieved 2021-07-10.
  32. ↑ "Hacking FreeBSD / sysctlfs / [9a7ced]". SourceForge. Retrieved 2021-07-10.
  33. ↑ Terrehon Bowden; Bodo Bauer; Shen Feng. "Chapter 2: Modifying System Parameters - The /proc Filesystem". The Linux Kernel. Retrieved 2021-07-10.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.