XFS

Last updated
XFS
Developer(s)
Full nameXFS
Introduced1994;30 years ago (1994) with IRIX  5.3
Partition IDs
  • MBR : 0x83: Linux filesystem
  • GPT : 0FC63DAF-8483-4772-8E79-3D69D8477DE4: Linux filesystem [1]
Structures
Directory contents B+ trees
File allocationB+ trees
Limits
Max volume size8  exbibytes  − 1 byte
Max file size8  exbibytes  − 1 byte
Max no. of files264 [2]
Max filename length255  bytes
Allowed filename
characters		All except NUL and "/"
Features
Dates recordedatime, mtime, ctime, [3] version 5: crtime [4]
Date rangeDecember 13, 1901 – July 2, 2486 [5]
Date resolution1 ns
AttributesYes
File system
permissions 		Yes
Transparent
compression		No
Transparent
encryption 		No (provided at the block device level)
Data deduplication Experimental, Linux only [6]
Other
Supported
operating systems

XFS is a high-performance 64-bit journaling file system created by Silicon Graphics, Inc (SGI) in 1993. [7] It was the default file system in SGI's IRIX operating system starting with its version 5.3. XFS was ported to the Linux kernel in 2001; as of June 2014, XFS is supported by most Linux distributions; Red Hat Enterprise Linux uses it as its default file system.

Contents

XFS excels in the execution of parallel input/output (I/O) operations due to its design, which is based on allocation groups (a type of subdivision of the physical volumes in which XFS is used- also shortened to AGs). Because of this, XFS enables extreme scalability of I/O threads, file system bandwidth, and size of files and of the file system itself when spanning multiple physical storage devices. XFS ensures the consistency of data by employing metadata journaling and supporting write barriers. Space allocation is performed via extents with data structures stored in B+ trees, improving the overall performance of the file system, especially when handling large files. Delayed allocation assists in the prevention of file system fragmentation; online defragmentation is also supported.

History

First generation XFS

Silicon Graphics began development of XFS [8] ("X" was meant to be filled in later but never was) in 1993 for its UNIX System V based IRIX operating system. The file system was released under the GNU General Public License (GPL) in May 1999. [9]

Second generation XFS

A team led by Steve Lord at SGI ported XFS to Linux, [10] and first support by a Linux distribution came in 2001. This support gradually became available in almost all Linux distributions.[ citation needed ]

Initial support for XFS in the Linux kernel came through patches from SGI. It merged into the Linux kernel mainline for the 2.6 series, and separately merged in February 2004 into the 2.4 series in version 2.4.25, [11] making XFS almost universally available on Linux systems. [12] Gentoo Linux became the first Linux distribution to introduce an option for XFS as the default filesystem in mid-2002. [13]

FreeBSD added read-only support for XFS in December 2005, and in June 2006 introduced experimental write support. However, this was intended only as an aid in migration from Linux, not as a "main" file system. FreeBSD 10 removed support for XFS. [14]

In 2009, version 5.4 of 64-bit Red Hat Enterprise Linux (RHEL) Linux distribution contained the necessary kernel support for the creation and usage of XFS file systems, but lacked the corresponding command-line tools. The tools available from CentOS could operate for that purpose, and Red Hat also provided them to RHEL customers on request. [15] RHEL 6.0, released in 2010, includes XFS support for a fee as part of Red Hat's "scalable file system add-on". [16] Oracle Linux 6, released in 2011, also includes an option for using XFS. [17]

RHEL 7.0, released in June 2014, uses XFS as its default file system, [18] including support for using XFS for the /boot partition, which previously was not practical due to bugs in the GRUB bootloader. [19]

Linux kernel 4.8 in August 2016 added a new feature, "reverse mapping". This is the foundation for a large set of planned features: snapshots, copy-on-write (COW) data, data deduplication, reflink copies, online data and metadata scrubbing, highly accurate reporting of data loss or bad sectors, and significantly improved reconstruction of damaged or corrupted filesystems. This work required changes to XFS's on-disk format. [20] [21]

Third generation XFS

XFS v4 is deprecated. Partitions have to be reformatted with XFS v5. XFS v4 Linux Kernel Option.jpg
XFS v4 is deprecated. Partitions have to be reformatted with XFS v5.

Linux kernel 5.10, released in December 2020, included the new on-disk format XFS v5. This was a hard break, since the deprecated XFS v4 can not be converted to XFS v5. Data on partitions formatted with XFS v4 has to be backed up to another partition or media in order to restore it after formatting the old partition with XFS v5, which completely wipes all data on it. The support for XFS v4 will be removed from the Linux kernel in September 2030. [22]

XFS v5 introduced "bigtime", to store inode timestamps as a 64-bit nanosecond counter instead of the traditional 32-bit seconds counter. This postpones the previous Year 2038 problem until the year 2486. [5] It also introduced metadata checksums.

The Gentoo Handbook, Gentoo Linux's official installation manual, has recommended XFS as the "all-purpose all-platform filesystem" since 28 Jun 2023, succeeding Ext4. [23]

Features

Capacity

XFS is a 64-bit file system [24] and supports a maximum file system size of 8 exbibytes minus one byte (263 − 1 bytes), but limitations imposed by the host operating system can decrease this limit. 32-bit Linux systems limit the size of both the file and file system to 16 tebibytes.

Journaling

In modern computing, journaling is a capability which ensures consistency of data in the file system, despite any power outages or system crash that may occur. XFS provides journaling for file system metadata, where file system updates are first written to a serial journal before the actual disk blocks are updated. The journal is a circular buffer of disk blocks that is not read in normal file system operation.

The XFS journal can be stored within the data section of the file system (as an internal log), or on a separate device to minimize disk contention.

In XFS, the journal primarily contains entries that describe the portions of the disk blocks changed by filesystem operations. Journal updates are performed asynchronously to avoid a decrease in performance speed.

In the event of a system crash, file system operations which occurred immediately prior to the crash can be reapplied and completed as recorded in the journal, which is how data stored in XFS file systems remain consistent. Recovery is performed automatically the first time the file system is mounted after the crash. The speed of recovery is independent of the size of the file system, instead depending on the amount of file system operations to be reapplied.

Allocation groups

XFS file systems are internally partitioned into allocation groups, which are equally sized linear regions within the file system. Files and directories can span allocation groups. Each allocation group manages its own inodes and free space separately, providing scalability and parallelism so multiple threads and processes can perform I/O operations on the same file system simultaneously.

This architecture helps to optimize parallel I/O performance on systems with multiple processors and/or cores, as metadata updates can also be parallelized. The internal partitioning provided by allocation groups can be especially beneficial when the file system spans multiple physical devices, allowing optimal usage of throughput of the underlying storage components.

Striped allocation

If an XFS file system is to be created on a striped RAID array, a stripe unit can be specified when the file system is created. This maximizes throughput by ensuring that data allocations, inode allocations and the internal log (the journal) are aligned with the stripe unit.

Extent-based allocation

Blocks used in files stored on XFS file systems are managed with variable length extents where one extent describes one or more contiguous blocks. This can shorten the list of blocks considerably, compared to file systems that list all blocks used by a file individually.

Block-oriented file systems manage space allocation with one or more block-oriented bitmaps; in XFS, these structures are replaced with an extent oriented structure consisting of a pair of B+ trees for each file system allocation group. One of the B+ trees is indexed by the length of the free extents, while the other is indexed by the starting block of the free extents. This dual indexing scheme allows for the highly efficient allocation of free extents for file system operations.

Variable block sizes

The file system block size represents the minimum allocation unit. XFS allows file systems to be created with block sizes ranging between 512 bytes and 64 KB, allowing the file system to be tuned for the expected degree of usage. When many small files are expected, a small block size would typically maximize capacity, but for a system dealing mainly with large files, a larger block size can provide a performance efficiency advantage.

Delayed allocation

XFS makes use of lazy evaluation techniques for file allocation. When a file is written to the buffer cache, rather than allocating extents for the data, XFS simply reserves the appropriate number of file system blocks for the data held in memory. The actual block allocation occurs only when the data is finally flushed to disk. This improves the chance that the file will be written in a contiguous group of blocks, reducing fragmentation problems and increasing performance.

Sparse files

XFS provides a 64-bit sparse address space for each file, which allows both for very large file sizes, and for "holes" within files in which no disk space is allocated. As the file system uses an extent map for each file, the file allocation map size is kept small. Where the size of the allocation map is too large for it to be stored within the inode, the map is moved into a B+ tree which allows for rapid access to data anywhere in the 64-bit address space provided for the file.

Extended attributes

XFS provides multiple data streams for files; this is made possible by its implementation of extended attributes. These allow the storage of a number of name/value pairs attached to a file. Names are nul-terminated printable character strings which are up to 256 bytes in length, while their associated values can contain up to 64  KB of binary data.

They are further subdivided into two namespaces: root and user. Extended attributes stored in the root namespace can be modified only by the superuser, while attributes in the user namespace can be modified by any user with permission to write to the file.

Extended attributes can be attached to any kind of XFS inode, including symbolic links, device nodes, directories, etc. The attr utility can be used to manipulate extended attributes from the command line, and the xfsdump and xfsrestore utilities are aware of extended attributes, and will back up and restore their contents. Many other backup systems do not support working with extended attributes.

Direct I/O

For applications requiring high throughput to disk, XFS provides a direct I/O implementation that allows non-cached I/O operations to be applied directly to the userspace. Data is transferred between the buffer of the application and the disk using DMA, which allows access to the full I/O bandwidth of the underlying disk devices.

Snapshots

XFS does not yet [25] provide direct support for snapshots, as it currently expects the snapshot process to be implemented by the volume manager. Taking a snapshot of an XFS filesystem involves temporarily halting I/O to the filesystem using the xfs_freeze utility, having the volume manager perform the actual snapshot, and then resuming I/O to continue with normal operations. The snapshot can then be mounted read-only for backup purposes.

Releases of XFS in IRIX incorporated an integrated volume manager called XLV. This volume manager has not been ported to Linux, and XFS works with standard LVM in Linux systems instead.

In recent Linux kernels, the xfs_freeze functionality is implemented in the VFS layer, and is executed automatically when the Volume Manager's snapshot functionality is invoked. This was once a valuable advantage as the ext3 file system could not be suspended [26] and the volume manager was unable to create a consistent "hot" snapshot to back up a heavily busy database. [27] Fortunately this is no longer the case. Since Linux 2.6.29, the file systems ext3, ext4, GFS2 and JFS have the freeze feature as well. [28]

Online defragmentation

Although the extent-based nature of XFS and the delayed allocation strategy it uses significantly improves the file system's resistance to fragmentation problems, XFS provides a filesystem defragmentation utility (xfs_fsr, short for XFS filesystem reorganizer) that can defragment the files on a mounted and active XFS filesystem. [29]

Online growth

XFS provides the xfs_growfs utility to perform online expansion of XFS file systems. XFS filesystems can be grown so long as there is remaining unallocated space on the device holding the filesystem. This feature is typically used in conjunction with volume management, as otherwise the partition holding the filesystem will need enlarging separately.

IRIX-only features

DMAPI

XFS implemented the DMAPI interface to support Hierarchical Storage Management in IRIX. As of October 2010, the Linux implementation of XFS supported the required on-disk metadata for DMAPI implementation, but the kernel support was reportedly not usable. For some time, SGI hosted a kernel tree which included the DMAPI hooks, but this support has not been adequately maintained, although kernel developers have stated an intention to bring this support up to date. [30]

Guaranteed-rate I/O

The XFS guaranteed-rate I/O system provides an API that allows applications to reserve bandwidth to the filesystem. XFS dynamically calculates the performance available from the underlying storage devices, and will reserve bandwidth sufficient to meet the requested performance for a specified time. This is a feature unique to the XFS file system. Guaranteed rates can be "hard" or "soft", representing a trade off between reliability and performance; however, XFS will only allow "hard" guarantees if the underlying storage subsystem supports it. This facility is used mostly for real-time applications, such as video streaming.

Guaranteed-rate I/O was only supported under IRIX, and required special hardware for that purpose. [31]

Disadvantages

Historic Disadvantages

See also

Related Research Articles

<span class="mw-page-title-main">IRIX</span> Computer operating system

IRIX is a discontinued operating system developed by Silicon Graphics (SGI) to run on the company's proprietary MIPS workstations and servers. It is based on UNIX System V with BSD extensions. In IRIX, SGI originated the XFS file system and the industry-standard OpenGL graphics API.

ReiserFS is a general-purpose, journaling file system initially designed and implemented by a team at Namesys led by Hans Reiser and licensed under GPLv2. Introduced in version 2.4.1 of the Linux kernel, it was the first journaling file system to be included in the standard kernel. ReiserFS was the default file system in Novell's SUSE Linux Enterprise until Novell decided to move to ext3 for future releases on October 12, 2006.

ext2, or second extended file system, is a file system for the Linux kernel. It was initially designed by French software developer Rémy Card as a replacement for the extended file system (ext). Having been designed according to the same principles as the Berkeley Fast File System from BSD, it was the first commercial-grade filesystem for Linux.

ext3, or third extended filesystem, is a journaled file system that is commonly used with the Linux kernel. It used to be the default file system for many popular Linux distributions but generally has been supplanted by its successor version ext4. The main advantage of ext3 over its predecessor, ext2, is journaling, which improves reliability and eliminates the need to check the file system after an improper, a.k.a. unclean, shutdown.

Journaled File System (JFS) is a 64-bit journaling file system created by IBM. There are versions for AIX, OS/2, eComStation, ArcaOS and Linux operating systems. The latter is available as free software under the terms of the GNU General Public License (GPL). HP-UX has another, different filesystem named JFS that is actually an OEM version of Veritas Software's VxFS.

The Unix file system (UFS) is a family of file systems supported by many Unix and Unix-like operating systems. It is a distant descendant of the original filesystem used by Version 7 Unix.

Reiser4 is a computer file system, successor to the ReiserFS file system, developed from scratch by Namesys and sponsored by DARPA as well as Linspire. Reiser4 was named after its former lead developer Hans Reiser. As of 2021, the Reiser4 patch set is still being maintained, but according to Phoronix, it is unlikely to be merged into mainline Linux without corporate backing.

The inode is a data structure in a Unix-style file system that describes a file-system object such as a file or a directory. Each inode stores the attributes and disk block locations of the object's data. File-system object attributes may include metadata, as well as owner and permission data.

In computing, the Global File System 2 or GFS2 is a shared-disk file system for Linux computer clusters. GFS2 allows all members of a cluster to have direct concurrent access to the same shared block storage, in contrast to distributed file systems which distribute data throughout the cluster. GFS2 can also be used as a local file system on a single computer.

HFS Plus or HFS+ is a journaling file system developed by Apple Inc. It replaced the Hierarchical File System (HFS) as the primary file system of Apple computers with the 1998 release of Mac OS 8.1. HFS+ continued as the primary Mac OS X file system until it was itself replaced with the Apple File System (APFS), released with macOS High Sierra in 2017. HFS+ is also one of the formats supported by the iPod digital music player.

In computing, an extent is a contiguous area of storage reserved for a file in a file system, represented as a range of block numbers, or tracks on count key data devices. A file can consist of zero or more extents; one file fragment requires one extent. The direct benefit is in storing each range compactly as two numbers, instead of canonically storing every block number in the range. Also, extent allocation results in less file fragmentation.

In Linux, Logical Volume Manager (LVM) is a device mapper framework that provides logical volume management for the Linux kernel. Most modern Linux distributions are LVM-aware to the point of being able to have their root file systems on a logical volume.

NILFS or NILFS2 is a log-structured file system implementation for the Linux kernel. It was developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories and a community from all over the world. NILFS was released under the terms of the GNU General Public License (GPL).

The following tables compare general and technical information for a number of file systems.

ext4 is a journaling file system for Linux, developed as the successor to ext3.

Btrfs is a computer storage format that combines a file system based on the copy-on-write (COW) principle with a logical volume manager, developed together. It was created by Chris Mason in 2007 for use in Linux, and since November 2013, the file system's on-disk format has been declared stable in the Linux kernel.

HAMMER is a high-availability 64-bit file system developed by Matthew Dillon for DragonFly BSD using B+ trees. Its major features include infinite NFS-exportable snapshots, master–multislave operation, configurable history retention, fsckless-mount, and checksums to deal with data corruption. HAMMER also supports data block deduplication, meaning that identical data blocks will be stored only once on a file system. A successor, HAMMER2, was announced in 2011 and became the default in Dragonfly 5.2.

Next3 is a journaling file system for Linux based on ext3 which adds snapshots support, yet retains compatibility to the ext3 on-disk format. Next3 is implemented as open-source software, licensed under the GPL license.

Apple File System (APFS) is a proprietary file system developed and deployed by Apple Inc. for macOS Sierra (10.12.4) and later, iOS 10.3, tvOS 10.2, watchOS 3.2, and all versions of iPadOS. It aims to fix core problems of HFS+, APFS's predecessor on these operating systems. APFS is optimized for solid-state drive storage and supports encryption, snapshots, and increased data integrity, among other capabilities.

References

  1. ↑ "GPT fdisk - ArchWiki".
  2. ↑ "What is the maximum number of inodes in Linux filesystems?". 2014-06-17.
  3. ↑ "XFS Filesystem Structure 2nd Edition, Revision 1" (PDF). p. 25. Archived from the original (PDF) on 2017-10-31.
  4. ↑ "ondisk_inode.asciidoc\XFS_Filesystem_Structure\design - xfs/xfs-documentation.git - XFS AsciiDoc Documentation tree". git.kernel.org.
  5. 1 2 Darrick J. Wong (2020-08-10). "xfs: widen timestamps to deal with y2038".
  6. ↑ "Duperemove". GitHub . Archived from the original on 6 March 2016. Retrieved 21 August 2016.
  7. ↑ "xFS: The Extension of EFS - "x" for To-be-Determined (but the Name Stuck)". XFS.org. Archived from the original on 2014-07-14.
  8. ↑ Smith, Roderick W. (2007). Linux Administrator Street Smarts: A Real World Guide to Linux Certification Skills. Street smarts series. John Wiley & Sons. p. 204. ISBN   9780470116746. Archived from the original on 2016-08-21. Retrieved 2016-03-21. Silicon Graphics (SGI) created its Extents File System (XFS) for its IRIX OS and [...] later donated the code to Linux.
  9. ↑ "SGI open-sourcing XFS". slashdot.org . 1999-05-19. Retrieved 2023-04-12.
  10. ↑ "Porting XFS to Linux". Olstrans.SourceForge.net. 2000-07-21. Archived from the original on 2013-02-25. Retrieved 2013-04-29.
  11. ↑ "Linux kernel 2.4.25 changelog". kernel.org . 2004-02-18. Archived from the original on 2014-08-19. Retrieved 2014-08-14.
  12. ↑ Daniel Robbins (January 1, 2002). "Common threads: Advanced filesystem implementor's guide, Part 9, Introducing XFS". Developer Works. IBM. Archived from the original on September 4, 2015. Retrieved November 6, 2011.
  13. ↑ Daniel Robbins (April 1, 2002). "Common threads: Advanced filesystem implementor's guide, Part 10, Deploying XFS". Developer Works. IBM. Archived from the original on December 24, 2011. Retrieved November 6, 2011.
  14. ↑ "Has FreeBSD 10 Dropped Support For XFS?". Lists.freebsd.org. 2013-10-27. Archived from the original on 2014-03-30. Retrieved 2014-03-30.
  15. ↑ "Bug 521173 -xfsprogs is missing in RHEL-5.4". RedHat.com. May 24, 2010. Archived from the original on July 10, 2012. Retrieved November 6, 2011.
  16. ↑ "Red Hat Enterprise Linux Scalable File System Add-On". RedHat.com. Archived from the original on 2014-05-29. Retrieved 2014-05-22.
  17. ↑ "Oracle Linux 6 Release Notes". Oracle Corporation. February 2011. Archived from the original on 2012-03-28. Retrieved 2013-04-07. Oracle Linux 6 includes many new features, including [...] XFS [:] Oracle Linux 6 includes XFS as an optional filesystem.
  18. ↑ "Red Hat Unveils Red Hat Enterprise Linux 7, Redefining the Enterprise Operating System". Red Hat. 2014-06-10. Archived from the original on 2014-06-13. Retrieved 2014-06-10.
  19. ↑ "Bug 250843 -grub-install hangs on xfs". Redhat.com. May 4, 2009. Archived from the original on July 10, 2012. Retrieved November 6, 2011.
  20. ↑ "kernel/git/torvalds/linux.git - Linux kernel source tree". git.kernel.org.
  21. ↑ "Linux_4.8 - Linux Kernel Newbies". Archived from the original on 2018-10-19. Retrieved 2018-10-19.
  22. ↑ https://www.haiku-os.org/docs/develop/file_systems/xfs.html
  23. ↑ "Difference between revisions of "Handbook:Parts/Installation/Disks" - Gentoo wiki". wiki.gentoo.org. Retrieved 2024-04-25.
  24. ↑ "XFS Overview". Silicon Graphics International Corp. 2013-07-02. Archived from the original on 2013-06-06. Retrieved 2013-07-02.
  25. ↑ "XFS: There and back ... and there again? [LWN.net]". lwn.net. Archived from the original on 2016-10-27. Retrieved 2016-10-27.
  26. ↑ "How to freeze ext3 file system". www.linuxquestions.org. Archived from the original on 2011-04-28. Retrieved 2011-08-24.
  27. ↑ "LVM snapshots: How to use?". www.linuxquestions.org. Archived from the original on 2011-01-17. Retrieved 2010-04-17.
  28. ↑ "kernel/git/torvalds/linux.git - Linux kernel source tree". git.kernel.org.
  29. ↑ Bitubique.com Archived April 1, 2009, at the Wayback Machine
  30. ↑ Christoph Hellwig (October 3, 2010). "Re: Linux and DMAPI". XFS mailing list (Mailing list). SGI. Archived from the original on September 27, 2011. Retrieved November 6, 2011.
  31. ↑ John Nelson (July 30, 2012). "Re: Re: realtime section bugs still around". XFS mailing list (Mailing list). SGI. Archived from the original on April 14, 2014. Retrieved April 13, 2014.
  32. ↑ XFS.org Archived 2009-01-05 at the Wayback Machine , FAQ
  33. ↑ Eric Sandeen (Jan 18, 2002). "Re: Shrink an XFS filesystem? (LVM)". XFS mailing list (Mailing list). SGI. Archived from the original on 2016-02-03.
  34. ↑ "About External XFS Journals". oracle.com. Retrieved 16 November 2022.
  35. ↑ Nelson, John (December 23, 2010). "Improving Metadata Performance By Reducing Journal Overhead". XFS.org wiki. Archived from the original on October 6, 2011. Retrieved November 6, 2011.
  36. ↑ Nelson, John (May 24, 2010). "Re: PATCH 0/12 xfs: delayed logging V6". XFS Mailing List Message (Mailing list). Archived from the original on December 5, 2011. Retrieved November 6, 2011.

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.