Unix File System

Last updated
UFS
Developer(s) CSRG
Full nameUNIX file system
Introducedwith 4.2BSD
Structures
Directory contentstables
Limits
Max volume size273 bytes (8 ZiB)
Max file size273 bytes (8 ZiB)
Max filename length255 bytes
Features
Dates recordedUFS1 and UFS2: last access time (atime), last modified time (mtime), last inode change time (ctime), UFS2: inode creation time (birthtime) [1]
Date rangeUFS1: December 14, 1901–January 18, 2038, UFS2: 64-bit signed integer offset from epoch [1]
Date resolutionUFS1 and UFS2: Nanosecond [1]
Other
Supported
operating systems 		A/UX, DragonFly BSD, FreeBSD, FreeNAS, NAS4Free, HP-UX, NetBSD, NeXTSTEP, Linux, OpenBSD, illumos, Solaris, SunOS, Tru64 UNIX, UNIX System V, Orbis OS, and others

The Unix file system (UFS) is a family of file systems supported by many Unix and Unix-like operating systems. It is a distant descendant of the original filesystem used by Version 7 Unix.

Contents

Design

A UFS volume is composed of the following parts:

Inodes are numbered sequentially, starting at 0. Inode 0 is reserved for unallocated directory entries, inode 1 was the inode of the bad block file in historical UNIX versions, followed by the inode for the root directory, which is always inode 2 and the inode for the lost+found directory which is inode 3.

Directory files contain only the list of filenames in the directory and the inode associated with each file. All file metadata are kept in the inode.

History and evolution

Early Unix filesystems were referred to simply as FS. FS only included the boot block, superblock, a clump of inodes, and the data blocks. This worked well for the small disks early Unixes were designed for, but as technology advanced and disks grew larger, moving the head back and forth between the clump of inodes and the data blocks they referred to caused thrashing. Marshall Kirk McKusick, then a Berkeley graduate student, optimized the V7 FS layout to create BSD 4.2's FFS (Fast File System) by inventing cylinder groups, which break the disk up into smaller chunks, with each group having its own inodes and data blocks. [2] [3]

The intent of BSD FFS is to try to localize associated data blocks and metadata in the same cylinder group and, ideally, all of the contents of a directory (both data and metadata for all the files) in the same or nearby cylinder group, thus reducing fragmentation caused by scattering a directory's contents over a whole disk.

Some of the performance parameters in the superblock included number of tracks and sectors, disk rotation speed, head speed, and alignment of the sectors between tracks. In a fully optimized system, the head could be moved between close tracks to read scattered sectors from alternating tracks while waiting for the platter to spin around.

As disks grew larger and larger, sector-level optimization became obsolete (especially with disks that used linear sector numbering and variable sectors per track). With larger disks and larger files, fragmented reads became more of a problem. To combat this, BSD originally increased the filesystem block size from one sector to 1 K in 4.0 BSD; and, in FFS, increased the filesystem block size from 1 K to 8 K. This has several effects. The chance of a file's sectors being contiguous is much greater. The amount of overhead to list the file's blocks is reduced, while the number of bytes representable by any given number of blocks is increased.

Larger disk sizes are also possible, since the maximum number of blocks is limited by a fixed bit-width block number. However, with larger block sizes, disks with many small files will waste space, since each file must occupy at least one block. Because of this, BSD added block-level fragmentation, also called block suballocation, tail merging, or tail packing, where the last partial block of data from several files may be stored in a single "fragment" block instead of multiple mostly empty blocks. [4]

The work on Berkeley FFS was widely adopted by other Unix vendors, and the family of filesystems derived from it are collectively known as UFS.

Implementations

Vendors of some proprietary Unix systems, such as SunOS / Solaris, System V Release 4, HP-UX, and Tru64 UNIX, and open Unix derived systems like illumos, have adopted UFS. Most of them adapted UFS to their own uses, adding proprietary extensions that may not be recognized by other vendors' versions of Unix. Many[ which? ] have continued to use the original block size and data field widths as the original UFS, so some degree of read compatibility remains across platforms.[ which? ][ citation needed ][ according to whom? ] Compatibility between implementations as a whole is spotty at best.[ according to whom? ]

As of Solaris 7, Sun Microsystems included UFS Logging, which brought filesystem journaling to UFS, which is still available in current versions of Solaris and illumos. [5] Solaris UFS also has extensions for large files and large disks and other features.

In 4.4BSD and BSD Unix systems derived from it, such as FreeBSD, NetBSD, OpenBSD, and DragonFlyBSD, the implementation of UFS1 and UFS2 is split into two layers: an upper layer that provides the directory structure and supports metadata (permissions, ownership, etc.) in the inode structure, and lower layers that provide data containers implemented as inodes. This was done to support both the traditional FFS and the LFS log-structured file system with shared code for common functions. The upper layer is called "UFS", and the lower layers are called "FFS" and "LFS". In some of those systems, the term "FFS" is used for the combination of the FFS lower layer and the UFS upper layer, and the term "LFS" is used for the combination of the LFS lower layer and the UFS upper layer.

Kirk McKusick implemented block reallocation, a technique that reorders the blocks in the file system just before the writes are done to reduce fragmentation and control file system aging. He also implemented soft updates, a mechanism that maintains the file system consistency without limiting the performance in the way the traditional sync mode did. This has the side effect of reducing the requirement of file system checking after a crash or power failure. To overcome the remaining issues after a failure, a background fsck utility was introduced.

In UFS2, Kirk McKusick and Poul-Henning Kamp extended the FreeBSD FFS and UFS layers to add 64-bit block pointers (allowing volumes to grow up to 8 zebibytes), variable-sized blocks (similar to extents), extended flag fields, additional 'birthtime' stamps, extended attribute support and POSIX1.e ACLs. UFS2 became the supported UFS version starting with FreeBSD 5.0. FreeBSD also introduced soft updates and the ability to make file system snapshots for both UFS1 and UFS2. These have since been ported to NetBSD, but eventually soft updates (called soft dependencies in NetBSD) was removed from NetBSD 6.0 in favor of the less complex file system journaling mechanism called WAPBL (also referred as logging), which was added to FFS in NetBSD 5.0. OpenBSD has supported soft updates since version 2.9 [6] and has had UFS2 (FFS2) support (no ACLs) since version 4.2. [7] OpenBSD has now made UFS2 the default UFS version and will be included with the 6.7 release. [8] Since FreeBSD 7.0, UFS also supports filesystem journaling using the gjournal GEOM provider. FreeBSD 9.0 adds support for lightweight journaling on top of soft updates (SU+J), which greatly reduces the need for background fsck, and NFSv4 ACLs.

FreeBSD, NetBSD, OpenBSD, and DragonFly BSD also include the Dirhash system, developed by Ian Dowse. This system maintains an in-memory hash table to speed up directory lookups. Dirhash alleviates a number of performance problems associated with large directories in UFS.

Linux includes a UFS implementation for binary compatibility at the read level with other Unixes, but since there is no standard implementation for the vendor extensions to UFS, Linux does not have full support for writing to UFS. The native Linux ext2 filesystem was inspired by UFS1 but does not support fragments and there are no plans to implement soft updates.[ citation needed ] (In some 4.4BSD-derived systems, the UFS layer can use an ext2 layer as a container layer, just as it can use FFS and LFS.)

NeXTStep, which was BSD-derived, also used a version of UFS. In Apple's Mac OS X, it was available as an alternative to HFS+, their proprietary filesystem. However, as of Mac OS X Leopard, it was no longer possible to install Mac OS X on a UFS-formatted volume. In addition, one cannot upgrade older versions of Mac OS X installed on UFS-formatted volumes to Leopard; upgrading requires reformatting the startup volume. [9] There was a 4 GB file limit for disks formatted as UFS in Mac OS X. As of Mac OS X Lion, UFS support was completely dropped. [10]

See also

Related Research Articles

ext2, or second extended file system, is a file system for the Linux kernel. It was initially designed by French software developer Rémy Card as a replacement for the extended file system (ext). Having been designed according to the same principles as the Berkeley Fast File System from BSD, it was the first commercial-grade filesystem for Linux.

fsck System tool for checking the consistency of a file system

The system utility fsck is a tool for checking the consistency of a file system in Unix and Unix-like operating systems, such as Linux, macOS, and FreeBSD. The equivalent programs on MS-DOS and Microsoft Windows are CHKDSK, SFC, and SCANDISK.

<span class="mw-page-title-main">Virtual file system</span> Abstract layer on top of a more concrete file system

A virtual file system (VFS) or virtual filesystem switch is an abstract layer on top of a more concrete file system. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, classic Mac OS/macOS and Unix filesystems, so that applications can access files on local file systems of those types without having to know what type of file system they are accessing.

tmpfs is a temporary file storage paradigm implemented in many Unix-like operating systems. It is intended to appear as a mounted file system, but data is stored in volatile memory instead of a persistent storage device. A similar construction is a RAM disk, which appears as a virtual disk drive and hosts a disk file system.

A disk quota is a limit set by a system administrator that restricts certain aspects of file system usage on modern operating systems. The function of using disk quotas is to allocate limited disk space in a reasonable way.

stat (system call) Unix system call

stat is a Unix system call that returns file attributes about an inode. The semantics of stat vary between operating systems. As an example, Unix command ls uses this system call to retrieve information on files that includes:

Soft updates is an approach to maintaining file system meta-data integrity in the event of a crash or power outage. Soft updates work by tracking and enforcing dependencies among updates to file system meta-data. Soft updates are an alternative to the more commonly used approach of journaling file systems.

<span class="mw-page-title-main">File system</span> Format or program for storing files and directories

In computing, a file system or filesystem is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified. Taking its name from the way a paper-based data management system is named, each group of data is called a "file". The structure and logic rules used to manage the groups of data and their names is called a "file system."

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

The Log-Structured File System is an implementation of a log-structured file system, originally developed for BSD. It was removed from FreeBSD and OpenBSD; the NetBSD implementation was nonfunctional until work leading up to the 4.0 release made it viable again as a production file system.

NILFS or NILFS2 is a log-structured file system implementation for the Linux kernel. It was developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories and a community from all over the world. NILFS was released under the terms of the GNU General Public License (GPL).

Extended file attributes are file system features that enable users to associate computer files with metadata not interpreted by the filesystem, whereas regular attributes have a purpose strictly defined by the filesystem. Unlike forks, which can usually be as large as the maximum file size, extended attributes are usually limited in size to a value significantly smaller than the maximum file size. Typical uses include storing the author of a document, the character encoding of a plain-text document, or a checksum, cryptographic hash or digital certificate, and discretionary access control information.

The following tables compare general and technical information for a number of file systems.

<span class="mw-page-title-main">Marshall Kirk McKusick</span> American computer scientist (born 1954)

Marshall Kirk McKusick is a computer scientist, known for his extensive work on BSD UNIX, from the 1980s to FreeBSD in the present day. He was president of the USENIX Association from 1990 to 1992 and again from 2002 to 2004, and still serves on the board. He is on the editorial board of ACM Queue Magazine. He is known to friends and colleagues as "Kirk".

chattr is the command in Linux that allows a user to set certain attributes of a file. lsattr is the command that displays the attributes of a file.

The Orlov block allocator is an algorithm to define where a particular file will reside on a given file system (blockwise), so as to speed up disk operations.

Write Ahead Physical Block Logging (WAPBL) provides meta data journaling for file systems in conjunction with Fast File System (FFS) to accomplish rapid filesystem consistency after an unclean shutdown of the filesystem and better general use performance over regular FFS. With the journal, fsck is no longer required at system boot; instead, the system can replay the journal in order to correct any inconsistencies in the filesystem if the system has been shut down in an unclean fashion.

<span class="mw-page-title-main">Unix filesystem</span> Directory structure used by a Unix-like operating system

In Unix and operating systems inspired by it, the file system is considered a central component of the operating system. It was also one of the first parts of the system to be designed and implemented by Ken Thompson in the first experimental version of Unix, dated 1969.

A journaling file system is a file system that keeps track of changes not yet committed to the file system's main part by recording the goal of such changes in a data structure known as a "journal", which is usually a circular log. In the event of a system crash or power failure, such file systems can be brought back online more quickly with a lower likelihood of becoming corrupted.

References

Citations

  1. 1 2 3 "[base] Contents of /Head/Sys/Ufs/Ufs/Dinode.h".
  2. ↑ "Open Sources: Voices from the Open Source Revolution". 29 March 1999.
  3. ↑ McKusick, K M; Joy, W; Leffler, S; Fabry, R (August 1984). "A Fast File System for UNIX" (PDF). ACM Transactions on Computer Systems. 2 (3): 181–197. doi:10.1145/989.990. S2CID   222285164 . Retrieved 2013-04-08.
  4. ↑ Allen, Hervey (2005-06-20). "UFS2 and Soft Updates make for a powerful combination" (PDF). Introduction to FreeBSD, PacNOG I Workshop, Additional Topics. Network Startup Resource Center. p. 23. Retrieved 2013-04-08.
  5. ↑ "UFS Logging". Oracle Documentation. Retrieved 2022-09-27.
  6. ↑ "OpenBSD 2.9 Release". OpenBSD. 2001-06-01. Retrieved 2013-04-08.
  7. ↑ "OpenBSD 4.2 Release". OpenBSD. 2007-11-01. Retrieved 2013-04-08.
  8. ↑ "Make FFS2 the default filesystem". OpenBSD. 2020-04-05. Retrieved 2020-04-07.
  9. ↑ "Archived — Mac OS X 10.5 Leopard: Installing on a UFS-formatted volume". Apple, Inc. 2012-06-12. Retrieved 2013-04-08.
  10. ↑ "Lion won't mount any disk images with the built-in utility or Disk Utility". Apple Support Communities. Apple, Inc. 2011-08-05. Retrieved 2013-12-24.

Bibliography

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.