Extended file attributes

Last updated

Extended file attributes are file system features that enable users to associate computer files with metadata not interpreted by the filesystem, whereas regular attributes have a purpose strictly defined by the filesystem (such as permissions or records of creation and modification times). Unlike forks, which can usually be as large as the maximum file size, extended attributes are usually limited in size to a value significantly smaller than the maximum file size. Typical uses include storing the author of a document, the character encoding of a plain-text document, or a checksum, cryptographic hash or digital certificate, and discretionary access control information.

Contents

In Unix-like systems, extended attributes are usually abbreviated as xattr. [1]

Implementations

AIX

In AIX, the JFS2 v2 filesystem supports extended attributes, which are accessible using the getea command. [2] The getea, [3] setea, [4] listea, [5] statea, [6] and removeea [7] APIs support fetching, setting, listing, getting information about, and removing extended attributes.

BeOS/Haiku

In the now-defunct BeOS and successors like Haiku, extended file attributes are widely used in base and third-party programs. The Be File System allows the indexing and querying of attributes, essentially giving the filesystem database-like characteristics. The uses of extended attributes in Be-like systems are varied: For example, Tracker and OpenTracker, the file-managers of BeOS and Haiku respectively, both store the locations of file icons in attributes, [8] Haiku's "Mail" service stores all message content and metadata in extended file attributes, [9] and the MIME types of files are stored in their attributes. Extended file attributes can be viewed and edited in Be-like systems' GUI through the file-manager, often Tracker or derivatives thereof.

FreeBSD

In FreeBSD 5.0 and later, the UFS1, UFS2, and ZFS filesystems support extended attributes, using the extattr_ [10] family of system calls. Any regular file may have a list of extended attributes. Each attribute consists of a name and the associated data. The name must be a null-terminated string, and exists in a namespace identified by a small-integer namespace identifier. Currently, two namespaces exist: user and system. The user namespace has no restrictions with regard to naming or contents. The system namespace is primarily used by the kernel for access control lists and mandatory access control.

Linux

In Linux, the ext2, ext3, ext4, JFS, Squashfs, UBIFS, Yaffs2, ReiserFS, Reiser4, XFS, Btrfs, OrangeFS, Lustre, OCFS2 1.6, ZFS, and F2FS [11] filesystems support extended attributes (abbreviated xattr) when enabled in the kernel configuration. Any regular file or directory may have extended attributes consisting of a name and associated data. The name must be a null-terminated string prefixed by a namespace identifier and a dot character. Currently, four namespaces exist: user, trusted, security and system. The user namespace has no restrictions with regard to naming or contents. The system namespace is primarily used by the kernel for access control lists. The security namespace is used by SELinux, for example.

Support for the extended attribute concept from a POSIX.1e draft[ citation needed ] that had been withdrawn [12] in 1997 was added to Linux around 2002. [13] [14] As of 2016, they are not yet in widespread use by user-space Linux programs, but are used by Beagle, OpenStack Swift, Dropbox, KDE's semantic metadata framework (Baloo), Chromium, Wget and cURL.

The Linux kernel allows extended attribute to have names of up to 255 bytes and values of up to 64 KiB, [15] as do XFS and ReiserFS, but ext2/3/4 and btrfs impose much smaller limits, requiring all the attributes (names and values) of one file to fit in one "filesystem block" (usually 4 KiB). Per POSIX.1e,[ citation needed ] the names are required to start with one of security, system, trusted, and user plus a period. This defines the four namespaces of extended attributes. [16]

Extended attributes can be accessed and modified using the getfattr and setfattr commands from the attr package on most distributions. [17] The APIs are called getxattr and setxattr.

macOS

Mac OS X 10.4 and later support extended attributes by making use of the HFS+ filesystem Attributes File B*-tree feature which allows for named forks. Although the named forks in HFS+ support arbitrarily large amounts of data through extents, the OS support for extended attributes only supports inline attributes, limiting their size to that which can fit within a single B*-tree node.[ citation needed ] Any regular file may have a list of extended attributes. HFS+ supports an arbitrary number of named forks, and it is unknown if macOS imposes any limit on the number of extended attributes.

Each attribute consists of a name and the associated data. The name is a null-terminated Unicode string. No namespace restrictions are present (making this an open xattr system) and the convention is to use a reverse DNS string (similar to Uniform Type Identifiers) as the attribute name.

macOS supports listing, [18] getting, [19] setting, [20] and removing [21] extended attributes from files or directories using a Linux-like API. From the command line, these abilities are exposed through the xattr utility. [22]

Since macOS 10.5, files originating from the web are marked with com.apple.quarantine via extended file attributes. [23] In some older versions of macOS (such as Mac OS X 10.6), user space extended attributes were not preserved on save in common Cocoa applications (TextEdit, Preview etc.).[ citation needed ]

OpenBSD

Support for extended file attributes was removed from the OpenBSD source code in 2005 due to a lack of interest in Access Control Lists. [24]

OS/2

In OS/2 version 1.2 and later, the High Performance File System was designed with extended attributes in mind, but support for them was also retro-fitted on the FAT filesystem of DOS. For compatibility with other operating systems using a FAT partition, OS/2 attributes are stored inside a single file "EA DATA. SF" located in the root directory. This file is normally inaccessible when an operating system supporting extended attributes manages the disk, but can be freely manipulated under, for example, DOS. Files and directories having extended attributes use one or more clusters inside this file. The logical cluster number of the first used cluster is stored inside the owning file's or directory's directory entry. [25] These two bytes are used for other purposes on the FAT32 filesystem, and hence OS/2 extended attributes cannot be stored on this filesystem.

Parts of OS/2 version 2.0 and later such as the Workplace Shell uses several standardized extended attributes (also called EAs) for purposes like identifying the filetype, comments, computer icons and keywords about the file. Programs written in the interpreted language Rexx store an already parsed version of the code as an extended attribute, to allow faster execution.

Solaris

Solaris version 9 and later allows files to have "extended attributes", which are actually forks; the maximum size of an "extended attribute" is the same as the maximum size of a file, and they are read and written in the same fashion as files. Internally, they are actually stored and accessed like normal files, so their names cannot contain "/" characters [26] and their ownership and permissions can differ from those of the parent file.

Version 4 of the Network File System supports extended attributes in much the same way as Solaris.

Windows NT

On Windows NT, limited-length extended attributes are supported by FAT, [25] HPFS, and NTFS. This was implemented as part of the OS/2 subsystem. They are notably used by the NFS server of the Interix POSIX subsystem in order to implement Unix-like permissions. The Windows Subsystem for Linux added in the Windows 10 Anniversary Update uses them for similar purposes, storing the Linux file mode, owner, device ID (if applicable), and file times in the extended attributes. [27]

Additionally, NTFS can store arbitrary-length extended attributes in the form of alternate data streams (ADS), a type of resource fork. Plugins for the file manager Total Commander, like NTFS Descriptions and QuickSearch eXtended support filtering the file list by or searching for metadata contained in ADS. [28] [29] NTFS-3G supports mapping ADS to extended attributes in FUSE; it also maps file attributes that way. [30]

See also

Related Research Articles

The Single UNIX Specification (SUS) is a standard for computer operating systems, compliance with which is required to qualify for using the "UNIX" trademark. The standard specifies programming interfaces for the C language, a command-line shell, and user commands. The core specifications of the SUS known as Base Specifications are developed and maintained by the Austin Group, which is a joint working group of IEEE, ISO/IEC JTC 1/SC 22/WG 15 and The Open Group. If an operating system is submitted to The Open Group for certification, and passes conformance tests, then it is deemed to be compliant with a UNIX standard such as UNIX 98 or UNIX 03.

New Technology File System (NTFS) is a proprietary journaling file system developed by Microsoft. Starting with Windows NT 3.1, it is the default file system of the Windows NT family. It superseded File Allocation Table (FAT) as the preferred filesystem on Windows and is supported in Linux and BSD as well. NTFS reading and writing support is provided using a free and open-source kernel implementation known as NTFS3 in Linux and the NTFS-3G driver in BSD. By using the convert command, Windows can convert FAT32/16/12 into NTFS without the need to rewrite all files. NTFS uses several files typically hidden from the user to store metadata about other files stored on the drive which can help improve speed and performance when reading data. Unlike FAT and High Performance File System (HPFS), NTFS supports access control lists (ACLs), filesystem encryption, transparent compression, sparse files and file system journaling. NTFS also supports shadow copy to allow backups of a system while it is running, but the functionality of the shadow copies varies between different versions of Windows.

Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems (Sun) in 1984, allowing a user on a client computer to access files over a computer network much like local storage is accessed. NFS, like many other protocols, builds on the Open Network Computing Remote Procedure Call system. NFS is an open IETF standard defined in a Request for Comments (RFC), allowing anyone to implement the protocol.

<span class="mw-page-title-main">Disk partitioning</span> Creation of separate accessible storage areas on a secondary computer storage device

Disk partitioning or disk slicing is the creation of one or more regions on secondary storage, so that each region can be managed separately. These regions are called partitions. It is typically the first step of preparing a newly installed disk, before any file system is created. The disk stores the information about the partitions' locations and sizes in an area known as the partition table that the operating system reads before any other part of the disk. Each partition then appears to the operating system as a distinct "logical" disk that uses part of the actual disk. System administrators use a program called a partition editor to create, resize, delete, and manipulate the partitions. Partitioning allows the use of different filesystems to be installed for different kinds of files. Separating user data from system data can prevent the system partition from becoming full and rendering the system unusable. Partitioning can also make backing up easier. A disadvantage is that it can be difficult to properly size partitions, resulting in having one partition with too much free space and another nearly totally allocated.

In computer security, an access-control list (ACL) is a list of permissions associated with a system resource. An ACL specifies which users or system processes are granted access to resources, as well as what operations are allowed on given resources. Each entry in a typical ACL specifies a subject and an operation. For instance,

The resource fork is a fork or section of a file on Apple's classic Mac OS operating system, which was also carried over to the modern macOS for compatibility, used to store structured data along with the unstructured data stored within the data fork.

In computing, tar is a computer software utility for collecting many files into one archive file, often referred to as a tarball, for distribution or backup purposes. The name is derived from "tape archive", as it was originally developed to write data to sequential I/O devices with no file system of their own, such as devices that use magnetic tape. The archive data sets created by tar contain various file system parameters, such as name, timestamps, ownership, file-access permissions, and directory organization. POSIX abandoned tar in favor of pax, yet tar sees continued widespread use.

In computing, a symbolic link is a file whose purpose is to point to a file or directory by specifying a path thereto.

<span class="mw-page-title-main">Filename</span> Text string used to uniquely identify a computer file

A filename or file name is a name used to uniquely identify a computer file in a file system. Different file systems impose different restrictions on filename lengths.

HFS Plus or HFS+ is a journaling file system developed by Apple Inc. It replaced the Hierarchical File System (HFS) as the primary file system of Apple computers with the 1998 release of Mac OS 8.1. HFS+ continued as the primary Mac OS X file system until it was itself replaced with the Apple File System (APFS), released with macOS High Sierra in 2017. HFS+ is also one of the formats supported by the iPod digital music player.

<span class="mw-page-title-main">File system</span> Format or program for storing files and directories

In computing, a file system or filesystem is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified. Taking its name from the way a paper-based data management system is named, each group of data is called a "file". The structure and logic rules used to manage the groups of data and their names is called a "file system."

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

Most file systems include attributes of files and directories that control the ability of users to read, change, navigate, and execute the contents of the file system. In some cases, menu options or functions may be made visible or hidden depending on a user's permission level; this kind of user interface is referred to as permission-driven.

File locking is a mechanism that restricts access to a computer file, or to a region of a file, by allowing only one user or process to modify or delete it at a specific time and to prevent reading of the file while it's being modified or deleted.

In computing, an extent is a contiguous area of storage reserved for a file in a file system, represented as a range of block numbers, or tracks on count key data devices. A file can consist of zero or more extents; one file fragment requires one extent. The direct benefit is in storing each range compactly as two numbers, instead of canonically storing every block number in the range. Also, extent allocation results in less file fragmentation.

File attributes are a type of meta-data that describe and may modify how files and/or directories in a filesystem behave. Typical file attributes may, for example, indicate or specify whether a file is visible, modifiable, compressed, or encrypted. The availability of most file attributes depends on support by the underlying filesystem where attribute data must be stored along with other control structures. Each attribute can have one of two states: set and cleared. Attributes are considered distinct from other metadata, such as dates and times, filename extensions or file system permissions. In addition to files, folders, volumes and other file system objects may have attributes.

GPFS is high-performance clustered file system software developed by IBM. It can be deployed in shared-disk or shared-nothing distributed parallel modes, or a combination of these. It is used by many of the world's largest commercial companies, as well as some of the supercomputers on the Top 500 List. For example, it is the filesystem of the Summit at Oak Ridge National Laboratory which was the #1 fastest supercomputer in the world in the November 2019 TOP500 list of supercomputers. Summit is a 200 Petaflops system composed of more than 9,000 POWER9 processors and 27,000 NVIDIA Volta GPUs. The storage filesystem called Alpine has 250 PB of storage using Spectrum Scale on IBM ESS storage hardware, capable of approximately 2.5TB/s of sequential I/O and 2.2TB/s of random I/O.

The following tables compare general and technical information for a number of file systems.

ptrace is a system call found in Unix and several Unix-like operating systems. By using ptrace one process can control another, enabling the controller to inspect and manipulate the internal state of its target. ptrace is used by debuggers and other code-analysis tools, mostly as aids to software development.

References

  1. ↑ xattr(7)  –  Linux Programmer's Manual – Overview, Conventions and Miscellanea
  2. ↑ "getea Command". IBM AIX V7.2 documentation. IBM . Retrieved 2017-07-11.
  3. ↑ "getea Subroutine". IBM AIX V7.2 documentation: Base Operating System and Extensions, Volume 1. IBM. Retrieved 2017-07-11.
  4. ↑ "setea Subroutine". IBM AIX V7.2 documentation: Base Operating System and Extensions, Volume 2. IBM. Retrieved 2017-07-11.
  5. ↑ "listea Subroutine". IBM AIX V7.2 documentation: Base Operating System and Extensions, Volume 1. IBM. Retrieved 2017-07-11.
  6. ↑ "statea Subroutine". IBM AIX V7.2 documentation: Base Operating System and Extensions, Volume 2. IBM. Retrieved 2017-07-11.
  7. ↑ "removeea Subroutine". IBM AIX V7.2 documentation: Base Operating System and Extensions, Volume 2. IBM. Retrieved 2017-07-11.
  8. ↑ "Attributes". Haiku User Guide. Haiku, Inc. Retrieved 28 January 2021.
  9. ↑ "Workshop: Managing Emails". Haiku User Guide. Haiku, Inc. Retrieved 28 January 2021.
  10. ↑ extattr(2)  –  FreeBSD System Calls Manual
  11. ↑ Kim, Jaegeuk (2012-10-05). "[PATCH 13/16] f2fs: add xattr and acl functionalities" . Retrieved 2017-07-11.
  12. ↑ Schaufler, Casey (2002-04-30). "Posix.1e?". Newsgroup:  comp.security.unix.
  13. ↑ "v2.5.3 git commit log". 2002-02-05. Retrieved 2017-07-11.
  14. ↑ "Red Hat Enterprise Linux AS 3 Release Notes (x86 Edition)". Red Hat. 2003. Archived from the original on 2017-07-09. Retrieved 2017-07-11. EA (Extended Attributes) and ACL (Access Control Lists) functionality is now available for ext3 file systems. In addition, ACL functionality is available for NFS.
  15. ↑ "linux/include/uapi/linux/limits.h". 2012-10-13. Retrieved 2017-07-11.
  16. ↑ xattr(7)  –  Linux Programmer's Manual – Overview, Conventions and Miscellanea
  17. ↑ "Commands for Manipulating Filesystem Extended Attributes". GNU Savannah . Retrieved 2017-07-11.
  18. ↑ listxattr(2)  –  Darwin and macOS System Calls Manual
  19. ↑ getxattr(2)  –  Darwin and macOS System Calls Manual
  20. ↑ setxattr(2)  –  Darwin and macOS System Calls Manual
  21. ↑ removexattr(2)  –  Darwin and macOS System Calls Manual
  22. ↑ xattr(1)  –  Darwin and macOS General Commands Manual
  23. ↑ "Launch Services Release Notes". developer.apple.com.
  24. ↑ Dale Rahn (2005-07-03). "CVS log for src/sys/ufs/ufs/Attic/extattr.h".
  25. 1 2 Eager, Bob (2000-10-28). "Implementation of extended attributes on the FAT file system" . Retrieved 2017-07-11.
  26. ↑ fsattr(7)  –  Solaris 11.4 Standards, Environments, Macros, Character Sets, and Miscellany Reference Manual
  27. ↑ Hammons, Jack (2016-06-15). "WSL File System Support". MSDN . Retrieved 2017-07-11.
  28. ↑ "NTFS Descriptions 1.2.1". TotalCmd.net. Retrieved 16 April 2018.
  29. ↑ "QuickSearch eXtended". TotalCmd.net. Retrieved 16 April 2018.
  30. ↑ "Extended Attributes". Tuxera. Archived from the original on 2020-08-14.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.