Resource fork

Last updated

A resource fork is a fork of a file on Apple's classic Mac OS operating system that is used to store structured data. It is one of the two forks of a file, along with the data fork, which stores data that the operating system treats as unstructured. Resource fork capability has been carried over to the modern macOS for compatibility.

Contents

A resource fork stores information in a specific form, containing details such as icon bitmaps, the shapes of windows, definitions of menus and their contents, and application code (machine code). For example, a word processing file might store its text in the data fork, while storing any embedded images in the same file's resource fork. The resource fork is used mostly by executables, but any file can have a resource fork.

In a 1986 technical note, Apple strongly recommended that developers do not put general data into the resource fork of a file. According to Apple, there are parts of the system software that rely on resource forks having only valid Resource Manager information in them. [1]

The resource fork was conceived and implemented by Apple programmer Bruce Horn.

Macintosh file systems

The resource fork has three purposes in classic Macintosh file systems:

The resource fork is implemented in all of the file systems used for system drives in the classic Mac OS (MFS, HFS and HFS Plus), and in the macOS-only APFS. The presence of a resource fork makes it easy to store a variety of additional information, such as an icon that the desktop should display for that file. While the data fork allows random access to any offset within it, access to the resource fork works like extracting structured records from a database. (Microsoft Windows also has a concept of "resources", but these are completely unrelated to resources in Mac OS.)

The Macintosh file systems store metadata distinct from either the data or resource fork, such as the creation and modification timestamps, the file type and creator codes, and fork lengths.

Some files have only a resource fork. One example is a font file in the classic Mac OS. Another example is a Classic 68k application, where even the executable code is contained in resources of type 'CODE'. Later PowerPC binaries stored the executable code in the data fork.

Since resource forks were supported only on Macintosh file systems including MFS, HFS, HFS Plus, and APFS, they could not be copied to the file systems of other operating systems. The Mac BinHex and MacBinary formats were invented to encode resource and data forks into one file, for transfer between systems. A/UX supported resource forks on Unix file systems via the AppleSingle and AppleDouble formats. Starting with Mac OS X Tiger, AppleDouble was used to store resource forks on file systems such as Windows SMB shares and FAT32 (File Allocation Table) volumes.

In the HFS Plus file system, settings can be made to allow other forks in addition to the data and resource forks, to create a "multi-fork" application. [2]

As of August 7, 2002, Apple recommended that developers should not build resources into resource forks in Mach-O binaries on Mac OS X. [3]

Resource identifiers

Each resource has an OSType identifier (a four byte value), an ID (a signed 16-bit word), and an optional name. There are standardized resource types for dialog boxes (DITL), images ( PICT ), sounds (snd )  and executable binaries (CODE) which, until the advent of the PowerPC processor, were without exception stored in the resource fork. Subroutines for rendering windows are stored in their own type of resources (WDEF), and subroutines for rendering menus in theirs (MDEF). This arrangement enabled users to easily customize not only individual applications but also the operating system itself, using tools such as ResEdit to modify the resources of an application file or any of the system files.

Within an application or other code, resources can be loaded simply using a combination of their type, ID or name, without regard to how and where they are stored in the resource fork. The client is returned a handle to the loaded resource which can then be accessed like any other heap-based data. The OS component that facilitates this is the Resource Manager. In addition to abstracting the details of the data storage from the data, the Resource Manager also arranges sets of open resource forks into a stack, with the most recently opened file on top. When trying to load a resource, it will look in the top of the stack first, (perhaps the current document's resource fork), then the next one down (the application's resource fork), then the next one (system resource forks). This arrangement is very powerful  it permits local resources to override more global ones lower down  so an application can provide its own icons or fonts in place of the standard system ones, for example. It also allows an application to load resources from the system using the same API as any other resource, without regard to where or how that resource is stored  to the application, all resources are equally available and easy to use. The system reserves resource IDs in a certain range to help avoid resource conflicts arising from this. Resource Manager APIs allow the programmer to manipulate the stack and modify the search behaviour.

Editing

As the resource fork can be edited with a resource editor such as ResEdit, it can be used to localize and customize software. In addition, most resource editors allow visual editing of data. In macOS, it is possible to use resources when developing an application. However, if the application may need to be used in UFS, it is also possible to configure it so that the entire resource fork is moved to the data fork, using the Raw Resource File setting. The integrated development environments distributed for free by Apple Inc., which include MPW and Apple Developer's Tools, include a compiler called Rez. This uses a dedicated language, also called Rez, which can be used to create a resource fork by compiling source code. A decompiler, DeRez, which can be used to change a resource fork back into Rez code is also included.

In the structure of the resource fork, there is a piece of data called a "resource map" which stores the positions of resource data items. This can be used to allow random access to resource data based on the defined IDs and names. The resource fork can be thought of as consisting of essentially two objects, the resource map and the resource data itself, but in fact each data type is a hierarchical structure which stores multiple items of data. The format in which the information in the resource data is stored is defined based on the types of information, which are known as "resource types." Resource data often makes references to other types of data.

In macOS, forks are named file/..namedfork/forkname, e.g., the resource fork of the file IMG_0593.jpg is IMG_0593.jpg/..namedfork/rsrc. The ls command supports a -l@ option which lists a file's forks.

Accessing

Resource forks appear as the extended attribute com.apple.ResourceFork. [4]

Previously resource forks were accessed via the 'Resource Manager' API. This API is now deprecated. [5]

Under the deprecated API:

  1. When a resource fork is accessed, data including the start position and length of the resource data and resource map is read in from the header.
  2. If a resource type to read in has been specified, a check is performed to make sure that type is present in the resource list, and the number of items of data containing that type and their offsets in the resource reference list from the start position of the resource map is found.
  3. The resource ID, the offset of the resource name, the resource properties, and the offset of the data from the start position of the resource data is found.
  4. If resource data with the specified ID or name is present in the resource data, the offset obtained above is accessed, the data length is found, and all the data stored there is read in, and returned as the return value.

File Manager APIs such as PBOpenRF() also allowed access to the raw resource fork; however, they should be used only for applications such as copying a file  Apple strongly warns against using the resource fork as a "second data fork."

From the POSIX interface, the resource fork could be accessed as filename/..namedfork/rsrc or as filename/rsrc; the shorter form was deprecated in Mac OS X v10.4 and removed completely in Mac OS X v10.7. [6]

Data types

The smallest elements making up a resource fork are called data types. There are several data types. After a resource fork is accessed, its contents can be found by reading it in as appropriate for the data types defined in advance. Placing definitions inside the program stating how data is to be treated makes it possible to store resources called TMPL resources as well. Using this method increases the visibility of the data when viewed with a program such as ResEdit, making later editing simpler. As the Macintosh platform originated with Motorola-based processors (68k and PPC), the data is serialized to disk in big-endian format.

The following is a list of the major data types, in alphabetical order.

Data typeactual nameDescription
BBITbinary bitRepresents a single boolean bit (true or false). Normally the number of BBITs must be a multiple of 8.
BOOLbooleanRepresents a boolean value. It consists of 2 bytes; 256 is true, and 0 is false.
CHARcharacterRepresents a one-byte character.
CSTRC stringRepresents a string of the form used in the C programming language: a null-terminated string of bytes.
DLNGdecimal long word integerA decimal long word (4 byte integer). Represents values between approximately − 2.1 billion and 2.1 billion.
HEXDhex dumpIndicates that the data from this position to the end is hexadecimal. This is used to represent code resources or compressed data.
HLNGlong word hexadecimalThis data is treated as a 4 byte hexadecimal value. It is used, among other things, to represent integers greater than 2.1 billion, such as unsigned long values in C.
PSTRPascal stringRepresents a Pascal string, with the first byte giving the length of the string.
TNAMtype nameA string representing a value such as a creator code, which is always 4 bytes long.
RECTrectangleRepresents the coordinates of the corners of a rectangle (top, left, bottom, right). Always 8 bytes long.

Types

The type codes below, like the above datatypes, are used as type identifiers for more than resource forks themselves: they are used to identify file themselves, to describe data in the clipboard, and much more.

Types must be 4 bytes long, so types like snd and STR actually have a space (0x20) at the end.

Name of resource typeactual nameDescription
alisaliasStores an alias to another file, in a resource fork of a file whose "alias" attribute bit is set
ALRTalertDefines the shape of an application alert box
APPLapplicationStores application information
BNDLbundleDefines data such as a file type icon used in an application
cicncolor iconDefines a color icon used in data
clutcolor look-up tableDefines a color palette used in data
CNTLcontrolDefines the details of a component positioned in a window
CODEcode resourceStores the machine code for the program
CURScursorDefines the shape of a monochrome cursor (8 × 8 bit square)
DITLdialog item listDefines a component of a window
DLOGdialogDefines the shape of a dialog box for an application
FREFfile referenceDefines a file type handled by an application
hfdricon balloon helpDefines the contents and shape of the balloon help displayed when the cursor hovers over the file in the Finder
icl88-bit icon listDefines an icon displayed in the Finder
icns32-bit icon listDefines an icon displayed in the Finder
ICONiconDefines a monochrome item used in data
kindfile descriptionDefines a description of a file type
MBARmenu barDefines a menu and menu bar for an application
MDEFmenu definitionDefines a menu for an application. Can also be used to define menus with complex shapes such as color palettes.
MENUmenuDefines the menu items in an application
MooVmovieStores a QuickTime movie
openopenDefines a file type which the application can open
PICTpictureStores a PICT image contained in the file
PREFpreferenceStores the environment settings for an application
sndsoundStores a sound used in the file
STRstringStores a string or hexadecimal data used in the file
STR#string listStores multiple strings used in the file
stylstyleDefines style information, such as the font, color and size of text
TEXTtextStores text
TMPLtemplateDefines the format for the resource data
versversionDefines the version or region of use of the file
WDEFwindow definitionDefines a window for the application. Windows of an unspecified shape can also be defined.
WINDwindowDefines the shape of an application window

Editors

ResEdit
Distributed free of charge by Apple. Can be used for visual editing of resource data. If the structure of data is known, it can display a range of different types of data in a visual format. Does not run on modern macOS.
Resorcerer
Expensive, but popular, as it can be used for visual editing of many more types of data than ResEdit.
HexEdit
A binary editor, which in fact is normally used more for editing the data fork rather than the resource fork.
ResKnife
Open-source editor for Mac OS X; no longer maintained.
Rezycle
A macOS tool that extracts resources from a resource fork into separate binary files while converting many types into formats suitable for modern development.
resource_dasm
An open-source resource extractor for macOS and Linux, also capable of converting many resources into modern formats. [7]
ResForge
resource editor for macOS, capable of editing classic resource fork files and related formats. Compatible with macOS 10.14 or later. Runs natively on both 64-bit Intel and Apple Silicon. [8]

Compatibility

The complexity of programming with resource forks has led to compatibility problems when accessing other file systems via file sharing protocols such as AFP, SMB, NFS and FTP, when storing to non-HFS volumes, or when transmitting files to other systems in other ways (such as via email). The AFP protocol natively supports Resource Forks, and so resource forks are typically transmitted to these volumes as-is, and stored by the server transparently to clients. The SMB protocol supports a file metadata system similar to Macintosh forks known as Alternate Data Streams (ADSes hereafter). macOS did not support storing resource forks in ADSes on SMB volumes by default until Mac OS X v10.6. In previous versions of the OS, including upgraded versions of 10.6, this feature can be enabled with a param change or by creating a special file. [9]

Networked file sharing protocols such as NFSv3 and FTP do not have a concept of file metadata, and so there is no way to natively store resource forks. This is also true when writing to certain types of local file systems, including UFS, and on SMB volumes where Alternate Data Stream support is not enabled. In those cases, macOS stores metadata and resource forks using a technique called AppleDouble, in which the data fork is written as one file, and the resource fork and metadata are written as an entirely separate file preceded by a "._" naming convention. For example: ExampleFile.psd would contain the data fork, and ._ExampleFile.psd would contain the resource fork and metadata.

Compatibility problems can arise because macOS will handle storage of resource forks differently, depending on macOS version, settings, and file system type. For example, on an SMB network with a mixture of 10.5 and 10.6 clients. A freshly installed 10.6 client will look for and store resource forks on an SMB volume in ADSes, but the 10.5 client will (by default) ignore ADSes and use AppleDouble format to handle forks. If a fileserver supports both AFP and NFS, then clients using NFS will store files in AppleDouble format, whereas AFP users will stored the resource fork natively. In those cases, compatibility can sometimes be maintained by forcing clients to use, or not use, AppleDouble format.

Many fileservers providing AFP support do not natively support resource forks on their local file systems. In those cases the forks may be stored in special ways, such as specially named files, special directories, or even Alternate Data Streams.

Another challenge is preserving resource forks when transmitting files using non-resource fork-aware applications or with certain transfer methods, including email and FTP. A number of file formats, such as MacBinary and BinHex, have been created to handle this. Command-line system tools SplitForks and FixupResourceForks allow manual flattening and merging of resource forks. In addition, a file server seeking to present file systems to Macintosh clients must accommodate the resource fork as well as the data fork of files; UNIX servers providing AFP support usually implement this with hidden directories.

Older applications written with the Carbon API have a potential issue when being ported to the current Intel Macs. While the Resource Manager and operating system know how to deserialize data correctly for common resources like 'snd ' or 'moov', resources created using TMPL resources have to be byte swapped manually to ensure file interoperability between PPC and Intel-based versions of an application. (While the resource map and other implementation details are big-endian, the Resource Manager by itself does not have any knowledge of the contents of a generic resource, and so cannot perform the byte swapping automatically.)

Until the advent of Mac OS X v10.4, the standard UNIX command-line utilities in macOS (such as cp and mv) did not respect resource forks. To copy files with resource forks, one had to use ditto or CpMac and MvMac.

Other operating systems

The concept of a resource manager for graphics objects, to save memory, originated in the OOZE package on the Xerox Alto in Smalltalk-76. [10] The concept is now largely universal in all modern operating systems. However, the concept of the resource fork remains peculiar to the Macintosh. Most operating systems used a binary file containing resources, which is then "tacked onto" the end of an existing program file. This solution is used on Microsoft Windows for instance, and similar solutions are used with the X Window System, although the resources are often left as a separate file.

The Windows NT NTFS can support forks (and so can be a file server for Mac files), the native feature providing that support is called an alternate data stream. Windows operating system features (such as the standard Summary tab in the Properties page for non-Office files) and Windows applications use them and Microsoft was developing a next-generation file system that has this sort of feature as basis.

Early versions of the BeOS implemented a database within the file system, which could be used in a manner analogous to a resource fork. Performance issues led to a change in later releases to a system of complex file system attributes. Under this system resources were handled in a fashion somewhat more analogous to the Mac.

AmigaOS does not use forked files. Its executable files are internally divided into a modular structure of large pieces (hunk) capable of storing code, data, and additional information. Similarly, data and project files have a chunk structure codified in the IFF standard. Other file types are stored similarly to other operating systems. Though not strictly a resource fork, AmigaOS stores meta data in files known as .info files. .info files can be identified by the .info extension; for example, if you save a project to a disk, two files will be saved, MyProject and MyProject.info. MyProject would be the actual project data and MyProject.info would contain the project icon, information regarding which program is needed to open the project (since there is no application binding in AmigaOS), special project options and any user comments. .info files are invisible on the Amiga's desktop (Workbench). The icon on the desktop, taken from the .info itself, is the interface metaphor through which the user interacts both with the project itself and its associated .info file. A dialog box accessible by right-clicking the icon allows the user to see and modify the metadata present in the .info file. .info files can be seen as individual files in the command-line interface or a File manager. Modern AmigaOS clones (AROS, MorphOS and AOS4) inherit the structure (complete with metadata) of the .info files of older AmigaOS versions, and can also accept standard PNG graphic files as icon bitmaps in their .info files.

NeXT operating systems NeXTSTEP and OPENSTEP, their successor, macOS, and other systems like RISC OS implemented another solution. Under these systems the resources are left in an original format, for instance, pictures are included as complete TIFF files instead of being encoded into some sort of container. These resources are then placed in a directory along with the executable code and "raw data". The directory (called a "bundle" or "application directory") is then presented to the user as the application itself. This solution provides all of the same functionality as the resource fork, but allows the resources to be easily manipulated by any application  a "resource editor" (like ResEdit) is not needed. From the command-line interface, the bundle appears to be a normal directory. This approach was not an option on the classic Mac OS, since the file system (MFS) did not support separate catalog directories. When catalog file support was included in Mac OS, with the HFS filesystem, the resource fork was retained. macOS does retain the classic Resource Manager API as part of its Carbon libraries for backward compatibility. However, the resources themselves can now be stored in separate data files within the file system  the Resource Manager now hides this implementation change from the client code.

See also

Related Research Articles

Hierarchical File System (HFS) is a proprietary file system developed by Apple Inc. for use in computer systems running Mac OS. Originally designed for use on floppy and hard disks, it can also be found on read-only media such as CD-ROMs. HFS is also referred to as Mac OS Standard, while its successor, HFS Plus, is also called Mac OS Extended.

In classic Mac OS System 7 and later, and in macOS, an alias is a small file that represents another object in a local, remote, or removable file system and provides a dynamic link to it; the target object may be moved or renamed, and the alias will still link to it. In Windows, a "shortcut", a file with a .lnk extension, performs a similar function.

<span class="mw-page-title-main">ResEdit</span>

ResEdit is a discontinued developer tool application for the Apple Macintosh, used to create and edit resources directly in the Mac's resource fork architecture. It was an alternative to tools such as REdit, and the resource compiler Rez. For the average user, ResEdit was generally easier to use, because it used a graphical user interface. Although it had been intended to be a developer tool, power users often used it to edit icons, menus, and other elements of an application's GUI, customizing it to their own preferences.

MacBinary is a file format that combines the data fork and the resource fork of a classic Mac OS file into a single file, along with HFS's extended metadata. The resulting file is suitable for transmission over FTP, the World Wide Web, and electronic mail. The documents can also be stored on computers that run operating systems with no HFS support, such as Unix or Windows.

Macintosh File System (MFS) is a volume format created by Apple Computer for storing files on 400K floppy disks. MFS was introduced with the original Apple Macintosh computer in January 1984.

HFS Plus or HFS+ is a journaling file system developed by Apple Inc. It replaced the Hierarchical File System (HFS) as the primary file system of Apple computers with the 1998 release of Mac OS 8.1. HFS+ continued as the primary Mac OS X file system until it was itself replaced with the Apple File System (APFS), released with macOS High Sierra in 2017. HFS+ is also one of the formats supported by the iPod digital music player.

A FourCC is a sequence of four bytes used to uniquely identify data formats. It originated from the OSType or ResType metadata system used in classic Mac OS and was adopted for the Amiga/Electronic Arts Interchange File Format and derivatives. The idea was later reused to identify compressed data types in QuickTime and DirectShow.

In a computer file system, a fork is a set of data associated with a file-system object. File systems without forks only allow a single set of data for the contents, while file systems with forks allow multiple such contents. Every non-empty file must have at least one fork, often of default type, and depending on the file system, a file may have one or more other associated forks, which in turn may contain primary data integral to the file, or just metadata.

On the classic Mac OS, extensions were small pieces of code that extended the system's functionality. They were run initially at start-up time, and operated by a variety of mechanisms, including trap patching and other code modifying techniques. Initially an Apple developer hack, extensions became the standard way to provide a modular operating system. Large amounts of important system services such as the TCP/IP network stacks and USB and FireWire support were optional components implemented as extensions. The phrase "system extension" later came to encompass faceless background applications as well.

The Extensible Metadata Platform (XMP) is an ISO standard, originally created by Adobe Systems Inc., for the creation, processing and interchange of standardized and custom metadata for digital documents and data sets.

In the macOS, iOS, NeXTSTEP, and GNUstep programming frameworks, property list files are files that store serialized objects. Property list files use the filename extension .plist, and thus are often referred to as p-list files.

In NeXTSTEP, OPENSTEP, and their lineal descendants macOS, iOS, iPadOS, tvOS, and watchOS, and in GNUstep, a bundle is a file directory with a defined structure and file extension, allowing related files to be grouped together as a conceptually single item.

Extended file attributes are file system features that enable users to associate computer files with metadata not interpreted by the filesystem, whereas regular attributes have a purpose strictly defined by the filesystem. Unlike forks, which can usually be as large as the maximum file size, extended attributes are usually limited in size to a value significantly smaller than the maximum file size. Typical uses include storing the author of a document, the character encoding of a plain-text document, or a checksum, cryptographic hash or digital certificate, and discretionary access control information.

AppleSingle Format and AppleDouble Format are file formats developed by Apple Computer to store Mac OS "dual-forked" files on the Unix filesystem being used in A/UX, the Macintosh platform's first Unix-like operating system. AppleSingle combined both file forks and the related Finder meta-file information into a single file, whereas AppleDouble stored them as two separate files. Support for the formats was later added to Unix software such as NFS and MAE, but they saw little use outside this small market.

In the Apple macOS operating system, .DS_Store is a file that stores custom attributes of its containing folder, such as folder view options, icon positions, and other visual information. The name is an abbreviation of Desktop Services Store, reflecting its purpose. It is created and maintained by the Finder application in every folder, and has functions similar to the file desktop.ini in Microsoft Windows. Starting with a period . character, it is hidden in Finder and many Unix utilities. Its internal structure is proprietary, but has been reverse-engineered. Starting at macOS 10.12 16A238m, Finder will not display .DS_Store files.

Sidecar files, also known as buddy files or connected files, are computer files that store data which is not supported by the format of a source file.

A file format is a standard way that information is encoded for storage in a computer file. It specifies how bits are used to encode information in a digital storage medium. File formats may be either proprietary or free.

<span class="mw-page-title-main">Apple Disk Image</span> File format developed by Apple and used by macOS

AppleDisk Image is a disk image format commonly used by the macOS operating system. When opened, an Apple Disk Image is mounted as a volume within the Finder.

<span class="mw-page-title-main">Classic Mac OS</span> Original operating system of Apple Mac (1984–2001)

Mac OS is the series of operating systems developed for the Macintosh family of personal computers by Apple Computer, Inc. from 1984 to 2001, starting with System 1 and ending with Mac OS 9. The Macintosh operating system is credited with having popularized the graphical user interface concept. It was included with every Macintosh that was sold during the era in which it was developed, and many updates to the system software were done in conjunction with the introduction of new Macintosh systems.

Apple File System (APFS) is a proprietary file system developed and deployed by Apple Inc. for macOS Sierra (10.12.4) and later, iOS 10.3, tvOS 10.2, watchOS 3.2, and all versions of iPadOS. It aims to fix core problems of HFS+, APFS's predecessor on these operating systems. APFS is optimized for solid-state drive storage and supports encryption, snapshots, and increased data integrity, among other capabilities.

References

  1. "Technical Note FL19: Data in Resource Fork: Don't do It".
  2. "Technical Note TN1150: HFS Plus Volume Format". Apple, Inc. Retrieved 11 February 2024.
  3. "Technical Q&A QA1175: Resource forks in Mach-O binaries".
  4. "Mac OS X Resource Forks" . Retrieved 2012-10-22.
  5. "Resource Manager Reference" . Retrieved 2012-10-22.
  6. "Using Pathnames". developer.apple.com. 2002-12-18. Archived from the original on 2002-12-18. Retrieved 2002-12-18.
  7. "resource_dasm". GitHub .
  8. "ResForge". GitHub .
  9. "OS X v10.5, v10.6: About named streams on SMB-mounted NAS, OS X, and Windows servers" . Retrieved 2010-04-19.
  10. "The Early History of Smalltalk" . Retrieved 2008-07-24.