Sidecar file

Last updated

Sidecar files, also known as buddy files or connected files, are computer files that store data (often metadata) which is not supported by the format of a source file.

Contents

There may be one or more sidecar files for each source file. There may also be "metadata databases" where one database contains metadata for several source files.

In most cases the relationship between the source file and the sidecar file is based on the file name; sidecar files have the same base name as the source file, but with a different extension. The problem with this system is that most operating systems and file managers have no knowledge of these relationships, and might allow the user to rename or move one of the files thereby breaking the relationship.

Examples

Amiga Hunk metadata
In AmigaOS, a file with a .info extension contains metadata for a companion Amiga Hunk executable file.
Extensible Metadata Platform
Extensible Metadata Platform (XMP) metadata is stored in a sidecar file when either a file format does not support embedded XMP metadata or if the workflow requires this.
DxO sidecar
Similar to the XMP format, DOP sidecars store metadata and edits made through DxO PhotoLab [1] and other DxO tools [2]
Connected Web Files and Folders
A file system object that associated two or more files. The file system treats connected files as a unit for purposes of moving, copying, and deleting. Some versions of Internet Explorer and Microsoft Word can save an HTML and its hyperlinked assets as such a unit.
THM
Many digital cameras will store a .thm (thumbnail) file alongside a recorded movie, with the same base filename as the movie file. These thumbnail files are JFIF-encoded image files. This system allows for quickly displaying a still preview of the movie, and storing camera data which is not supported by the AVI file format.
INF
Acorn filesystems support metadata such as load and execution addresses that may not be natively supported on other filesystems. A .inf file is used to store this metadata in text format, stored in a file with the same base filename, e.g. Menu and Menu.inf, Build.src and Build.src.inf.
JPEG + WAV
Some digital cameras allow for voice/audio annotations with photos. These are then stored as WAV audio files alongside the JPEG photo file, with the same base filename.
PDF + annotations

PDF viewers which allow the reader to annotate documents with comments and drawings may store these in a sidecar file, such as Xournal's .xoj files.

RunPacker
The MS-DOS-based self-extracting archive generator RunPacker relied heavily on sidecar files because the proprietary package format PFA (packfile archive) used in it did not natively store file attributes or timestamps. To address this limitation, a backward-compatible system of auxiliary files (as called in official documentation) was implemented to store aforementioned data in volumes without changing their format. These were added to the PFA volume the same way as ordinary files, except they were marked by special characters in their name, and software made aware of this system (sometimes called ePFA, Extended PFA) processed them transparently. Older programs relying on the PFA format would simply read or extract the file along with all others found in the volume. The ePFA format was designed with further extensibility in mind, and has been actually extended with several features over time.
Meta Information Encapsulation (MIE)
Meta Information Encapsulation sidecar files. The MIE format is an extensible, dedicated meta information format part of ExifTool. MIE files can be used to encapsulate meta information from many sources and bundle it together with any type of file.

A variation of this are copies of the source file which contain largely the same information, but in a different format or from a previous version:

Exif
Since many JPEG editing software used to destroy Exif metadata stored in digital photos, some photo cataloging applications can extract the Exif data and store that in an .exf file, so that the metadata can later be re-inserted into the JPEG file.
Raw + JPEG
Many digital cameras allow to store both uncompressed raw data and a JFIF-encoded image file when shooting in raw mode. This allows for faster previewing the photo, and support by applications that do not support the (often undocumented) raw format.
TIF + TFW or JPG + JGW
Aerial photos may be supplied with a World File that determines the location, size and rotation of the image.

Alternatives

Rather than storing data separately, it can be stored as part of the main file. This is particularly done for container files, which allow certain types of data to be stored in them. Instead of separate files on the file system, multiple files can be combined into an archive file, which keeps them together, but requires that software processes the archive file, rather than individual files. This is a generic solution, as archive files can contain arbitrary files from the file system.

Forks

A file system level solution for the same problem are forks, which allow multiple pieces of data to be associated with a single file. Sidecar files can be seen as "forks for file systems without native support for forks".

These can then be manipulated with usual file system tools: because the support is built into the operating system, these resource forks will not show up as separate files, and all applications inherit support for resource forks.[ clarification needed ] However, forks cannot be copied to file systems without support for forks, or transmitted over a channel that does not support forks. For interchange forks are generally instead stored as sidecar file.

The classic Mac OS and macOS are notable examples of operating systems with support for forks, in the HFS file system. However, this causes problems with exchanging over ISO 9660 format CD-ROM, FAT format MS-DOS disks, and over internet email, and requires the use of sidecar files to store this information. Microsoft NTFS supports Alternate Data Streams which are similar.

Related Research Articles

The JPEG File Interchange Format (JFIF) is an image file format standard published as ITU-T Recommendation T.871 and ISO/IEC 10918-5. It defines supplementary specifications for the container format that contains the image data encoded with the JPEG algorithm. The base specifications for a JPEG container format are defined in Annex B of the JPEG standard, known as JPEG Interchange Format (JIF). JFIF builds over JIF to solve some of JIF's limitations, including unnecessary complexity, component sample registration, resolution, aspect ratio, and color space. Because JFIF is not the original JPG standard, one might expect another mime-type. However, it is still registered as "image/jpeg".

The resource fork is a fork or section of a file on Apple's classic Mac OS operating system, which was also carried over to the modern macOS for compatibility, used to store structured data along with the unstructured data stored within the data fork.

<span class="mw-page-title-main">Exif</span> Metadata standard in digital images

Exchangeable image file format is a standard that specifies formats for images, sound, and ancillary tags used by digital cameras, scanners and other systems handling image and sound files recorded by digital cameras. The specification uses the following existing encoding formats with the addition of specific metadata tags: JPEG lossy coding for compressed image files, TIFF Rev. 6.0 for uncompressed image files, and RIFF WAV for audio files. It does not support JPEG 2000 or GIF encoded images.

<span class="mw-page-title-main">Picasa</span> Image organizer and image viewer (2002–2016)

Picasa was a cross-platform image organizer and image viewer for organizing and editing digital photos, integrated with a now defunct photo-sharing website, originally created by a company named Lifescape in 2002. "Picasa" is a blend of the name of Spanish painter Pablo Picasso, the word casa and "pic" for pictures.

<span class="mw-page-title-main">Geotagged photograph</span>

A geotagged photograph is a photograph which is associated with a geographic position by geotagging. Usually this is done by assigning at least a latitude and longitude to the image, and optionally elevation, compass bearing and other fields may also be included.

A container format or metafile is a file format that allows multiple data streams to be embedded into a single file, usually along with metadata for identifying and further detailing those streams. Notable examples of container formats include archive files and formats used for multimedia playback. Among the earliest cross-platform container formats were Distinguished Encoding Rules and the 1985 Interchange File Format.

<span class="mw-page-title-main">Geeqie</span>

Geeqie is a free software image viewer and image organiser program for Unix-like operating systems, which includes Linux-based systems and Apple's OS X. It was first released in March 2010, having been created as a fork of GQview, which appeared to have ceased development. It uses the GTK toolkit. In September 2015, development was moved from SourceForge to GitHub.

The Extensible Metadata Platform (XMP) is an ISO standard, originally created by Adobe Systems Inc., for the creation, processing and interchange of standardized and custom metadata for digital documents and data sets.

An image file format is a file format for a digital image. There are many formats that can be used, such as JPEG, PNG, and GIF. Most formats up until 2022 were for storing 2D images, not 3D ones. The data stored in an image file format may be compressed or uncompressed. If the data is compressed, it may be done so using lossy compression or lossless compression. For graphic design applications, vector formats are often used. Some image file formats support transparency.

<span class="mw-page-title-main">XnView</span> Software for reading, viewing, and processing images

XnView is an image organizer and general-purpose file manager used for viewing, converting, organizing and editing raster images, as well as general purpose file management. It comes with built-in hex inspection, batch renaming and screen capture tools. It is licensed as freeware for private, educational and non-profit uses. For other uses, it is licensed as commercial software.

A camera raw image file contains unprocessed or minimally processed data from the image sensor of either a digital camera, a motion picture film scanner, or other image scanner. Raw files are so named because they are not yet processed, and contain large amounts of potentially redundant data. Normally, the image is processed by a raw converter, in a wide-gamut internal color space where precise adjustments can be made before conversion to a viewable file format such as JPEG or PNG for storage, printing, or further manipulation. There are dozens of raw formats in use by different manufacturers of digital image capture equipment.

<span class="mw-page-title-main">Windows Photo Gallery</span> Discontinued image organizer, photo editor and photo sharing program

Windows Photo Gallery is a discontinued image organizer, photo editor and photo sharing program. It is a part of Microsoft's Windows Essentials software suite. The product has been unavailable for download since January 10, 2017, as the Windows Essentials line of products have been discontinued.

Windows Imaging Component (WIC) is a COM-based imaging codec framework introduced in Windows Vista (and later available in Windows XP Service Pack 3) for working with and processing digital images and image metadata.

<span class="mw-page-title-main">FastPictureViewer</span>

FastPictureViewer is a freemium image viewer for Windows XP and later. Its aim is to facilitate quick review, rating and annotation of large quantities of digital images in the early steps of the digital workflow, with an emphasis on simplicity and speed. As an app with a freemium license, a basic version is available cost-free for personal, non-profit or educational uses, while a commercial license is required for the professional version with additional features. The basic version starts as a full version trial.

<span class="mw-page-title-main">ExifTool</span> Software

ExifTool is a free and open-source software program for reading, writing, and manipulating image, audio, video, and PDF metadata. It is platform independent, available as both a Perl library (Image::ExifTool) and command-line application. ExifTool is commonly incorporated into different types of digital workflows and supports many types of metadata including Exif, IPTC, XMP, JFIF, GeoTIFF, ICC Profile, Photoshop IRB, FlashPix, AFCP and ID3, as well as the manufacturer-specific metadata formats of many digital cameras.

The Information Interchange Model (IIM) is a file structure and set of metadata attributes that can be applied to text, images and other media types. It was developed in the early 1990s by the International Press Telecommunications Council (IPTC) to expedite the international exchange of news among newspapers and news agencies.

A file format is a standard way that information is encoded for storage in a computer file. It specifies how bits are used to encode information in a digital storage medium. File formats may be either proprietary or free.

<span class="mw-page-title-main">RawTherapee</span> Raw photo processing software

RawTherapee is application software for processing photographs in raw image formats, as created by many digital cameras. It comprises a subset of image editing operations specifically aimed at non-destructive post-production of raw photos and is primarily focused on improving a photographer's workflow by facilitating the handling of large numbers of images. It is notable for the advanced control it gives the user over the demosaicing and developing process. It is cross-platform, with versions for Microsoft Windows, macOS and Linux.

<span class="mw-page-title-main">Extensible Device Metadata</span>

The Extensible Device Metadata (XDM) specification is an open file format for embedding device-related metadata in JPEG and other common image files without breaking compatibility with ordinary image viewers. The metadata types include: depth map, camera pose, point cloud, lens model, image reliability data, and identifying info about the hardware components. This metadata can be used, for instance, to create depth effects such as a bokeh filter, recreate the exact location and position of the camera when the picture was taken, or create 3D data models of environments or objects.

JPEG XT is an image compression standard which specifies backward-compatible extensions of the base JPEG standard.

References