High Performance File System

Last updated
HPFS
Developer(s) Microsoft, IBM
Full nameHigh Performance File System
IntroducedNovember 1989;33 years ago (1989-11) with OS/2 1.2
Partition IDs 0x07 (MBR)
Structures
Directory contents B tree
File allocation B+ tree
Bad blocks List
Limits
Max volume size64 GB (as implemented)
2 TB (theoretical)
Max file size2 GB
Max no. of filesUnlimited
Max filename length255 characters
Allowed filename
characters
Single-byte from 0x20 to 0xFF
Features
Dates recordedAccess, Creation, Modified
Forks Yes
AttributesRead-only, hidden, system, archive
File system
permissions
Yes (only in HPFS386)
Transparent
compression
No
Transparent
encryption
No
Other
Supported
operating systems
OS/2, Windows NT, Linux, DragonFly BSD, eComStation, ArcaOS

HPFS (High Performance File System) is a file system created specifically for the OS/2 operating system to improve upon the limitations of the FAT file system. It was written by Gordon Letwin and others at Microsoft and added to OS/2 version 1.2, at that time still a joint undertaking of Microsoft and IBM, and released in 1988.

Contents

Overview

Compared with FAT, HPFS provided a number of additional capabilities:

HPFS also can keep 64 KB of metadata ("extended attributes") per file.

IBM offers two kinds of IFS drivers for this file system:

HPFS386's cache is limited by the amount of available memory in OS/2's system memory arena [1] and was implemented in 32-bit assembly language. HPFS386 is a ring 0 driver (allowing direct hardware access and direct interaction with the kernel) with built-in SMB networking properties that are usable by various server daemons, whereas HPFS is a ring 3 driver. Thus, HPFS386 is faster than HPFS and highly optimized for server applications. It is also highly tunable by experienced administrators.

Though IBM still had rights to HPFS, its agreement with Microsoft to continue licensing the HPFS386 version was contingent upon the company paying Microsoft a licensing fee for each copy sold. This was a result of the Microsoft and IBM collaboration that gave both the right to use Windows and OS/2 technology.

Due to the Microsoft dependence, limited partition size, file size limit of 2 GB and the long disk-check times after a crash, IBM ported the journaling file system, JFS, to OS/2 as a substitute.

DOS and Linux support HPFS via third-party drivers. Windows NT versions 3.51 and earlier had native support for HPFS.

Native support under Windows

Windows 95 and its successors Windows 98 and Windows Me can read and write HPFS only when mapped via a network share; they cannot read it from a local disk. They listed the NTFS partitions of networked computers as "HPFS"[ citation needed ], because NTFS and HPFS share the same filesystem identification number in the partition table.

Windows NT 3.1 and 3.5 have native read/write support for local disks and can even be installed onto an HPFS partition.

Windows NT 3.51 can also read and write from local HPFS formatted drives. Starting with Windows NT 4 the filesystem driver PINBALL.SYS enabling the read/write access is not shipped anymore. Later Windows versions do not ship with this driver. Note that this driver is limited to 4GB HPFS volumes.

Microsoft retained rights to OS/2 technologies, including the HPFS filesystem, after they ceased collaboration with IBM. Since Windows NT 3.1 was designed for more rigorous (enterprise-class) use than previous versions of Windows, it included support for HPFS (and NTFS) giving it a larger storage capacity than the FAT12 and FAT16 filesystems. However, since HPFS lacks a journal, any recovery after an unexpected shutdown or other error state takes progressively longer as the filesystem grows. A utility such as CHKDSK would need to scan each entry in the filesystem to ensure no errors are present, a problem which is vastly reduced on NTFS, which simply replays the journal.

See also

Related Research Articles

New Technology File System (NTFS) is a proprietary journaling file system developed by Microsoft. Starting with Windows NT 3.1, it is the default file system of the Windows NT family. It superseded File Allocation Table (FAT) as the preferred filesystem on Windows and is supported in Linux and BSD as well. NTFS reading and writing support is provided using a free and open-source kernel implementation known as NTFS3 in Linux and the NTFS-3G driver in BSD. By using the convert command, Windows can convert FAT32/16/12 into NTFS without the need to rewrite all files. NTFS uses several files typically hidden from the user to store metadata about other files stored on the drive which can help improve speed and performance when reading data. Unlike FAT and High Performance File System (HPFS), NTFS supports access control lists (ACLs), filesystem encryption, transparent compression, sparse files and file system journaling. NTFS also supports shadow copy to allow backups of a system while it is running, but the functionality of the shadow copies varies between different versions of Windows.

Universal Disk Format (UDF) is an open, vendor-neutral file system for computer data storage for a broad range of media. In practice, it has been most widely used for DVDs and newer optical disc formats, supplanting ISO 9660. Due to its design, it is very well suited to incremental updates on both write-once and re-writable optical media. UDF was developed and maintained by the Optical Storage Technology Association (OSTA).

File Allocation Table (FAT) is a file system developed for personal computers and was the default filesystem for MS-DOS and Windows 9x operating systems. Originally developed in 1977 for use on floppy disks, it was adapted for use on hard disks and other devices. The increase in disk drives capacity required three major variants: FAT12, FAT16 and FAT32. FAT was replaced with NTFS as the default file system on Microsoft operating systems starting with Windows XP. Nevertheless, FAT continues to be used on flash and other solid-state memory cards and modules, many portable and embedded devices because of its compatibility and ease of implementation.

<span class="mw-page-title-main">Disk partitioning</span> Creation of separate accessible storage areas on a secondary computer storage device

Disk partitioning or disk slicing is the creation of one or more regions on secondary storage, so that each region can be managed separately. These regions are called partitions. It is typically the first step of preparing a newly installed disk, before any file system is created. The disk stores the information about the partitions' locations and sizes in an area known as the partition table that the operating system reads before any other part of the disk. Each partition then appears to the operating system as a distinct "logical" disk that uses part of the actual disk. System administrators use a program called a partition editor to create, resize, delete, and manipulate the partitions. Partitioning allows the use of different filesystems to be installed for different kinds of files. Separating user data from system data can prevent the system partition from becoming full and rendering the system unusable. Partitioning can also make backing up easier. A disadvantage is that it can be difficult to properly size partitions, resulting in having one partition with too much free space and another nearly totally allocated.

<span class="mw-page-title-main">Defragmentation</span> Rearrangement of sectors on a hard disk into contiguous units

In the maintenance of file systems, defragmentation is a process that reduces the degree of fragmentation. It does this by physically organizing the contents of the mass storage device used to store files into the smallest number of contiguous regions. It also attempts to create larger regions of free space using compaction to impede the return of fragmentation. Some defragmentation utilities try to keep smaller files within a single directory together, as they are often accessed in sequence.

<span class="mw-page-title-main">PartitionMagic</span>

PartitionMagic is a utility software program for hard disk drive partitioning originally made by PowerQuest, but subsequently owned by Symantec. As of December 8, 2009, the Symantec website stated that they no longer offer PartitionMagic.

The Encrypting File System (EFS) on Microsoft Windows is a feature introduced in version 3.0 of NTFS that provides filesystem-level encryption. The technology enables files to be transparently encrypted to protect confidential data from attackers with physical access to the computer.

The Installable File System (IFS) is a filesystem API in MS-DOS/PC DOS 4.x, IBM OS/2 and Microsoft Windows that enables the operating system to recognize and load drivers for file systems.

HFS Plus or HFS+ is a journaling file system developed by Apple Inc. It replaced the Hierarchical File System (HFS) as the primary file system of Apple computers with the 1998 release of Mac OS 8.1. HFS+ continued as the primary Mac OS X file system until it was itself replaced with the Apple File System (APFS), released with macOS High Sierra in 2017. HFS+ is also one of the formats supported by the iPod digital music player.

<span class="mw-page-title-main">File system</span> Format or program for storing files and directories

In computing, a file system or filesystem is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified. Taking its name from the way a paper-based data management system is named, each group of data is called a "file". The structure and logic rules used to manage the groups of data and their names is called a "file system."

In computing, the BIOS parameter block, often shortened to BPB, is a data structure in the volume boot record (VBR) describing the physical layout of a data storage volume. On partitioned devices, such as hard disks, the BPB describes the volume partition, whereas, on unpartitioned devices, such as floppy disks, it describes the entire medium. A basic BPB can appear and be used on any partition, including floppy disks where its presence is often necessary; however, certain filesystems also make use of it in describing basic filesystem structures. Filesystems making use of a BIOS parameter block include FAT12, FAT16, FAT32, HPFS, and NTFS. Due to different types of fields and the amount of data they contain, the length of the BPB is different for FAT16, FAT32, and NTFS boot sectors. Combined with the 11-byte data structure at the very start of volume boot records immediately preceding the BPB or EBPB, this is also called FDC descriptor or extended FDC descriptor in ECMA-107 or ISO/IEC 9293.

<span class="mw-page-title-main">CHKDSK</span> System tool in DOS, OS/2 and Windows

In computing, CHKDSK is a system tool and command in DOS, Digital Research FlexOS, IBM/Toshiba 4690 OS, IBM OS/2, Microsoft Windows and related operating systems. It verifies the file system integrity of a volume and attempts to fix logical file system errors. It is similar to the fsck command in Unix and similar to Microsoft ScanDisk, which co-existed with CHKDSK in Windows 9x and MS-DOS 6.x.

In computer data storage, a volume or logical drive is a single accessible storage area with a single file system, typically resident on a single partition of a hard disk. Although a volume might be different from a physical disk drive, it can still be accessed with an operating system's logical interface. However, a volume differs from a partition.

Extended file attributes are file system features that enable users to associate computer files with metadata not interpreted by the filesystem, whereas regular attributes have a purpose strictly defined by the filesystem. Unlike forks, which can usually be as large as the maximum file size, extended attributes are usually limited in size to a value significantly smaller than the maximum file size. Typical uses include storing the author of a document, the character encoding of a plain-text document, or a checksum, cryptographic hash or digital certificate, and discretionary access control information.

The following tables compare general and technical information for a number of file systems.

<span class="mw-page-title-main">Microsoft Drive Optimizer</span> Windows utility which defragments a hard drive

Microsoft Drive Optimizer is a utility in Microsoft Windows designed to increase data access speed by rearranging files stored on a disk to occupy contiguous storage locations, a technique called defragmentation. Defragmenting a disk minimizes head travel, which reduces the time it takes to read files from and write files to the disk. As a result of the decreased read and write times, Microsoft Drive Optimizer decreases system startup times for systems starting from magnetic storage devices such as a hard drive. However, defragmentation is not helpful on storage devices such as solid state drives, USB drives or SD cards that use flash memory to increase speeds, as these drives do not use a head. Defragmentation may decrease lifespan for certain technologies, e.g. solid state drives. Microsoft Drive Optimizer was first officially shipped with Windows XP.

An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter driver identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage Hierarchical Storage System. They also can act as hard links, but are not limited to pointing to files on the same volume: they can point to directories on any local volume. The feature is inherited to ReFS.

<span class="mw-page-title-main">NTFS-3G</span>

NTFS-3G is an open-source cross-platform implementation of the Microsoft Windows NTFS file system with read/write support. NTFS-3G often uses the FUSE file system interface, so it can run unmodified on many different operating systems. It is runnable on Linux, FreeBSD, NetBSD, OpenSolaris, illumos, BeOS, QNX, WinCE, Nucleus, VxWorks, Haiku, MorphOS, Minix, macOS and OpenBSD. It is licensed under the GNU General Public License. It is a partial fork of ntfsprogs and is under active maintenance and development.

Resilient File System (ReFS), codenamed "Protogon", is a Microsoft proprietary file system introduced with Windows Server 2012 with the intent of becoming the "next generation" file system after NTFS.

References

  1. "Virtual Memory Problems under OS/2". www.os2voice.org. Archived from the original on 24 September 2015. Retrieved 11 June 2015.

Further reading