NTFS reparse point

Last updated

An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter driver identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage Hierarchical Storage System. They also can act as hard links [ citation needed ], but are not limited to pointing to files on the same volume: they can point to directories on any local volume. The feature[ which? ] is inherited to ReFS. [1]

Contents

The open source NTFS-3G driver implements built-in support for the link-type reparse points, namely symbolic links and junction points. A plugin filter system is available to handle additional types of reparse points, allowing for chunk-deduplicated files, system-compressed files, and OneDrive files to be read. [2]

Structure

A reparse point has the following general structure, in C struct form:

structREPARSE_BUFFER{uint32_tReparseTag;uint32_tReparseDataLength;uint16_tReserved;uint8_tDataBuffer[];// flexible array member}

The reparse tag [3] is unique to each type of reparse point. It defines to which reparse point handler (usually a file system filter driver) the I/O manager delegates the processing. [4] Microsoft provides documentation on some "public" tag types. [5]

Types

Volume mount points

Volume mount points are similar to Unix mount points, where the root of another file system is attached to a directory. In NTFS, this allows additional file systems to be mounted without requiring a separate drive letter (such as C: or D:) for each.

Once a volume has been mounted on top of an existing directory of another volume, the contents previously listed in that directory become invisible and are replaced by the content of the root directory of the mounted volume.[ citation needed ] The mounted volume could still have its own drive letter assigned separately. The file system does not allow volumes to be mutually mounted on each other. Volume mount points can be made to be either persistent (remounted automatically after system reboot) or not persistent (must be manually remounted after reboot).[ citation needed ]

Mounted volumes may use other file systems than just NTFS, possibly with their own security settings and remapping of access rights according to the remote file system policy.

The substitute names of volume mount points use the NT namespace form \??\DeviceName\. [6] [7] [4] Junctions generally use \??\<drive>:\ to refer to a volume with an existing driver letter, while true volume mount points use \??\Volume{<guid>} to refer to any volume. UNC paths are invalid for junctions. [8]

Directory junctions

Directory junctions are defined using the exact same mechanism (and reparse tag: IO_REPARSE_TAG_MOUNT_POINT) as volume mount points are. The only difference is that their substitute names point to a subdirectory of another volume that usually already has a drive letter. This function is conceptually similar to symbolic links to directories in Unix, except that the target in NTFS must always be another directory (typical Unix file systems allow the target of a symbolic link to be any type of file). [4]

For instance, the directory C:\exampledir with a directory junction attribute that contains a link to D:\linkeddir will automatically refer to the directory D:\linkeddir when it is accessed by a user-mode application. [9]

Directory junctions (which can be created with the command MKLINK /J junctionName targetDirectory and removed with RMDIR junctionName from a console prompt) are persistent, and resolved on the server side as they share the same security realm of the local system or domain on which the parent volume is mounted and the same security settings for its contents as the content of the target directory; however the junction itself may have distinct security settings. Unlinking a directory junction does not delete files in the target directory.

Some directory junctions are installed by default on Windows Vista, for compatibility with previous versions of Windows, such as Documents and Settings in the root directory of the system drive, which links to the Users physical directory in the root directory of the same volume. However they are hidden by default, and their security settings are set up so that the Windows Explorer will refuse to open them from within the Shell or in most applications, except for the local built-in SYSTEM user or the local Administrators group (both user accounts are used by system software installers). This additional security restriction has probably been made to avoid users of finding apparent duplicate files in the joined directories and deleting them by error, because the semantics of directory junctions are not the same as for hard links; the reference counting is not used on the target contents and not even on the referenced container itself.[ citation needed ]

Directory junctions are soft links (they will persist even if the target directory is removed), working as a limited form of symbolic links (with an additional restriction on the location of the target), but it is an optimized version allowing faster processing of the reparse point with which they are implemented, with less overhead than the newer NTFS symbolic links, and can be resolved on the server side (when they are found in remote shared directories).[ citation needed ]

Symbolic links (or soft links) were introduced in Windows Vista. [10] Symbolic links are resolved on the client side. So when a symbolic link is shared, the target is subject to the access restrictions on the client, and not the server.[ citation needed ]

Symbolic links can be created either to files (created with MKLINK symLink targetFilename) or to directories (created with MKLINK /D symLinkD targetDirectory), but (unlike Unix symbolic links) the semantic of the link must be provided with the created link. The target however need not exist or be available when the symbolic link is created: when the symbolic link will be accessed and the target will be checked for availability, NTFS will also check if it has the correct type (file or directory); it will return a not-found error if the existing target has the wrong type.[ citation needed ]

They can also reference shared directories on remote hosts or files and subdirectories within shared directories: their target is not mounted immediately at boot, but only temporarily on demand while opening them with the OpenFile() or CreateFile() API. Their definition is persistent on the NTFS volume where they are created (all types of symbolic links can be removed as if they were files, using DEL symLink from a command line prompt or batch).[ citation needed ]

The symbolic link data is similar to mount point data, in that both use an NT namespace path. The difference is that symbolic links accepts UNC paths, but not Volume{guid} mounts. [8]

Distributed link tracking allows applications to track files, shell shortcuts or OLE links even if they were renamed or moved to another volume within the same machine, domain or workgroup. [11] Tracking is implemented as a system service, which uses the object identifier (OID) index stored in a metafile. [12] When the application requests a track to a file or directory, the tracking service creates the OID entry, which points to the file, and file rename, copy or move operation to a NTFS v3 volume also copies the object ID. This allows the tracking service to eventually find the target file.

Data deduplication

When there are several directories that have different but similar files, some of these files may have identical content. Single instance storage, found in Windows Server 2000 through Windows Storage Server 2008, allows identical files to be merged to one file and create references to that merged file. SIS consists of a file system filter that manages copies, modification and merges to files; and a user space service (or groveler) that searches for files that are identical and need merging. SIS was mainly designed for remote installation servers as these may have multiple installation images that contain many identical files; SIS allows these to be consolidated but, unlike for example hard links, each file remains distinct; changes to one copy of a file will leave others unaltered. This is similar to copy-on-write, which is a technique by which memory copying is not really done until one copy is modified. [13]

Since Windows Server 2012, there is a new chunk-based data deduplication mechanism (tag 0x80000013) that allows files with similar content to be deduplicated as long as they have stretches of identical data. [2] This mechanism is more powerful than SIS. [14] Since Windows Server 2019, the feature is fully supported on ReFS. [15]

Hierarchical Storage Management (HSM)

Hierarchical Storage Management is a means of transferring files that are not used for some period of time to less expensive storage media. When the file is next accessed, the reparse point on that file determines that it is needed and retrieves it from storage.[ citation needed ]

Native Structured Storage (NSS)

NSS was an ActiveX document storage technology that has since been discontinued by Microsoft.[ citation needed ] It allowed ActiveX Documents to be stored in the same multi-stream format that ActiveX uses internally. An NSS file system filter was loaded and used to process the multiple streams transparently to the application, and when the file was transferred to a non-NTFS formatted disk volume it would also transfer the multiple streams into a single stream. [16]

Unix domain socket (socket)

With Windows 10 build 17063 (for stable release version 1803), Microsoft introduced Unix domain sockets to Windows. These are realized by using the afunix.sys kernel driver and a new reparse point in the file system. Unix domain sockets are common on BSD and Linux systems and can be seen as the standard for inter process communication on these systems; therefore their introduction to Windows will allow simplified adoption of code and cross-platform portability. [17]

System compression

Windows 10 introduces read-only CompactOS compression algorithms for NTFS file system, taken from Windows Imaging Format (WIM); they are designed to compress Windows system files and reduce disk space usage. [18]

Internally, the compressed file is recorded as a reparse point with tag IO_REPARSE_TAG_WOF (0x80000017), where WoF stands for Windows Overlay Filter, [19] and the actual data is stored in an alternate data stream named "WofCompressedData", which is processed by WOF file system filter. [20] [21] [2]

CompactOS is an improved variant of WIMBoot from Windows 8.1, where system files could be stored in a compressed WIM image on a hidden disk partition, [22] and the WOF filter driver would decompress file contents from that WIM file; using alternate data streams instead of read-only WIM images allows CompactOS to recompress system files when they need to be rewritten with an updated version. [23]

OneDrive

OneDrive tags the files and directories it has downloaded to the local storage as a reparse point with tag 0x9000001a. The actual data is stored normally. [2]

Known risks

The Stuxnet as part of its series of Win32 exploits uses NTFS junction points as part of its overall mode of operation.[ citation needed ]

See also

Related Research Articles

<span class="mw-page-title-main">Windows 2000</span> Fifth major release of Windows NT, released in 2000

Windows 2000 is a major release of the Windows NT operating system developed by Microsoft and designed for businesses. It was the direct successor to Windows NT 4.0, and was released to manufacturing on December 15, 1999, and was officially released to retail on February 17, 2000 and September 26, 2000 for Windows 2000 Datacenter Server. It was Microsoft's business operating system until the introduction of Windows XP Professional in 2001.

New Technology File System (NTFS) is a proprietary journaling file system developed by Microsoft. Starting with Windows NT 3.1, it is the default file system of the Windows NT family. It superseded File Allocation Table (FAT) as the preferred filesystem on Windows and is supported in Linux and BSD as well. NTFS reading and writing support is provided using a free and open-source kernel implementation known as NTFS3 in Linux and the NTFS-3G driver in BSD. By using the convert command, Windows can convert FAT32/16/12 into NTFS without the need to rewrite all files. NTFS uses several files typically hidden from the user to store metadata about other files stored on the drive which can help improve speed and performance when reading data. Unlike FAT and High Performance File System (HPFS), NTFS supports access control lists (ACLs), filesystem encryption, transparent compression, sparse files and file system journaling. NTFS also supports shadow copy to allow backups of a system while it is running, but the functionality of the shadow copies varies between different versions of Windows.

<span class="mw-page-title-main">Disk partitioning</span> Creation of separate accessible storage areas on a secondary computer storage device

Disk partitioning or disk slicing is the creation of one or more regions on secondary storage, so that each region can be managed separately. These regions are called partitions. It is typically the first step of preparing a newly installed disk, before any file system is created. The disk stores the information about the partitions' locations and sizes in an area known as the partition table that the operating system reads before any other part of the disk. Each partition then appears to the operating system as a distinct "logical" disk that uses part of the actual disk. System administrators use a program called a partition editor to create, resize, delete, and manipulate the partitions. Partitioning allows the use of different filesystems to be installed for different kinds of files. Separating user data from system data can prevent the system partition from becoming full and rendering the system unusable. Partitioning can also make backing up easier. A disadvantage is that it can be difficult to properly size partitions, resulting in having one partition with too much free space and another nearly totally allocated.

In computing, a symbolic link is a file whose purpose is to point to a file or directory by specifying a path thereto.

ln (Unix) Unix file management utility

The ln command is a standard Unix command utility used to create a hard link or a symbolic link (symlink) to an existing file or directory. The use of a hard link allows multiple filenames to be associated with the same file since a hard link points to the inode of a given file, the data of which is stored on disk. On the other hand, symbolic links are special files that refer to other files by name.

<span class="mw-page-title-main">Filename</span> Text string used to uniquely identify a computer file

A filename or file name is a name used to uniquely identify a computer file in a file system. Different file systems impose different restrictions on filename lengths.

In computing, a hard link is a directory entry that associates a name with a file. Thus, each file must have at least one hard link. Creating additional hard links for a file makes the contents of that file accessible via additional paths. This causes an alias effect: a process can open the file by any one of its paths and change its content. By contrast, a soft link or “shortcut” to a file is not a direct link to the data itself, but rather a reference to a hard link or another soft link.

HFS Plus or HFS+ is a journaling file system developed by Apple Inc. It replaced the Hierarchical File System (HFS) as the primary file system of Apple computers with the 1998 release of Mac OS 8.1. HFS+ continued as the primary Mac OS X file system until it was itself replaced with the Apple File System (APFS), released with macOS High Sierra in 2017. HFS+ is also one of the formats supported by the iPod digital music player.

<span class="mw-page-title-main">File system</span> Format or program for storing files and directories

In computing, a file system or filesystem is a method and data structure that the operating system uses to control how data is stored and retrieved. Without a file system, data placed in a storage medium would be one large body of data with no way to tell where one piece of data stopped and the next began, or where any piece of data was located when it was time to retrieve it. By separating the data into pieces and giving each piece a name, the data are easily isolated and identified. Taking its name from the way a paper-based data management system is named, each group of data is called a "file". The structure and logic rules used to manage the groups of data and their names is called a "file system."

Filesystem in Userspace (FUSE) is a software interface for Unix and Unix-like computer operating systems that lets non-privileged users create their own file systems without editing kernel code. This is achieved by running file system code in user space while the FUSE module provides only a bridge to the actual kernel interfaces.

<span class="mw-page-title-main">Shadow Copy</span> Microsoft technology for storage snapshots

Shadow Copy is a technology included in Microsoft Windows that can create backup copies or snapshots of computer files or volumes, even when they are in use. It is implemented as a Windows service called the Volume Shadow Copy service. A software VSS provider service is also included as part of Windows to be used by Windows applications. Shadow Copy technology requires either the Windows NTFS or ReFS filesystems in order to create and store shadow copies. Shadow Copies can be created on local and external volumes by any Windows component that uses this technology, such as when creating a scheduled Windows Backup or automatic System Restore point.

File attributes are a type of meta-data that describe and may modify how files and/or directories in a filesystem behave. Typical file attributes may, for example, indicate or specify whether a file is visible, modifiable, compressed, or encrypted. The availability of most file attributes depends on support by the underlying filesystem where attribute data must be stored along with other control structures. Each attribute can have one of two states: set and cleared. Attributes are considered distinct from other metadata, such as dates and times, filename extensions or file system permissions. In addition to files, folders, volumes and other file system objects may have attributes.

In computing, a file shortcut is a handle in a user interface that allows the user to find a file or resource located in a different directory or folder from the place where the shortcut is located. Similarly, an Internet shortcut allows the user to open a page, file or resource located at a remote Internet location or Web site.

In computer data storage, a volume or logical drive is a single accessible storage area with a single file system, typically resident on a single partition of a hard disk. Although a volume might be different from a physical disk drive, it can still be accessed with an operating system's logical interface. However, a volume differs from a partition.

The following tables compare general and technical information for a number of file systems.

The NTFS file system defines various ways to redirect files and folders, e.g., to make a file point to another file or its contents without making a copy of it. The object being pointed to is called the target. Such file is called a hard or symbolic link depending on a way it's stored on the filesystem.

<span class="mw-page-title-main">NTFS-3G</span>

NTFS-3G is an open-source cross-platform implementation of the Microsoft Windows NTFS file system with read/write support. NTFS-3G often uses the FUSE file system interface, so it can run unmodified on many different operating systems. It is runnable on Linux, FreeBSD, NetBSD, OpenSolaris, illumos, BeOS, QNX, WinCE, Nucleus, VxWorks, Haiku, MorphOS, Minix, macOS and OpenBSD. It is licensed under the GNU General Public License. It is a partial fork of ntfsprogs and is under active maintenance and development.

NTBackup is the built-in backup application introduced in Windows NT 3.51 and included in Windows NT 4.0, Windows 2000, Windows XP, and Windows Server 2003. NTBackup comprises a command-line utility and a set of wizard interfaces that provide multiple options to create, customize, and manage backups, and it is integrated with Shadow Copy and Task Scheduler. NTBackup creates backups in a proprietary BKF file format to external sources including floppy disks, hard drives, tape drives, and ZIP drives.

NTFS volume mount points are specialized NTFS filesystem objects which are used to mount and provide an entry point to other volumes.

Resilient File System (ReFS), codenamed "Protogon", is a Microsoft proprietary file system introduced with Windows Server 2012 with the intent of becoming the "next generation" file system after NTFS.

References

  1. ↑ "Microsoft Windows Vista Client Configuration Study Guide" Wiley Publishing, Inc. 2007 p.285
  2. 1 2 3 4 André, Jean-Pierre (March 1, 2019). "NTFS-3G: Junction Points, Symbolic Links and Reparse Points". jp-andre.pagesperso-orange.fr.
  3. ↑ "Reparse Point Tags" . Retrieved 12 December 2019.
  4. 1 2 3 "NTFS Links, Directory Junctions, and Windows Shortcuts". www.flexhex.com.
  5. ↑ "[MS-FSCC] Reparse Tags" . Retrieved 12 December 2019.
  6. ↑ "Naming Files, Paths, and Namespaces/NT Namespaces". Microsoft Windows Dev Center. Retrieved 12 December 2019.
  7. ↑ "winapi - Does the "SubstituteName" string in the PathBuffer of a REPARSE_DATA_BUFFER structure always start with the prefix "\??\", and if so, why?". Stack Overflow. Retrieved 4 October 2019.
  8. ↑ Mark Russinovich. "Inside Win2K NTFS, Part 1". Microsoft Developer Network . Retrieved 2008-04-18.
  9. ↑ "Symbolic Links (Windows)". MSDN.
  10. ↑ "Distributed Link Tracking and Object Identifiers".
  11. ↑ "Distributed Link Tracking Client (System Services for the Windows Server 2003 Family and Windows XP Operating Systems)". Archived from the original on 2016-03-07. Retrieved 2017-08-26.
  12. ↑ Bolosky, Bill; Corbin, Scott; Goebel, David; Douceur, John (January 2000). Single Instance Storage in Windows 2000 (PDF). Proceedings of 4th USENIX Windows Systems Symposium. Seattle, Washington: Microsoft Research and Balder Technology Group.
  13. ↑ FileCAB-Team (10 April 2019). "Introduction to Data Deduplication in Windows Server 2012". Microsoft Tech Community.
  14. ↑ "Data Deduplication interoperability". docs.microsoft.com.
  15. ↑ Saville, John (date unknown). What is Native Structured Storage? Windows IT Pro. Retrieved from "What is Native Structured Storage?". Archived from the original on 2007-09-27. Retrieved 2007-12-03..
  16. ↑ "AF_UNIX comes to Windows". Windows Command Line Tools For Developers. Retrieved 2018-07-03.
  17. ↑ "Compact OS, single-instancing, and image optimization". Microsoft. Retrieved 1 October 2019.
  18. ↑ "What is WofCompressedData? Does WOF mean that Windows is a dog?". 18 June 2019.
  19. ↑ "Re: [ntfs-3g-devel] Experimental support for Windows 10 "System Compressed" files". sourceforge.net. Retrieved 1 October 2019.
  20. ↑ Biggers, Eric (29 April 2019). "ntfs-3g-system-compression: NTFS-3G plugin for reading "system compressed" files". GitHub. Retrieved 1 October 2019.
  21. ↑ Windows Image File Boot (WIMBoot) Overview
  22. ↑ Raymond Chen.What is WofCompressedData? Does WOF mean that Windows is a dog? Microsoft DevBlogs.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.