NTFS links

Last updated

The NTFS file system defines various ways to redirect files and folders, e.g., to make a file point to another file or its contents without making a copy of it. The object being pointed to is called the target. Such file is called a hard or symbolic link depending on a way it's stored on the filesystem.

Contents

History

Symbolic links to directories or volumes, called junction points and mount points, were introduced with NTFS 3.0 that shipped with Windows 2000. From NTFS 3.1 onwards, symbolic links can be created for any kind of file system object. NTFS 3.1 was introduced together with Windows XP, but the functionality was not made available (through ntfs.sys) to user mode applications. Third-party filter drivers such as Masatoshi Kimura's opensource senable driver could however be installed to make the feature available in user mode as well. The ntfs.sys released with Windows Vista made the functionality available to user mode applications by default.

Since NTFS 3.1, a symbolic link can also point to a file or remote SMB network path. While NTFS junction points support only absolute paths on local drives, the NTFS symbolic links allow linking using relative paths. Additionally, the NTFS symbolic link implementation provides full support for cross-filesystem links. However, the functionality enabling cross-host symbolic links requires that the remote system also support them, which effectively limits their support to Windows Vista and later Windows operating systems.

Types

There are three classes of links:

All NTFS links are designed to be transparent to applications. This means that the application accessing a link will be seamlessly redirected by the file system driver, and no special handling is needed. To users, they appear as normal directories or files. This also leads to an aliasing effect: writes to a link will pass the write to the underlying, linked file or MFT entry.

Symbolic links and junction points contain the path to the linked file, and a tag identifying the driver which implements the behaviour. Because they record the path, they can link to files on other volumes or even remote files. However this also means that if the referenced file is deleted or renamed, the link becomes invalid, and if the referenced file or directory is replaced with another, the link will now refer to the new file or directory.

Hard links create a new directory entry referring to the same MFT record, and increasing the reference count of the MFT record. This means they are restricted to files in the same filesystem, but the original file can be deleted without making the hard link invalid, since deletion reduces the reference count by one, and the file is not deleted in the MFT until the count reaches zero.

Shortcut files

An NTFS symbolic link is not the same as a Windows shortcut file, which is a regular file. The latter may be created on any filesystem (such as the earlier FAT32), may contain metadata (such as an icon to display when the shortcut is viewed in Remove links), and is not transparent to applications.

Implementations of unix-like environments for Windows such as Cygwin and Mingw can use shortcut files to emulate symbolic links where the host operating system does not support them, if configured to do so.

Examples of use

Built-in uses

Program redirection

By setting a junction point that points to a directory containing a particular version of a piece of software, it may be possible to add another version of the software and redirect the junction point to point to the version desired.

Saving storage space

The contents of a junction use almost no storage space (they simply point to the original directory). If an administrator needs to have multiple points of entry to a large directory, junction points can be an effective solution. Junction points should not be confused with a copy of something as junctions simply point to the original. If directories need to be modified separately a junction cannot be used as it does not provide a distinct copy of the directory or files within.

Likewise, symbolic links and hard links are useful for merging the contents of individual files.

Circumventing predefined paths

Since reinstalling Windows (or installing a new version) often requires deleting the contents of the C: drive, it is advantageous to create multiple partitions so only one partition needs to be deleted during the installation. However, some programs don't let the user choose the installation directory, or install some of their files to the C: drive even when they are installed to a different drive. By creating a junction point, the program can be tricked into installing to a different directory.

Command-line tools

Windows comes with several tools capable of creating and manipulating NTFS links.

In addition, the following utilities can create NTFS links, even though they don't come with Windows.

APIs

To create hard links, apps may use the CreateHardLink() function of Windows API. All versions of the Windows NT family can use GetFileInformationByHandle() to determine the number of hard links associated with a file. There can be up to 1024 links associated with an MFT entry. Similarly, the CreateSymbolicLink() function can create symbolic links. Junctions are more complex to create. They require manual reparse point information filling. [15] A code example is found in libuv. [16] Junctions are defined for directories only: although the API does not fail when one creates a junction pointing to a file, the junction will not be interpreted successfully when used later.

Junctions and symbolic links, even those pointing to directories, can be removed with pNtSetInformationFile. Libuv's implementation of unlink on Windows demonstrates this use. [17] Alternatively, the .NET System.IO.Directory.Delete() method works on them as well. [18]

Hazards

Consistency

Symbolic links and NTFS junctions can point to non-existent targets because the operating system does not continuously ensure that the target exists. [19]

Additional hazards lurk in the use of NTFS directory junctions that:

Recursive structure

The problem in the first case is that it creates recursive paths, which further implies infinite recursion in the directory structure. By introducing reentrancy, the presence of one or more directory junctions changes the structure of the file system from a simple proper tree into a directed graph, but recursive linking further complicates the graph-theoretical character from acyclic to cyclic. Since the same files and directories can now be encountered through multiple paths, applications which traverse reentrant or recursive structures naively may give incorrect or incoherent results, or may never terminate. Worse, if recursively deleting, such programs may attempt to delete a parent of the directory it is currently traversing.

Note that both of the conditions listed above exist in the system of hard links established on the C: drive in the default Windows setup. For example, every Windows 10 installation defines the recursive path:

C:\ProgramData\ C:\ProgramData\Application Data\ C:\ProgramData\Application Data\Application Data\ C:\ProgramData\Application Data\Application Data\Application Data\ C:\ProgramData\Application Data\Application Data\Application Data\Application Data\ C:\ProgramData\Application Data\Application Data\Application Data\Application Data\Application Data\ ...

Each additional path name in this seemingly infinite set is an actual valid Windows path which refers to the same location. In practice, path names are limited by the 260-character DOS path limit (or newer 32,767 character limit), but truncation may result in incomplete or invalid path and file names. Whenever a copy of a Windows installation is archived, with directory junctions intact, to another volume on the same—or worse—another computer, the archived copy may still incorporate active folders from the running installation. For example, depending on the method used for copying, a backup copy of a Windows drive X:\archive\... will include a hard link called X:\archive\Users\USERNAME\My Documents which still points to folder C:\Users\USERNAME\Documents\ in the current, active installation.

Cross-volume traversal

The second form of deferred target mis-referral, while conceptually simpler, can have more severe consequences. When a self-consistent volume or directory structure containing hard links which use volume drive-letter path names is copied or moved to another volume (or when the drive letter of a volume is reassigned by some other means), such links may no longer point to the corresponding target in the copied structure. Again the results depend on the software that was used for copying; while some programs may intercede by modifying any fully subsumed hard links in the copy in order to preserve structural consistency, others may ignore, copy exactly, or even traverse into hard links, copying their contents.

The serious problems occur if hard links are copied exactly such that they become, in the new copy, cross-volume hard links which still point to original files and folders on the source volume. Unintentional cross-volume hard links, such as hard links in an "archive" folder which still point to locations on the original volume (according to drive letter), are catastrophes waiting to happen. For example, deleting what is much later presumed to be an unused archive directory on a disused backup volume may result in deleting current, active user data or system files.

A preventative measure for the drive-letter hazard is to use volume GUID path syntax, [20] rather than paths containing volume drive letters, when specifying the target path for a directory junction. For example, consider creating an alias for X:\Some\Other\Path at X:\Some\Path\Foo:

X:\Some\Path> linkd Foo X:\Some\Other\Path

As described above, if the folder structure that contains the resulting link is moved to a disk with a drive letter other than X:, or if the letter is changed on drive X: itself, the data content at the target location is vulnerable to accidental corruption or malicious abuse. A more resilient version of this link can partially mitigate this risk by referencing the target volume by its GUID identifier value (which can be discovered by running the fsutil volume list command).

X:\Some\Path> linkd Foo \\?\Volume{12345678-abcd-1234--abcdefghijkl}\Some\Other\Path

Doing so ensures that the junction will remain valid if drive letter X: is changed by any means.

As for a proactive means of avoiding directory junction disasters, the command dir /AL /S /B "X:\Some\Path" can be used to obtain, for careful analysis prior to committing any irreversible file system alterations, a list of all hard links "below" a certain file system location. While by definition every link in the resulting list has a path name that starts with X:\Some\Path\, if any of those hard links contains a target which is not subsumed by X:\Some\Path, then the specified scope has been escaped, and the starting directory you specified is not fully-subsuming. Extra caution may be indicated in this case, since the specified directory includes files and directories which reside on other physical volumes, or whose own parent-traversal-to-root does not include the specified directory.

Limitations

Privilege requirements

The default security settings in Windows disallow non-elevated administrators and all non-administrators from creating symbolic links but not junctions. This behavior can be changed running "secpol.msc", the Local Security Policy management console (under: Security Settings\Local Policies\User Rights Assignment\Create symbolic links). It can be worked around by starting cmd.exe with Run as administrator option or the runas command. Starting with Windows 10 Insiders build 14972 the requirement for elevated administrator privileges was removed in Windows "Developer Mode", allowing symlinks to be created without needing to elevate the console as administrator. At the API level, a SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag is supplied for this purpose. [21]

Boot time

The Windows startup process does not support junction points, so it is impossible to redirect certain system folders:

Other critical system boot files, such as The sleep image file hiberfil.sys, also do not support redirecetion.

System-defined locations

It is technically possible to redirect the following non-critical system folder locations:

Doing this may lead to long-term Windows reliability or compatibility issues. Creating junctions for \Users and \ProgramData pointing to another drive is not recommended as it breaks updates and Windows Store Apps. [22]

Creating junctions for \Users, \ProgramData, \Program Files or \Program Files (x86) pointing to other locations breaks installation or upgrade of Windows. [23]

Creating junctions for \Program Files or \Program Files (x86) pointing to another drive breaks Windows' Component Based Servicing which hardlinks files from its repository \Windows\SxS to their installation directory.[ citation needed ]

Windows installer

Windows Installer does not fully support symbolic links. Redirecting \Windows\Installer will cause most .msi-based Windows installers to fail with error 2755 and/or error 1632.

Since Windows XP uses the same NTFS format version as later releases, it's feasible to enable symbolic links support in it. For using NTFS symbolic links under Windows 2000 and XP, a third-party driver exists that does it by installing itself as a file system filter. [24] [25]

See also

Related Research Articles

In computing, a computer file is a resource for recording data on a computer storage device, primarily identified by its filename. Just as words can be written on paper, so too can data be written to a computer file. Files can be shared with and transferred between computers and mobile devices via removable media, networks, or the Internet.

New Technology File System (NTFS) is a proprietary journaling file system developed by Microsoft. Starting with Windows NT 3.1, it is the default file system of the Windows NT family. It superseded File Allocation Table (FAT) as the preferred filesystem on Windows and is supported in Linux and BSD as well. NTFS reading and writing support is provided using a free and open-source kernel implementation known as NTFS3 in Linux and the NTFS-3G driver in BSD. By using the convert command, Windows can convert FAT32/16/12 into NTFS without the need to rewrite all files. NTFS uses several files typically hidden from the user to store metadata about other files stored on the drive which can help improve speed and performance when reading data. Unlike FAT and High Performance File System (HPFS), NTFS supports access control lists (ACLs), filesystem encryption, transparent compression, sparse files and file system journaling. NTFS also supports shadow copy to allow backups of a system while it is running, but the functionality of the shadow copies varies between different versions of Windows.

In computing, a symbolic link is a file whose purpose is to point to a file or directory by specifying a path thereto.

ln (Unix) Unix file management utility

The ln command is a standard Unix command utility used to create a hard link or a symbolic link (symlink) to an existing file or directory. The use of a hard link allows multiple filenames to be associated with the same file since a hard link points to the inode of a given file, the data of which is stored on disk. On the other hand, symbolic links are special files that refer to other files by name.

In computing, a hard link is a directory entry that associates a name with a file. Thus, each file must have at least one hard link. Creating additional hard links for a file makes the contents of that file accessible via additional paths. This causes an alias effect: a process can open the file by any one of its paths and change its content. By contrast, a soft link or “shortcut” to a file is not a direct link to the data itself, but rather a reference to a hard link or another soft link.

<span class="mw-page-title-main">File system</span> Computer filing system

In computing, a file system or filesystem governs file organization and access. A local file system is a capability of an operating system that services the applications running on the same computer. A distributed file system is a protocol that provides file access between networked computers.

<span class="mw-page-title-main">Shadow Copy</span> Microsoft technology for storage snapshots

Shadow Copy is a technology included in Microsoft Windows that can create backup copies or snapshots of computer files or volumes, even when they are in use. It is implemented as a Windows service called the Volume Shadow Copy service. A software VSS provider service is also included as part of Windows to be used by Windows applications. Shadow Copy technology requires either the Windows NTFS or ReFS filesystems in order to create and store shadow copies. Shadow Copies can be created on local and external volumes by any Windows component that uses this technology, such as when creating a scheduled Windows Backup or automatic System Restore point.

File attributes are a type of meta-data that describe and may modify how files and/or directories in a filesystem behave. Typical file attributes may, for example, indicate or specify whether a file is visible, modifiable, compressed, or encrypted. The availability of most file attributes depends on support by the underlying filesystem where attribute data must be stored along with other control structures. Each attribute can have one of two states: set and cleared. Attributes are considered distinct from other metadata, such as dates and times, filename extensions or file system permissions. In addition to files, folders, volumes and other file system objects may have attributes.

In computing, a file shortcut is a handle in a user interface that allows the user to find a file or resource located in a different directory or folder from the place where the shortcut is located. Similarly, an Internet shortcut allows the user to open a page, file or resource located at a remote Internet location or Web site.

<span class="mw-page-title-main">System Restore</span> System recovery feature in Microsoft Windows

System Restore is a feature in Microsoft Windows that allows the user to revert their computer's state to that of a previous point in time, which can be used to recover from system malfunctions or other problems. First included in Windows Me, it has been included in all following desktop versions of Windows released since, excluding Windows Server. In Windows 10, System Restore is turned off by default and must be enabled by users in order to function. This does not affect personal files such as documents, music, pictures, and videos.

In computer data storage, a volume or logical drive is a single accessible storage area with a single file system, typically resident on a single partition of a hard disk. Although a volume might be different from a physical disk drive, it can still be accessed with an operating system's logical interface. However, a volume differs from a partition.

In computing, SUBST is a command on the DOS, IBM OS/2, Microsoft Windows and ReactOS operating systems used for substituting paths on physical and logical drives as virtual drives.

Windows Vista introduced a number of new I/O functions to the Microsoft Windows line of operating systems. They are intended to shorten the time taken to boot the system, improve the responsiveness of the system, and improve the reliability of data storage.

An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter driver identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage Hierarchical Storage System. They also can act as hard links, but are not limited to pointing to files on the same volume: they can point to directories on any local volume. The feature is inherited to ReFS.

Windows Vista has many significant new features compared with previous Microsoft Windows versions, covering most aspects of the operating system.

NTBackup is the first built-in backup utility of the Windows NT family. It was introduced with Windows NT 3.51. NTBackup comprises a GUI (wizard-style) and a command-line utility to create, customize, and manage backups. It takes advantage of Shadow Copy and Task Scheduler. NTBackup stores backups in the BKF file format on external sources, e.g., floppy disks, hard drives, tape drives, and Zip drives. When used with tape drives, NTBackup uses the Microsoft Tape Format (MTF), which is also used by BackupAssist, Backup Exec, and Veeam Backup & Replication and is compatible with BKF.

Windows Vista contains a range of new technologies and features that are intended to help network administrators and power users better manage their systems. Notable changes include a complete replacement of both the Windows Setup and the Windows startup processes, completely rewritten deployment mechanisms, new diagnostic and health monitoring tools such as random access memory diagnostic program, support for per-application Remote Desktop sessions, a completely new Task Scheduler, and a range of new Group Policy settings covering many of the features new to Windows Vista. Subsystem for UNIX Applications, which provides a POSIX-compatible environment is also introduced.

<span class="mw-page-title-main">Trash (computing)</span> Temporary storage for deleted files

In computing, the trash, also known by other names such as dustbin, wastebasket, and others, is a graphical user interface desktop metaphor for temporary storage for files set aside by the user for deletion, but not yet permanently erased. The concept and name is part of Mac operating systems, a similar implementation is called the Recycle Bin in Microsoft Windows, and other operating systems use other names.

Program Files is the directory name of a standard folder in Microsoft Windows operating systems in which applications that are not part of the operating system are conventionally installed. Typically, each application installed under the 'Program Files' directory will have a subdirectory for its application-specific resources. Shared resources, for example resources used by multiple applications from one company, are typically stored in the 'Common Files' directory.

NTFS volume mount points are specialized NTFS filesystem objects which are used to mount and provide an entry point to other volumes.

References

  1. "Hard Links and Junctions". Windows Developer. Microsoft. 31 May 2018.
  2. 1 2 "Junction v1.07". Sysinternals. Microsoft. 4 July 2016.
  3. "New-Item (PowerShell 3.0)". Microsoft Docs. Microsoft. 22 June 2020. If your location is in a FileSystem drive, the following values are allowed: If your location is in a FileSystem drive, the following values are allowed: File[,] Directory[,] Junction[,] HardLink
  4. "New-Item (PowerShell 5.0)". Microsoft Docs. Microsoft. 22 June 2020.
  5. "Fix Remove-Item <symbolic link to directory>". PowerShell repo. Microsoft. 4 March 2016 via GitHub.
  6. "mklink.c". reactos repo. ReactOS Deutschland e.V. 3 October 2017 via GitHub.com.
  7. "Dir". Microsoft Docs. Microsoft. 18 April 2012.
  8. "Fsutil hardlink". Microsoft Docs. Microsoft. 18 April 2012.
  9. "Fsutil reparsepoint". Microsoft Docs. Microsoft. 18 April 2012.
  10. "Download Windows Server 2003 Resource Kit Tools". Download Center. 2015-03-10. Archived from the original on 2015-03-10.
  11. "How to create and manipulate NTFS junction points". 2015-03-09. Archived from the original on 2015-03-09.
  12. "Pscx". PowerShell Gallery. Microsoft. 17 January 2018.
  13. Hill, Keith; Grehan, Oisin (17 January 2018). "README.md". pscx repo via GitHub.
  14. "NTFS Links, Directory Junctions, and Windows Shortcuts". www.flexhex.com. Archived from the original on 25 January 2021. Retrieved 4 October 2019.
  15. "libuv/src/win/fs.c". GitHub. Retrieved 12 December 2019.
  16. "libuv/src/win/fs.c fs__unlink". GitHub. Retrieved 12 December 2019.
  17. "PowerShell Remove Junction". StackOverflow. Retrieved 12 December 2019.
  18. "Programming Considerations (Windows)". msdn.microsoft.com.
  19. "File path formats on Windows systems". docs.microsoft.com.
  20. "Symlinks in Windows 10! - Windows Developer BlogWindows Developer Blog". blogs.windows.com. 2 December 2016.
  21. "Relocation of the Users directory and the ProgramData directory to a drive other than the drive that contains the Windows directory" . Retrieved 2015-03-12.
  22. "You encounter an error when trying to install Windows 8.1 due to redirecting the Users or Program Files folder to another partition" . Retrieved 2015-03-12.
  23. Hermann Schinagl (August 23, 2013). "ln - commandline hardlinks - Symbolic links for Windows XP".
  24. "リンク/ジャンクション作成ツール". emk.name.