Named pipe

Last updated

In computing, a named pipe (also known as a FIFO for its behavior) is an extension to the traditional pipe concept on Unix and Unix-like systems, and is one of the methods of inter-process communication (IPC). The concept is also found in OS/2 and Microsoft Windows, although the semantics differ substantially. A traditional pipe is "unnamed" and lasts only as long as the process. A named pipe, however, can last as long as the system is up, beyond the life of the process. It can be deleted if no longer used. Usually a named pipe appears as a file, and generally processes attach to it for IPC.

Contents

In Unix

Instead of a conventional, unnamed, shell pipeline, a named pipeline makes use of the filesystem. It is explicitly created using mkfifo() [1] or mknod(), [2] and two separate processes can access the pipe by name — one process can open it as a reader, and the other as a writer.

For example, one can create a pipe and set up gzip to compress things piped to it:

mkfifo my_pipe gzip -9 -c < my_pipe > out.gz &

In a separate process shell, independently, one could send the data to be compressed:

cat file > my_pipe

The named pipe can be deleted just like any file:

rm my_pipe

A named pipe can be used to transfer information from one application to another without the use of an intermediate temporary file. For example, you can pipe the output of gzip into a named pipe like so (here out.gz is from above example but it can be any gz):

mkfifo -m 0666 /tmp/namedPipe gzip -d < out.gz > /tmp/namedPipe

Then load the uncompressed data into a MySQL table [3] like so:

LOADDATAINFILE'/tmp/namedPipe'INTOTABLEtableName;

Without this named pipe one would need to write out the entire uncompressed version of file.gz before loading it into MySQL. Writing the temporary file is both time consuming and results in more I/O and less free space on the hard drive.

PostgreSQL's command line utility, psql, also supports loading data from named pipes. [4]

In Windows

A named pipe can be accessed much like a file. Win32 SDK functions CreateFile, ReadFile, WriteFile and CloseHandle open, read from, write to, and close a pipe, respectively. Unlike Unix, there is no command line interface, except for PowerShell.

Named pipes cannot be created as files within a normal filesystem, unlike in Unix. Also unlike their Unix counterparts, named pipes are volatile (removed after the last reference to them is closed). Every pipe is placed in the root directory of the named pipe filesystem (NPFS), mounted under the special path \\.\pipe\ (that is, a pipe named "foo" would have a full path name of \\.\pipe\foo). Anonymous pipes used in pipelining are actually named pipes with a random name.

They are very rarely seen by users, but there are notable exceptions. The VMware Workstation PC hardware virtualization tool, for instance, can expose emulated serial ports to the host system as named pipes, and the WinDbg kernel mode debugger from Microsoft supports named pipes as a transport for debugging sessions (in fact, VMware and WinDbg can be coupled together – as WinDbg normally requires a serial connection to the target computer – letting driver developers do their development and testing on a single computer). Both programs require the user to enter names in the \\.\pipe\name form.

Windows NT named pipes can inherit a security context.

Summary of named pipes on Microsoft Windows:

The .NET Framework 3.5 has added named pipe support. [5]

Named pipes can also be used as an endpoint in Microsoft SQL Server. [6]

Named pipes are also a networking protocol in the Server Message Block (SMB) suite, based on the use of a special inter-process communication (IPC) share. SMB's IPC can seamlessly and transparently pass the authentication context of the user across to Named Pipes. Windows NT's entire NT Domain protocol suite of services are implemented as DCE/RPC service over Named Pipes, as are the Exchange 5.5 Administrative applications.

See also

Related Research Articles

In computer security, an access-control list (ACL) is a list of permissions associated with a system resource (object). An ACL specifies which users or system processes are granted access to objects, as well as what operations are allowed on given objects. Each entry in a typical ACL specifies a subject and an operation. For instance, if a file object has an ACL that contains (Alice: read,write; Bob: read), this would give Alice permission to read and write the file and only give Bob permission to read it.

HPFS is a file system created specifically for the OS/2 operating system to improve upon the limitations of the FAT file system. It was written by Gordon Letwin and others at Microsoft and added to OS/2 version 1.2, at that time still a joint undertaking of Microsoft and IBM, and released in 1988.

Inter-process communication How computer operating systems enable data sharing

In computer science, inter-process communication or interprocess communication (IPC) refers specifically to the mechanisms an operating system provides to allow the processes to manage shared data. Typically, applications can use IPC, categorized as clients and servers, where the client requests data and the server responds to client requests. Many applications are both clients and servers, as commonly seen in distributed computing.

In computing, a symbolic link is a term for any file that contains a reference to another file or directory in the form of an absolute or relative path and that affects pathname resolution.

A database transaction symbolizes a unit of work performed within a database management system against a database, and treated in a coherent and reliable way independent of other transactions. A transaction generally represents any change in a database. Transactions in a database environment have two main purposes:

  1. To provide reliable units of work that allow correct recovery from failures and keep a database consistent even in cases of system failure. For example: when execution prematurely and unexpectedly stops in which case many operations upon a database remain uncompleted, with unclear status.
  2. To provide isolation between programs accessing a database concurrently. If this isolation is not provided, the programs' outcomes are possibly erroneous.
Defragmentation Rearrangement of sectors on a hard disk into contiguous units

In the maintenance of file systems, defragmentation is a process that reduces the degree of fragmentation. It does this by physically organizing the contents of the mass storage device used to store files into the smallest number of contiguous regions. It also attempts to create larger regions of free space using compaction to impede the return of fragmentation. Some defragmentation utilities try to keep smaller files within a single directory together, as they are often accessed in sequence.

In computing, the process identifier is a number used by most operating system kernels—such as those of Unix, macOS and Windows—to uniquely identify an active process. This number may be used as a parameter in various function calls, allowing processes to be manipulated, such as adjusting the process's priority or killing it altogether.

In computing, a hard link is a directory entry that associates a name with a file. Thus, each file must have at least one hard link. Creating additional hard links for a file makes the contents of that file accessible via additional paths. This causes an alias effect: a process can open the file by any one of its paths and change its content. By contrast, a soft link or “shortcut” to a file is not a direct link to the data itself, but rather a reference to a hard link or another soft link.

A temporary file is a file created to store information temporarily, either for a program's intermediate use or for transfer to a permanent file when complete. It may be created by computer programs for a variety of purposes, such as when a program cannot allocate enough memory for its tasks, when the program is working on data bigger than the architecture's address space, or as a primitive form of inter-process communication.

The Installable File System (IFS) is a filesystem API in MS-DOS/PC DOS 4.x, IBM OS/2 and Microsoft Windows that enables the operating system to recognize and load drivers for file systems.

pax (command)

pax is an archiving utility available for various operating systems and defined since 1995. Rather than sort out the incompatible options that have crept up between tar and cpio, along with their implementations across various versions of Unix, the IEEE designed a new archive utility that could support various archive formats with useful options from both archivers. The pax command is available on Unix and Unix-like operating systems and on IBM i, Microsoft Windows NT, and Windows 2000.

In software engineering, a pipeline consists of a chain of processing elements, arranged so that the output of each element is the input of the next; the name is by analogy to a physical pipeline. Usually some amount of buffering is provided between consecutive elements. The information that flows in these pipelines is often a stream of records, bytes, or bits, and the elements of a pipeline may be called filters; this is also called the pipes and filters design pattern. Connecting elements into a pipeline is analogous to function composition.

In computing, a temporary folder or temporary directory is a directory used to hold temporary files. Many operating systems and some software automatically delete the contents of this directory at bootup or at regular intervals, leaving the directory itself intact.

LAMP (software bundle) Acronym for a common web hosting solution

LAMP is an acronym denoting one of the most common software stacks for many of the web's most popular applications. However, LAMP now refers to a generic software stack model and its components are largely interchangeable.

In computing, tee is a command in command-line interpreters (shells) using standard streams which reads standard input and writes it to both standard output and one or more files, effectively duplicating its input. It is primarily used in conjunction with pipes and filters. The command is named after the T-splitter used in plumbing.

In computer science, an anonymous pipe is a simplex FIFO communication channel that may be used for one-way interprocess communication (IPC). An implementation is often integrated into the operating system's file IO subsystem. Typically a parent program opens anonymous pipes, and creates a new process that inherits the other ends of the pipes, or creates several new processes and arranges them in a pipeline.

An NTFS reparse point is a type of NTFS file system object. It is available with the NTFS v3.0 found in Windows 2000 or later versions. Reparse points provide a way to extend the NTFS filesystem. A reparse point contains a reparse tag and data that are interpreted by a filesystem filter identified by the tag. Microsoft includes several default tags including NTFS symbolic links, directory junction points, volume mount points and Unix domain sockets. Also, reparse points are used as placeholders for files moved by Windows 2000's Remote Storage Hierarchical Storage System. They also can act as hard links, but aren't limited to point to files on the same volume: they can point to directories on any local volume. The feature is inherited to ReFS.

In Unix-like operating systems, a device file or special file is an interface to a device driver that appears in a file system as if it were an ordinary file. There are also special files in DOS, OS/2, and Windows. These special files allow an application program to interact with a device by using its device driver via standard input/output system calls. Using standard system calls simplifies many programming tasks, and leads to consistent user-space I/O mechanisms regardless of device features and functions.

Strozzi NoSQL is a shell-based relational database management system initialized and developed by Carlo Strozzi that runs under Unix-like operating systems, or others with compatibility layers. Its file name NoSQL merely reflects the fact that it does not express its queries using Structured Query Language; the NoSQL RDBMS is distinct from the circa-2009 general concept of NoSQL databases, which are typically non-relational, unlike the NoSQL RDBMS. Strozzi NoSQL is released under the GNU GPL.

In computing, process substitution is a form of inter-process communication that allows the input or output of a command to appear as a file. The command is substituted in-line, where a file name would normally occur, by the command shell. This allows programs that normally only accept files to directly read from or write to another program.

References

  1. ↑ "mkfifo, mkfifoat - make a FIFO special file". IEEE Std 1003.1-2017. The Open Group.
  2. ↑ "mknod, mknodat - make directory, special file, or regular file". IEEE Std 1003.1-2017. The Open Group.
  3. ↑ "13.2.7 LOAD DATA Statement". MySQL 8.0 Reference Manual. MySQL.
  4. ↑ Aidan Van Dyk (2008-03-27). "Re: psql and named pipes". pgsql-hackers. PostgreSQL.
  5. ↑ "System.IO.Pipes Namespace". Microsoft Developer Network.
  6. ↑ "How to connect to SQL Server by using an earlier version of SQL Server". Microsoft. 2019-11-19. Retrieved 2020-05-19.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.