STREAMS

Last updated

In computer networking, STREAMS is the native framework in Unix System V for implementing character device drivers, network protocols, and inter-process communication. In this framework, a stream is a chain of coroutines that pass messages between a program and a device driver (or between a pair of programs). STREAMS originated in Version 8 Research Unix, as Streams (not capitalized).

Contents

STREAMS's design is a modular architecture for implementing full-duplex I/O between kernel and device drivers. Its most frequent uses have been in developing terminal I/O (line discipline) and networking subsystems. In System V Release 4, the entire terminal interface was reimplemented using STREAMS. [1] An important concept in STREAMS is the ability to push drivers – custom code modules which can modify the functionality of a network interface or other device – together to form a stack. Several of these drivers can be chained together in order.

History

STREAMS was based on the Streams I/O subsystem introduced in the Eighth Edition Research Unix (V8) by Dennis Ritchie, where it was used for the terminal I/O subsystem and the Internet protocol suite. This version, not yet called STREAMS in capitals, fit the new functionality under the existing device I/O system calls (open, close, read, write, and ioctl), [2] and its application was limited to terminal I/O and protocols providing pipe-like I/O semantics.

This I/O system was ported to System V Release 3 by Robert Israel, Gil McGrath, Dave Olander, Her-Daw Che, and Maury Bach as part of a wider framework intended to support a variety of transport protocols, including TCP, ISO Class 4 transport, SNA LU 6.2, and the AT&T NPACK protocol (used in RFS). [3] It was first released with the Network Support Utilities (NSU) package of UNIX System V Release 3. [4] This port added the putmsg, getmsg, and poll system calls, which are nearly equivalent in purpose to the send, recv, and select calls from Berkeley sockets. The putmsg and getmsg system calls were originally called send and recv, [5] but were renamed to avoid namespace conflict. [6] In System V Release 4, STREAMS was extended and used for the terminal I/O framework and pipes, providing useful new functionality like bidirectional pipes and file descriptor passing. [3] A port for UNICOS was also produced. Eric S. Raymond quotes Ritchie as saying about the complexity of System V STREAMS when compared to his V8 Streams that "Streams means something different when shouted". [7]

Concurrent with the System V Release 3 port, AT&T developed protocol-independent STREAMS message passing guidelines for the link, [8] network, [9] and transport layers [10] of the OSI model (layers 2-4). Due to the typically close implementation coupling of the network and transport protocols in a given protocol stack, and the typical practice of implementing layers 5-7 outside of the kernel, only the link [8] and transport layer [11] STREAMS service interfaces were later standardized by X/Open. In conjunction with the transport message passing model, the Transport Layer Interface (later adopted as the X/Open Transport Interface) was defined to provide a transport protocol-independent API for application development. Also, a library supporting the session, presentation and application layers [12] was defined and later standardized by The Open Group. [13]

STREAMS was required for conformance with the Single UNIX Specification versions 1 (UNIX 95) and 2 (UNIX 98), but as a result of the refusal of the BSD and Linux developers to provide STREAMS,[ citation needed ] was marked as optional for POSIX compliance by the Austin Group in version 3 (UNIX 03). POSIX.1-2008 with TC1 (IEEE Std 1003.1, 2013 edition) has designated STREAMS as 'marked obsolescent' [14] [15] meaning that said functionality may be removed in a future version of the specification. However, the specific definition of 'obsolescent' used [16] also says that strictly conforming POSIX applications 'shall not use obsolescent features'.

Technical overview

Example use of Streams to implement remote command execution over a network, after (Ritchie 1984) Unix Streams.svg
Example use of Streams to implement remote command execution over a network, after ( Ritchie 1984 )

In Version 7 Unix, a command was connected to a terminal (keyboard and screen, or keyboard and printer) through a mechanism called the line discipline, which would buffer a single line of input, i.e., wait for the user to press the Return key before sending input to the program for processing; this allowed simple error correction. Streams replaced this with a set of processing modules organized in a linear chain that allowed bidirectional communication between neighboring modules. Programs could "push" a new module onto one end of the chain to change the behavior of a terminal or other character device. Ritchie gives the example chain of a terminal module chained with a Datakit network module to achieve remote login over a network. [5] Aside from characters (bytes) going from program to device and vice versa, Streams could carry control messages such as "hangup" (drop connection) and ioctl messages.

Streams could also be used for inter-process communication, by connecting two processes to pseudoterminals. This functionality was implemented in the mpx window system for the Blit graphics terminal, which could display multiple terminal emulator windows. Each window was a process that communicated with the window system through a pseudoterminal that had the line discipline driver installed, sending typed characters to it and receiving text (and graphics) to display. Control signals designated the user's wish to switch between windows or close them. [17] [18] : 348–350 

The actual Streams modules live in kernel space on Unix, and are installed (pushed) and removed (popped) by the ioctl system call. For example, to install the aforementioned line discipline on a file descriptor fd referring to a terminal device, one would write (in C): [18] : 347 

ioctl(fd,PUSH,TTYLD);

To perform input/output on a stream, one either uses the read and write system calls as with regular file descriptors, or a set of STREAMS-specific functions to send control messages. [19]

Ritchie admitted to regretting having to implement Streams in the kernel, rather than as processes, but felt compelled to do so for reasons of efficiency. [5] A later Plan 9 implementation did implement modules as user-level processes. [20]

Implementations

STREAMS has mostly been used in the System V Unix world; however, other implementations exist:

Linux does not include STREAMS functionality without third-party add-ons. Caldera had "pushed" for STREAMS to be included in Linux ca. 1998, to support its Netware for Linux, but it was rejected outright by the Linux kernel developers on technical grounds (mainly performance). [25] The compatibility layers in Linux for other operating systems convert STREAMS operations into sockets as early as possible. [26] The implementation used by Caldera was "LiS", by a company called GCOM; it later figured in the legal battles by Caldera's successor, the SCO Group, against Linux, with SCO claiming that Linux with STREAMS infringed what it believed to be its copyrights to System V. [25]

Notes

  1. ↑ ( Goodheart 1994 , pp. 51–53, 403–527)
  2. ↑ ( Goodheart 1994 , pp. 52–53)
  3. 1 2 ( Goodheart 1994 , p. 17)
  4. ↑ ( Goodheart 1994 , p. 51)
  5. 1 2 3 ( Ritchie 1984 )
  6. ↑ ( Goodheart 1994 )
  7. ↑ Eric S. Raymond (2003). "Chapter 7. Multiprogramming". The Art of Unix Programming. Addison-Wesley.
  8. 1 2 ( DLPI & 2.0.0 )
  9. ↑ ( NPI & 2.0.0 )
  10. ↑ ( TPI & 1.5 )
  11. ↑ ( TPI & 2.0.0 )
  12. ↑ ( APLI 1990 )
  13. ↑ ( XAP 1993 )
  14. ↑ "Base Specifications, Issue 7, 2013 Edition, Section B.2.6 STREAMS". The Open Group. Retrieved 9 March 2015.
  15. ↑ "The Austin Common Standards Revision Group". The Open Group. Retrieved 9 March 2015.
  16. ↑ "The Open Group Base Specifications Issue 7, Codes". The Open Group. Retrieved 9 March 2015.
  17. ↑ Pike, Rob (1984). "The Blit: A Multiplexed Graphics Terminal". AT&T Bell Laboratories Technical Journal. 63 (8): 1607–1631. doi:10.1002/j.1538-7305.1984.tb00056.x. S2CID   34062559.
  18. 1 2 Bach, Maurice J. (1986). The Design of the UNIX Operating System . Prentice Hall. Bibcode:1986duos.book.....B. ISBN   9780132017992.
  19. ↑ See: putmsg  – System Interfaces Reference, The Single UNIX Specification , Version 3 from The Open Group, and getmsg  – System Interfaces Reference, The Single UNIX Specification , Version 3 from The Open Group.
  20. 1 2 Presotto, David L. (1990). Multiprocessor streams for Plan 9. Proc. UKUUG Summer Conf. CiteSeerX   10.1.1.42.1172 .
  21. ↑ Newton, Mark. "FreeBSD SysVR4 Emulation". Mark Newton's FreeBSD pages.
  22. ↑ ( Barr 2001 )
  23. ↑ ( Valentine 2001 )
  24. ↑ "Alpha Micro Phun Machine: Primer to AMOS" . Retrieved 5 March 2022.
  25. 1 2 "STREAMS, LiS, and Caldera's Netware for Linux - Updated". Groklaw . 3 July 2006. Retrieved 14 July 2022.
  26. ↑ Alan Cox, Streams and Linux, Linux Kernel Mailing List, 28 June 1998

Related Research Articles

Berkeley sockets is an application programming interface (API) for Internet sockets and Unix domain sockets, used for inter-process communication (IPC). It is commonly implemented as a library of linkable modules. It originated with the 4.2BSD Unix operating system, which was released in 1983.

In computing, the Windows Sockets API (WSA), later shortened to Winsock, is an application programming interface (API) that defines how Windows network application software should access network services, especially TCP/IP. It defines a standard interface between a Windows TCP/IP client application and the underlying TCP/IP protocol stack. The nomenclature is based on the Berkeley sockets API used in BSD for communications between programs.

Open Transport was the name given by Apple Inc. to its implementation of the Unix-originated System V STREAMS networking stack. Based on code licensed from Mentat's Portable Streams product, Open Transport was built to provide the classic Mac OS with a modern TCP/IP implementation, replacing MacTCP. Apple also added its own implementation of AppleTalk to the stack to support legacy networks.

IPX/SPX stands for Internetwork Packet Exchange/Sequenced Packet Exchange. IPX and SPX are networking protocols used initially on networks using the Novell NetWare operating systems. They also became widely used on networks deploying Microsoft Windows LANS, as they replaced NetWare LANS, but are no longer widely used. IPX/SPX was also widely used prior to and up to Windows XP, which supported the protocols, while later Windows versions do not, and TCP/IP took over for networking.

In computer networking, the Transport Layer Interface (TLI) was the networking API provided by AT&T UNIX System V Release 3 (SVR3) in 1987 and continued into Release 4 (SVR4). TLI was the System V counterpart to the BSD sockets programming interface, which was also provided in UNIX System V Release 4 (SVR4). TLI was later standardized as XTI, the X/Open Transport Interface.

<span class="mw-page-title-main">Pseudoterminal</span>

In some operating systems, including Unix and Linux, a pseudoterminal, pseudotty, or PTY is a pair of pseudo-device endpoints (files) which establish asynchronous, bidirectional communication (IPC) channel between two or more processes. The master provides means by which a terminal emulator process controls the slave. The slave emulates a hardware text terminal device. PTY are similar to bidirectional pipes.

TCP offload engine (TOE) is a technology used in some network interface cards (NIC) to offload processing of the entire TCP/IP stack to the network controller. It is primarily used with high-speed network interfaces, such as gigabit Ethernet and 10 Gigabit Ethernet, where processing overhead of the network stack becomes significant. TOEs are often used as a way to reduce the overhead associated with Internet Protocol (IP) storage protocols such as iSCSI and Network File System (NFS).

OpenVPN is a virtual private network (VPN) system that implements techniques to create secure point-to-point or site-to-site connections in routed or bridged configurations and remote access facilities. It implements both client and server applications.

<span class="mw-page-title-main">Linux kernel interfaces</span> An overview and comparison of the Linux kernal APIs and ABIs.

The Linux kernel provides multiple interfaces to user-space and kernel-mode code that are used for varying purposes and that have varying properties by design. There are two types of application programming interface (API) in the Linux kernel:

  1. the "kernel–user space" API; and
  2. the "kernel internal" API.

In computing, ioctl is a system call for device-specific input/output operations and other operations which cannot be expressed by regular file semantics. It takes a parameter specifying a request code; the effect of a call depends completely on the request code. Request codes are often device-specific. For instance, a CD-ROM device driver which can instruct a physical device to eject a disc would provide an ioctl request code to do so. Device-independent request codes are sometimes used to give userspace access to kernel functions which are only used by core system software or still under development.

Netlink is a socket family used for inter-process communication (IPC) between both the kernel and userspace processes, and between different userspace processes, in a way similar to the Unix domain sockets available on certain Unix-like operating systems, including its original incarnation as a Linux kernel interface, as well as in the form of a later implementation on FreeBSD. Similarly to the Unix domain sockets, and unlike INET sockets, Netlink communication cannot traverse host boundaries. However, while the Unix domain sockets use the file system namespace, Netlink sockets are usually addressed by process identifiers (PIDs).

In computer networking, TUN and TAP are kernel virtual network devices. Being network devices supported entirely in software, they differ from ordinary network devices which are backed by physical network adapters.

A network socket is a software structure within a network node of a computer network that serves as an endpoint for sending and receiving data across the network. The structure and properties of a socket are defined by an application programming interface (API) for the networking architecture. Sockets are created only during the lifetime of a process of an application running in the node.

A line discipline (LDISC) is a layer in the terminal subsystem in some Unix-like systems. The terminal subsystem consists of three layers: the upper layer to provide the character device interface, the lower hardware driver to communicate with the hardware or pseudo terminal, and the middle line discipline to implement behavior common to terminal devices.

<span class="mw-page-title-main">Network block device</span> Network storage protocol

On Linux, network block device (NBD) is a network protocol that can be used to forward a block device from one machine to a second machine. As an example, a local machine can access a hard disk drive that is attached to another computer.

In Unix-like operating systems, a device file or special file is an interface to a device driver that appears in a file system as if it were an ordinary file. There are also special files in DOS, OS/2, and Windows. These special files allow an application program to interact with a device by using its device driver via standard input/output system calls. Using standard system calls simplifies many programming tasks, and leads to consistent user-space I/O mechanisms regardless of device features and functions.

<span class="mw-page-title-main">Unix</span> Family of computer operating systems

Unix is a family of multitasking, multi-user computer operating systems that derive from the original AT&T Unix, whose development started in 1969 at the Bell Labs research center by Ken Thompson, Dennis Ritchie, and others.

The X/Open Transport Interface (XTI) is an Open Group specification for network application programming present in UNIX System V operating systems. It provides OSI transport layer services with protocol independence. Although Open Group considers this specification withdrawn, an implementation is part of the standard programming interfaces on modern UNIX System V operating systems where it is implemented using the STREAMS character input/output mechanism.

The POSIX terminal interface is the generalized abstraction, comprising both an application programming interface for programs, and a set of behavioural expectations for users of a terminal, as defined by the POSIX standard and the Single Unix Specification. It is a historical development from the terminal interfaces of BSD version 4 and Seventh Edition Unix.

The Stream Control Transmission Protocol (SCTP) is a computer networking communications protocol in the transport layer of the Internet protocol suite. Originally intended for Signaling System 7 (SS7) message transport in telecommunication, the protocol provides the message-oriented feature of the User Datagram Protocol (UDP), while ensuring reliable, in-sequence transport of messages with congestion control like the Transmission Control Protocol (TCP). Unlike UDP and TCP, the protocol supports multihoming and redundant paths to increase resilience and reliability.

References

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.