Transaction Processing Facility

Last updated
z/TPF
IBM logo.svg
Developer IBM
Written in z/Architecture Assembly language, C, C++
OS familyz/Architecture assembly language (z/TPF), ESA/390 assembly language (TPF4)
Working stateCurrent
Source model Closed source (Source code is available to licensed users with restrictions)
Initial release1979;45 years ago (1979)
Latest release 1.1.0.2023 [1]
Platforms IBM System z (z/TPF), ESA/390 (TPF4)
Kernel type Real-time
Default
user interface 		3215 3270
License Proprietary monthly license charge (MLC)
Official website z/TPF Product Page

Transaction Processing Facility (TPF) [2] is an IBM real-time operating system for mainframe computers descended from the IBM System/360 family, including zSeries and System z9.

Contents

TPF delivers fast, high-volume, high-throughput transaction processing, handling large, continuous loads of essentially simple transactions across large, geographically dispersed networks.

While there are other industrial-strength transaction processing systems, notably IBM's own CICS and IMS, TPF's specialty is extreme volume, large numbers of concurrent users, and very fast response times. For example, it handles VISA credit card transaction processing during the peak holiday shopping season. [3] [2]

The TPF passenger reservation application PARS, or its international version IPARS, is used by many airlines. PARS is an application program; TPF is an operating system.

One of TPF's major optional components is a high performance, specialized database facility called TPF Database Facility (TPFDF). [4]

A close cousin of TPF, the transaction monitor ALCS, was developed by IBM to integrate TPF services into the more common mainframe operating system MVS, now z/OS.

History

TPF evolved from the Airlines Control Program (ACP), a free package developed in the mid-1960s by IBM in association with major North American and European airlines. In 1979, IBM introduced TPF as a replacement for ACP — and as a priced software product. The new name suggests its greater scope and evolution into non-airline related entities.

TPF was traditionally an IBM System/370 assembly language environment for performance reasons, and many TPF assembler applications persist. However, more recent versions of TPF encourage the use of C. Another programming language called SabreTalk was born and died on TPF.

IBM announced the delivery of the current release of TPF, dubbed z/TPF V1.1, in September 2005. Most significantly, z/TPF adds 64-bit addressing and mandates use of the 64-bit GNU development tools. [5] [6]

The GCC compiler and the DIGNUS Systems/C++ and Systems/C are the only supported compilers for z/TPF. The Dignus compilers offer reduced source code changes when moving from TPF 4.1 to z/TPF.

Users

Current users include Sabre (reservations), VISA Inc. (authorizations), American Airlines, [7] American Express (authorizations), DXC Technology SHARES (reservations), Amtrak, Marriott International, Travelport (Galileo, Apollo, Worldspan), Citibank, Trenitalia (reservations), Delta Air Lines (reservations and operations) and Japan Airlines. [8]

Operating environment

Tightly coupled

Although IBM's 3083 was aimed at running TPF on a "fast... uniprocessor", [9] TPF is capable of running on a multiprocessor, that is, on systems in which there is more than one CPU. Within the LPAR, the CPUs are referred to as instruction streams or simply I-streams. When running on a LPAR with more than one I-stream, TPF is said to be running tightly coupled. TPF adheres to SMP concepts; no concept of NUMA-based distinctions between memory addresses exist.

The depth of the CPU ready list is measured as any incoming transaction is received, and queued for the I-stream with the lowest demand, thus maintaining continuous load balancing among available processors. In cases where loosely coupled configurations are populated by multiprocessor CPCs (Central Processing Complex, i.e. the physical machine packaged in one system cabinet), SMP takes place within the CPC as described here, whereas sharing of inter-CPC resources takes place as described under Loosely coupled, below.

In the TPF architecture, all memory (except for a 4KB-sized prefix area) is shared among all I-streams. In instances where memory-resident data must or should be kept separated by I-stream, the programmer typically allocates a storage area into a number of subsections equal to the number of I-streams, then accesses the desired I-stream associated area by taking the base address of the allocated area, and adding to it the product of the I-stream relative number times the size of each subsection.

Loosely coupled

TPF is capable of supporting multiple mainframes (of any size themselves — be it single I-stream to multiple I-stream) connecting to and operating on a common database. Currently, 32 IBM mainframes may share the TPF database; if such a system were in operation, it would be called 32-way loosely coupled. The simplest loosely coupled system would be two IBM mainframes sharing one DASD (Direct Access Storage Device). In this case, the control program would be equally loaded into memory and each program or record on DASD could be potentially accessed by either mainframe.

In order to serialize accesses between data records on a loosely coupled system, a practice known as record locking must be used. This means that when one mainframe processor obtains a hold on a record, the mechanism must prevent all other processors from obtaining the same hold and communicate to the requesting processors that they are waiting. Within any tightly coupled system, this is easy to manage between I-streams via the use of the Record Hold Table. However, when the lock is obtained offboard of the TPF processor in the DASD control unit, an external process must be used. Historically, the record locking was accomplished in the DASD control unit via an RPQ known as LLF (Limited Locking Facility) and later ELLF (extended). LLF and ELLF were both replaced by the Multipathing Lock Facility (MPLF). To run clustered (loosely coupled) z/TPF requires either MPLF in all disk control units or an alternative locking device called a Coupling Facility. [10] [11]

Processor shared records

Records that absolutely must be managed by a record locking process are those which are processor shared. In TPF, most record accesses are done by using record type and ordinal. Given a record type in the TPF system of 'FRED' with 100 records or ordinals, in a processor shared scheme, record type 'FRED' ordinal '5' would resolve to exactly the same file address on DASD — necessitating the use of a record locking mechanism.

All processor shared records on a TPF system will be accessed via the same file address which will resolve to the same location.

Processor unique records

A processor unique record is one that is defined such that each processor expected to be in the loosely coupled complex has a record type of 'FRED' and perhaps 100 ordinals. However, if a user on any 2 or more processors examines the file address that record type 'FRED', ordinal '5' resolves to, they will note a different physical address is used.

TPF attributes

What TPF is not

TPF is not a general-purpose operating system. TPF's specialized role is to process transaction input messages, then return output messages on a 1:1 basis at extremely high volume with short maximum elapsed time limits.

TPF has no built-in graphical user interface functionality, and TPF has never offered direct graphical display facilities: to implement it on the host would be considered an unnecessary and potentially harmful diversion of real-time system resources. TPF's user interface is command-line driven with simple text display terminals that scroll upward, and there are no mouse-driven cursors, windows, or icons on a TPF Prime CRAS [12] (Computer room agent set— which is best thought of as the "operator's console"). Character messages are intended to be the mode of communications with human users. All work is accomplished via the use of the command line, similar to UNIX without X. There are several products available which connect to Prime CRAS and provide graphical interface functions to the TPF operator, such as TPF Operations Server. [13] Graphical interfaces for end users, if desired, must be provided by external systems. Such systems perform analysis on character content (see Screen scrape) and convert the message to/from the desired graphical form, depending on its context.

Being a specialized purpose operating system, TPF does not host a compiler/assembler, text editor, nor implement the concept of a desktop as one might expect to find in a general-purpose operating system. TPF application source code is commonly stored in external systems, and likewise built "offline". Starting with z/TPF 1.1, Linux is the supported build platform; executable programs intended for z/TPF operation must observe the ELF format for s390x-ibm-linux.

Using TPF requires a knowledge of its Command Guide [14] since there is no support for an online command "directory" or "man"/help facility to which users might be accustomed. Commands created and shipped by IBM for the system administration of TPF are called "functional messages"—commonly referred to as "Z-messages", as they are all prefixed with the letter "Z". Other letters are reserved so that customers may write their own commands.

TPF implements debugging in a distributed client-server mode, which is necessary because of the system's headless, multi-processing nature: pausing the entire system in order to trap a single task would be highly counterproductive. Debugger packages have been developed by third party vendors who took very different approaches to the "break/continue" operations required at the TPF host, implementing unique communications protocols used in traffic between the human developer running the debugger client and the server-side debug controller, as well as the form and function of debugger program operations at the client side. Two examples of third party debugger packages are Step by Step Trace from Bedford Associates [15] and CMSTPF, TPF/GI, and zTPFGI, all from TPF Software, Inc. [16] Neither package is wholly compatible with the other, nor with IBM's own offering. IBM's debugging client offering is packaged in an IDE called IBM TPF Toolkit. [17]

What TPF is

TPF is highly optimized to permit messages from the supported network to either be switched out to another location, routed to an application (specific set of programs) or to permit extremely efficient accesses to database records.

Data records

Historically, all data on the TPF system had to fit in fixed record (and memory block) sizes of 381, 1055 and 4K bytes. This was due in part to the physical record sizes of blocks located on DASD. Much overhead was saved by freeing up any part of the operating system from breaking large data entities into smaller ones during file operations, and reassembling the same during read operations. Since IBM hardware does I/O via the use of channels and channel programs, TPF would generate very small and efficient channel programs to do its I/O — all in the name of speed. Since the early days also placed a premium on the size of storage media — be it memory or disk, TPF applications evolved into doing very powerful things while using very little resource.

Today, much of these limitations are removed. In fact, only because of legacy support are smaller-than-4K DASD records still used. With the advances made in DASD technology, a read/write of a 4K record is just as efficient as a 1055 byte record. The same advances have increased the capacity of each device so that there is no longer a premium placed on the ability to pack data into the smallest model as possible.

Programs and residency

TPF also had its program segments allocated as 381, 1055 and 4K byte-sized records at different points in its history. Each segment consisted of a single record; with a typically comprehensive application requiring perhaps tens or even hundreds of segments. For the first forty years of TPF's history, these segments were never link-edited. Instead, the relocatable object code (direct output from the assembler) was laid out in memory, had its internally (self-referential) relocatable symbols resolved, then the entire image was written to file for later loading into the system. This created a challenging programming environment in which segments related to one another could not directly address each other, with control transfer between them implemented as the ENTER/BACKsystem service.

In ACP/TPF's earliest days (circa 1965), memory space was severely limited, which gave rise to a distinction between file-resident and core-resident programs—only the most frequently used application programs were written into memory and never removed (core-residency); the rest were stored on file and read in on demand, with their backing memory buffers released post-execution.

The introduction of C language to TPF at version 3.0 was first implemented conformant to segment conventions, including the absence of linkage editing. This scheme quickly demonstrated itself to be impractical for anything other than the simplest of C programs. At TPF 4.1, truly and fully linked load modules were introduced to TPF. These were compiled with the z/OS C/C++ compiler using TPF-specific header files and linked with IEWL, resulting in a z/OS-conformant load module, which in no manner could be considered a traditional TPF segment. The TPF loader was extended to read the z/OS-unique load module file format, then lay out file-resident load modules' sections into memory; meanwhile, assembly language programs remained confined to TPF's segment model, creating an obvious disparity between applications written in assembler and those written in higher level languages (HLL).

At z/TPF 1.1, all source language types were conceptually unified and fully link-edited to conform to the ELF specification. The segment concept became obsolete, meaning that any program written in any source language—including Assembler—may now be of any size. Furthermore, external references became possible, and separate source code programs that had once been segments could now be directly linked together into a shared object. A value point is that critical legacy applications can benefit from improved efficiency through simple repackaging—calls made between members of a single shared object module now have a much shorter pathlength at run time as compared to calling the system's ENTER/BACK service. Members of the same shared object may now share writeable data regions directly thanks to copy-on-write functionality also introduced at z/TPF 1.1; which coincidentally reinforces TPF's reentrancy requirements.

The concepts of file- and memory- residency were also made obsolete, due to a z/TPF design point which sought to have all programs resident in memory at all times.

Since z/TPF had to maintain a call stack for high-level language programs, which gave HLL programs the ability to benefit from stack-based memory allocation, it was deemed beneficial to extend the call stack to assembly language programs on an optional basis, which can ease memory pressure and ease recursive programming.

All z/TPF executable programs are now packaged as ELF shared objects.

Memory usage

Historically and in step with the previous, core blocks— memory— were also 381, 1055 and 4 K bytes in size. Since ALL memory blocks had to be of this size, most of the overhead for obtaining memory found in other systems was discarded. The programmer merely needed to decide what size block would fit the need and ask for it. TPF would maintain a list of blocks in use and simply hand out the first block on the available list.

Physical memory was divided into sections reserved for each size so a 1055 byte block always came from a section and returned there, the only overhead needed was to add its address to the appropriate physical block table's list. No compaction or data collection was required.

As applications got more advanced demands for memory increased, and once C became available memory chunks of indeterminate or large size were required. This gave rise to the use of heap storage and some memory management routines. To ease the overhead, TPF memory was broken into frames— 4 KB in size (1 MB with z/TPF). If an application needs a certain number of bytes, the number of contiguous frames required to fill that need are granted.

Related Research Articles

IBM mainframes are large computer systems produced by IBM since 1952. During the 1960s and 1970s, IBM dominated the computer market with the 7000 series and the later System/360, followed by the System/370. Current mainframe computers in IBM's line of business computers are developments of the basic design of the System/360.

<span class="mw-page-title-main">MVS</span> Operating system for IBM mainframes

Multiple Virtual Storage, more commonly called MVS, is the most commonly used operating system on the System/370, System/390 and IBM Z IBM mainframe computers. IBM developed MVS, along with OS/VS1 and SVS, as a successor to OS/360. It is unrelated to IBM's other mainframe operating system lines, e.g., VSE, VM, TPF.

<span class="mw-page-title-main">Mainframe computer</span> Large computer

A mainframe computer, informally called a mainframe or big iron, is a computer used primarily by large organizations for critical applications like bulk data processing for tasks such as censuses, industry and consumer statistics, enterprise resource planning, and large-scale transaction processing. A mainframe computer is large but not as large as a supercomputer and has more processing power than some other classes of computers, such as minicomputers, servers, workstations, and personal computers. Most large-scale computer-system architectures were established in the 1960s, but they continue to evolve. Mainframe computers are often used as servers.

Computerized batch processing is a method of running software programs called jobs in batches automatically. While users are required to submit the jobs, no other interaction by the user is required to process the batch. Batches may automatically be run at scheduled times as well as being run contingent on the availability of computer resources.

In computing, a core dump, memory dump, crash dump, storage dump, system dump, or ABEND dump consists of the recorded state of the working memory of a computer program at a specific time, generally when the program has crashed or otherwise terminated abnormally. In practice, other key pieces of program state are usually dumped at the same time, including the processor registers, which may include the program counter and stack pointer, memory management information, and other processor and operating system flags and information. A snapshot dump is a memory dump requested by the computer operator or by the running program, after which the program is able to continue. Core dumps are often used to assist in diagnosing and debugging errors in computer programs.

A direct-access storage device (DASD) is a secondary storage device in which "each physical record has a discrete location and a unique address". The term was coined by IBM to describe devices that allowed random access to data, the main examples being drum memory and hard disk drives. Later, optical disc drives and flash memory units are also classified as DASD.

<span class="mw-page-title-main">VM (operating system)</span> Family of IBM operating systems

VM is a family of IBM virtual machine operating systems used on IBM mainframes System/370, System/390, zSeries, System z and compatible systems, including the Hercules emulator for personal computers.

Virtual Storage Access Method (VSAM) is an IBM direct-access storage device (DASD) file storage access method, first used in the OS/VS1, OS/VS2 Release 1 (SVS) and Release 2 (MVS) operating systems, later used throughout the Multiple Virtual Storage (MVS) architecture and now in z/OS. Originally a record-oriented filesystem, VSAM comprises four data set organizations: key-sequenced (KSDS), relative record (RRDS), entry-sequenced (ESDS) and linear (LDS). The KSDS, RRDS and ESDS organizations contain records, while the LDS organization contains a sequence of pages with no intrinsic record structure, for use as a memory-mapped file.

<span class="mw-page-title-main">CICS</span> IBM mainframe transaction monitor

IBM CICS is a family of mixed-language application servers that provide online transaction management and connectivity for applications on IBM mainframe systems under z/OS and z/VSE.

In computer systems a loader is the part of an operating system that is responsible for loading programs and libraries. It is one of the essential stages in the process of starting a program, as it places programs into memory and prepares them for execution. Loading a program involves either memory-mapping or copying the contents of the executable file containing the program instructions into memory, and then carrying out other required preparatory tasks to prepare the executable for running. Once loading is complete, the operating system starts the program by passing control to the loaded program code.

In the context of IBM mainframe computers in the S/360 line, a data set or dataset is a computer file having a record organization. Use of this term began with, e.g., DOS/360, OS/360, and is still used by their successors, including the current z/OS. Documentation for these systems historically preferred this term rather than file.

This article discusses support programs included in or available for OS/360 and successors. IBM categorizes some of these programs as utilities and others as service aids; the boundaries are not always consistent or obvious. Many, but not all, of these programs match the types in utility software.

z/Architecture, initially and briefly called ESA Modal Extensions (ESAME), is IBM's 64-bit complex instruction set computer (CISC) instruction set architecture, implemented by its mainframe computers. IBM introduced its first z/Architecture-based system, the z900, in late 2000. Later z/Architecture systems include the IBM z800, z990, z890, System z9, System z10, zEnterprise 196, zEnterprise 114, zEC12, zBC12, z13, z14, z15 and z16.

In computing, channel I/O is a high-performance input/output (I/O) architecture that is implemented in various forms on a number of computer architectures, especially on mainframe computers. In the past, channels were generally implemented with custom devices, variously named channel, I/O processor, I/O controller, I/O synchronizer, or DMA controller.

Count key data (CKD) is a direct-access storage device (DASD) data recording format introduced in 1964, by IBM with its IBM System/360 and still being emulated on IBM mainframes. It is a self-defining format with each data record represented by a Count Area that identifies the record and provides the number of bytes in an optional Key Area and an optional Data Area. This is in contrast to devices using fixed sector size or a separate format track.

An access method is a function of a mainframe operating system that enables access to data on disk, tape or other external devices. Access methods were present in several mainframe operating systems since the late 1950s, under a variety of names; the name access method was introduced in 1963 in the IBM OS/360 operating system. Access methods provide an application programming interface (API) for programmers to transfer data to or from device, and could be compared to device drivers in non-mainframe operating systems, but typically provide a greater level of functionality.

A Supervisor Call instruction (SVC) is a hardware instruction used by the System/360 family of IBM mainframe computers up to contemporary zSeries, the Amdahl 470V/5, 470V/6, 470V/7, 470V/8, 580, 5880, 5990M, and 5990A, and others; Univac 90/60, 90/70 and 90/80, and possibly others; the Fujitsu M180 (UP) and M200 (MP), and others; and is also used in the Hercules open source mainframe emulation software. It causes an interrupt to request a service from the operating system. The system routine providing the service is called an SVC routine. SVC is a system call.

The history of IBM mainframe operating systems is significant within the history of mainframe operating systems, because of IBM's long-standing position as the world's largest hardware supplier of mainframe computers. IBM mainframes run operating systems supplied by IBM and by third parties.

<span class="mw-page-title-main">OS/360 and successors</span> Operating system for IBM S/360 and later mainframes

OS/360, officially known as IBM System/360 Operating System, is a discontinued batch processing operating system developed by IBM for their then-new System/360 mainframe computer, announced in 1964; it was influenced by the earlier IBSYS/IBJOB and Input/Output Control System (IOCS) packages for the IBM 7090/7094 and even more so by the PR155 Operating System for the IBM 1410/7010 processors. It was one of the earliest operating systems to require the computer hardware to include at least one direct access storage device.

<span class="mw-page-title-main">Distributed Data Management Architecture</span> Open, published architecture for creating, managing and accessing data on a remote computer

Distributed Data Management Architecture (DDM) is IBM's open, published software architecture for creating, managing and accessing data on a remote computer. DDM was initially designed to support record-oriented files; it was extended to support hierarchical directories, stream-oriented files, queues, and system command processing; it was further extended to be the base of IBM's Distributed Relational Database Architecture (DRDA); and finally, it was extended to support data description and conversion. Defined in the period from 1980 to 1993, DDM specifies necessary components, messages, and protocols, all based on the principles of object-orientation. DDM is not, in itself, a piece of software; the implementation of DDM takes the form of client and server products. As an open architecture, products can implement subsets of DDM architecture and products can extend DDM to meet additional requirements. Taken together, DDM products implement a distributed file system.

References

  1. ↑ "z/TPF, z/TPFDF, TPF Operations Server, and TPF Toolkit 4.6 for 2023". IBM.
  2. 1 2 Steve Lohr (October 4, 2004). "IBM Updates Old Workhorse to Use Linux". The New York Times .
  3. ↑ Michelle Louzoun (August 24, 1987). "Visa Is Everywhere It Wants To Be". InformationWeek . p. 19.
  4. ↑ IBM Corporation. "TPF Database Facility (TPFDF)". z/Transaction Processing Facility. Retrieved November 11, 2016.
  5. ↑ "IBM bolsters its mainframe platform". Computerworld .
  6. ↑ Jennifer Mears. "IBM pumps up Linux virtual machines on mainframe OS". Computerworld .
  7. ↑ "TPF Users Group, Job Corner". Archived from the original on 2000-01-15.
  8. ↑ "IBM News room - 2008-04-14 Japan Airlines International to Upgrade Reservation and Ticketing System With IBM Mainframe - United States". 03.ibm.com. 2008-04-14. Archived from the original on September 24, 2009. Retrieved 2017-03-15.
  9. ↑ Anne & Lynn Wheeler. "IBM 9020 computers used by FAA (was Re: EPO stories (was: HELP IT'S HOT!!!!!))". Newsgroup:  alt.folklore.computers.
  10. ↑ "IBM Knowledge Center". Publib.boulder.ibm.com. 2014-10-24. Retrieved 2017-03-15.
  11. ↑ "IBM z/Transaction Processing Facility Enterprise Edition V1.1 hardware requirements - United States". www-01.ibm.com. Archived from the original on 7 October 2012. Retrieved 17 January 2022.
  12. ↑ IBM Corporation (19 Apr 2018). "z/TPF Glossary". IBM . Retrieved 10 May 2018.
  13. ↑ IBM Corporation (19 April 2018). "IBM TPF Operations Server". IBM . Retrieved 10 May 2018.
  14. ↑ IBM Corporation (29 January 2019). "z/TPF Operations Command Guide". IBM .
  15. ↑ Bedford Associates. "Bedford Associates, Inc" . Retrieved October 17, 2012.
  16. ↑ TPF Software. "TPF Software, Inc" . Retrieved October 17, 2012.
  17. ↑ IBM Corporation (Dec 2017). "IBM TPF Toolkit Overview". IBM . Retrieved 10 May 2018.

Bibliography

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.