Count key data (CKD) is a direct-access storage device (DASD)[lower-alpha 1] data recording format introduced in 1964, by IBM with its IBM System/360 and still being emulated on IBM mainframes. It is a self-defining format with each data record represented by a Count Area that identifies the record and provides the number of bytes in an optional Key Area and an optional Data Area. This is in contrast to devices using fixed sector size or a separate format track.
Count key data (CKD) also refers to the set of channel commands (collectively Channel Command Words, CCWs) that are generated by an IBM mainframe for execution by a DASD subsystem employing the CKD recording format.[1] The initial set of CKD CCWs, introduced in 1964, was substantially enhanced and improved into the 1990s.
CKD track format
Block diagram of count key data track format used on IBM mainframe computers beginning with S/360 shipment in 1965
The reason for CKD track format is to allow data field lengths to vary, each recorded block of data on a DASD track, called a record has an associated count field which identifies the record and indicates the size of the key, if used (user-defined up to 255 bytes), and the size of the data area, if used.[2][lower-alpha 2] The count field has the identification of the record[lower-alpha 3] in cylinder-head-record format, the length of the key, and the length of the data. The key may be omitted or consist of a string of characters.
"The beginning of a track is signalled when the index marker (index point) is detected. ... The marker is automatically recognized by a special sensing device."[3]:5 Following the index marker is the home address, which indicates the location of this track on the disk, and contains other control information internal to the control unit. A fixed-length gap follows the home address. Next, each track contains a Record 0 (R0), the track descriptor record, which is "designed to enable the entire content of a track to be moved to alternate tracks if a portion of the primary track becomes defective."[3]:7 Following R0 are the data records, separated by gaps.[3]:9
Because of the gaps and other information, the recorded space is larger than that required for just the count data, key data, or user data. IBM provides a "reference card" for each device, which can be used to compute the number of records per track for various key and data field sizes, and to optimize the capacity of the device.[4] Later, programs were written to do these calculations. Because records are normally not split between tracks, specification of an incorrect record size create problems.
Most often, the key is omitted and the record is located sequentially or by direct cylinder-head-record addressing. If it is present, the key is any data used to find the record, usually using the Search Key Equal or Search Key High or Equal CCW. The key (and hence the record) is locatable via hardware commands.[5][pageneeded] Since the introduction of IBM's System/360 in 1964, nearly all IBM large and intermediate system DASDs have used the count key data record format.[6][pageneeded]
The advantages of count key data record format are:
The record size can be exactly matched to the application data block sizes
CPU and memory requirements can be reduced by exploiting search-key commands.
IBM CKD subsystems initially operated synchronously with the system channel and can process information in the gaps between the various fields, thereby achieving higher performance by avoiding the redundant transfer of information to the host.[6] Both synchronous and asynchronous operations are supported on later subsystems.[6]
Reduced CPU and memory prices and higher device and interface speeds have somewhat nullified the advantages of CKD, and it is retained only because IBM's flagship operating system z/OS does not support sector-oriented interfaces.
Originally CKD records had a one-to-one correspondence to a physical track of a DASD device; however over time the records have become more and more virtualized such that in modern IBM mainframes there is no longer a direct correspondence between a CKD record ID and the physical layout of a track.
IBM's CKD DASD subsystems
Packaging
IBM S/360 & S/370 Input/Output operations for CKD DASD showing channel, storage control unit and DASD device
Initially there was a high degree of correspondence between the logical view of DASD accesses and the actual hardware, as shown in the illustration. Three digit labels were typically affixed[lower-alpha 4] to identify the address of channel, control unit and device.
On low end systems the Channel and the Control Unit were frequently physically integrated but remained logically separate. IBM's New Attachment Strategy[7] beginning with the 3830 Model 2 in 1972 physically separated the SCU into two physical entities, a director and a controller while keeping them logically the same. The controller handles the CKD track formatting and is packaged with the first drive or drives in a string of drives and having a model number with the letter "A" as a prefix, an "A-Unit" (or "A-Box") as in 3350 Model A2 containing a controller and two DASDs. DASD without a controller, that is B-Units, have a "B" prefix in their model number.
CKD subsystems[lower-alpha 5] and directors were offered by IBM and plug compatible competitors until at least 1996 (2301 to 3390 Model 9);[8] in total 22 unique[lower-alpha 6] DASD offered by IBM configured in at least 35 different subsystem configurations. Plug-compatible offered many of the same DASD including 4 CKD subsystems featuring unique DASD.[lower-alpha 7]
Access to specific classes of I/O devices by an IBM mainframe is under the control of Channel Command Words (CCWs), some of which are generic (e.g. No Operation) but many of which are specific to the type of I/O device (e.g. Read Backwards for a tape drive). The group of CCWs defined by IBM for DASD fall into five broad categories:
Control– control of the DASD including the path thereto
Sense– sense status of the DASD including the path thereto; some sense commands affect the status of the controller and DASD in a fashion more in keeping with a control command, e.g., RESERVE, RELEASE
Write– write information to the controller or DASD (which may be buffered or cached in the path)
Search[lower-alpha 8]– compare information from the CPU with information stored in the DASD; the Channel operates in the Write mode while the storage unit operates in the Read mode.
Read– read information from the DASD (which may be buffered or cached in the path)
CKD CCWs are the specific set of CCWs used to access CKD DASD subsystems. This is in contrast to fixed block architecture (FBA) CCWs which are used to access FBA DASD subsystems.
CKD DASD are addressed like other Input/Output devices; for System/360 and System/370 DASD are addressed directly, through channels and the associated control units[9][10][11] (SCU or Storage Control Unit), initially using three hexadecimal digits, one for channel and two for control unit and device, providing addressing for up to 16 channels, for up to 256 DASD access mechanisms/channel and 4,096 DASD addresses total. Modern IBM mainframes use four hexadecimal digits as an arbitrary subchannel number within a channel subsystem subset, whose definition includes the actual channels, control units and device, providing addressing for up to 65,536 DASD per channel subsystem subset. In practice, physical and design constraints of the channel and of the controllers limited the maximum number of attached DASD attachable to a system to a smaller amount than the number that could be addressed.
Initial CKD feature set
The initial feature set provided by IBM with its 1964 introduction of the CKD track format and associated CCWs included: .
Defective/Alternative Track– enables an alternate track to replace a defective track transparent to the access method in use.
Record overflow– AKA track overflow– The Write Special Count, Key and Data CCW can write a record segment that is logically part of a record segment on the next track. While an individual record segment can not exceed the maximum track length of a DASD[12] track, a sequence of record segments is not so limited.
Multitrack operations– specific CCWs can continue onto the next sequential head[12]
Command chaining– CCWs could be chained together to construct complex channel programs. The gaps in a CKD track format provided sufficient time between the commands so that all channel and SCU activity necessary to complete a command can be performed in the a gap between appropriate fields.[13] Such programs can search a large amount of information stored on a DASD, upon successful completion returning only the desired data and thereby freeing CPU resources for other activity.[14] This mode of operating synchronous to the gap was later enhanced by additional CCWs enabling a nonsynchronous mode of operation.
Channel switching– an SCU can be shared between channels– initially two channel switching was provided and it was expanded to up to eight channels in later SCUs. The channels can be on the same or different CPUS.
A Scan feature set was also provided but not continued into future CKD subsystems beyond the 2314.
MT = multitrack: when supported CCW will continue to operate on next heads in sequence to end of cylinder
‡ = TIC (Transfer In Channel) and other standard commands not shown.
† = code same as MT Off except as listed
1. File Scan Feature (9 CCWs) only available on 2841 for 2302, 2311 and 2321; they were not available on subsequent DASD controllers for DASD later than 2314.
2. Count is number of bytes in search argument, including mask bytes
The CCWs were initially executed by two types of SCU attached to the system's high speed Selector Channels. The 2820 SCU[16] controlled the 2301 Drum while the 2841 SCU[12] controlled combinations of the 2302 Disk Storage, 2311 Disk Drive, 2321 Data Cell and/or 7320 Drum Storage. IBM quickly replaced the 7320 with the faster and larger 2303.
The following example of a channel program[18] reads a disk record identified by a Key field. The track containing the record and the desired value of the key is known. The SCU will search the track to find the requested record. In this example <> indicate that the channel program contains the storage address of the specified field.
SEEK <cylinder/head number> SEARCH KEY EQUAL <key value> TIC *-8 Back to search if not equal READ DATA <buffer>
The TIC (transfer in channel) will cause the channel program to branch to the SEARCH command until a record with a matching key (or the end of the track) is encountered. When a record with a matching key is found the SCU will include Status Modifier in the channel status, causing the channel to skip the TIC CCW; thus the channel program will not branch and the channel will execute the READ command.
Block multiplexer channel enhancements
The block multiplexor channel was introduced beginning in 1971 on some high end System/360 systems along with the 2835 Control Unit and associated 2305 DASD,[19] This channel was then standard on IBM System/370 and subsequent mainframes; when contrasted to the prior Selector channel it offered performance improvements for high speed devices such as DASD, including:
Multiple Requesting
Allowed multiple channel programs,[lower-alpha 10] to be simultaneously active in the facility[19] as opposed to only one with a Selector channel. The actual number of subchannels provided depends upon the system model and its configuration.[20] Sometimes described as disconnected command chaining, the control unit could disconnect at various times during a chained set of CCWs, for example, disconnection for a Seek CCW, freeing the channel for another subchannel.
Command Retry
The channel and storage control under certain conditions can inter-operate to cause a CCW to be retried without an I/O interruption.[19] This procedure is initiated by the storage control and used to recover from correctable errors.
Rotational Position Sensing
Rotational position sensing (RPS) was implemented with two new CCWs, SET SECTOR and READ SECTOR enabled the channel to delay command chaining until the disk rotated to a specified angular track position. RPS permits channel disconnection during most of the rotational delay period and thus contributes to increased channel utilization. The control unit implements RPS by dividing each track into equal angular segments.[19]
Example Channel Program
The following example channel program will format a track with an R0 and three CKD records.[19]
SEEK <cylinder/head number> SET FILE MASK <allow write operations> SET SECTOR <sector number=0> WRITE R0 <cylinder/head/R0, key length=0, data length=6> WRITE CKD <cylinder/head/R1, key length, data length> WRITE CKD <cylinder/head/R2, key length, data length> WRITE CKD <cylinder/head/R3, key length, data length>
In this example the Record 0 conforms to IBM programming standards. With a block multiplexer channel the channel is free during the time the DASD is seeking and again while the disk rotates to beginning of the track. A selector channel would be busy for the entire duration of this sample program.
Defect skipping
Defect skipping allows data to be written before and after one of more surface defects[lower-alpha 11] allowing all of a track to be used except for that portion that has the defect. This also eliminates the time that was formerly required to seek to an alternate track.[21] Only a limited number of defects could be skipped so alternate tracks remained supported for those tracks with excess defects.
Defect skipping was introduced in 1974 with the 3340 attached via the 3830 Model 2 Storage Control Unit[21] or integrated attachments on small systems. Defect skipping was essentially a factory only feature until 1981 when CCWs for management along with associated utilities were released.[22]
Dynamic paths
First introduced with the 3380 DASD on the 3880 Storage Control Unit[23] in 1981 the feature was included with the later CKD DASD subsystems. The dynamic path selection function controls operation of the two controllers, including simultaneous data transfer over the two paths. When supported by the operating system, each controller can serve as an alternate path in the event the other controller is unavailable.[24]
Three additional commands, Set Path Group ID, Sense Path Group ID, and Suspend Multipath Reconnection, are used to support attachment of the 3380 Models having two controllers at the head of a string.[23]
The Set Path Group ID command, with the dynamic path selection (DPS) function, provides greater flexibility in operations on reserved devices. Once a path group for a device has been established, it may be accessed over any path which is a member of the group to which it is reserved. In addition, on 370-XA systems which set the multipath mode bit in the function control byte (byte 0) to a 1, block multiplex reconnections will occur on the first available path which is a member of the group over which the channel program was initiated (regardless of the reservation state of the device).[23]
If the controller designated in the I/O address is busy or disabled, the dynamic path selection allows an alternate path to the device to be established via another storage director and the other controller in the model AA.[24]
Nonsynchronous operation
Prior to the 1981 introduction of the 3880 director, CKD records were synchronously accessed, all activities required that one CCW be ended and the next initiated in the gaps between the CKD fields.[13] The gap size placed limitations on cable length but did provide for very high performance since complex chains of CCWs could be performed by the subsystem in real time without use of CPU memory or cycles.
Nonsynchronous operation provided by the Extended CKD ("ECKD") set of CCWs removed the gap timing constraint.[13] The five additional ECKD CCWs are Define Extent, Locate Record, Write Update Data, Write Update Key and Data, and Write CKD Next Track.[23]
In nonsynchronous operation, the transfer of data between the channel and the storage control is not synchronized with the transfer of data between the storage control and the device. Channel programs can be executed such that channel and storage control activities required to end execution of one command and advance to the next do not have to occur during the inter-record gap between two adjacent fields. An intermediate buffer in the storage control allows independent operations between the channel and the device. A major advantage of ECKDs is far longer cables; depending upon application it may improve performance.[13]
ECKD CCWs are supported on all subsequent CKD subsystems.
This example nonsynchronous channel program reads records R1 and R2 from track X'0E' in cylinder X'007F'. Both records have a key length of 8 and a data length of X'64' (10010) bytes.[23]
Define Extent <extent= X'007F 0000' through track X'0081 000E'> Locate Record <cylinder = X'007F', head = X'000E' Read Key and Data <key record = X'001038'> Read Data <record = X'001108'>
Caching
Caching first introduced in DASD CKD subsystems by Memorex[25] (1978) and StorageTek[lower-alpha 12] (1981) was subsequently introduced in late 1981 by IBM on the 3880 Model 13 for models of the 3380 with dynamic pathing.[lower-alpha 13]
The cache is dynamically managed by an algorithm; high activity data is accessed from the high-performance cache and low activity data is accessed from less-expensive DASD storage. A large memory in the Director, the cache, is divided into track slots that store data from the 3380 tracks. A smaller area is a directory that contains entries that allow data to be located in the cache.[26]
Caches were also provided on subsequently introduced storage controls.
Other extensions
Over time a number of path control, diagnostic and/or error recovery CCWs were implemented on one or more storage controls. For example:
Unconditional Reserve allowed the releasing a device reserved to another channel and reserving the device to the channel issuing the command.
Read Multiple Count Key Data could more efficiently read full tracks allowing for more efficient backups.
Beyond System/370
Reduced CPU and memory prices and higher device and interface speeds have somewhat nullified the advantages of CKD, and support continues by IBM to this date because its flagship operating system z/OS continues to use CKD CCWs for many functions.
Originally CKD records had a one-to-one correspondence to a physical track of a DASD device; however over time the records have become more and more virtualized such that in a modern IBM mainframe there is no longer a direct correspondence between the a CKD record ID and a physical layout of a track. An IBM mainframe constructs CKD track images in memory and executes the ECKD and CKD channel programs against the image. To bridge between the native fixed block sized disks and the variable length ECKD/CKD record format, the CKD track images in memory are mapped onto a series of fixed blocks suitable for transfer to and from an FBA disk subsystem.[27]
Of the 83 CKD CCWs implemented for System/360 and System/370 channels 56 are emulated on System/390 and later systems.[27]
↑ Because records are variable length and because the record number on the track need not be unique, the record number does not correspond to an angular displacement.
↑ In some early DASD the label was on a plug which allowed the address to be moved between DASD
↑ That is the combination of an SCU and one or more DASDs or an A-Unit with any attached B-Units.
↑ A unique combination of number of tracks and maximum track length. With this definition a double density DASD counts as a unique DASD.
↑ double density 2314, triple density 3330, double density 3350 and solid state disk
↑ Simultaneously announced was the 3880-11 using its cache in a paging mode 3350s as paging devices
Related Research Articles
The IBM System/360 (S/360) is a family of mainframe computer systems that was announced by IBM on April 7, 1964, and delivered between 1965 and 1978. It was the first family of computers designed to cover both commercial and scientific applications and a complete range of applications from small to large. The design distinguished between architecture and implementation, allowing IBM to release a suite of compatible designs at different prices. All but the only partially compatible Model 44 and the most expensive systems use microcode to implement the instruction set, featuring 8-bit byte addressing and binary, decimal and hexadecimal floating-point calculations.
A direct-access storage device (DASD) is a secondary storage device in which "each physical record has a discrete location and a unique address". The term was coined by IBM to describe devices that allowed random access to data, the main examples being drum memory and hard disk drives. Later, optical disc drives and flash memory units are also classified as DASD.
Disk formatting is the process of preparing a data storage device such as a hard disk drive, solid-state drive, floppy disk, memory card or USB flash drive for initial use. In some cases, the formatting operation may also create one or more new file systems. The first part of the formatting process that performs basic medium preparation is often referred to as "low-level formatting". Partitioning is the common term for the second part of the process, dividing the device into several sub-devices and, in some cases, writing information to the device allowing an operating system to be booted from it. The third part of the process, usually termed "high-level formatting" most often refers to the process of generating a new file system. In some operating systems all or parts of these three processes can be combined or repeated at different levels and the term "format" is understood to mean an operation in which a new disk medium is fully prepared to store files. Some formatting utilities allow distinguishing between a quick format, which does not erase all existing data and a long option that does erase all existing data.
Virtual Storage Access Method (VSAM) is an IBM direct-access storage device (DASD) file storage access method, first used in the OS/VS1, OS/VS2 Release 1 (SVS) and Release 2 (MVS) operating systems, later used throughout the Multiple Virtual Storage (MVS) architecture and now in z/OS. Originally a record-oriented filesystem, VSAM comprises four data set organizations: key-sequenced (KSDS), relative record (RRDS), entry-sequenced (ESDS) and linear (LDS). The KSDS, RRDS and ESDS organizations contain records, while the LDS organization simply contains a sequence of pages with no intrinsic record structure, for use as a memory-mapped file.
IBM manufactured magnetic disk storage devices from 1956 to 2003, when it sold its hard disk drive business to Hitachi. Both the hard disk drive (HDD) and floppy disk drive (FDD) were invented by IBM and as such IBM's employees were responsible for many of the innovations in these products and their technologies. The basic mechanical arrangement of hard disk drives has not changed since the IBM 1301. Disk drive performance and characteristics are measured by the same standards now as they were in the 1950s. Few products in history have enjoyed such spectacular declines in cost and physical size along with equally dramatic improvements in capacity and performance.
This article discusses support programs included in or available for OS/360 and successors. IBM categorizes some of these programs as utilities and others as service aids; the boundaries are not always consistent or obvious. Many, but not all, of these programs match the types in utility software.
In computing, channel I/O is a high-performance input/output (I/O) architecture that is implemented in various forms on a number of computer architectures, especially on mainframe computers. In the past, channels were generally implemented with custom devices, variously named channel, I/O processor, I/O controller, I/O synchronizer, or DMA controller.
In the IBM System/360 storage architecture, the Volume Table of Contents (VTOC), is a data structure that provides a way of locating the data sets that reside on a particular DASD volume. With the exception of the IBM Z compatible disk layout in Linux on Z, it is the functional equivalent of the MS/PC DOS File Allocation Table (FAT), the NTFS Master File Table (MFT), and an inode table in a file system for a Unix-like system. The VTOC is not used to contain any IPLTEXT and does not have any role in the IPL process, therefore does not have any data used by or functionally equivalent to the MBR. It lists the names of each data set on the volume as well as size, location, and permissions. Additionally, it contains an entry for every area of contiguous free space on the volume. The third record on the first track of the first cylinder of any DASD volume is known as the volume label and must contain a pointer to the location of the VTOC. The location of the VTOC may be specified when the volume is initialized. For performance reasons it may be located as close to the center of the volume as possible, since it is referenced frequently. A VTOC is added to a DASD volume when it is initialized using the Device Support Facilities program, ICKDSF, in current systems.
The IBM SAN Volume Controller (SVC) is a block storage virtualization appliance that belongs to the IBM System Storage product family. SVC implements an indirection, or "virtualization", layer in a Fibre Channel storage area network (SAN).
In computer disk storage, a sector is a subdivision of a track on a magnetic disk or optical disc. For most disks, each sector stores a fixed amount of user-accessible data, traditionally 512 bytes for hard disk drives (HDDs) and 2048 bytes for CD-ROMs and DVD-ROMs. Newer HDDs and SSDs use 4096-byte (4 KiB) sectors, which are known as the Advanced Format (AF).
In IBM mainframe operating systems OS/360 and its successors, a Unit Control Block (UCB) is a memory structure, or a control block, that describes any single input/output peripheral device (unit), or an exposure (alias), to the operating system. Certain data within the UCB also instructs the Input/Output Supervisor (IOS) to use certain closed subroutines in addition to normal IOS processing for additional physical device control.
An access method is a function of a mainframe operating system that enables access to data on disk, tape or other external devices. Access methods were present in several mainframe operating systems since the late 1950s, under a variety of names; the name access method was introduced in 1963 in the IBM OS/360 operating system. Access methods provide an application programming interface (API) for programmers to transfer data to or from device, and could be compared to device drivers in non-mainframe operating systems, but typically provide a greater level of functionality.
In IBM mainframe operating systems, Execute Channel Program (EXCP) is a macro generating a system call, implemented as a Supervisor Call instruction, for low-level device access, where the programmer is responsible for providing a channel program—a list of device-specific commands (CCWs)—to be executed by I/O channels, control units and devices. EXCP for OS/360 and successors is more specifically described in the OS System Programmer's Guide.; EXCP for DOS/360 and successors is more specifically described in DOS Supervisor and I/O Macros. This article mostly reflects OS/360 through z/OS; some details are different for TOS/360 and DOS/360 through z/VSE.
Fixed-block architecture (FBA) is an IBM term for the hard disk drive (HDD) layout in which each addressable block on the disk has the same size, utilizing 4 byte block numbers and a new set of command codes. FBA as a term was created and used by IBM for its 3310 and 3370 HDDs beginning in 1979 to distinguish such drives as IBM transitioned away from their variable record size format used on IBM's mainframe hard disk drives beginning in 1964 with its System/360.
The IBM System/360 architecture is the model independent architecture for the entire S/360 line of mainframe computers, including but not limited to the instruction set architecture. The elements of the architecture are documented in the IBM System/360 Principles of Operation and the IBM System/360 I/O Interface Channel to Control Unit Original Equipment Manufacturers' Information manuals.
Beginning with its 1964 System/360 announcement, IBM's mainframes initially accessed count key data (CKD) subsystems via a channel connected to separate Storage Control Units (SCUs) with attached Direct Access Storage Devices (DASD), typically a hard disk drive. This practice continued in IBM's larger mainframes thru System/370; however low end systems generally used lower cost integrated attachments where the function of the SCU was combined with that of the channel, typically called an Integrated File Adapter.
Bus and Tag is an "IBM standard for a computer peripheral interface", and was commonly used to connect their mainframe computers to peripheral devices such as line printers, disk storage, magnetic tape drives and IBM 3270 display controllers. The technology uses two sets of thick, multi-connector copper cables, one set, carrying data, called the bus, and the other set, carrying control information, called the tag.
In addition to the drums used as main memory by IBM, e.g., IBM 305, IBM 650, IBM offered drum devices as secondary storage for the 700/7000 series and System/360 series of computers.
A track is a path on a recording medium. There are some variations in nomenclature; for some media a track is a logical path and for others it is based on the geometry of the medium. The term is not used for punched cards.
References
↑ IBM 3990 Storage Control Introduction – 6th Ed. IBM. February 1994. GA32-0098-05.
↑ Houtekamer, Gilbert E.; Artis, H. Pat (1993). MVS I/O Subsystems: Configuration Management and Performance Analysis. New York: McGraw-Hill. ISBN978-0-07-002553-0. OCLC26096983.
1 2 3 "Synchronous DASD Operations". Introduction to Nonsynchronous Direct Access Storage Subsystems. International Business Machines Corporation. January 1990. GC46–4519–0.
This page is based on this Wikipedia article Text is available under the CC BY-SA 4.0 license; additional terms may apply. Images, videos and audio are available under their respective licenses.
