![]() | |
Developer(s) | CA Technologies |
---|---|
Initial release | 1973 |
Stable release | 18.5 |
Operating system | z/OS, z/VM, z/VSE, BS2000/OSD |
Platform | Mainframe computer |
Type | Network DBMS |
Website | ca |
The Integrated Database Management System (IDMS) is a network model (CODASYL) database management system for mainframes. It was first developed at B.F. Goodrich and later marketed by Cullinane Database Systems (renamed Cullinet in 1983). Since 1989 the product has been owned by Computer Associates (now CA Technologies), who renamed it Advantage CA-IDMS and later simply to CA IDMS. In 2018 Broadcom acquired CA Technologies, renaming it back to IDMS. [1]
The roots of IDMS go back to the pioneering database management system called Integrated Data Store (IDS), developed at General Electric by a team led by Charles Bachman and first released in 1964. [2] In the early 1960s IDS was taken from its original form, by the computer group of the B.F. Goodrich Chemical Division, and re-written in a language called Intermediate System Language (ISL). [3] ISL was designed as a portable system programming language able to produce code for a variety of target machines. Since ISL was actually written in ISL, it was able to be ported to other machine architectures with relative ease, and then to produce code that would execute on them.
The Chemical Division computer group had given some thought to selling copies of IDMS to other companies, but was told by management that they were not in the software products business. Eventually, a deal was struck with John Cullinane to buy the rights and market the product. Because Cullinane was required to remit royalties back to B.F. Goodrich, all add-on products were listed and billed as separate products – even if they were mandatory for the core IDMS product to work. This sometimes confused customers.
The original platforms were the GE 235 computer and GE DATANET-30 message switching computer: later the product was ported to IBM mainframes and to DEC and ICL hardware. The IBM-ported version runs on IBM mainframe systems (System/360, System/370, System/390, zSeries, System z9). In the mid-1980s, it was claimed that some 2,500 IDMS licenses had been sold. Users included the Strategic Air Command, Ford of Canada, Ford of Europe, Jaguar Cars, Clarks Shoes UK, AXA/PPP, MAPFRE, Royal Insurance, Tesco, Manulife, Hudson's Bay Company, Cleveland Clinic, Bank of Canada, General Electric, Aetna and BT in the UK.
A version for use on the Digital Equipment Corporation PDP-11 series of computers was sold to DEC and was marketed as DBMS-11. In 1976 the source code was licensed to ICL, who ported the software to run on their 2900 series mainframes, and subsequently also on the older 1900 range. ICL continued development of the software independently of Cullinane, selling the original ported product under the name ICL 2900 IDMS and an enhanced version as IDMSX. In this form it was used by many large UK users, an example being the Pay-As-You-Earn system operated by Inland Revenue. Many of these IDMSX systems for UK Government were still running in 2013.
In the early to mid-1980s, relational database management systems started to become more popular, encouraged by increasing hardware power and the move to minicomputers and client–server architecture. Relational databases offered improved development productivity over CODASYL systems, and the traditional objections based on poor performance were slowly diminishing.
Cullinet attempted to continue competing against IBM's DB2 and other relational databases by developing a relational front-end and a range of productivity tools. These included Automatic System Facility (ASF), which made use of a pre-existing IDMS feature called LRF (Logical Record Facility). ASF was a fill-in-the-blanks database generator that would also develop a mini-application to maintain the tables.
It is difficult to judge whether such features may have been successful in extending the selling life of the product, but they made little impact in the long term. Those users who stayed with IDMS were primarily interested in its high performance, not in its relational capabilities. It was widely recognized (helped by a high-profile campaign by E. F. Codd, the father of the relational model) that there was a significant difference between a relational database and a network database with a relational veneer.
In 1989 Computer Associates continued after Cullinet acquisition [4] with the development and released Release 12.0 with full SQL in 1992–93. CA Technologies continued to market and support the CA IDMS and enhanced IDMS in subsequent releases by TCP/IP support, two phase commit support, XML publishing, zIIP specialty processor support, Web-enabled access in combination with CA IDMS Server, SQL Option and GUI database administration via CA IDMS Visual DBA tool. CA-IDMS systems are today still running businesses worldwide. Many customers have opted to web-enable their applications via the CA-IDMS SQL Option which is part of CA Technologies' Dual Database Strategy.
One of the sophisticated features of IDMS was its built-in Integrated data dictionary (IDD). The IDD was primarily developed to maintain database definitions. It was itself an IDMS database. DBAs (database administrators) and other users interfaced with the IDD using a language called Data Dictionary Definition Language (DDDL). IDD was also used to store definitions and code for other products in the IDMS family such as ADS/Online and IDMS-DC. IDD's power was that it was extensible and could be used to create definitions of just about anything. Some companies used it to develop in-house documentation.
The data model offered to users is the CODASYL network model. The main structuring concepts in this model are records and sets. Records essentially follow the COBOL pattern, consisting of fields of different types: this allows complex internal structure such as repeating items and repeating groups.
The most distinctive structuring concept in the Codasyl model is the set. Not to be confused with a mathematical set, a Codasyl set represents a one-to-many relationship between records: one owner, many members. The fact that a record can be a member in many different sets is the key factor that distinguishes the network model from the earlier hierarchical model. As with records, each set belongs to a named set type (different set types model different logical relationships). Sets are in fact ordered, and the sequence of records in a set can be used to convey information. A record can participate as an owner and member of any number of sets.
Records have identity, the identity being represented by a value known as a database key. In IDMS, as in most other Codasyl implementations, the database key is directly related to the physical address of the record on disk. Database keys are also used as pointers to implement sets in the form of linked lists and trees. This close correspondence between the logical model and the physical implementation (which is not a strictly necessary part of the Codasyl model, but was a characteristic of all successful implementations) is responsible for the efficiency of database retrieval, but also makes operations such as database loading and restructuring rather expensive.
Records can be accessed directly by database key, by following set relationships, or by direct access using key values. Initially the only direct access was through hashing, a mechanism known in the Codasyl model as CALC access. In IDMS, CALC access is implemented through an internal set, linking all records that share the same hash value to an owner record that occupies the first few bytes of every disk page.
In subsequent years, some versions of IDMS added the ability to access records using BTree-like indexes.
IDMS organizes its databases as a series of files. These files are mapped and pre-formatted into so-called areas. The areas are subdivided into pages which correspond to physical blocks on the disk. The database records are stored within these blocks.
The DBA allocates a fixed number of pages in a file for each area. The DBA then defines which records are to be stored in each area, and details of how they are to be stored.
IDMS intersperses special space-allocation pages throughout the database. These pages are used to keep track of the free space available in each page in the database. To reduce I/O requirements, the free space is only tracked for all pages when the free space for the area falls below 30%.
Four methods are available for storing records in an IDMS database: Direct, Sequential, CALC, and VIA. The Fujitsu/ICL IDMSX version extends this with two more methods, Page Direct, and Random.
In direct mode the target database key is specified by the user and is stored as close as possible to that DB key, with the actual DB key on which the record is stored being returned to the application program.
Sequential placement (not to be confused with indexed sequential), simply places each new record at the end of the area. This option is rarely used.
CALC uses a hashing algorithm to decide where to place the record; the hash key then provides efficient retrieval of the record. The entire CALC area is preformatted each with a header consisting of a special CALC "owner" record. The hashing algorithm determines a page number (from which the physical disk address can be determined), and the record is then stored on this page, or as near as possible to it, and is linked to the header record on that page using the CALC set. The CALC records are linked to the page's CALC Owner record using a single link-list (pointers). The CALC Owner located in the page header thus owns the set of all records which target to its particular page (whether the records are stored on that page or, in the case of an overflow, on another page ).
CALC provides extremely efficient storage and retrieval: IDMS can retrieve a CALC record in 1.1 I/O operations. However, the method does not cope well with changes to the value of the primary key, and expensive reorganization is needed if the number of pages needs to be expanded. A work-around is to expand the area, and then run an application program which scans the area sequentially for each CALC record, and then uses the MODIFY verb to update each record. This results in each CALC record being connected to the CALC Set for the correct target page as calculated for the Area's new page range. The downside to this method is that vanishingly few CALC records will now be on their target pages, and navigating each page's CALC set is likely to involve many IO operations. As a result, it is recommended only to use this work-around in extreme circumstances as performance will suffer.
VIA placement attempts to store a record near its owner in a particular set. Usually the records are clustered on the same physical page as the owner. This leads to efficient navigation when the record is accessed by following that set relationship. (VIA allows records to be stored in a different IDMS area so that they can be stored separately from the owner, yet remain clustered together for efficiency. Within IDMSX they may also be offset from the owner by a set number of pages).
Page Direct (IDMSX only) is similar to Direct mode, however a target Database page number is specified and the record is connected to the CALC chain for that page.
Random (IDMSX only) allocates a target page number to the record occurrence when it is stored using the CALC algorithm (this either uses a Key within the record or in the case of un-keyed random, uses the date & time of storage as a seed for the CALC algorithm).
Sets are generally maintained as linked lists, using the database key as a pointer. Every record includes a forward link to the next record; the database designer can choose whether to include owner pointers and prior pointers (if not provided, navigation in those directions will be slower).
Some versions of IDMS subsequently included the ability to define indexes: either record indexes, allowing records to be located from knowledge of a secondary key, or set indexes, allowing the members of a set to be retrieved by key value.
The IDMSX Page Direct and Random placement records are typically used in conjunction with Record Indexes as described above. The Indexes themselves are subject to placement rules, either Direct (which really means "CALC using the Index ID as the key") or CALC.
Release version | Release date | Example features |
---|---|---|
R12 | 1992 | 24-hour processing, logical physical separation, catalog management, deadlock management, centralized security facility, SQL |
R14 | 1999-01-09 [5] | Parallel sysplex exploitation, multitasking |
R15 | 2001-04-25 [6] | Improved performance, data sharing |
R16 | 2004-04-13 [7] | Two phase commit, TCP/IP, Parallel Access Volumes exploitation, XML publishing |
R17 | 2008-10-30 [8] | Improved performance, zIIP support, improved auto-recovery features [9] |
R18 | 2011-06-02 | Enhanced zIIP support, automatic system tuning, improved performance, simplified installation and maintenance |
R19 |
IDMS has non-profit user associations who use or support CA IDMS or related products. They include:
In computing, a database is an organized collection of data or a type of data store based on the use of a database management system (DBMS), the software that interacts with end users, applications, and the database itself to capture and analyze the data. The DBMS additionally encompasses the core facilities provided to administer the database. The sum total of the database, the DBMS and the associated applications can be referred to as a database system. Often the term "database" is also used loosely to refer to any of the DBMS, the database system or an application associated with the database.
A mainframe computer, informally called a mainframe or big iron, is a computer used primarily by large organizations for critical applications like bulk data processing for tasks such as censuses, industry and consumer statistics, enterprise resource planning, and large-scale transaction processing. A mainframe computer is large but not as large as a supercomputer and has more processing power than some other classes of computers, such as minicomputers, servers, workstations, and personal computers. Most large-scale computer-system architectures were established in the 1960s, but they continue to evolve. Mainframe computers are often used as servers.
A relational database (RDB) is a database based on the relational model of data, as proposed by E. F. Codd in 1970. A database management system used to maintain relational databases is a relational database management system (RDBMS). Many relational database systems are equipped with the option of using SQL for querying and updating the database.
Ingres Database is a proprietary SQL relational database management system intended to support large commercial and government applications.
Db2 is a family of data management products, including database servers, developed by IBM. It initially supported the relational model, but was extended to support object–relational features and non-relational structures like JSON and XML. The brand name was originally styled as DB2 until 2017, when it changed to its present form.
In computing, the network model is a database model conceived as a flexible way of representing objects and their relationships. Its distinguishing feature is that the schema, viewed as a graph in which object types are nodes and relationship types are arcs, is not restricted to being a hierarchy or lattice.
A hierarchical database model is a data model in which the data are organized into a tree-like structure. The data are stored as records which are connected to one another through links. A record is a collection of fields, with each field containing only one value. The type of a record defines which fields the record contains.
Indexed Sequential Access Method (ISAM) is a method for creating, maintaining, and manipulating computer files of data so that records can be retrieved sequentially or randomly by one or more keys. Indexes of key fields are maintained to achieve fast retrieval of required file records in indexed files. IBM originally developed ISAM for mainframe computers, but implementations are available for most computer systems.
Charles William Bachman III was an American computer scientist, who spent his entire career as an industrial researcher, developer, and manager rather than in academia. He was particularly known for his work in the early development of database management systems. His techniques of layered architecture include his namesake Bachman diagrams.
CODASYL, the Conference/Committee on Data Systems Languages, was a consortium formed in 1959 to guide the development of a standard programming language that could be used on many computers. This effort led to the development of the programming language COBOL, the CODASYL Data Model, and other technical standards.
International Computers Limited (ICL) was a British computer hardware, computer software and computer services company that operated from 1968 until 2002. It was formed through a merger of International Computers and Tabulators (ICT), English Electric Computers (EEC) and Elliott Automation in 1968. The company's most successful product line was the ICL 2900 Series range of mainframe computers.
A navigational database is a type of database in which records or objects are found primarily by following references from other objects. The term was popularized by the title of Charles Bachman's 1973 Turing Award paper, The Programmer as Navigator. This paper emphasized the fact that the new disk-based database systems allowed the programmer to choose arbitrary navigational routes following relationships from record to record, contrasting this with the constraints of earlier magnetic-tape and punched card systems where data access was strictly sequential.
The Content Addressable File Store (CAFS) was a hardware device developed by International Computers Limited (ICL) that provided a disk storage with built-in search capability. The motivation for the device was the discrepancy between the high speed at which a disk could deliver data, and the much lower speed at which a general-purpose processor could filter the data looking for records that matched a search condition. Development of CAFS started in ICL's Research and Advanced Development Centre under Gordon Scarrott in the late 1960s following research by George Coulouris and John Evans who had completed a field study at Imperial College and Queen Mary College on database systems and applications. Their study had revealed the potential for substantial performance improvements in large-scale database applications by the inclusion of search logic in the disk controller.
Cullinet was a software company whose products included the database management system IDMS and the integrated software package Goldengate. In 1989, the company was bought by Computer Associates. Cullinet was headquartered at 400 Blue Hill Drive in Westwood, Massachusetts.
NOMAD is a relational database and fourth-generation language (4GL), originally developed in the 1970s by time-sharing vendor National CSS. While it is still in use today, its widest use was in the 1970s and 1980s. NOMAD supports both the relational and hierarchical database models.
A database model is a type of data model that determines the logical structure of a database. It fundamentally determines in which manner data can be stored, organized and manipulated. The most popular example of a database model is the relational model, which uses a table-based format.
Datacom/DB is a relational database management system for mainframe computers. It was developed in the early 1970s by Computer Information Management Company and was subsequently owned by Insyte, Applied Data Research, Ameritech, and Computer Associates International, Inc. Datacom was acquired by CA Technologies, which renamed it to CA-Datacom/DB and later to CA Datacom/DB. In 2018, Broadcom acquired CA Technologies which included the CA Datacom product family. In 2021, Broadcom has dropped the CA and now refers to the product family as Datacom or Datacom/DB.
Britton Lee Inc. was a pioneering relational database company. Renamed ShareBase, it was acquired by Teradata in June, 1990.
Rocket U2 is a suite of database management (DBMS) and supporting software now owned by Rocket Software. It includes two MultiValue database platforms: UniData and UniVerse. Both of these products are operating environments which run on current Unix, Linux and Windows operating systems. They are both derivatives of the Pick operating system. The family also includes developer and web-enabling technologies including SB/XA, U2 Web Development Environment (WebDE), UniObjects connectivity API and wIntegrate terminal emulation software.
Distributed Data Management Architecture (DDM) is IBM's open, published software architecture for creating, managing and accessing data on a remote computer. DDM was initially designed to support record-oriented files; it was extended to support hierarchical directories, stream-oriented files, queues, and system command processing; it was further extended to be the base of IBM's Distributed Relational Database Architecture (DRDA); and finally, it was extended to support data description and conversion. Defined in the period from 1980 to 1993, DDM specifies necessary components, messages, and protocols, all based on the principles of object-orientation. DDM is not, in itself, a piece of software; the implementation of DDM takes the form of client and server products. As an open architecture, products can implement subsets of DDM architecture and products can extend DDM to meet additional requirements. Taken together, DDM products implement a distributed file system.