Ada Conformity Assessment Test Suite

Last updated

The Ada Conformity Assessment Test Suite (ACATS) is the test suite used for Ada processor conformity testing. A prior test suite was known as the Ada Compiler Validation Capability (ACVC).

Contents

ACVC era

Example of Ada validation certificate awarded to vendor after passing the ACVC tests, 1986 AJPO validation certificate for ACT Ada 1750A compiler July 1986.jpg
Example of Ada validation certificate awarded to vendor after passing the ACVC tests, 1986

The Ada Compiler Validation Capability test suite, commonly referred to as the ACVC tests, [1] was the original test suite developed for the Ada language. The ACVC system was organized under the aegis of the Ada Joint Program Office. [2]

The tests were developed by the American company SofTech, beginning around 1980. [1] The test suites were modeled on a VAX/VMS system, [3] which was the dominant host platform for such defense-related applications at the time. Some of the tests were composed using orthogonal Latin squares as an approach towards get the most effective coverage of language feature combinations without employing an exhaustive enumeration of them. [1]

The individual test files were based on the section of the Ada reference manual they pointed to, for instance C45210A.ADA. [1] The suite included both positive tests and negative tests. [4] There was an organization set up to review queries vendors raised as to whether a certain aspect of one or more tests was an accurate reflection of the language standard. [2]

The year 1985 saw the issuing of the first Ada validation certificates. [2] At the height of the language's use, which corresponded to the years 1985 through 1993, there were five Ada Validation Facilities around the world that could process vendor ACVC submissions: the Language Control Facility at Wright Patterson Air Force Base (United States), the National Bureau of Standards (United States, soon renamed to the National Institute of Standards), the National Computing Centre (United Kingdom), AFNOR (France), and IABG (Germany). [2] However once Ada use for defense or similar applications began falling, the number of validation contracts fell as well, and several of these facilities became inoperative or transferred their responsibilities. [4]

The Ada Compiler Validation Capability came to an end with the closure of the Ada Joint Program Office in 1998. [2] Ada compiler vendors still wanted a validation mechanism, however, so a new validation system was devised to replace it, the Ada Conformity Assessment Test Suite. [2]

ACATS era

The preface to the test report includes the following:

Conformity assessment does not ensure that a processor has no nonconformities to the Ada standard other than those, if any, documented in this report. The compiler vendor declares that the tested processor contains no deliberate deviation from the Ada standard; a copy of this Declaration of Conformity is presented immediately after the certificate.

The second paragraph of the background of the current ACAA procedures says:

It is important to note the scope and intent of conformity assessment. The purpose of conformity assessment is to ensure that Ada processors achieve a high degree of conformity with the Ada standard (Ada95 as corrected by [TC1]). Characteristics such as performance and suitability for a particular application are not specified by the standard, and thus are outside the scope of Ada conformity assessment. Moreover, the ACATS is a set of test programs intended to check broadly for correct implementation; it is not possible to exhaustively test for conformity. Thus, conformity is checked only to the extent of these tests; processors that are certified as conforming may fail to conform to the standard in ways peculiar to each, under particular circumstances.

The ACATS consists of 1821 tests with 255,838 lines of code occupying 30 MB of storage. The test suite is publicly available, for example as a part of the source distribution of the GNU Compiler Collection, which also contains the Ada compiler GNAT.

See also

Overview document for the final release of the original ACVC tests, 1989 README file for ACVC Version 1.11.pdf
Overview document for the final release of the original ACVC tests, 1989

Related Research Articles

<span class="mw-page-title-main">Ada (programming language)</span> High-level programming language first released in 1980

Ada is a structured, statically typed, imperative, and object-oriented high-level programming language, inspired by Pascal and other languages. It has built-in language support for design by contract (DbC), extremely strong typing, explicit concurrency, tasks, synchronous message passing, protected objects, and non-determinism. Ada improves code safety and maintainability by using the compiler to find errors in favor of runtime errors. Ada is an international technical standard, jointly defined by the International Organization for Standardization (ISO), and the International Electrotechnical Commission (IEC). As of 2020, the standard, called Ada 2012 informally, is ISO/IEC 8652:2012.

In computing, a compiler is a computer program that translates computer code written in one programming language into another language. The name "compiler" is primarily used for programs that translate source code from a high-level programming language to a low-level programming language to create an executable program.

Conformance testing — an element of conformity assessment, and also known as compliance testing, or type testing — is testing or other activities that determine whether a process, product, or service complies with the requirements of a specification, technical standard, contract, or regulation. Testing is often either logical testing or physical testing. The test procedures may involve other criteria from mathematical testing or chemical testing. Beyond simple conformance, other requirements for efficiency, interoperability, or compliance may apply. Conformance testing may be undertaken by the producer of the product or service being assessed, by a user, or by an accredited independent organization, which can sometimes be the author of the standard being used. When testing is accompanied by certification, the products or services may then be advertised as being certified in compliance with the referred technical standard. Manufacturers and suppliers of products and services rely on such certification including listing on the certification body's website, to assure quality to the end user and that competing suppliers are on the same level.

<span class="mw-page-title-main">GNAT</span> Free-software compiler for the Ada programming language

GNAT is a free-software compiler for the Ada programming language which forms part of the GNU Compiler Collection (GCC). It supports all versions of the language, i.e. Ada 2012, Ada 2005, Ada 95 and Ada 83. Originally its name was an acronym that stood for GNU NYU Ada Translator, but that name no longer applies. The front-end and run-time are written in Ada.

ECMAScript is a standard for scripting languages, including JavaScript, JScript, and ActionScript. It is best known as a JavaScript standard intended to ensure the interoperability of web pages across different web browsers. It is standardized by Ecma International in the document ECMA-262.

Error: no inner hatnotes detected (help).

A domain-specific language (DSL) is a computer language specialized to a particular application domain. This is in contrast to a general-purpose language (GPL), which is broadly applicable across domains. There are a wide variety of DSLs, ranging from widely used languages for common domains, such as HTML for web pages, down to languages used by only one or a few pieces of software, such as MUSH soft code. DSLs can be further subdivided by the kind of language, and include domain-specific markup languages, domain-specific modeling languages, and domain-specific programming languages. Special-purpose computer languages have always existed in the computer age, but the term "domain-specific language" has become more popular due to the rise of domain-specific modeling. Simpler DSLs, particularly ones used by a single application, are sometimes informally called mini-languages.

The man or boy test was proposed by computer scientist Donald Knuth as a means of evaluating implementations of the ALGOL 60 programming language. The aim of the test was to distinguish compilers that correctly implemented "recursion and non-local references" from those that did not.

There are quite a few ALGOL60 translators in existence which have been designed to handle recursion and non-local references properly, and I thought perhaps a little test-program may be of value. Hence I have written the following simple routine, which may separate the man-compilers from the boy-compilers.

<span class="mw-page-title-main">Ada Semantic Interface Specification</span> Interface

The Ada Semantic Interface Specification (ASIS) is a layered, open architecture providing vendor-independent access to the Ada Library Environment. It allows for the static analysis of Ada programs and libraries. It is an open, published interface library that consists of the Ada environment and their tools and applications.

The Ravenscar profile is a subset of the Ada tasking features designed for safety-critical hard real-time computing. It was defined by a separate technical report in Ada 95; it is now part of the Ada 2012 Standard. It has been named after the English village of Ravenscar, the location of the 8th International Real-Time Ada Workshop.

ACATS may refer to:

<span class="mw-page-title-main">Benchmark (computing)</span> Comparing the relative performance of computers by running the same program on all of them

In computing, a benchmark is the act of running a computer program, a set of programs, or other operations, in order to assess the relative performance of an object, normally by running a number of standard tests and trials against it.

In computing, compiler correctness is the branch of computer science that deals with trying to show that a compiler behaves according to its language specification. Techniques include developing the compiler using formal methods and using rigorous testing on an existing compiler.

DDC-I, Inc. is a privately held company providing software development of real-time operating systems, software development tools, and software services for safety-critical embedded applications, headquartered in Phoenix, Arizona. It was first created in 1985 as the Danish firm DDC International A/S, a commercial outgrowth of Dansk Datamatik Center, a Danish software research and development organization of the 1980s. The American subsidiary was created in 1986. For many years, the firm specialized in language compilers for the programming language Ada.

<span class="mw-page-title-main">Robert Dewar</span> Computer scientist

Robert Berriedale Keith Dewar was an American computer scientist and educator. He helped to develop programming languages and compilers and was an outspoken advocate of freely licensed open-source software. He was a cofounder, CEO, and president of the AdaCore software company. He was also an enthusiastic amateur performer and musician, especially with the Village Light Opera Group in New York City.

<span class="mw-page-title-main">Concurrent Computer Corporation</span> American computing vendor

Concurrent Computer Corporation was an American computer company, in existence from 1985 to 2017, that made real-time computing and parallel processing systems. Its products powered a variety of applications including process control, simulators, data acquisition, and video-on-demand. It was based in Monmouth County, New Jersey, initially, and then later in Fort Lauderdale, Florida and Duluth, Georgia.

iMAX 432 was an operating system developed by Intel for digital electronic computers based on the 1980s Intel iAPX 432 32-bit microprocessor. The term micromainframe was an Intel marketing designation describing the iAPX 432 processor's capabilities as being comparable to a mainframe. The iAPX 432 processor and the iMAX 432 operating system were incompatible with the x86 architecture commonly found in personal computers. iMAX 432 was implemented in a subset of the original (1980) version of the Ada, extended with runtime type checking and dynamic package creation.

<span class="mw-page-title-main">Dansk Datamatik Center</span> Danish software research and development centre

Dansk Datamatik Center (DDC) was a Danish software research and development centre that existed from 1979 to 1989. Its main purpose was to demonstrate the value of using modern techniques, especially those involving formal methods, in software design and development.

<span class="mw-page-title-main">TLD Systems</span>

TLD Systems, Ltd. was an American software company active in the 1980s and 1990s and based in Torrance, California, that specialized in language compilers for the JOVIAL and Ada programming languages that were targeted to embedded systems.

Verdix Corporation was an American software company active in the 1980s and 1990s and based in Fairfax County, Virginia, that specialized in language compilers for the Ada programming language.

References

  1. 1 2 3 4 Mandl, Robert (October 1985). "Orthogonal Latin squares: An application of experiment design to compiler testing". Communications of the ACM. 28 (10): 1054–1058. doi: 10.1145/4372.4375 .
  2. 1 2 3 4 5 6 Brashear, Phil (March 1999). "Ada validation := Ada conformity assessment". Ada Letters. XIX (1): 48–51. doi: 10.1145/311457.311470 .
  3. ↑ "Ada Compiler Validation Capability (ACVC) Version 1.11 (Field-Test Release)". Defense Technical Information Center. June 1, 1989.
  4. 1 2 Tonndorf, Michael (1999). "Ada conformity assessments: A model for other programming languages?". Proceedings of the 1999 annual ACM SIGAda international conference on Ada (SIGAda '99). Association for Computing Machinery. pp. 89–99. doi:10.1145/319294.319310.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.