Operational acceptance testing

Last updated
Operational testing a jet engine US Navy 070804-N-1745W-122 A Sailor assigned to Aircraft Intermediate Maintenance Department (AIMD) tests an aircraft jet engine for defects while performing Jet Engine Test Instrumentation, (JETI) Certification-Engine Runs.jpg
Operational testing a jet engine

Operational acceptance testing (OAT) is used to conduct operational readiness (pre-release) of a product, service, or system as part of a quality management system. OAT is a common type of non-functional software testing, used mainly in software development and software maintenance projects. This type of testing focuses on the operational readiness of the system to be supported, and/or to become part of the production environment. Hence, it is also known as operational readiness testing (ORT) or operations readiness and assurance testing (OR&A). Functional testing within OAT is limited to those tests which are required to verify the non-functional aspects of the system.

OAT elaborates upon and compartmentalises operational aspects of acceptance testing. [1]

According to the International Software Testing Qualifications Board (ISTQB), OAT may include checking the backup/restore facilities, IT disaster recovery procedures, maintenance tasks and periodic check of security vulnerabilities., [2] and whitepapers on ISO 29119 and Operational Acceptance by Anthony Woods, [3] and ISO 25000 and Operational Acceptance Testing by Dirk Dach et al., OAT generally includes: [4]

  • Component fail-over
  • Network fail-over
  • Accessibility
  • Conversion
  • Stability
  • Usability
  • Compatibility
  • Interoperability
  • Installation and Backout
  • Localization
  • Application/system recovery
  • Data recovery
  • Backup and Restoration (Recovery)
  • Disaster Recovery
  • Maintainability
  • Performance, Stress and Volume,
  • Procedures (Operability) and Supporting Documentation (Supportability)
  • Security and Penetration

During OAT changes may be made to environmental parameters which the application uses to run smoothly. For example, with Microsoft Windows applications with a mixed or hybrid architecture, this may include: Windows services, configuration files, web services, XML files, COM+ components, web services, IIS, stored procedures in databases, etc. Typically OAT should occur after each main phase of the development life cycle: design, build, and functional testing. In sequential projects it is often viewed as a final verification before a system is released; where in agile and iterative projects, a more frequent execution of OAT occurs providing stakeholders with assurance of continued stability of the system and its operating environment.

An approach used in OAT may follow these steps:

For running the OAT test cases, the tester normally has exclusive access to the system or environment. This means that a single tester would be executing the test cases at a single point of time. For OAT the exact Operational Readiness quality gates are defined: both entry and exit gates. The primary emphasis of OAT should be on the operational stability, portability and reliability of the system.

Related Research Articles

<span class="mw-page-title-main">Acceptance testing</span> Test to determine if the requirements of a specification or contract are met

In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. It may involve chemical tests, physical tests, or performance tests.

<span class="mw-page-title-main">Software testing</span> Checking software against a standard

Software testing is the act of checking whether software satisfies expectations.

<span class="mw-page-title-main">Configuration management</span> Process for maintaining consistency of a product attributes with its design

Configuration management (CM) is a management process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life. The CM process is widely used by military engineering organizations to manage changes throughout the system lifecycle of complex systems, such as weapon systems, military vehicles, and information systems. Outside the military, the CM process is also used with IT service management as defined by ITIL, and with other domain models in the civil engineering and other industrial engineering segments such as roads, bridges, canals, dams, and buildings.

A software company is an organisation — owned either by the state or private — established for profit whose primary products are various forms of software, software technology, distribution, and software product development. They make up the software industry.

Black-box testing, sometimes referred to as specification-based testing, is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. Black-box testing is also used as a method in penetration testing, where an ethical hacker simulates an external hacking or cyber warfare attack with no knowledge of the system being attacked.

In the context of software engineering, software quality refers to two related but distinct notions:

Software quality assurance (SQA) is a means and practice of monitoring all software engineering processes, methods, and work products to ensure compliance against defined standards. It may include ensuring conformance to standards or models, such as ISO/IEC 9126, SPICE or CMMI.

Integrated logistics support (ILS) is a technology in the system engineering to lower a product life cycle cost and decrease demand for logistics by the maintenance system optimization to ease the product support. Although originally developed for military purposes, it is also widely used in commercial customer service organisations.

In systems engineering and requirements engineering, a non-functional requirement (NFR) is a requirement that specifies criteria that can be used to judge the operation of a system, rather than specific behaviours. They are contrasted with functional requirements that define specific behavior or functions. The plan for implementing functional requirements is detailed in the system design. The plan for implementing non-functional requirements is detailed in the system architecture, because they are usually architecturally significant requirements.

Portability testing is the process of determining the degree of ease or difficulty to which a software component or application can be effectively and efficiently transferred from one hardware, software or other operational or usage environment to another. The test results, defined by the individual needs of the system, are some measurement of how easily the component or application will be to integrate into the environment and these results will then be compared to the software system's non-functional requirement of portability for correctness. The levels of correctness are usually measured by the cost to adapt the software to the new environment compared to the cost of redevelopment.

The International Software Testing Qualifications Board (ISTQB) is a software testing certification board that operates internationally. Founded in Edinburgh in November 2002, the ISTQB is a non-profit association legally registered in Belgium.

In software development, functional testing is a form of software system testing that verifies whether software matches its design.

Software Quality Management (SQM) is a management process that aims to develop and manage the quality of software in such a way so as to best ensure that the product meets the quality standards expected by the customer while also meeting any necessary regulatory and developer requirements, if any. Software quality managers require software to be tested before it is released to the market, and they do this using a cyclical process-based quality assessment in order to reveal and fix bugs before release. Their job is not only to ensure their software is in good shape for the consumer but also to encourage a culture of quality throughout the enterprise.

Software quality control is the set of procedures used by organizations to ensure that a software product will meet its quality goals at the best value to the customer, and to continually improve the organization’s ability to produce software products in the future.

API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. Since APIs lack a GUI, API testing is performed at the message layer. API testing is now considered critical for automating testing because APIs serve as the primary interface to application logic and because GUI tests are difficult to maintain with the short release cycles and frequent changes commonly used with Agile software development and DevOps.

ISO 26262, titled "Road vehicles – Functional safety", is an international standard for functional safety of electrical and/or electronic systems that are installed in serial production road vehicles, defined by the International Organization for Standardization (ISO) in 2011, and revised in 2018.

ISO/IEC/IEEE 29119Software and systems engineering -- Software testing is a series of five international standards for software testing. First developed in 2007 and released in 2013, the standard "defines vocabulary, processes, documentation, techniques, and a process assessment model for testing that can be used within any software development lifecycle."

This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to software quality assurance and general application of the test method.

<span class="mw-page-title-main">Standardisation Testing and Quality Certification</span> Science and technology agency of the Government of India

Standardisation Testing and Quality Certification (STQC) Directorate, established in 1980, is an authoritative body offering quality assurance services to IT and Electronics domains.

References

  1. "atos-operational-acceptance-testing-whitepaper.pdf" (PDF).
  2. ISTQB http://istqbexamcertification.com/what-is-acceptance-testing/
  3. Anthony Woods (2015). "Operational Acceptance - an application of the ISO 29119 Software Testing standard" (Document). Capgemini and Sogeti. pp. 1–12.
  4. White Paper: Operational Acceptance Testing, Business Continuity Assurance. December 2012 Dirk Dach, Dr Kai-Uwe Gawlik, Mark Mevert