Operational acceptance testing

Last updated
Operational testing a jet engine US Navy 070804-N-1745W-122 A Sailor assigned to Aircraft Intermediate Maintenance Department (AIMD) tests an aircraft jet engine for defects while performing Jet Engine Test Instrumentation, (JETI) Certification-Engine Runs.jpg
Operational testing a jet engine

Operational acceptance testing (OAT) is used to conduct operational readiness (pre-release) of a product, service, or system as part of a quality management system. OAT is a common type of non-functional software testing, used mainly in software development and software maintenance projects. This type of testing focuses on the operational readiness of the system to be supported, and/or to become part of the production environment. Hence, it is also known as operational readiness testing (ORT) or operations readiness and assurance testing (OR&A). Functional testing within OAT is limited to those tests which are required to verify the non-functional aspects of the system.

OAT elaborates upon and compartmentalises operational aspects of acceptance testing [1] .

According to the International Software Testing Qualifications Board (ISTQB), OAT may include checking the backup/restore facilities, IT disaster recovery procedures, maintenance tasks and periodic check of security vulnerabilities., [2] and whitepapers on ISO 29119 and Operational Acceptance by Anthony Woods, [3] and ISO 25000 and Operational Acceptance Testing by Dirk Dach et al., OAT generally includes: [4]

  • Component fail-over
  • Network fail-over
  • Accessibility
  • Conversion
  • Stability
  • Usability
  • Compatibility
  • Interoperability
  • Installation and Backout
  • Localization
  • Application/system recovery
  • Data recovery
  • Backup and Restoration (Recovery)
  • Disaster Recovery
  • Maintainability
  • Performance, Stress and Volume,
  • Procedures (Operability) and Supporting Documentation (Supportability)
  • Security and Penetration

During OAT changes may be made to environmental parameters which the application uses to run smoothly. For example, with Microsoft Windows applications with a mixed or hybrid architecture, this may include: Windows services, configuration files, web services, XML files, COM+ components, web services, IIS, stored procedures in databases, etc. Typically OAT should occur after each main phase of the development life cycle: design, build, and functional testing. In sequential projects it is often viewed as a final verification before a system is released; where in agile and iterative projects, a more frequent execution of OAT occurs providing stakeholders with assurance of continued stability of the system and its operating environment.

An approach used in OAT may follow these steps:

For running the OAT test cases, the tester normally has exclusive access to the system or environment. This means that a single tester would be executing the test cases at a single point of time. For OAT the exact Operational Readiness quality gates are defined: both entry and exit gates. The primary emphasis of OAT should be on the operational stability, portability and reliability of the system.

Related Research Articles

<span class="mw-page-title-main">Acceptance testing</span> Test to determine if the requirements of a specification or contract are met

In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. It may involve chemical tests, physical tests, or performance tests.

Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to:

<span class="mw-page-title-main">Configuration management</span> Process for maintaining consistency of a product attributes with its design

Configuration management (CM) is a systems engineering process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life. The CM process is widely used by military engineering organizations to manage changes throughout the system lifecycle of complex systems, such as weapon systems, military vehicles, and information systems. Outside the military, the CM process is also used with IT service management as defined by ITIL, and with other domain models in the civil engineering and other industrial engineering segments such as roads, bridges, canals, dams, and buildings.

Black-box testing is a method of software testing that examines the functionality of an application without peering into its internal structures or workings. This method of test can be applied virtually to every level of software testing: unit, integration, system and acceptance. It is sometimes referred to as specification-based testing.

In the context of software engineering, software quality refers to two related but distinct notions:

Software quality assurance (SQA) is a means and practice of monitoring all software engineering processes, methods, and work products to ensure compliance against defined standards. It may include ensuring conformance to standards or models, such as ISO/IEC 9126, SPICE or CMMI.

Integrated logistics support (ILS) is a technology in the system engineering to lower a product life cycle cost and decrease demand for logistics by the maintenance system optimization to ease the product support. Although originally developed for military purposes, it is also widely used in commercial customer service organisations.

In systems engineering and requirements engineering, a non-functional requirement (NFR) is a requirement that specifies criteria that can be used to judge the operation of a system, rather than specific behaviours. They are contrasted with functional requirements that define specific behavior or functions. The plan for implementing functional requirements is detailed in the system design. The plan for implementing non-functional requirements is detailed in the system architecture, because they are usually architecturally significant requirements.

A hazard analysis is used as the first step in a process used to assess risk. The result of a hazard analysis is the identification of different types of hazards. A hazard is a potential condition and exists or not. It may, in single existence or in combination with other hazards and conditions, become an actual Functional Failure or Accident (Mishap). The way this exactly happens in one particular sequence is called a scenario. This scenario has a probability of occurrence. Often a system has many potential failure scenarios. It also is assigned a classification, based on the worst case severity of the end condition. Risk is the combination of probability and severity. Preliminary risk levels can be provided in the hazard analysis. The validation, more precise prediction (verification) and acceptance of risk is determined in the risk assessment (analysis). The main goal of both is to provide the best selection of means of controlling or eliminating the risk. The term is used in several engineering specialties, including avionics, food safety, occupational safety and health, process safety, reliability engineering.

Portability testing is the process of determining the degree of ease or difficulty to which a software component or application can be effectively and efficiently transferred from one hardware, software or other operational or usage environment to another. The test results, defined by the individual needs of the system, are some measurement of how easily the component or application will be to integrate into the environment and these results will then be compared to the software system's non-functional requirement of portability for correctness. The levels of correctness are usually measured by the cost to adapt the software to the new environment compared to the cost of redevelopment.

The International Software Testing Qualifications Board (ISTQB) is a software testing certification board that operates internationally. Founded in Edinburgh in November 2002, the ISTQB is a non-profit association legally registered in Belgium.

In software development, functional testing is a quality assurance (QA) process and a type of black-box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered. Functional software testing is conducted to evaluate the compliance of a system or component with specified functional requirements. Functional testing usually describes what the system does.

Software Quality Management (SQM) is a management process that aims to develop and manage the quality of software in such a way so as to best ensure that the product meets the quality standards expected by the customer while also meeting any necessary regulatory and developer requirements, if any. Software quality managers require software to be tested before it is released to the market, and they do this using a cyclical process-based quality assessment in order to reveal and fix bugs before release. Their job is not only to ensure their software is in good shape for the consumer but also to encourage a culture of quality throughout the enterprise.

Software quality control is the set of procedures used by organizations to ensure that a software product will meet its quality goals at the best value to the customer, and to continually improve the organization’s ability to produce software products in the future.

API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. Since APIs lack a GUI, API testing is performed at the message layer. API testing is now considered critical for automating testing because APIs now serve as the primary interface to application logic and because GUI tests are difficult to maintain with the short release cycles and frequent changes commonly used with Agile software development and DevOps.

ISO/IEC/IEEE 29119Software and systems engineering -- Software testing is a series of five international standards for software testing. First developed in 2007 and released in 2013, the standard "defines vocabulary, processes, documentation, techniques, and a process assessment model for testing that can be used within any software development lifecycle."

This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to Software Quality Assurance (more widely colloquially known as Quality Assurance and general application of the test method.

TestOps refers to the discipline of managing the operational aspects of testing within the software delivery lifecycle.

<span class="mw-page-title-main">Standardisation Testing and Quality Certification</span> Science and technology agency of the Government of India

Standardisation Testing and Quality Certification (STQC) Directorate, established in 1980, is an authoritative body offering quality assurance services to IT and Electronics domains.

References

  1. "atos-operational-acceptance-testing-whitepaper.pdf" (PDF).
  2. ISTQB http://istqbexamcertification.com/what-is-acceptance-testing/
  3. Anthony Woods (2015). "Operational Acceptance - an application of the ISO 29119 Software Testing standard" (Document). Capgemini and Sogeti. pp. 1–12.
  4. White Paper: Operational Acceptance Testing, Business Continuity Assurance. December 2012 Dirk Dach, Dr Kai-Uwe Gawlik, Mark Mevert