Orthogonal array testing

Last updated November 23, 2023

Orthogonal array testing is a systematic and statistically-driven black-box testing technique employed in the field of software testing.^[1]^[2] This method is particularly valuable in scenarios where the number of inputs to a system is substantial enough to make exhaustive testing impractical.

Overview

Orthogonal array testing works on the premise of selecting a subset of test cases from a large pool of potential inputs. This selection is based on statistical methods to ensure that the chosen subset represents the whole input space. As a result, serious bugs can be identified while the number of tests necessary to do so is greatly reduced.

Benefits

Reduced testing cycle time: By strategically selecting test cases, the testing process becomes more efficient, leading to time savings.
Simplified analysis: The structured nature of orthogonal array testing makes analysis straightforward and less complex.
Balanced test cases: This technique ensures that test cases are well-balanced, making it easier to isolate defects and assess performance.
Cost savings: It offers a significant cost advantage over pair-wise testing, making it an economical choice for testing large-scale software systems.

Cons

Limited applicability: This technique is most effective when the number of inputs is relatively small. In cases with an extremely large number of inputs, it may not be as efficient.
Complex implementation: Properly designing orthogonal arrays requires a good understanding of statistical principles, which may pose a challenge for some testing teams.
May miss specific edge cases: While orthogonal arrays are designed to cover a broad range of scenarios, they may not capture very specific edge cases that could be crucial in certain applications.

Applications

User Interface Testing: Orthogonal array testing is employed to evaluate the user interface of software applications. It aids in identifying interface-related anomalies and inconsistencies.
System Testing : It is used to validate the functionality of entire systems, ensuring that they perform as specified in their requirements.
Regression Testing : Orthogonal array testing is effective in the detection of regressions, ensuring that new updates or modifications do not introduce unintended consequences.
Configuration Testing: This technique is valuable in assessing different configurations of software, ensuring compatibility across diverse environments.
Performance Testing : It can be applied to evaluate the performance characteristics of software systems, helping to identify potential bottlenecks or performance issues.

Principle of Orthogonality

Orthogonal array testing works based on something called orthogonal arrays.^[3] These are organized lists of different factors. When we use them, we make sure that the results we get from each factor are not connected or related. This means each test gives us new and unique information. This way of organizing inputs helps us avoid repeating tests and get the same info with the least number of experiments.

Orthogonal vector

The concept of orthogonal vectors in orthogonal arrays is fundamental to understanding orthogonal array testing. Orthogonal vectors possess key properties:

Unique Information: Each vector imparts information distinct from any other vector in the sequence, thereby avoiding redundancy.
Separability: Through linear addition, the signals can be easily separated.
Statistical Independence: Each vector is statistically independent of the others, signifying a lack of correlation between them.
Resultant Summation: When subjected to linear addition, the resultant is the arithmetic sum of the individual components

Related Research Articles

Multivariate statistics is a subdivision of statistics encompassing the simultaneous observation and analysis of more than one outcome variable, i.e., multivariate random variables. Multivariate statistics concerns understanding the different aims and background of each of the different forms of multivariate analysis, and how they relate to each other. The practical application of multivariate statistics to a particular problem may involve several types of univariate and multivariate analyses in order to understand the relationships between variables and their relevance to the problem being studied.

Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not necessarily limited to:

Regression testing is re-running functional and non-functional tests to ensure that previously developed and tested software still performs as expected after a change. If not, that would be called a regression.

A web service (WS) is either:

SciPy is a free and open-source Python library used for scientific computing and technical computing.

R is a programming language for statistical computing and graphics supported by the R Core Team and the R Foundation for Statistical Computing. Created by statisticians Ross Ihaka and Robert Gentleman, R is used among data miners, bioinformaticians and statisticians for data analysis and developing statistical software. The core R language is augmented by a large number of extension packages containing reusable code and documentation.

SAS is a statistical software suite developed by SAS Institute for data management, advanced analytics, multivariate analysis, business intelligence, criminal investigation, and predictive analytics.

In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.

In machine learning and pattern recognition, a feature is an individual measurable property or characteristic of a phenomenon. Choosing informative, discriminating and independent features is a crucial element of effective algorithms in pattern recognition, classification and regression. Features are usually numeric, but structural features such as strings and graphs are used in syntactic pattern recognition. The concept of "feature" is related to that of explanatory variable used in statistical techniques such as linear regression.

In statistics, classification is the problem of identifying which of a set of categories (sub-populations) an observation belongs to. Examples are assigning a given email to the "spam" or "non-spam" class, and assigning a diagnosis to a given patient based on observed characteristics of the patient.

In computer science, all-pairs testing or pairwise testing is a combinatorial method of software testing that, for each pair of input parameters to a system, tests all possible discrete combinations of those parameters. Using carefully chosen test vectors, this can be done much faster than an exhaustive search of all combinations of all parameters by "parallelizing" the tests of parameter pairs.

In software engineering, graphical user interface testing is the process of testing a product's graphical user interface (GUI) to ensure it meets its specifications. This is normally done through the use of a variety of test cases.

Manual testing is the process of manually testing software for defects. It requires a tester to play the role of an end user where by they use most of the application's features to ensure correct behaviour. To guarantee completeness of testing, the tester often follows a written test plan that leads them through a set of important test cases.

STARMAD deals with the latest trend in the space industry is towards space missions, spacecraft, systems and products, which require quick solutions for system design and software development.

In mathematics, an orthogonal array is a "table" (array) whose entries come from a fixed finite set of symbols, arranged in such a way that there is an integer t so that for every selection of t columns of the table, all ordered t-tuples of the symbols, formed by taking the entries in each row restricted to these columns, appear the same number of times. The number t is called the strength of the orthogonal array. Here are two examples:

Gray-box testing is a combination of white-box testing and black-box testing. The aim of this testing is to search for the defects, if any, due to improper structure or improper usage of applications.

Software reliability testing is a field of software-testing that relates to testing a software's ability to function, given environmental conditions, for a particular amount of time. Software reliability testing helps discover many problems in the software design and functionality.

This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to Software Quality Assurance (more widely colloquially known as Quality Assurance and general application of the test method.

References

↑ Pressman, Roger S. (2005). Software engineering: a practitioner's approach. McGraw-Hill series in computer science (6th ed.). Boston: McGraw-Hill. ISBN 978-0-07-285318-6.
↑ Phadke, Madhav S.; Phadke, Kedar M. (January 2014). "Utilizing design of experiments to reduce IT system testing cost". 2014 Reliability and Maintainability Symposium. IEEE. pp. 1–6. doi:10.1109/rams.2014.6798451. ISBN 978-1-4799-2848-4. S2CID 8081137.
↑ "What is Orthogonal Array Testing? (Example)". www.guru99.com. 2023-09-30. Retrieved 2023-10-05.

External links

Rao, Calyampudi Radhakrishna (2009). "Orthogonal arrays". Scholarpedia. 4 (7): 9076. Bibcode:2009SchpJ...4.9076R. doi: 10.4249/scholarpedia.9076 .
Delius, Gustav W (May 2004). "Orthogonal Arrays (Taguchi Designs)". University of York.
Kuhfeld, Warren F. "Orthogonal Arrays". SAS Institute Inc. SAS provides a catalog of over 117,000 orthogonal arrays.
Phadke, Madhav S. "Planning Efficient Software Tests". Phadke Associates, Inc. Numerous articles on utilizing Orthogonal Arrays for Software and System Testing.
"rdExpert Software for Orthogonal Array Testing". Phadke Associates, Inc. Archived from the original on 2011-01-14. Commercial toolset for Orthogonal Array Testing.

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] Pressman, Roger S. (2005). Software engineering: a practitioner's approach. McGraw-Hill series in computer science (6th ed.). Boston: McGraw-Hill. ISBN 978-0-07-285318-6.

[2] Phadke, Madhav S.; Phadke, Kedar M. (January 2014). "Utilizing design of experiments to reduce IT system testing cost". 2014 Reliability and Maintainability Symposium. IEEE. pp. 1–6. doi:10.1109/rams.2014.6798451. ISBN 978-1-4799-2848-4. S2CID 8081137.

[3] "What is Orthogonal Array Testing? (Example)". www.guru99.com. 2023-09-30. Retrieved 2023-10-05.

[1]

[2]

[3]

v t e Software testing
The "box" approach	Black-box testing All-pairs testing Exploratory testing Fuzz testing Model-based testing Scenario testing Grey-box testing White-box testing API testing Mutation testing Static testing
Testing levels	Acceptance testing Integration testing System testing Unit testing
Testing types, techniques, and tactics	A/B testing Benchmark Compatibility testing Concolic testing Concurrent testing Conformance testing Continuous testing Destructive testing Development testing Differential testing Dynamic program analysis Installation testing Negative testing Random testing Regression testing Security testing Smoke testing (software) Software performance testing Stress testing Symbolic execution Test automation Usability testing
See also	Graphical user interface testing Manual testing Orthogonal array testing Pair testing Soak testing Software reliability testing Stress testing Web testing