Modified condition/decision coverage

Last updated

Modified condition/decision coverage (MC/DC) is a code coverage criterion used in software testing.

Contents

Overview

MC/DC requires all of the below during testing: [1]

  1. Each entry and exit point is invoked
  2. Each decision takes every possible outcome
  3. Each condition in a decision takes every possible outcome
  4. Each condition in a decision is shown to independently affect the outcome of the decision.

Independence of a condition is shown by proving that only one condition changes at a time.

MC/DC is used in avionics software development guidance DO-178B and DO-178C to ensure adequate testing of the most critical (Level A) software, which is defined as that software which could provide (or prevent failure of) continued safe flight and landing of an aircraft. It is also highly recommended for SIL 4 in part 3 Annex B of the basic safety publication [2] and ASIL D in part 6 of automotive standard ISO 26262. [3]

Additionally, NASA requires 100% MC/DC coverage for any safety critical software component in Section 3.7.4 of NPR 7150.2D. [4]

Definitions

Condition
A condition is a leaf-level Boolean expression (it cannot be broken down into simpler Boolean expressions).
Decision
A Boolean expression composed of conditions and zero or more Boolean operators. A decision without a Boolean operator is a condition. A decision does not imply a change of control flow, e.g. an assignment of a boolean expression to a variable is a decision for MC/DC.
Condition coverage
Every condition in a decision in the program has taken all possible outcomes at least once.
Decision coverage
Every point of entry and exit in the program has been invoked at least once, and every decision in the program has taken all possible outcomes at least once.
Condition/decision coverage
Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and every decision in the program has taken all possible outcomes at least once.
Modified condition/decision coverage
Every point of entry and exit in the program has been invoked at least once, every condition in a decision in the program has taken all possible outcomes at least once, and each condition has been shown to affect that decision outcome independently. A condition is shown to affect a decision's outcome independently by varying just that condition while holding fixed all other possible conditions. The condition/decision criterion does not guarantee the coverage of all conditions in the module because in many test cases, some conditions of a decision are masked by the other conditions. Using the modified condition/decision criterion, each condition must be shown to be able to act on the decision outcome by itself, everything else being held fixed. The MC/DC criterion is thus much stronger than the condition/decision coverage.

Criticism

It is a misunderstanding that by purely syntactic rearrangements of decisions (breaking them into several independently evaluated conditions using temporary variables, the values of which are then used in the decision) which do not change the semantics of a program can lower the difficulty of obtaining complete MC/DC coverage. [5]

This is because MC/DC is driven by the program syntax. However, this kind of "cheating" can be done to simplify expressions, not simply to avoid MC/DC complexities. For example, assignment of the number of days in a month (excluding leap years) could be achieved by using either a switch statement or by using a table with an enumeration value as an index. The number of tests required based on the source code could be considerably different depending upon the coverage required, although semantically we would want to test both approaches with a minimum number of tests.[ citation needed ]

Another example that could be considered as "cheating" to achieve higher MC/DC is:

/* Function A */voidfunction_a(inta,boolb,boolc,boold,boole,boolf){if(a==100){if(b||c)// statement 1if(d||e||f)// statement 2}}
/* Function B */voidfunction_b(inta,boolb,boolc,boold,boole,boolf){boola_is_equal_to_100=a==100;boolb_or_c=b||c;boold_or_e_or_f=d||e||f;if(a_is_equal_to_100){if(b_or_c)// statement 1if(d_or_e_or_f)// statement 2}}

if the definition of a decision is treated as if it is an boolean expression that changes the control flow of the program (the text in brackets in an 'if' statement) then one may think that Function B is likely to have higher MC/DC than Function A for a given set of test cases (easier to test because it needs less tests to achieve 100% MC/DC coverage), even though functionally both are the same. [6]

However, what is wrong in the previous statement is the definition of decision. A decision includes 'any' boolean expression, even for assignments to variables. In this case, the three assignments should be treated as a decision for MC/DC purposes and therefore the changed code needs exactly the same tests and number of tests to achieve MC/DC than the first one. Some code coverage tools do not use this strict interpretation of a decision and may produce false positives (reporting 100% code coverage when indeed this is not the case). [ citation needed ]

RC/DC

In 2002 Sergiy Vilkomir proposed reinforced condition/decision coverage (RC/DC) as a stronger version of the MC/DC coverage criterion that is suitable for safety-critical systems. [7] [8]

Jonathan Bowen and his co-author analyzed several variants of MC/DC and RC/DC and concluded that at least some MC/DC variants have superior coverage over RC/DC. [9]

See also

Related Research Articles

<span class="mw-page-title-main">Boolean algebra (structure)</span> Algebraic structure modeling logical operations

In abstract algebra, a Boolean algebra or Boolean lattice is a complemented distributive lattice. This type of algebraic structure captures essential properties of both set operations and logic operations. A Boolean algebra can be seen as a generalization of a power set algebra or a field of sets, or its elements can be viewed as generalized truth values. It is also a special case of a De Morgan algebra and a Kleene algebra.

In software engineering, code coverage is a percentage measure of the degree to which the source code of a program is executed when a particular test suite is run. A program with high test coverage has more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs compared to a program with low test coverage. Many different metrics can be used to calculate test coverage. Some of the most basic are the percentage of program subroutines and the percentage of program statements called during execution of the test suite.

In computer programming, unit testing is a software testing method by which individual units of source code—sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures—are tested to determine whether they are fit for use. It is a standard step in development and implementation approaches such as Agile.

<span class="mw-page-title-main">Decision table</span>

Decision tables are a concise visual representation for specifying which actions to perform depending on given conditions. They are algorithms whose output is a set of actions. The information expressed in decision tables could also be represented as decision trees or in a programming language as a series of if-then-else and switch-case statements.

<span class="mw-page-title-main">Conditional (computer programming)</span> Control flow statement that executes code according to some condition(s)

In computer science, conditionals are programming language commands for handling decisions. Specifically, conditionals perform different computations or actions depending on whether a programmer-defined Boolean condition evaluates to true or false. In terms of control flow, the decision is always achieved by selectively altering the control flow based on some condition . Although dynamic dispatch is not usually classified as a conditional construct, it is another way to select between alternatives at runtime. Conditional statements are the checkpoints in the programe that determines behaviour according to situation.

In computer programming, the ternary conditional operator is a ternary operator that is part of the syntax for basic conditional expressions in several programming languages. It is commonly referred to as the conditional operator, ternary if, or inline if. An expression a ? b : c evaluates to b if the value of a is true, and otherwise to c. One can read it aloud as "if a then b otherwise c". The form a ? b : c is by far and large the most common, but alternative syntaxes do exist; for example, Raku uses the syntax a ?? b !! c to avoid confusion with the infix operators ? and !, whereas in Visual Basic .NET, it instead takes the form If(a, b, c).

Short-circuit evaluation, minimal evaluation, or McCarthy evaluation is the semantics of some Boolean operators in some programming languages in which the second argument is executed or evaluated only if the first argument does not suffice to determine the value of the expression: when the first argument of the AND function evaluates to false, the overall value must be false; and when the first argument of the OR function evaluates to true, the overall value must be true.

In computer programming, conditional loops or repetitive control structures are a way for computer programs to repeat one or more various steps depending on conditions set either by the programmer initially or real-time by the actual program.

White-box testing is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality. In white-box testing, an internal perspective of the system is used to design test cases. The tester chooses inputs to exercise paths through the code and determine the expected outputs. This is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). White-box testing can be applied at the unit, integration and system levels of the software testing process. Although traditional testers tended to think of white-box testing as being done at the unit level, it is used for integration and system testing more frequently today. It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it has the potential to miss unimplemented parts of the specification or missing requirements. Where white-box testing is design-driven, that is, driven exclusively by agreed specifications of how each component of software is required to behave, white-box test techniques can accomplish assessment for unimplemented or missing requirements.

Cyclomatic complexity is a software metric used to indicate the complexity of a program. It is a quantitative measure of the number of linearly independent paths through a program's source code. It was developed by Thomas J. McCabe, Sr. in 1976.

In computer science, the Boolean is a data type that has one of two possible values which is intended to represent the two truth values of logic and Boolean algebra. It is named after George Boole, who first defined an algebraic system of logic in the mid 19th century. The Boolean data type is primarily associated with conditional statements, which allow different actions by changing control flow depending on whether a programmer-specified Boolean condition evaluates to true or false. It is a special case of a more general logical data type—logic does not always need to be Boolean.

In computer science, a relational operator is a programming language construct or operator that tests or defines some kind of relation between two entities. These include numerical equality and inequalities.

Mutation testing is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways. Each mutated version is called a mutant and tests detect and reject mutants by causing the behaviour of the original version to differ from the mutant. This is called killing the mutant. Test suites are measured by the percentage of mutants that they kill. New tests can be designed to kill additional mutants. Mutants are based on well-defined mutation operators that either mimic typical programming errors or force the creation of valuable tests. The purpose is to help the tester develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution. Mutation testing is a form of white-box testing.

Structured text, abbreviated as ST or STX, is one of the five languages supported by the IEC 61131-3 standard, designed for programmable logic controllers (PLCs). It is a high level language that is block structured and syntactically resembles Pascal, on which it is based. All of the languages share IEC61131 Common Elements. The variables and function calls are defined by the common elements so different languages within the IEC 61131-3 standard can be used in the same program.

Fault coverage refers to the percentage of some type of fault that can be detected during the test of any engineered system. High fault coverage is particularly valuable during manufacturing test, and techniques such as Design For Test (DFT) and automatic test pattern generation are used to increase it.

Linear code sequence and jump (LCSAJ), in the broad sense, is a software analysis method used to identify structural units in code under test. Its primary use is with dynamic software analysis to help answer the question "How much testing is enough?". Dynamic software analysis is used to measure the quality and efficacy of software test data, where the quantification is performed in terms of structural units of the code under test. When used to quantify the structural units exercised by a given set of test data, dynamic analysis is also referred to as structural coverage analysis.

A binary decision is a choice between two alternatives, for instance between taking some specific action or not taking it.

Elementary comparison testing (ECT) is a white-box, control-flow, test-design methodology used in software development. The purpose of ECT is to enable detailed testing of complex software. Software code or pseudocode is tested to assess the proper handling of all decision outcomes. As with multiple-condition coverage and basis path testing, coverage of all independent and isolated conditions is accomplished through modified condition/decision coverage (MC/DC). Isolated conditions are aggregated into connected situations creating formal test cases. The independence of a condition is shown by changing the condition value in isolation. Each relevant condition value is covered by test cases.

<span class="mw-page-title-main">Sergiy Vilkomir</span> Ukrainian-born computer scientist (1956–2020)

Sergiy A. Vilkomir was a Ukrainian-born computer scientist.

Flix is a functional, imperative, and logic programming language developed at Aarhus University, with funding from the Independent Research Fund Denmark, and by a community of open source contributors. The Flix language supports algebraic data types, pattern matching, parametric polymorphism, currying, higher-order functions, extensible records, channel and process-based concurrency, and tail call elimination. Two notable features of Flix are its type and effect system and its support for first-class Datalog constraints.

References

  1. ↑ Hayhurst, Kelly; Veerhusen, Dan; Chilenski, John; Rierson, Leanna (May 2001). "A Practical Tutorial on Modified Condition/ Decision Coverage" (PDF). NASA.
  2. ↑ IEC 61508-3:2010
  3. ↑ ISO 26262-2011 Part 6 Table 12
  4. ↑ "NASA Software Engineering Requirements". NASA.
  5. ↑ Rajan, Ajitha; Heimdahl, Mats; Whalen, Michael (March 2003). "The Effect of Program and Model Structure on MC⁄DC Test Adequacy Coverage" (PDF).{{cite journal}}: Cite journal requires |journal= (help)
  6. ↑ http://www.hbni.ac.in/phdthesis/engg/ENGG02201004005.pdf [ bare URL PDF ]
  7. ↑ Vilkomir, S.A.; Bowen, J.P. (2002). "Reinforced condition/decision coverage (RC/DC): A new criterion for software testing". International Conference of B and Z Users. Lecture Notes in Computer Science. Springer-Verlag. 2272: 291–308. doi:10.1007/3-540-45648-1_15. ISBN   978-3-540-43166-4.
  8. ↑ Vilkomir, S.A.; Bowen, J.P. (2006). "From MC/DC to RC/DC: formalization and analysis of control-flow testing criteria". Formal Aspects of Computing . Springer Nature. 18 (1): 42–62. doi: 10.1007/s00165-005-0084-7 . S2CID   10467796.
  9. ↑ Kapoor, Kalpesh; Bowen, Jonathan P (2005). "A formal analysis of MCDC and RCDC test criteria". Software Testing, Verification and Reliability. Wiley Online Library. 15 (1): 21–40. doi:10.1002/stvr.306. S2CID   35276126.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.