White-box testing

Black box systems
System
Black box, Oracle machine
Methods and techniques
Black-box testing, Blackboxing
Related techniques
Feed forward, Obfuscation, Pattern recognition, White box, White-box testing, Gray-box testing, System identification
Fundamentals
A priori information, Control systems, Open systems, Operations research, Thermodynamic systems
v t e

White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing). In white-box testing, an internal perspective of the system is used to design test cases. The tester chooses inputs to exercise paths through the code and determine the expected outputs. This is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). White-box testing can be applied at the unit, integration and system levels of the software testing process. Although traditional testers tended to think of white-box testing as being done at the unit level, it is used for integration and system testing more frequently today. It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it has the potential to miss unimplemented parts of the specification or missing requirements. Where white-box testing is design-driven, ^[1] that is, driven exclusively by agreed specifications of how each component of software is required to behave (as in DO-178C and ISO 26262 processes), white-box test techniques can accomplish assessment for unimplemented or missing requirements.

White-box test design techniques include the following code coverage criteria:

• Control flow testing
• Data flow testing
• Branch testing
• Statement coverage
• Decision coverage
• Modified condition/decision coverage
• Prime path testing
• Path testing

Overview

White-box testing is a method of testing the application at the level of the source code. These test cases are derived through the use of the design techniques mentioned above: control flow testing, data flow testing, branch testing, path testing, statement coverage and decision coverage as well as modified condition/decision coverage. White-box testing is the use of these techniques as guidelines to create an error-free environment by examining all code. These white-box testing techniques are the building blocks of white-box testing, whose essence is the careful testing of the application at the source code level to reduce hidden errors later on. ^[2] These different techniques exercise every visible path of the source code to minimize errors and create an error-free environment. The whole point of white-box testing is the ability to know which line of the code is being executed and being able to identify what the correct output should be. ^[2]

Levels

1. Unit testing. White-box testing is done during unit testing to ensure that the code is working as intended, before integration happens with previously tested code. White-box testing during unit testing potentially catches many defects early on and aids in addressing defects that happen later on after the code is integrated with the rest of the application and therefore reduces the impacts of errors later in development. ^[2]
2. Integration testing. White-box testing at this level is written to test the interactions of interfaces with each other. The unit level testing made sure that each code was tested and working accordingly in an isolated environment and integration examines the correctness of the behaviour in an open environment through the use of white-box testing for any interactions of interfaces that are known to the programmer. ^[2]
3. Regression testing. White-box testing during regression testing is the use of recycled white-box test cases at the unit and integration testing levels. ^[2]

Basic procedure

White-box testing's basic procedures require the tester to have an in-depth knowledge of the source code being tested. The programmer must have a deep understanding of the application to know what kinds of test cases to create so that every visible path is exercised for testing. Once the source code is understood then it can be analyzed for test cases to be created. The following are the three basic steps that white-box testing takes in order to create test cases:

1. Input involves different types of requirements, functional specifications, detailed designing of documents, proper source code and security specifications.^{[ citation needed ]} This is the preparation stage of white-box testing to lay out all of the basic information.
2. Processing involves performing risk analysis to guide whole testing process, proper test plan, execute test cases and communicate results.^{[ citation needed ]} This is the phase of building test cases to make sure they thoroughly test the application the given results are recorded accordingly.
3. Output involves preparing final report that encompasses all of the above preparations and results.^{[ citation needed ]}

Advantages

1. Side effects of having the knowledge of the source code is beneficial to thorough testing.^{[ citation needed ]}
2. Optimization of code becomes easy as inconspicuous bottlenecks are exposed.^{[ citation needed ]}
3. Gives the programmer introspection because developers carefully describe any new implementation.^{[ citation needed ]}
4. Provides traceability of tests from the source, thereby allowing future changes to the source to be easily captured in the newly added or modified tests. ^[3]
5. Easy to automate. ^[4]
6. Provides clear, engineering-based rules for when to stop testing. ^{[5] [4]}

Disadvantages

1. White-box tests are written to test the details of a specific implementation. This means that the tests will fail when the implementation changes as the test is tightly coupled to the implementation. Additional work has to be done to update the tests so they match the implementation again when it is changed. On the other hand with black-box testing, tests are independent of the implementation, and so they will still run successfully if the implementation changes but the output or side-effects of the implementation do not.
2. The code under test could be rewritten to implement the same functionality in a different way that invalidates the assumptions baked into the test. This could result in tests that fail unnecessarily or, in the worst case, tests that now give false positives and mask errors in the code. The white-box test never was written such that it tests the intended behavior of the code under test, but instead only such that the specific implementation does what it does.
3. White-box testing brings complexity to testing because the tester must have knowledge of the program, or the test team needs to have at least one very good programmer who can understand the program at the code level. White-box testing requires a programmer with a high level of knowledge due to the complexity of the level of testing that needs to be done.
4. On some occasions, it is not realistic to be able to test every single existing condition of the application and some conditions will be untested.
5. The tests focus on the software as it exists, and missing functionality may not be discovered.

Modern view

A more modern view is that the dichotomy between white-box testing and black-box testing has blurred and is becoming less relevant. Whereas "white-box" originally meant using the source code, and black-box meant using requirements, tests are now derived from many documents at various levels of abstraction. The real point is that tests are usually designed from an abstract structure such as the input space, a graph, or logical predicates, and the question is what level of abstraction we derive that abstract structure from. ^[4] That can be the source code, requirements, input space descriptions, or one of dozens of types of design models. Therefore, the "white-box / black-box" distinction is less important and the terms are less relevant.^{[ citation needed ]}

Hacking

In penetration testing, white-box testing refers to a method where a white hat hacker has full knowledge of the system being attacked. The goal of a white-box penetration test is to simulate a malicious insider who has knowledge of and possibly basic credentials for the target system.

See also

• Black-box testing
• Gray-box testing
• White-box cryptography

References

1. Stacy Nelson (June 2003), NASA/CR–2003-212806 Certification Processes for Safety-Critical and Mission-Critical Aerospace Software (PDF), Ames Research Center, p. 25, [Glossary] White Box Testing: Design-driven testing where engineers examine internal workings of code
2. Williams, Laurie. "White-Box Testing" (PDF). pp. 60–61, 69. Archived from the original (PDF) on 3 March 2016. Retrieved 13 February 2013.^{[ verification needed ]}
3. Binder, Bob (2000). Testing Object-oriented Systems . Addison-Wesley Publishing Company Inc. ISBN   9780201809381.
4. Ammann, Paul; Offutt, Jeff (2008). Introduction to Software Testing. Cambridge University Press. ISBN   978-0-521-88038-1.
5. Myers, Glenford (1979). The Art of Software Testing . John Wiley and Sons. ISBN   9780471043287.

External links

• BCS SIGIST (British Computer Society Specialist Interest Group in Software Testing): http://www.testingstandards.co.uk/Component%20Testing.pdf Standard for Software Component Testing], Working Draft 3.4, 27. April 2001.
• http://agile.csc.ncsu.edu/SEMaterials/WhiteBox.pdf has more information on control flow testing and data flow testing.
• http://research.microsoft.com/en-us/projects/pex/ Pex – Automated white-box testing for .NET