| Part of a series on |
| Software development |
|---|
In computer programming, unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source code is tested to validate expected behavior. [1]
It is a common step in development processes such as Agile.
Unit testing, as principle for testing separately smaller parts of large software systems dates back to the early days of software engineering. In June 1956, H.D. Benington presented at US Navy's Symposium on Advanced Programming Methods for Digital Computers the SAGE project and its specification based approach where the coding phase was followed by "parameter testing" to validate component subprograms against their specification, followed then by an "assembly testing" for parts put together. [2] [3]
In 1964, a similar approach is described for the software of the Mercury project, where individual units developed by different programmes underwent "unit tests" before being integrated together. [4] In 1969, testing methodologies appear more structured, with unit tests, component tests and integration tests with the purpose of validating individual parts written separately and their progressive assembly into larger blocks. [5] Some public standards adopted end of the 60's, such as MIL-STD-483 [6] and MIL-STD-490 contributed further to a wide acceptance of unit testing in large projects.
Unit testing was in those times interactive [3] or automated, [7] using either coded tests or capture and replay testing tools. In 1989, Kent Beck described a testing framework for Smalltalk (later called SUnit) in "Simple Smalltalk Testing: With Patterns". In 1997, Kent Beck and Erich Gamma developed and released JUnit, a unit test framework that became popular with Java developers. [8] Google embraced automated testing around 2005–2006. [9]
Unit generally implies a relatively small amount of code; code that can be isolated from the rest of a codebase which may be a large and complex system. [10] In procedural programming, a unit is typically a function or a module. In object-oriented programming, a unit is typically a method, object or class. [11]
Unit tests can be performed manually or via automated test execution. Automated tests include benefits such as: running tests often, running tests without staffing cost, consistent and repeatable testing.
Testing is often performed by the programmer who writes and modifies the code under test. Unit testing may be viewed as part of the process of writing code.
 | This section needs additional citations for verification .(September 2019) |
During development, a programmer may code criteria, or results that are known to be good, into the test to verify the unit's correctness.
During test execution, frameworks log tests that fail any criterion and report them in a summary.
For this, the most commonly used approach is test - function - expected value.
A test case describes the expected behavior (i.e. output) of the code under test for a particular setup (i.e. input).
Practitioners recommend organizing test cases to be independent of each other and running them independently so that when one fails other test cases can be still be run and potentially still pass.
A parameterized test is a test that accepts a set of values that can be used to enable the test to run with multiple, different input values. A testing framework that supports parametrized tests supports a way to encode parameter sets and to run the test with each set.
Use of parametrized tests can reduce test code duplication.
Parameterized tests are supported by TestNG, JUnit, [12] XUnit and NUnit, as well as in various JavaScript test frameworks.[ citation needed ]
Parameters for the unit tests may be coded manually or in some cases are automatically generated by the test framework. In recent years support was added for writing more powerful (unit) tests, leveraging the concept of theories, test cases that execute the same steps, but using test data generated at runtime, unlike regular parameterized tests that use the same execution steps with input sets that are pre-defined.[ citation needed ]
Sometimes, in the agile software development, unit testing is done per user story and comes in the later half of the sprint after requirements gathering and development are complete. Typically, the developers or other members from the development team, such as consultants, will write step-by-step 'test scripts' for the developers to execute in the tool. Test scripts are generally written to prove the effective and technical operation of specific developed features in the tool, as opposed to full fledged business processes that would be interfaced by the end user, which is typically done during user acceptance testing. If the test-script can be fully executed from start to finish without incident, the unit test is considered to have "passed", otherwise errors are noted and the user story is moved back to development in an 'in-progress' state. User stories that successfully pass unit tests are moved on to the final steps of the sprint - Code review, peer review, and then lastly a 'show-back' session demonstrating the developed tool to stakeholders.
In test-driven development (TDD), unit tests are written while the production code is written. Starting with working code, the developer adds test code for a required behavior, then adds just enough code to make the test pass, then refactors the code (including test code) as makes sense and then repeats by adding another test.
Unit testing is intended to ensure that the units meet their design and behave as intended. [13]
By writing tests first for the smallest testable units, then the compound behaviors between those, one can build up comprehensive tests for complex applications. [13]
One goal of unit testing is to isolate each part of the program and show that the individual parts are correct. [1] A unit test provides a strict, written contract that the piece of code must satisfy.
Unit testing finds problems early in the development cycle. This includes both bugs in the programmer's implementation and flaws or missing parts of the specification for the unit. The process of writing a thorough set of tests forces the author to think through inputs, outputs, and error conditions, and thus more crisply define the unit's desired behavior.[ citation needed ]
The cost of finding a bug before coding begins or when the code is first written is considerably lower than the cost of detecting, identifying, and correcting the bug later. Bugs in released code may also cause costly problems for the end-users of the software. [14] [15] [16] Code can be impossible or difficult to unit test if poorly written, thus unit testing can force developers to structure functions and objects in better ways.
Unit testing enables more frequent releases in software development. By testing individual components in isolation, developers can quickly identify and address issues, leading to faster iteration and release cycles. [17]
Unit testing allows the programmer to refactor code or upgrade system libraries at a later date, and make sure the module still works correctly (e.g., in regression testing). The procedure is to write test cases for all functions and methods so that whenever a change causes a fault, it can be identified quickly.
Unit tests detect changes which may break a design contract.
Unit testing may reduce uncertainty in the units themselves and can be used in a bottom-up testing style approach. By testing the parts of a program first and then testing the sum of its parts, integration testing becomes much easier.[ citation needed ]
Some programmers contend that unit tests provide a form of documentation of the code. Developers wanting to learn what functionality is provided by a unit, and how to use it, can review the unit tests to gain an understanding of it.[ citation needed ]
Test cases can embody characteristics that are critical to the success of the unit. These characteristics can indicate appropriate/inappropriate use of a unit as well as negative behaviors that are to be trapped by the unit. A test case documents these critical characteristics, although many software development environments do not rely solely upon code to document the product in development.[ citation needed ]
In some processes, the act of writing tests and the code under test plus associated refactoring may take the place of formal design. Each unit test can be seen as a design element specifying classes, methods, and observable behavior.[ citation needed ]
Testing will not catch every error in the program, because it cannot evaluate every execution path in any but the most trivial programs. This problem is a superset of the halting problem, which is undecidable. The same is true for unit testing. Additionally, unit testing by definition only tests the functionality of the units themselves. Therefore, it will not catch integration errors or broader system-level errors (such as functions performed across multiple units, or non-functional test areas such as performance). Unit testing should be done in conjunction with other software testing activities, as they can only show the presence or absence of particular errors; they cannot prove a complete absence of errors. To guarantee correct behavior for every execution path and every possible input, and ensure the absence of errors, other techniques are required, namely the application of formal methods to proving that a software component has no unexpected behavior.[ citation needed ]
An elaborate hierarchy of unit tests does not equal integration testing. Integration with peripheral units should be included in integration tests, but not in unit tests.[ citation needed ] Integration testing typically still relies heavily on humans testing manually; high-level or global-scope testing can be difficult to automate, such that manual testing often appears faster and cheaper.[ citation needed ]
Software testing is a combinatorial problem. For example, every Boolean decision statement requires at least two tests: one with an outcome of "true" and one with an outcome of "false". As a result, for every line of code written, programmers often need 3 to 5 lines of test code.[ citation needed ] This obviously takes time and its investment may not be worth the effort. There are problems that cannot easily be tested at all – for example those that are nondeterministic or involve multiple threads. In addition, code for a unit test is as likely to be buggy as the code it is testing. Fred Brooks in The Mythical Man-Month quotes: "Never go to sea with two chronometers; take one or three." [18] Meaning, if two chronometers contradict, how do you know which one is correct?
Another challenge related to writing the unit tests is the difficulty of setting up realistic and useful tests. It is necessary to create relevant initial conditions so the part of the application being tested behaves like part of the complete system. If these initial conditions are not set correctly, the test will not be exercising the code in a realistic context, which diminishes the value and accuracy of unit test results.[ citation needed ]
To obtain the intended benefits from unit testing, rigorous discipline is needed throughout the software development process.
It is essential to keep careful records not only of the tests that have been performed, but also of all changes that have been made to the source code of this or any other unit in the software. Use of a version control system is essential. If a later version of the unit fails a particular test that it had previously passed, the version-control software can provide a list of the source code changes (if any) that have been applied to the unit since that time.[ citation needed ]
It is also essential to implement a sustainable process for ensuring that test case failures are reviewed regularly and addressed immediately. [19] If such a process is not implemented and ingrained into the team's workflow, the application will evolve out of sync with the unit test suite, increasing false positives and reducing the effectiveness of the test suite.
Unit testing embedded system software presents a unique challenge: Because the software is being developed on a different platform than the one it will eventually run on, you cannot readily run a test program in the actual deployment environment, as is possible with desktop programs. [20]
Unit tests tend to be easiest when a method has input parameters and some output. It is not as easy to create unit tests when a major function of the method is to interact with something external to the application. For example, a method that will work with a database might require a mock up of database interactions to be created, which probably won't be as comprehensive as the real database interactions. [21] [ better source needed ]
Below is an example of a JUnit test suite. It focuses on the Adder class.
classAdder{publicintadd(inta,intb){returna+b;}}The test suite uses assert statements to verify the expected result of various input values to the sum method.
import staticorg.junit.Assert.assertEquals;importorg.junit.Test;publicclassAdderUnitTest{@TestpublicvoidsumReturnsZeroForZeroInput(){Adderadder=newAdder();assertEquals(0,adder.add(0,0));}@TestpublicvoidsumReturnsSumOfTwoPositiveNumbers(){Adderadder=newAdder();assertEquals(3,adder.add(1,2));}@TestpublicvoidsumReturnsSumOfTwoNegativeNumbers(){Adderadder=newAdder();assertEquals(-3,adder.add(-1,-2));}@TestpublicvoidsumReturnsSumOfLargeNumbers(){Adderadder=newAdder();assertEquals(2222,adder.add(1234,988));}}| |
Using unit-tests as a design specification has one significant advantage over other design methods: The design document (the unit-tests themselves) can itself be used to verify the implementation. The tests will never pass unless the developer implements a solution according to the design.
Unit testing lacks some of the accessibility of a diagrammatic specification such as a UML diagram, but they may be generated from the unit test using automated tools. Most modern languages have free tools (usually available as extensions to IDEs). Free tools, like those based on the xUnit framework, outsource to another system the graphical rendering of a view for human consumption.
Unit testing is the cornerstone of extreme programming, which relies on an automated unit testing framework. This automated unit testing framework can be either third party, e.g., xUnit, or created within the development group.
Extreme programming uses the creation of unit tests for test-driven development. The developer writes a unit test that exposes either a software requirement or a defect. This test will fail because either the requirement isn't implemented yet, or because it intentionally exposes a defect in the existing code. Then, the developer writes the simplest code to make the test, along with other tests, pass.
Most code in a system is unit tested, but not necessarily all paths through the code. Extreme programming mandates a "test everything that can possibly break" strategy, over the traditional "test every execution path" method. This leads developers to develop fewer tests than classical methods, but this isn't really a problem, more a restatement of fact, as classical methods have rarely ever been followed methodically enough for all execution paths to have been thoroughly tested.[ citation needed ] Extreme programming simply recognizes that testing is rarely exhaustive (because it is often too expensive and time-consuming to be economically viable) and provides guidance on how to effectively focus limited resources.
Crucially, the test code is considered a first class project artifact in that it is maintained at the same quality as the implementation code, with all duplication removed. Developers release unit testing code to the code repository in conjunction with the code it tests. Extreme programming's thorough unit testing allows the benefits mentioned above, such as simpler and more confident code development and refactoring, simplified code integration, accurate documentation, and more modular designs. These unit tests are also constantly run as a form of regression test.
Unit testing is also critical to the concept of Emergent Design. As emergent design is heavily dependent upon refactoring, unit tests are an integral component.[ citation needed ]
An automated testing framework provides features for automating test execution and can accelerate writing and running tests. Frameworks have been developed for a wide variety of programming languages.
Generally, frameworks are third-party; not distributed with a compiler or integrated development environment (IDE).
Tests can be written without using a framework to exercise the code under test using assertions, exception handling, and other control flow mechanisms to verify behavior and report failure. Some note that testing without a framework is valuable since there is a barrier to entry for the adoption of a framework; that having some tests is better than none, but once a framework is in place, adding tests can be easier. [22]
In some frameworks advanced test features are missing and must be hand-coded.
Some programming languages directly support unit testing. Their grammar allows the direct declaration of unit tests without importing a library (whether third party or standard). Additionally, the Boolean conditions of the unit tests can be expressed in the same syntax as Boolean expressions used in non-unit test code, such as what is used for if and while statements.
Languages with built-in unit testing support include:
Languages with standard unit testing framework support include:
Some languages do not have built-in unit-testing support but have established unit testing libraries or frameworks. These languages include:
JUnit is a test automation framework for the Java programming language. JUnit is often used for unit testing, and is one of the xUnit frameworks.
In computer programming and software design, code refactoring is the process of restructuring existing computer code—changing the factoring—without changing its external behavior. Refactoring is intended to improve the design, structure, and/or implementation of the software, while preserving its functionality. Potential advantages of refactoring may include improved code readability and reduced complexity; these can improve the source code's maintainability and create a simpler, cleaner, or more expressive internal architecture or object model to improve extensibility. Another potential goal for refactoring is improved performance; software engineers face an ongoing challenge to write programs that perform faster or use less memory.
Software testing is the act of examining the artifacts and behavior of software via verification and validation.
Design by contract (DbC), also known as contract programming, programming by contract and design-by-contract programming, is an approach for designing software.
xUnit is a label used for an automated testing software framework that shares significant structure and functionality with SUnit; designed by Kent Beck in 1998.
Software development is the process used to create software. Programming and maintaining the source code is the central step of this process, but it also includes conceiving the project, evaluating its feasibility, analyzing the business requirements, software design, testing, to release. Software engineering, in addition to development, also includes project management, employee management, and other overhead functions. Software development may be sequential, in which each step is complete before the next begins, but iterative development methods where multiple steps can be executed at once and earlier steps can be revisited have also been devised to improve flexibility, efficiency, and scheduling.
Test-driven development (TDD) a way of writing code that involves writing an automated test case that fails, then writing just enough code to make the test pass, then refactoring both the test code and the production code, then repeating with another new test case.
A programming tool or software development tool is a computer program that software developers use to create, debug, maintain, or otherwise support other programs and applications. The term usually refers to relatively simple programs, that can be combined to accomplish a task, much as one might use multiple hands to fix a physical object. The most basic tools are a source code editor and a compiler or interpreter, which are used ubiquitously and continuously. Other tools are used more or less depending on the language, development methodology, and individual engineer, often used for a discrete task, like a debugger or profiler. Tools may be discrete programs, executed separately – often from the command line – or may be parts of a single large program, called an integrated development environment (IDE). In many cases, particularly for simpler use, simple ad hoc techniques are used instead of a tool, such as print debugging instead of using a debugger, manual timing instead of a profiler, or tracking bugs in a text file or spreadsheet instead of a bug tracking system.
Web development is the work involved in developing a website for the Internet or an intranet. Web development can range from developing a simple single static page of plain text to complex web applications, electronic businesses, and social network services. A more comprehensive list of tasks to which Web development commonly refers, may include Web engineering, Web design, Web content development, client liaison, client-side/server-side scripting, Web server and network security configuration, and e-commerce development.
White-box testing is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality. In white-box testing, an internal perspective of the system is used to design test cases. The tester chooses inputs to exercise paths through the code and determine the expected outputs. This is analogous to testing nodes in a circuit, e.g. in-circuit testing (ICT). White-box testing can be applied at the unit, integration and system levels of the software testing process. Although traditional testers tended to think of white-box testing as being done at the unit level, it is used for integration and system testing more frequently today. It can test paths within a unit, paths between units during integration, and between subsystems during a system–level test. Though this method of test design can uncover many errors or problems, it has the potential to miss unimplemented parts of the specification or missing requirements. Where white-box testing is design-driven, that is, driven exclusively by agreed specifications of how each component of software is required to behave, white-box test techniques can accomplish assessment for unimplemented or missing requirements.
In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.
In object-oriented programming, a mock object is an object that simulates the behavior of a production code object in limited ways.
Mutation testing is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways. Each mutated version is called a mutant and tests detect and reject mutants by causing the behaviour of the original version to differ from the mutant. This is called killing the mutant. Test suites are measured by the percentage of mutants that they kill. New tests can be designed to kill additional mutants. Mutants are based on well-defined mutation operators that either mimic typical programming errors or force the creation of valuable tests. The purpose is to help the tester develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution. Mutation testing is a form of white-box testing.
Behavior-driven development (BDD) involves naming software tests using domain language to describe the behavior of the code.
Game testing, also called quality assurance (QA) testing within the video game industry, is a software testing process for quality control of video games. The primary function of game testing is the discovery and documentation of software defects. Interactive entertainment software testing is a highly technical field requiring computing expertise, analytic competence, critical evaluation skills, and endurance. In recent years the field of game testing has come under fire for being extremely strenuous and unrewarding, both financially and emotionally.
Extreme programming (XP) is an agile software development methodology used to implement software systems. This article details the practices used in this methodology. Extreme programming has 12 practices, grouped into four areas, derived from the best practices of software engineering.
Software construction is a software engineering discipline. It is the detailed creation of working meaningful software through a combination of coding, verification, unit testing, integration testing, and debugging. It is linked to all the other software engineering disciplines, most strongly to software design and software testing.
Extreme programming (XP) is a software development methodology intended to improve software quality and responsiveness to changing customer requirements. As a type of agile software development, it advocates frequent releases in short development cycles, intended to improve productivity and introduce checkpoints at which new customer requirements can be adopted.
This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to Software Quality Assurance (more widely colloquially known as Quality Assurance and general application of the test method.
{{cite journal}}: CS1 maint: date and year (link)The contractor shall then code and test software Units, and enter the source and object code, and associated listings of each successfully tested Unit into the Developmental Configuration
{{cite book}}: CS1 maint: date and year (link){{cite web}}: CS1 maint: location (link)