Smoke testing (software)

Last updated

In computer programming and software testing, smoke testing (also confidence testing, sanity testing, [1] build verification test (BVT) [2] [3] [4] and build acceptance test) is preliminary testing or sanity testing to reveal simple failures severe enough to, for example, reject a prospective software release. Smoke tests are a subset of test cases that cover the most important functionality of a component or system, used to aid assessment of whether main functions of the software appear to work correctly. [1] [2] When used to determine if a computer program should be subjected to further, more fine-grained testing, a smoke test may be called a pretest [5] or an intake test. [1] Alternatively, it is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team. [6] In the DevOps paradigm, use of a build verification test step is one hallmark of the continuous integration maturity stage. [7]

Contents

For example, a smoke test may address basic questions like "does the program run?", "does the user interface open?", or "does clicking the main button do anything?" The process of smoke testing aims to determine whether the application is so badly broken as to make further immediate testing unnecessary. As the book Lessons Learned in Software Testing [8] puts it, "smoke tests broadly cover product features in a limited time [...] if key features don't work or if key bugs haven't yet been fixed, your team won't waste further time installing or testing". [3]

Smoke tests frequently run quickly, giving benefits of faster feedback, rather than running more extensive test suites, which would naturally take longer.

Frequent reintegration with smoke testing is among industry best practices. [9] [ need quotation to verify ] Ideally, every commit to a source code repository should trigger a Continuous Integration build, to identify regressions as soon as possible. If builds take too long, you might batch up several commits into one build, or very large systems might be rebuilt once a day. Overall, rebuild and retest as often as you can.

Smoke testing is also done by testers before accepting a build for further testing. Microsoft claims that after code reviews, "smoke testing is the most cost-effective method for identifying and fixing defects in software". [10]

One can perform smoke tests either manually or using an automated tool. In the case of automated tools, the process that generates the build will often initiate the testing.[ citation needed ]

Smoke tests can be functional tests or unit tests. Functional tests exercise the complete program with various inputs. Unit tests exercise individual functions, subroutines, or object methods. Functional tests may comprise a scripted series of program inputs, possibly even with an automated mechanism for controlling mouse movements. Unit tests can be implemented either as separate functions within the code itself, or else as a driver layer that links to the code without altering the code being tested.[ citation needed ]

Etymology

In Lessons Learned in Software Testing, Cem Kaner, James Bach, and Brett Pettichord provided the origin of the term: "The phrase smoke test comes from electronic hardware testing. You plug in a new board and turn on the power. If you see smoke coming from the board, turn off the power. You don't have to do any more testing." [3]

See also

Related Research Articles

<span class="mw-page-title-main">Acceptance testing</span> Test to determine if the requirements of a specification or contract are met

In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. It may involve chemical tests, physical tests, or performance tests.

Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not necessarily limited to:

A sanity check or sanity test is a basic test to quickly evaluate whether a claim or the result of a calculation can possibly be true. It is a simple check to see if the produced material is rational. The point of a sanity test is to rule out certain classes of obviously false results, not to catch every possible error. A rule-of-thumb or back-of-the-envelope calculation may be checked to perform the test. The advantage of performing an initial sanity test is that of speedily evaluating basic function.

System testing is testing conducted on a complete integrated system to evaluate the system's compliance with its specified requirements.

In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.

In software engineering, a test case is a specification of the inputs, execution conditions, testing procedure, and expected results that define a single test to be executed to achieve a particular software testing objective, such as to exercise a particular program path or to verify compliance with a specific requirement. Test cases underlie testing that is methodical rather than haphazard. A battery of test cases can be built to produce the desired coverage of the software being tested. Formally defined test cases allow the same tests to be run repeatedly against successive versions of the software, allowing for effective and consistent regression testing.

A test script in software testing is a set of instructions that will be performed on the system under test to test that the system functions as expected.

In software testing, a test harness is a collection of stubs and drivers configured to assist with the testing of an application or component. It acts as imitation infrastructure for test environments or containers where the full infrastructure is either not available or not desired.

Exploratory testing is an approach to software testing that is concisely described as simultaneous learning, test design and test execution. Cem Kaner, who coined the term in 1984, defines exploratory testing as "a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project."

<span class="mw-page-title-main">Certification</span> Formal confirmation of certain characteristics of an object, person or organization

Certification is part of testing, inspection and certification and the provision by an independent body of written assurance that the product, service or system in question meets specific requirements. It is the formal attestation or confirmation of certain characteristics of an object, person, or organization. This confirmation is often, but not always, provided by some form of external review, education, assessment, or audit. Accreditation is a specific organization's process of certification. According to the U.S. National Council on Measurement in Education, a certification test is a credentialing test used to determine whether individuals are knowledgeable enough in a given occupational area to be labeled "competent to practice" in that area.

The International Software Testing Qualifications Board (ISTQB) is a software testing certification board that operates internationally. Founded in Edinburgh in November 2002, the ISTQB is a non-profit association legally registered in Belgium.

Cem Kaner is a professor of software engineering at Florida Institute of Technology, and the Director of Florida Tech's Center for Software Testing Education & Research (CSTER) since 2004. He is perhaps best known outside academia as an advocate of software usability and software testing.

In software development, functional testing is a quality assurance (QA) process and a type of black-box testing that bases its test cases on the specifications of the software component under test. Functions are tested by feeding them input and examining the output, and internal program structure is rarely considered. Functional software testing is conducted to evaluate the compliance of a system or component with specified functional requirements. Functional testing usually describes what the system does.

Agile testing is a software testing practice that follows the principles of agile software development. Agile testing involves all members of a cross-functional agile team, with special expertise contributed by testers, to ensure delivering the business value desired by the customer at frequent intervals, working at a sustainable pace. Specification by example is used to capture examples of desired and undesired behavior and guide coding.

There is considerable variety among software testing writers and consultants about what constitutes responsible software testing. Proponents of a context-driven approach consider much of the writing about software testing to be doctrine, while others believe this contradicts the IEEE 829 documentation standard.

Gray-box testing is a combination of white-box testing and black-box testing. The aim of this testing is to search for the defects, if any, due to improper structure or improper usage of applications.

James Marcus Bach is an American software tester, author, trainer, and consultant.

Smoke testing refers to various classes of tests of systems, usually intended to determine whether they are ready for more robust testing. The expression probably was first used in plumbing in referring to tests for the detection of cracks, leaks or breaks in closed systems of pipes. By metaphorical extension the term is used in electronics. In Lessons Learned in Software Testing, Cem Kaner, James Bach, and Brett Pettichord provided the origin of the term: "The phrase smoke test comes from electronic hardware testing. You plug in a new board and turn on the power. If you see smoke coming from the board, turn off the power. You don't have to do any more testing."

This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to Software Quality Assurance (more widely colloquially known as Quality Assurance and general application of the test method.

In software testing a test driver is a software component or application that initiates and controls the execution of a program under test, especially when such components are part of a larger system and cannot run in isolation. Drivers control applications across various stages of software testing, from unit and integration testing right through to system integration testing and acceptance testing , especially when the target module is a component of a larger system that is not yet fully implemented or otherwise unavailable.

References

  1. 1 2 3 ISTQB® Glossary for the International Software Testing Qualification Board® software testing qualification scheme, ISTQB Glossary International Software Testing Qualification Board.
  2. 1 2 Dustin, Rashka, Paul. "Automated Software Testing -Introduction, Management, and Performance". Addison-Wesley 1999, p. 43-44. ISBN   0-201-43287-0.
  3. 1 2 3 Kaner, Cem; Bach, James; Pettichord, Bret (2002). Lessons Learned in Software Testing. Wiley Computer Publishing. p. 95. ISBN   0-471-08112-4.
  4. ↑ "How to: Configure and Run Build Verification Tests (BVTs)". MSDN Library for Visual Studio 2005. Retrieved 2010-11-20.
  5. ↑ 2013-03-20 ISTQB Terminologi for test av programvare, versjon 2.2 Glossary Working Party, International Software Testing Qualifications Board, Erik van Veenendaal (engelsk), Ernst von Düring (norsk) "intake test: A special instance of a smoke test to decide if the component or system is ready for detailed and further testing. An intake test is typically carried out at the start of the test execution phase. See also smoke test."
  6. ↑ Samuel Menaker; Sheetal Guttigoli (14 December 2014). Managing Software Development. Samuel Menaker, Sheetal Guttigoli. p. 40. GGKEY:JH61NP21TXJ.
  7. ↑ PowerShell Magazine, DevOps, Infrastructure as Code, and PowerShell DSC: The Introduction, by Ravikanth C, January 5, 2016
  8. ↑ Cem Kaner, James Bach, Bret Pettichord, Lessons learned in software testing: a context-driven approach. Wiley, 2001
  9. ↑ McConnell, Steve. "Rapid Development". Microsoft Press, p. 405
  10. ↑ "Guidelines for Smoke Testing". MSDN Library for Visual Studio 2005. 26 June 2007. Retrieved 2010-11-20.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.