Pytest

Last updated

Pytest
Original author(s) Krekel et al.
Stable release
8.2.1 [1]   OOjs UI icon edit-ltr-progressive.svg / 19 May 2024;19 days ago (19 May 2024)
Repository
Written in Python
Platform macOS, Windows, POSIX
Type Framework for software testing
License MIT License
Website pytest.org   OOjs UI icon edit-ltr-progressive.svg

Pytest is a Python testing framework that originated from the PyPy project. It can be used to write various types of software tests, including unit tests, integration tests, end-to-end tests, and functional tests. Its features include parametrized testing, fixtures, and assert re-writing.

Contents

Pytest fixtures provide the contexts for tests by passing in parameter names in test cases; its parametrization eliminates duplicate code for testing multiple sets of input and output; and its rewritten assert statements provide detailed output for causes of failures.

History

Pytest was developed as part of an effort by third-party packages to address Python's built-in module unittest's shortcomings. It originated as part of PyPy, an alternative implementation of Python to the standard CPython. Since its creation in early 2003, PyPy has had a heavy emphasis on testing. PyPy had unit tests for newly written code, regression tests for bugs, and integration tests using CPython's test suite. [2]

In mid 2004, a testing framework called utest emerged and contributors to PyPy began converting existing test cases to utest. Meanwhile, at EuroPython 2004 a complementary standard library for testing, named std, was invented. This package laid out the principles, such as assert rewriting, of what would later become pytest. In late 2004, the std project was renamed to py, std.utest became py.test, and the py library was separated from PyPy. In November 2010, pytest 2.0.0 was released as a package separate from py. It was still called py.test until August 2016, but following the release of pytest 3.0.0 the recommended command line entry point became pytest. [3]

Pytest has been classified by developer security platform Snyk as one of the key ecosystem projects in Python due to its popularity. Some well-known projects who switched to pytest from unittest and nose (another testing package) include those of Mozilla and Dropbox. [4] [5] [6] [7]

Features

Parametrized testing

It is a common pattern in software testing to send values through test functions and check for correct output. In many cases, in order to thoroughly test functionalities, one needs to test multiple sets of input/output, and writing such cases separately would cause duplicate code as most of the actions would remain the same, only differing in input/output values. Pytest's parametrized testing feature eliminates such duplicate code by combining different iterations into one test case, then running these iterations and displaying each test's result separately. [8]

Parametrized tests in pytest are marked by the @pytest.mark.parametrize(argnames, argvalues) decorator, where the first parameter, argnames, is a string of comma-separated names, and argvalues is a list of values to pass into argnames. When there are multiple names in argnames, argvalues would be a list of tuples where values in each tuple corresponds to the names in argnames by index. The names in argnames are then passed into the test function marked by the decorator as parameters. When pytest runs such decorated tests, each pair of argnames and argvalues would constitute a separate run with its own test output and unique identifier. The identifier can then be used to run individual data pairs. [8] :52–58 [9]

Assert rewriting

When writing software tests, the assert statement is a primary means for communicating test failure, where expected values are compared to actual values. [8] :32–34 While Python's built-in assert keyword would only raise AssertionError with no details in cases of failure, pytest rewrites Python's assert keyword and provides detailed output for the causes of failures, such as what expressions in the assert statement evaluate to. A comparison can be made with unittest (Python's built-in module for testing)'s assert statements: [8] :32

pytestunittest
assertxassertTrue(x)
assertx==yassertEqual(x, y)
assertx<=yassertLessEqual(x, y)

unittest adheres to a more verbose syntax because it is inspired by the Java programming language's JUnit, as are most unit testing libraries; pytest achieves the same while intercepting Python's built-in assert calls, making the approach more concise. [8] :32 [6]

Pytest fixtures

Pytest's tests verify that computer code performs as expected [10] using tests that are structured in an arrange, act and assert sequence known as AAA. [11] Its fixtures provide the context for tests. They can be used to put a system into a known state and to pass data into test functions. Fixtures practically constitute the arrange phase in the anatomy of a test (AAA, short for arrange, act, assert). [11] [10] Pytest fixtures can run before test cases as setup or after test cases for clean up, but are different from unittest and nose (another third-party Python testing framework)'s setups and teardowns. Functions declared as pytest fixtures are marked by the @pytest.fixture decorator, whose names can then be passed into test functions as parameters. [12] When pytest finds the fixtures' names in test functions' parameters, it first searches in the same module for such fixtures, and if not found, it searches for such fixtures in the conftest.py file. [8] :61

For example:

importpytest@pytest.fixturedefdataset():"""Return some data to test functions"""return{'data1':1,'data2':2}deftest_dataset(dataset):"""test and confirm fixture value"""assertdataset=={'data1':1,'data2':2}

In the above example, pytest fixture dataset returns a dictionary, which is then passed into test function test_dataset for assertion. In addition to fixture detection within the same file as test cases, pytest fixtures can also be placed in the conftest.py file in the tests directory. There can be multiple conftest.py files, each placed within a tests directory for fixtures to be detected for each subset of tests. [8] :63

Fixture scopes

In pytest, fixture scopes let the user define when a fixture should be called. There are four fixture scopes: function scope, class scope, module scope, and session scope. Function-scoped fixtures are default for all pytest fixtures, which are called every time a function having the fixture as a parameter runs. The goal of specifying a broader fixture scope is to eliminate repeated fixture calls, which could slow down test execution. Class-scoped fixtures are called once per test class, regardless of the number of times they are called, and the same logic goes for all other scopes. When changing fixture scope, one need only add the scope parameter to fixture decorators, for example, @pytest.fixture(scope="class"). [8] :72 [13]

Test filtering

Another feature of pytest is its ability to filter through tests, where only desired tests are selected to run, or behave in a certain way as desired by the developer. With the "k" option (e.g. pytest -k some_name), pytest would only run tests whose names include some_name. The opposite is true, where one can run pytest -k "not some_name", and pytest will run all tests whose names do not include some_name. [14]

Pytest's markers can, in addition to altering test behaviour, also filter tests. Pytest's markers are Python decorators starting with the @pytest.mark.<markername> syntax placed on top of test functions. With different arbitrarily named markers, running pytest -m <markername> on the command line will only run those tests decorated with such markers. [8] :13 All available markers can be listed by the pytest --markers along with their descriptions; custom markers can also be defined by users and registered in pytest.ini, in which case pytest --markers will also list those custom markers along with builtin markers. [8] :147

See also

Related Research Articles

<span class="mw-page-title-main">Python (programming language)</span> General-purpose programming language

Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation.

In object-oriented programming, the decorator pattern is a design pattern that allows behavior to be added to an individual object, dynamically, without affecting the behavior of other instances of the same class. The decorator pattern is often useful for adhering to the Single Responsibility Principle, as it allows functionality to be divided between classes with unique areas of concern as well as to the Open-Closed Principle, by allowing the functionality of a class to be extended without being modified. Decorator use can be more efficient than subclassing, because an object's behavior can be augmented without defining an entirely new object.

Unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source code is tested to validate expected behavior.

In computer programming, a parameter or a formal argument is a special kind of variable used in a subroutine to refer to one of the pieces of data provided as input to the subroutine. These pieces of data are the values of the arguments with which the subroutine is going to be called/invoked. An ordered list of parameters is usually included in the definition of a subroutine, so that, each time the subroutine is called, its arguments for that call are evaluated, and the resulting values can be assigned to the corresponding parameters.

Jython is an implementation of the Python programming language designed to run on the Java platform. It was known as JPython until 1999.

Resource acquisition is initialization (RAII) is a programming idiom used in several object-oriented, statically typed programming languages to describe a particular language behavior. In RAII, holding a resource is a class invariant, and is tied to object lifetime. Resource allocation is done during object creation, by the constructor, while resource deallocation (release) is done during object destruction, by the destructor. In other words, resource acquisition must succeed for initialization to succeed. Thus the resource is guaranteed to be held between when initialization finishes and finalization starts, and to be held only when the object is alive. Thus if there are no object leaks, there are no resource leaks.

The Web Server Gateway Interface is a simple calling convention for web servers to forward requests to web applications or frameworks written in the Python programming language. The current version of WSGI, version 1.0.1, is specified in Python Enhancement Proposal (PEP) 3333.

In computer programming, thread-local storage (TLS) is a memory management method that uses static or global memory local to a thread. The concept allows storage of data that appears to be global in a system with separate threads.

CherryPy is an object-oriented web application framework using the Python programming language. It is designed for rapid development of web applications by wrapping the HTTP protocol but stays at a low level and does not offer much more than what is defined in RFC 7231.

doctest is a module included in the Python programming language's standard library that allows the easy generation of tests based on output from the standard Python interpreter shell, cut and pasted into docstrings.

<span class="mw-page-title-main">Python syntax and semantics</span> Set of rules defining correctly structured programs

The syntax of the Python programming language is the set of rules that defines how a Python program will be written and interpreted. The Python language has many similarities to Perl, C, and Java. However, there are some definite differences between the languages. It supports multiple programming paradigms, including structured, object-oriented programming, and functional programming, and boasts a dynamic type system and automatic memory management.

Web2py is an open-source web application framework written in the Python programming language. Web2py allows web developers to program dynamic web content using Python. Web2py is designed to help reduce tedious web development tasks, such as developing web forms from scratch, although a web developer may build a form from scratch if required.

The Wing Python IDE is a family of integrated development environments (IDEs) from Wingware created specifically for the Python programming language with support for editing, testing, debugging, inspecting/browsing, and error-checking Python code.

Shed Skin is an experimental restricted-Python (3.8+) to C++ programming language compiler. It can translate pure, but implicitly statically typed Python programs into optimized C++. It can generate stand-alone programs or extension modules that can be imported and used in larger Python programs.

In software engineering, the module pattern is a design pattern used to implement the concept of software modules, defined by modular programming, in a programming language with incomplete direct support for the concept.

<span class="mw-page-title-main">Symbolic regression</span> Type of regression analysis

Symbolic regression (SR) is a type of regression analysis that searches the space of mathematical expressions to find the model that best fits a given dataset, both in terms of accuracy and simplicity.

<span class="mw-page-title-main">Nim (programming language)</span> Programming language

Nim is a general-purpose, multi-paradigm, statically typed, compiled high-level systems programming language, designed and developed by a team around Andreas Rumpf. Nim is designed to be "efficient, expressive, and elegant", supporting metaprogramming, functional, message passing, procedural, and object-oriented programming styles by providing several features such as compile time code generation, algebraic data types, a foreign function interface (FFI) with C, C++, Objective-C, and JavaScript, and supporting compiling to those same languages as intermediate representations.

MicroPython is a software implementation of a programming language largely compatible with Python 3, written in C, that is optimized to run on a microcontroller.

PyTorch is a machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, originally developed by Meta AI and now part of the Linux Foundation umbrella. It is recognized as one of the two most popular machine learning libraries alongside TensorFlow, offering free and open-source software released under the modified BSD license. Although the Python interface is more polished and the primary focus of development, PyTorch also has a C++ interface.

Nuitka is a source-to-source compiler which compiles Python code to C source code, applying some compile-time optimizations in the process such as constant folding and propagation, built-in call prediction, type inference, and conditional statement execution. Nuitka initially was designed to produce C++ code, but current versions produce C source code using only those features of C11 that are shared by C++03, enabling further compilation to a binary executable format by modern C and C++ compilers including gcc, clang, MinGW, or Microsoft Visual C++. It accepts Python code compatible with several different Python versions and optionally allows for the creation of standalone programs that do not require Python to be installed on the target computer.

References

  1. "Release 8.2.1". 19 May 2024. Retrieved 25 May 2024.
  2. Bolz-Tereick, Carl Friedrich (9 September 2018). "PyPy Status Blog". PyPy. Retrieved 12 May 2022.
  3. "History". pytest. Retrieved 13 April 2022.
  4. "Project examples". Pytest. Retrieved 1 February 2022.
  5. Koorapati, Nipunn. "Open Sourcing Pytest Tools". Dropbox . Retrieved 1 February 2022.
  6. 1 2 Oliveira, Bruno (August 2018). pytest Quick Start Guide. Packt Publishing. ISBN   978-1-78934-756-2 . Retrieved 1 February 2022.
  7. "pytest". Snyk. Retrieved 12 May 2022.
  8. 1 2 3 4 5 6 7 8 9 10 Okken, Brian (September 2017). Python Testing with Pytest (1st ed.). The Pragmatic Bookshelf. ISBN   9781680502404 . Retrieved 22 January 2022.
  9. "Parametrizing fixtures and test functions". pytest.org. Retrieved 24 May 2022.
  10. 1 2 Viafore, Patrick (12 July 2021). Robust Python. O'Reilly Media, Inc. ISBN   978-1-0981-0061-2 . Retrieved 3 July 2022. Tests verify that what you build is performing as you expect.
  11. 1 2 Khorikov, Vladimir (January 2020). Unit Testing Principles, Practices, and Patterns. Published by Manning Publications. ISBN   9781617296277 . Retrieved 4 June 2022.
  12. "About fixtures". Pytest. Retrieved 7 February 2022.
  13. Ashwin, Pajankar (27 February 2017). Python Unit Test Automation: Practical Techniques for Python Developers and Testers. Apress. ISBN   9781484226766 . Retrieved 7 March 2022.
  14. Molina, Alessandro (February 2021). Crafting Test-Driven Software with Python. Publisher(s): Packt Publishing. ISBN   9781838642655 . Retrieved 8 March 2022.