Non-functional testing

Last updated

Non-functional testing is testing software for its non-functional requirements: the way a system operates, rather than specific behaviors of that system. [1] This is in contrast to functional testing, which tests against functional requirements that describe the functions of a system and its components.

Contents

Types of non-functional testing

Accessibility testing

Accessibility testing is a non-functional testing activity that verifies whether a system, website, or application can be perceived, operated, and understood by people with a wide range of disabilities and whether it meets objective accessibility criteria such as the Web Content Accessibility Guidelines, WCAG success criteria. It typically combines automated checks, to detect obvious technical failures, manual inspection, to evaluate semantic structure, keyboard navigation, and ARIA usage, and human usability testing with people who have disabilities to assess real-world effectiveness and usability. Many government and organizational web standards now require WCAG conformance and explicitly treat accessibility as a mandatory non-functional quality attribute of public-facing digital services. Practical guides and industry overviews describe accessibility testing as a specialized subset of usability and non-functional testing that focuses on legal conformance, inclusive design, and measurable success criteria rather than functional feature behaviour alone.* [2] [3]

Baseline testing

Baseline testing is a non-functional activity that establishes a measured reference for key quality attributes (for example, response time, throughput, resource usage, error rates, and availability) against which future changes, releases, or configurations are compared. It usually occurs early in a release cycle or after a major environment change and combines controlled synthetic tests (benchmarks, scripted workloads, profiling) with monitored production observations to capture representative operating conditions. The baseline results are recorded as pass/fail thresholds, performance budgets, or trend series so regressions, capacity drift, and configuration-induced degradation can be detected quickly and triaged. Best practice is to store baseline test artifacts (test scripts, input datasets, environment definitions, and raw metrics) alongside the release so reruns are reproducible and comparisons remain meaningful across time. [4] [5]

Compliance testing

This section is an excerpt from Conformance testing.[ edit ]
Conformance testing and also known as compliance testing or type testing, is testing or other activities that determine whether a process, product, or service complies with the requirements of a specification, technical standard, contract, or regulation. [6] [7] [8] [9] [10] It is an element of the more general conformity assessment.

Documentation testing

This section is an excerpt from Documentation testing.[ edit ]
Documentation testing is part of the non-functional testing of a product. It may be a type of black-box testing that ensures that documentation about how to use the system matches with what the system does, providing proof that system changes and improvements have been documented. [11]

Endurance testing

This section is an excerpt from Soak testing.[ edit ]
Soak testing involves testing a system with a typical production load, over a continuous availability period, to validate system behavior under production use. [12]

Load testing

This section is an excerpt from Load testing.[ edit ]
Load testing a railroad bridge, 1883 Umgeni River Bridge Load Test.jpg
Load testing a railroad bridge, 1883
Load testing is the process of putting demand on a structure or system and measuring its response.

Localization testing

This section is an excerpt from Pseudolocalization.[ edit ]
Pseudolocalization (or pseudo-localization) is a software testing method used for testing internationalization aspects of software. Instead of translating the text of the software into a foreign language, as in the process of localization, the textual elements of an application are replaced with an altered version of the original language. For example, instead of "Account Settings", the text may be altered to display as "!!! Àççôûñţ Šéţţîñĝš !!!". [13]

Reliability testing

This section is an excerpt from Reliability engineering § Reliability testing.[ edit ]
The purpose of reliability testing or reliability verification is to discover potential problems with the design as early as possible and, ultimately, provide confidence that the system meets its reliability requirements. The reliability of the product in all environments such as expected use, transportation, or storage during the specified lifespan should be considered. [14] It is to expose the product to natural or artificial environmental conditions to undergo its action to evaluate the performance of the product under the environmental conditions of actual use, transportation, and storage, and to analyze and study the degree of influence of environmental factors and their mechanism of action. [15] Through the use of various environmental test equipment to simulate the high temperature, low temperature, and high humidity, and temperature changes in the climate environment, to accelerate the reaction of the product in the use environment, to verify whether it reaches the expected quality in R&D, design, and manufacturing. [16]

Recovery testing

These paragraphs are an excerpt from Recovery testing.[ edit ]
In software testing, recovery testing is the activity of testing how well an application is able to recover from crashes, hardware failures and other similar problems.

Software performance testing

This section is an excerpt from Software performance testing.[ edit ]
In software quality assurance, performance testing is in general a testing practice performed to determine how a system performs in terms of responsiveness and stability under a particular workload. [17] It can also serve to investigate, measure, validate or verify other quality attributes of the system, such as scalability, reliability and resource usage.

Security testing

This section is an excerpt from Security testing.[ edit ]
Security testing is a process intended to detect flaws in the security mechanisms of an information system and as such help enable it to protect data and maintain functionality as intended. [18] Due to the logical limitations of security testing, passing the security testing process is not an indication that no flaws exist or that the system adequately satisfies the security requirements.

Scalability testing

This section is an excerpt from Scalability testing.[ edit ]
Scalability testing is the testing of a software application to measure its capability to scale up or scale out in terms of any of its non-functional capability.

Stress testing

These paragraphs are an excerpt from Stress testing.[ edit ]
Stress testing is a form of deliberately intense or thorough testing, used to determine the stability of a given system, critical infrastructure or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results.

Usability testing

These paragraphs are an excerpt from Usability testing.[ edit ]
Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. [19] It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will not last for long. [20] This is in contrast with usability inspection methods where experts use different methods to evaluate a user interface without involving users.

Volume testing

These paragraphs are an excerpt from Volume testing.[ edit ]
Volume testing belongs to the group of non-functional tests, which are a group of tests often misunderstood and/or used interchangeably. Volume testing refers to testing a software application with a certain amount of data to assert the system performance with a certain amount of data in the database. Volume testing is regarded by some as a type of capacity testing, [21] and is often deemed necessary as other types of tests normally don't use large amounts of data, but rather typically use small amounts of data. [21] It is the only type of test which checks the ability of a system to handle large pools of data. [22] For example, the test can be used to stress the database to its maximum limit. [23] While the amount can, in generic terms, be the database size, it could also be the size of an interface file that is the subject of volume testing. For example, if one wants to volume test an application with a specific database size, the database will be expanded to that size and the application's performance will then be tested on it. Another example could be when there is a requirement for the application to interact with an interface file (could be any file such as .dat, .xml); this interaction could be reading and/or writing on to/from the file. A sample file of an intended size can then be created and used to test the application's functionality in order to test the performance.

See also

References

  1. ↑ "non-functional testing - ISTQB Glossary". ISTQB Glossary. International Software Testing Qualifications Board. Retrieved 2025-11-04.
  2. ↑ Lazar, Jonathan; Goldstein, Daniel; Taylor, Anne Marie (2015). Ensuring Digital Accessibility through Process and Policy. Elsevier. ISBN   978-0128006467.
  3. ↑ Yesilada, Yeliz; Harper, Simon (2019). Web Accessibility: A Foundation for Research. Springer. ISBN   978-1447174400.
  4. ↑ "Baseline Testing". GeeksforGeeks. GeeksforGeeks. 2025-07-23. Retrieved 2025-11-04.
  5. ↑ "Baseline Testing: What is it, and Why is it Important?". Testsigma Blog. Testsigma. 2025-06-26. Retrieved 2025-11-04.
  6. ↑ Trenkaev, V.; Kim, M.; Seol, S. (2003). "Interoperability Testing Based on a Fault Model for a System of Communicating FSMs". In Hogrefe, D.; Wiles, A. (eds.). Testing of Communicating Systems: 15th IFIP International Conference, TestCom 2003. Springer Science & Business Media. pp. 226–242. ISBN   9783540401230.
  7. ↑ Ware, J.E. (1990). "Chapter 2: The Need for Mutual Recognition of Test Laboratories". In Fickelson, M. (ed.). Test Quality for Construction, Materials and Structures: Proceedings of the International RILEM/ILAC Symposium. CRC Press. pp. 12–18. ISBN   9780412394508.
  8. ↑ Werstiuk, C. (2007). "Chapter 2: Relay Testing Fundamentals". The Relay Testing Handbook: Relay Testing Fundamentals. Valence Electrical Training Services. pp. 13–52. ISBN   9781934348017.
  9. ↑ Ad-Hoc Group on Issues for Harmonizing Conformity Assessment to Biometric Standards (16 March 2005). "Report on Issues for Harmonizing Conformity Assessment to Biometric Standards" (PDF). InterNational Committee for Information Technology Standards. Archived from the original (PDF) on 13 December 2006. Retrieved 22 February 2018.
  10. ↑ "What is conformity assessment?". International Organization for Standardization. Retrieved 22 February 2018.
  11. ↑ "What Is Documentation Testing? – Offshore Software Testing with QATestLab".
  12. ↑ Ellingwood, Justin (May 20, 2017). "An Introduction to Continuous Integration, Delivery, and Deployment". DigitalOcean . Archived from the original on December 8, 2017. Retrieved February 3, 2018. Soak testing involves testing software under significant production or production-like load for an extended period of time.
  13. ↑ Benjamin Zadik (12 April 2013). "Pseudolocalization: Prepare your app for localization" . Retrieved 13 April 2013.
  14. ↑ Tang, Jianfeng; Chen, Jie; Zhang, Chun; Guo, Qing; Chu, Jie (2013-03-01). "Exploration on process design, optimization and reliability verification for natural gas deacidizing column applied to offshore field" . Chemical Engineering Research and Design. 91 (3): 542–551. Bibcode:2013CERD...91..542T. doi:10.1016/j.cherd.2012.09.018. ISSN   0263-8762.
  15. ↑ Zhang, J.; Geiger, C.; Sun, F. (January 2016). "A system approach to reliability verification test design". 2016 Annual Reliability and Maintainability Symposium (RAMS). pp. 1–6. doi:10.1109/RAMS.2016.7448014. ISBN   978-1-5090-0249-8. S2CID   24770411.
  16. ↑ Dai, Wei; Maropoulos, Paul G.; Zhao, Yu (2015-01-02). "Reliability modelling and verification of manufacturing processes based on process knowledge management". International Journal of Computer Integrated Manufacturing. 28 (1): 98–111. doi:10.1080/0951192X.2013.834462. ISSN   0951-192X. S2CID   32995968.
  17. ↑ Thakur, Nitish (2012). "Rational Performance Tester: Tips & Tricks" (PDF). IBM. Retrieved 2024-02-03.
  18. ↑ M Martellini, & Malizia, A. (2017). Cyber and chemical, biological, radiological, nuclear, explosives challenges : threats and counter efforts. Springer.
  19. ↑ Nielsen, J. (1994). Usability Engineering, Academic Press Inc, p 165
  20. ↑ Mejs, Monika (2019-06-27). "Usability Testing: the Key to Design Validation". Mood Up team - software house. Retrieved 2019-09-11.
  21. 1 2 Teststrategi for ytelsestesting - IKT-testing i Helse Nord
  22. ↑ Volume Testing in Software Testing: Definition & Examples | PFLB
  23. ↑ Volume Testing Tutorial: Examples and Volume Testing Tools

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.