ISO/IEC 29119

Last updated

ISO/IEC/IEEE 29119Software and systems engineering -- Software testing [1] is a series of five international standards for software testing. First developed in 2007 [2] and released in 2013, the standard "defines vocabulary, processes, documentation, techniques, and a process assessment model for testing that can be used within any software development lifecycle." [3]

Contents

History and revisions

Development of the set of ISO/IEC/IEEE 29119 software testing standards began in May 2007, based on existing standards such as the Institute of Electrical and Electronics Engineers's IEEE 829 (test documentation), and IEEE 1008 (unit testing); and the BSI Group's BS 7925-1 (vocabulary) and -2 (software components). [2] [4]

At first the International Organization for Standardization (ISO) had no working group with significant software testing experience, so the ISO created WG26, which by 2011 was represented by more than 20 different countries. [2] Initially four sections were developed for the standard: Concepts and definitions (1), Test processes (2), Test documentation (3), and Test techniques (4). [2] A fifth part concerning process assessment was considered for addition, ultimately becoming ISO/IEC 33063:2015, which ties to 29119-2's test processes. [2] [5] The actual fifth part of 29119 was published in November 2016 concerning the concept of keyword-driven testing. [6]

As of June 2018, no major revisions have occurred to the five parts of the standard. These parts are, from most recent to oldest:

Structure and contents

ISO/IEC/IEEE 29119-1:2013, Part 1: Concepts and definitions

ISO/IEC/IEEE 29119 Part 1 facilitates the use of the other parts of the standard by introducing the vocabulary on which the standard is built and provides examples of its application in practice. [4] Part 1 provides definitions, a description of the concepts of software testing, and ways to apply these definitions and concepts to the other parts of the standard. [1]

ISO/IEC/IEEE 29119-2:2013, Part 2: Test processes

Part 2 defines a generic test process model for software testing that is intended for use by organizations when performing software testing. It comprises test process descriptions that define the software testing processes at the organizational level, test management (project) level, and dynamic test process levels (WG26 couldn't get consensus on the inclusion of static testing). [2] [3] [10] The processes defined in this standard can be used in conjunction with different software development lifecycle models. [9]

ISO/IEC/IEEE 29119-3:2013, Part 3: Test documentation

This part deals with software test documentation and includes templates and test documentation examples that are produced during the test process. The templates support the three primary test process levels of Part 2, and the standard also includes mapping to other existing standards. [2] [8]

The documents that are defined in ISO/IEC/IEEE 29119-3 are as follows: [2] [4]

Organizational Test Process Documentation:
- Test Policy
- Organizational Test Strategy
Test Management Process Documentation:
- Test Plan (including a Test Strategy)
- Test Status
- Test Completion
Dynamic Test Process Documentation:
- Test Design Specification
- Test Case Specification
- Test Procedure Specification
- Test Data Requirements
- Test Data Readiness Report
- Test Environment Requirements
- Test Environment Readiness Report
- Actual Results
- Test Result
- Test Execution Log
- Test Incident Report

ISO/IEC/IEEE 29119-4:2015, Part 4: Test techniques

Part 4 provides standard definitions of software test design techniques (also known as test case design techniques or test methods) and corresponding coverage measures that can be used during the test design and implementation processes defined in Part 2. [7] Techniques of Part 4 are intended to support or be used separately from Part 2. The standard's test design techniques are categorized into three main categories: Specification-, Structure-, and Experience-Based Test Design Techniques. [2] [11]

Specification-based test design techniques

These techniques are based on the (functional) specification of the system under test. They are also called black-box testing techniques.

Suggested test design techniques in this group are: [2] [11]

Structure-based test design techniques

These structural testing techniques are based on the (internal) structure of the system under test. They are also called white-box testing techniques.

Suggested test design techniques in this group are: [2] [11]

Experience-based test design techniques

These exploratory testing techniques rely on the experience of the human tester.

Suggested test design techniques in this group are: [11]

ISO/IEC/IEEE 29119-5:2016, Part 5: Keyword-driven testing

This standard covers keyword-driven testing, an approach to specifying software tests (normally automated) used in the software testing industry. [12] This standard is intended for users "who want to create keyword-driven test specifications, create corresponding frameworks, or build test automation based on keywords." [6]

Controversy

Upon introduction of ISO/IEC/IEEE 29119, culminating in the summer of 2014, some software testers and associated organizations began making calls for the ISO to rescind the standard. [13] [14] [15] [16] [17] [18] [19] Notable organizations that protested the standard included the Association for Software Testing [18] and the International Society for Software Testing [20] Some reasons for opposition to the standard included:

Dr. Stuart Reid, convenor of WG26, replied to some of these and other opposing viewpoints in September 2014, [21] though it's not clear if any further action by WG26 was taken due to software testers' complaints.

Related Research Articles

ANSI C, ISO C, and Standard C are successive standards for the C programming language published by the American National Standards Institute (ANSI) and ISO/IEC JTC 1/SC 22/WG 14 of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). Historically, the names referred specifically to the original and best-supported version of the standard. Software developers writing in C are encouraged to conform to the standards, as doing so helps portability between compilers.

<span class="mw-page-title-main">Acceptance testing</span> Test to determine if the requirements of a specification or contract are met

In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. It may involve chemical tests, physical tests, or performance tests.

<span class="mw-page-title-main">International Organization for Standardization</span> International standards development organization

The International Organization for Standardization is an international standard development organization composed of representatives from the national standards organizations of member countries. Membership requirements are given in Article 3 of the ISO Statutes.

Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not necessarily limited to:

ISO/IEC/IEEE 12207Systems and software engineering – Software life cycle processes is an international standard for software lifecycle processes. First introduced in 1995, it aims to be a primary standard that defines all the processes required for developing and maintaining software systems, including the outcomes and/or activities of each process.

A test script in software testing is a set of instructions that will be performed on the system under test to test that the system functions as expected.

Software quality assurance (SQA) is a means and practice of monitoring all software engineering processes, methods, and work products to ensure compliance against defined standards. It may include ensuring conformance to standards or models, such as ISO/IEC 9126, SPICE or CMMI.

Office Open XML is a zipped, XML-based file format developed by Microsoft for representing spreadsheets, charts, presentations and word processing documents. Ecma International standardized the initial version as ECMA-376. ISO and IEC standardized later versions as ISO/IEC 29500.

International standards in the ISO/IEC 19770 family of standards for IT asset management address both the processes and technology for managing software assets and related IT assets. Broadly speaking, the standard family belongs to the set of Software Asset Management standards and is integrated with other Management System Standards.

ISO/IEC/IEEE 42010Systems and software engineering — Architecture description is an international standard for architecture descriptions of systems and software.

ISO/IEC JTC 1/SC 22 Programming languages, their environments and system software interfaces is a standardization subcommittee of the Joint Technical Committee ISO/IEC JTC 1 of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) that develops and facilitates standards within the fields of programming languages, their environments and system software interfaces. ISO/IEC JTC 1/SC 22 is also sometimes referred to as the "portability subcommittee". The international secretariat of ISO/IEC JTC 1/SC 22 is the American National Standards Institute (ANSI), located in the United States.

A specification often refers to a set of documented requirements to be satisfied by a material, design, product, or service. A specification is often a type of technical standard.

ISO/IEC JTC 1/SC 27 Information security, cybersecurity and privacy protection is a standardization subcommittee of the Joint Technical Committee ISO/IEC JTC 1 of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC). ISO/IEC JTC 1/SC 27 develops International Standards, Technical Reports, and Technical Specifications within the field of information security. Standardization activity by this subcommittee includes general methods, management system requirements, techniques and guidelines to address information security, cybersecurity and privacy. Drafts of International Standards by ISO/IEC JTC 1 or any of its subcommittees are sent out to participating national standardization bodies for ballot, comments and contributions. Publication as an ISO/IEC International Standard requires approval by a minimum of 75% of the national bodies casting a vote. The international secretariat of ISO/IEC JTC 1/SC 27 is the Deutsches Institut für Normung (DIN) located in Germany.

ISO/IEC JTC 1/SC 7 Software and systems engineering is a standardization subcommittee of the Joint Technical Committee ISO/IEC JTC 1 of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), that develops and facilitates standards within the field of engineering of software products and systems. The international secretariat of ISO/IEC JTC 1/SC 7 is the Bureau of Indian Standards (BIS) located in India.

ISO/IEC JTC 1/SC 2 Coded character sets is a standardization subcommittee of the Joint Technical Committee ISO/IEC JTC 1 of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), that develops and facilitates standards within the field of coded character sets. The international secretariat of ISO/IEC JTC 1/SC 2 is the Japanese Industrial Standards Committee (JISC), located in Japan. SC 2 is responsible for the development of the Universal Coded Character Set which is the international standard corresponding to the Unicode Standard.

ISO/IEC JTC 1/SC 24 Computer graphics, image processing and environmental data representation is a standardization subcommittee of the joint subcommittee ISO/IEC JTC 1 of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), which develops and facilitates standards within the field of computer graphics, image processing, and environmental data representation. The international secretariat of ISO/IEC JTC 1/SC 24 is the British Standards Institute (BSI) located in the United Kingdom.

ISO/IEC JTC 1/SC 31 Automatic identification and data capture techniques is a subcommittee of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) Joint Technical Committee (JTC) 1, and was established in 1996. SC 31 develops and facilitates international standards, technical reports, and technical specifications in the field of automatic identification and data capture techniques. The first Plenary established three working groups (WGs): Data Carriers, Data Content, and Conformance. Subsequent Plenaries established other working groups: RFID, RTLS, Mobile Item Identification and Management, Security and File Management, and Applications.

References

  1. 1 2 3 "ISO/IEC/IEEE 29119-1:2013". Standards catalogue. International Organization for Standardization. September 2013. Retrieved 24 June 2018.
  2. 1 2 3 4 5 6 7 8 9 10 11 Reid, S. (2012). "The New Software Testing Standard". In Dale, C.; Anderson, T. (eds.). Achieving Systems Safety: Proceedings of the Twentieth Safety-Critical Systems Symposium, Bristol, UK, 7-9th February 2012. Springer Science & Business Media. pp. 237–56. ISBN   9781447124948.
  3. 1 2 Shoemaker, D.; Sigler, K. (2014). "Chapter 6: Software Impelementation Process Group". Cybersecurity: Engineering a Secure Information Technology Organization. Cengage Learning. pp. 117–34. ISBN   9781305443471.
  4. 1 2 3 Pröll, R.; Bauer, B. (2018). "Toward a Consistent and Strictly Model-Based Interpretation of the ISO/IEC/IEEE 29119 for Early Testing Activities". Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development (MODELSWARD 2018): 699–706. doi: 10.5220/0006749606990706 . ISBN   978-989-758-283-7.
  5. ↑ "ISO/IEC 33063:2015". Standards catalogue. International Organization for Standardization. August 2015. Retrieved 24 June 2018.
  6. 1 2 3 "ISO/IEC/IEEE 29119-5:2016". Standards catalogue. International Organization for Standardization. November 2016. Retrieved 24 June 2018.
  7. 1 2 "ISO/IEC/IEEE 29119-4:2015". Standards catalogue. International Organization for Standardization. December 2015. Retrieved 24 June 2018.
  8. 1 2 "ISO/IEC/IEEE 29119-3:2013". Standards catalogue. International Organization for Standardization. September 2013. Retrieved 24 June 2018.
  9. 1 2 "ISO/IEC/IEEE 29119-2:2013". Standards catalogue. International Organization for Standardization. September 2013. Retrieved 24 June 2018.
  10. ↑ Kasurinen, J.; Runeson, P.; Riungu, L.; et al. (2011). "A Self-assessment Framework for Finding Improvement Objectives with ISO/IEC 29119 Test Standard". In Connor, R.V.; Pries-Heje, J.; Messnarz, R. (eds.). Systems, Software and Services Process Improvement: 18th European Conference, EuroSPI 2011, Roskilde, Denmark, June 27-29, 2011, Proceedings. Springer Science & Business Media. pp. 25–36. ISBN   9783642222054.
  11. 1 2 3 4 "ISO/IEC/IEEE 29119-4:2015-12 (E) Table of Contents". Normenausschuss Informationstechnik und Anwendungen (NIA). Deutsches Institut für Normung e. V. Retrieved 24 June 2018.
  12. ↑ Faught, D.R. (5 November 2004). "Keyword-Driven Testing". StickyMinds. TechWell Corp. Retrieved 24 June 2018.
  13. 1 2 3 4 Krill, P. (22 August 2014). "Software testers balk at ISO 29119 standards proposal". InfoWorld. IDG Communications, Inc. Retrieved 24 June 2018.
  14. 1 2 Demman, J. (9 September 2014). "Software testers petition to stop ISO 29119". TechTarget SearchSoftwareQuality. TechTarget, Inc. Retrieved 24 June 2018.
  15. 1 2 3 McCowatt, I. (24 August 2014). "STOP 29119". Exploring Uncertainty: A Software Testing Blog. Archived from the original on 13 October 2014. Retrieved 24 June 2018.
  16. 1 2 3 Bach, J. (25 August 2014). "How Not to Standardize Testing (ISO 29119)". Satisfice Blog. Satisfice, Inc. Retrieved 24 June 2018.
  17. 1 2 3 Kaner, C. (28 August 2014). "Please sign the Petition to Stop ISO 29119". context-driven-testing.com. Retrieved 24 June 2018.
  18. 1 2 3 4 5 "The ISO29119 debate". Association for Software Testing. 5 September 2014. Retrieved 24 June 2018.
  19. 1 2 3 Christie, J. (28 August 2015). "Why ISO 29119 Is a Flawed Quality Standard". Better Software Magazine. Retrieved 23 July 2018.
  20. 1 2 "Petition against ISO 29119". International Society for Software Testing. 19 August 2014. Archived from the original on 8 February 2017. Retrieved 24 June 2018.
  21. ↑ Marnane, T.; Reid, S. (10 September 2014). "Response to Stop 29119 Petition". SoftwareTestingStandard.org. Archived from the original on 16 February 2020. Retrieved 24 June 2018.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.