Risk-based testing

Last updated

Risk-based testing (RBT) is a type of software testing that functions as an organizational principle used to prioritize the tests of features and functions in software, based on the risk of failure, the function of their importance and likelihood or impact of failure. [1] [2] [3] In theory, there are an infinite number of possible tests. Risk-based testing uses risk (re-)assessments to steer all phases of the test process, i.e., test planning, test design, test implementation, test execution and test evaluation. [4] This includes for instance, ranking of tests, and subtests, for functionality; test techniques such as boundary-value analysis, all-pairs testing and state transition tables aim to find the areas most likely to be defective.

Contents

Types of risk assessment

Light-weight risk assessment

Lightweight risk-based testing methods mainly concentrate on two important factors: likelihood and impact. [5] Likelihood means how likely it is for a risk to happen, while impact measures how serious the consequences could be if the risk actually occurs. Instead of using complicated math, these techniques rely on simple judgments and scales. [6] For instance, a team might rate the chance of risk as high, medium, or low and its impact as severe, moderate, or minor. These ratings help prioritize where testing efforts should be focused. [7]

Heavy-weight risk assessment

Heavy-weighted risk-based testing is a method used to test software by focusing on the areas where problems are most likely to happen. The testing team looks for the most important parts of the software that might fail and concentrates on testing those parts more thoroughly.[ citation needed ]

There are four main types of heavy-weight risk-based testing methods [7] :

  1. Cost of Exposure: This looks at how much money a problem in the software might cause. It figures this out by thinking about how likely a problem is and how much it might cost.
  2. Failure Mode and Effect Analysis (FMEA): This technique finds out what parts of the software might fail, why they might fail, and what might happen if they do. It helps find the important areas that need attention.
  3. Quality Functional Deployment (QFD): This method helps connect what the users need with what the software does. It looks at risks that might come from not understanding what the users really want.
  4. Fault Tree Analysis (FTA): This technique is used to figure out why something went wrong by looking at different reasons in a step-by-step way..

Types of risk

Risk can be identified as the probability that an undetected software bug may have a negative impact on the user of a system. [8]

The methods assess risks along a variety of dimensions:

Business or operational

Technical

External

[9]

Some considerations about prioritizing risks is written by Venkat Ramakrishnan in a blog. [10]

Related Research Articles

Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not necessarily limited to:

Regression testing is re-running functional and non-functional tests to ensure that previously developed and tested software still performs as expected after a change. If not, that would be called a regression.

<span class="mw-page-title-main">Prototype</span> Early sample or model built to test a concept or process

A prototype is an early sample, model, or release of a product built to test a concept or process. It is a term used in a variety of contexts, including semantics, design, electronics, and software programming. A prototype is generally used to evaluate a new design to enhance precision by system analysts and users. Prototyping serves to provide specifications for a real, working system rather than a theoretical one. In some design workflow models, creating a prototype is the step between the formalization and the evaluation of an idea.

In product development and process optimization, a requirement is a singular documented physical or functional need that a particular design, product or process aims to satisfy. It is commonly used in a formal sense in engineering design, including for example in systems engineering, software engineering, or enterprise engineering. It is a broad concept that could speak to any necessary function, attribute, capability, characteristic, or quality of a system for it to have value and utility to a customer, organization, internal user, or other stakeholder. Requirements can come with different levels of specificity; for example, a requirement specification or requirement "spec" refers to an explicit, highly objective/clear requirement to be satisfied by a material, design, product, or service.

In software development, agile practices include requirements discovery and solutions improvement through the collaborative effort of self-organizing and cross-functional teams with their customer(s)/end user(s), Popularized in the 2001 Manifesto for Agile Software Development, these values and principles were derived from and underpin a broad range of software development frameworks, including Scrum and Kanban.

Failure mode and effects analysis is the process of reviewing as many components, assemblies, and subsystems as possible to identify potential failure modes in a system and their causes and effects. For each component, the failure modes and their resulting effects on the rest of the system are recorded in a specific FMEA worksheet. There are numerous variations of such worksheets. An FMEA can be a qualitative analysis, but may be put on a quantitative basis when mathematical failure rate models are combined with a statistical failure mode ratio database. It was one of the first highly structured, systematic techniques for failure analysis. It was developed by reliability engineers in the late 1950s to study problems that might arise from malfunctions of military systems. An FMEA is often the first step of a system reliability study.

Software prototyping is the activity of creating prototypes of software applications, i.e., incomplete versions of the software program being developed. It is an activity that can occur in software development and is comparable to prototyping as known from other fields, such as mechanical engineering or manufacturing.

In the context of software engineering, software quality refers to two related but distinct notions:

Reliability engineering is a sub-discipline of systems engineering that emphasizes the ability of equipment to function without failure. Reliability describes the ability of a system or component to function under stated conditions for a specified period of time. Reliability is closely related to availability, which is typically described as the ability of a component or system to function at a specified moment or interval of time.

<span class="mw-page-title-main">ARP4761</span>

ARP4761, Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment is an Aerospace Recommended Practice from SAE International. In conjunction with ARP4754, ARP4761 is used to demonstrate compliance with 14 CFR 25.1309 in the U.S. Federal Aviation Administration (FAA) airworthiness regulations for transport category aircraft, and also harmonized international airworthiness regulations such as European Aviation Safety Agency (EASA) CS–25.1309.

Software project management is the process of planning and leading software projects. It is a sub-discipline of project management in which software projects are planned, implemented, monitored and controlled.

IEC 61508 is an international standard published by the International Electrotechnical Commission (IEC) consisting of methods on how to apply, design, deploy and maintain automatic protection systems called safety-related systems. It is titled Functional Safety of Electrical/Electronic/Programmable Electronic Safety-related Systems.

In computer science, fault injection is a testing technique for understanding how computing systems behave when stressed in unusual ways. This can be achieved using physical- or software-based means, or using a hybrid approach. Widely studied physical fault injections include the application of high voltages, extreme temperatures and electromagnetic pulses on electronic components, such as computer memory and central processing units. By exposing components to conditions beyond their intended operating limits, computing systems can be coerced into mis-executing instructions and corrupting critical data.

In software engineering, a software development process is a process of planning and managing software development. It typically involves dividing software development work into smaller, parallel, or sequential steps or sub-processes to improve design and/or product management. It is also known as a software development life cycle (SDLC). The methodology may include the pre-definition of specific deliverables and artifacts that are created and completed by a project team to develop or maintain an application.

ISO 26262, titled "Road vehicles – Functional safety", is an international standard for functional safety of electrical and/or electronic systems that are installed in serial production road vehicles, defined by the International Organization for Standardization (ISO) in 2011, and revised in 2018.

Risk management tools allow the uncertainty to be addressed by identifying and generating metrics, parameterizing, prioritizing, and developing responses, and tracking risk. These activities may be difficult to track without tools and techniques, documentation and information systems.

In computer security, a threat is a potential negative action or event facilitated by a vulnerability that results in an unwanted impact to a computer system or application.

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. Continuous testing was originally proposed as a way of reducing waiting time for feedback to developers by introducing development environment-triggered tests as well as more traditional developer/tester-triggered tests.

Malware analysis is the study or process of determining the functionality, origin and potential impact of a given malware sample such as a virus, worm, trojan horse, rootkit, or backdoor. Malware or malicious software is any computer software intended to harm the host operating system or to steal sensitive data from users, organizations or companies. Malware may include software that gathers user information without permission.

This article discusses a set of tactics useful in software testing. It is intended as a comprehensive list of tactical approaches to Software Quality Assurance (more widely colloquially known as Quality Assurance and general application of the test method.

References

  1. Bach, J. The Challenge of Good Enough Software (1995)
  2. Bach, J. and Kaner, C. Exploratory and Risk Based Testing (2004)
  3. Mika Lehto (October 25, 2011). "The concept of risk-based testing and its advantages and disadvantages". Ictstandard.org. Retrieved 2012-03-01.
  4. Felderer, Michael; Schieferdecker, Ina (2014). "A taxonomy of risk-based testing". International Journal on Software Tools for Technology Transfer. 16 (5): 559–568. arXiv: 1912.11519 . doi:10.1007/s10009-014-0332-3. S2CID   11598143.
  5. Mahesh, Hari (2023-11-03). "Risk-based Testing: A Strategic Approach to QA". testRigor AI-Based Automated Testing Tool. Retrieved 2023-11-18.
  6. Schmitz, Christopher; Pape, Sebastian (2020-03-01). "LiSRA: Lightweight Security Risk Assessment for decision support in information security". Computers & Security. 90: 101656. doi:10.1016/j.cose.2019.101656. ISSN   0167-4048. S2CID   208109813.
  7. 1 2 "What is Risk Based Testing: With Best Practices". www.lambdatest.com. Retrieved 2023-11-18.
  8. Stephane Besson (2012-01-03). "Article info : A Strategy for Risk-Based Testing". Software Quality Engineering IT. Stickyminds.com. Retrieved 2012-03-01.
  9. Gerrard, Paul and Thompson, Neil Risk-Based Testing E-Business (2002)
  10. On Risk-Based Testing