Orthogonal defect classification (ODC) [1] turns semantic information in the software defect stream into a measurement on the process. [2] The ideas were developed in the late 1980s and early 1990s by Ram Chillarege [3] at IBM Research. This has led to the development of new analytical methods used for software development and test process analysis. ODC is process model, language and domain independent. Applications of ODC have been reported by several corporations on a variety of platforms and development processes, ranging from waterfall, spiral, gated, and agile [4] [5] development processes. One of the popular applications of ODC is software root cause analysis.
ODC is known to reduce the time taken to perform root cause analysis by over a factor of 10. The gains come primarily from a different approach to root cause analysis, where the ODC data is generated rapidly (in minutes, as opposed to hours per defect) and analytics used for the cause and effect analysis. This shifts the burden of analysis from a purely human method to one that is more data intensive. [6]
ODC as proposed in its original papers have specific attribute-value sets that create measurements on the development process. Two of the five more well known categories are the defect type and defect trigger. The defect type captures the changes made in the code as a result of the defect. There are seven values for defect type and they have been empirically established to provide a measurement of the product through the process through their distribution. The concept is that changes in the defect type distribution is a function of the development process model, and thus provides an intrinsic measurement of progress of the product through the process.
The defect trigger, similarly provides a measurement of the Testing process. The concept of the trigger is a key contribution that came through ODC and is now fairly widely used in technical and research publications. [7] The software trigger is defined as the force that surfaced the Fault to create the failure. The full set of triggers is available in ODC Documentation.
The defect type and trigger collectively provide a large amount of causal information on defects. Additional information from the defect that is captured in standard ODC implementations includes "impact", "source" and "age". ODC training courses report that, once trained, an individual can categorize a defect via ODC in less than 3 minutes when performing the task retrospectively. [8] The time taken is far lower when done in-flight, or in-process. The categorization cannot be directly compared to root-cause-analysis, since ODC data is about "what-is", not "why". However, root cause analysis is very commonly performed using ODC. The analysis that studies ODC data is performing the first pass of root cause analysis, which is confirmed by discussing the results with the development team. This approach has five primary differences between the classical method and the ODC method. [9]
Root cause analysis is just one of the applications of ODC. The original design of ODC was to create a measurement system for software engineering using the defect stream as a source of intrinsic measurements. Thus, the attributes, either singularly, or in conjunction with one of the others provides specific measurements on certain aspects of the engineering process. These measurements can be used for one or more analytical methods, since they were designed with general measurement principles in mind. Todate, several research papers have applied these for a variety of purposes. More recently, there have been research articles that use ODC to assess the methods used for security evaluation, and expanded the scope of ODC. [10]
Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to:
A software bug is an error, flaw or fault in the design, development, or operation of computer software that causes it to produce an incorrect or unexpected result, or to behave in unintended ways. The process of finding and correcting bugs is termed "debugging" and often uses formal techniques or tools to pinpoint bugs. Since the 1950s, some computer systems have been designed to detect or auto-correct various software errors during operations.
Software development is the process used to conceive, specify, design, program, document, test, and bug fix in order to create and maintain applications, frameworks, or other software components. Software development involves writing and maintaining the source code, but in a broader sense, it includes all processes from the conception of the desired software through the final manifestation, typically in a planned and structured process often overlapping with software engineering. Software development also includes research, new development, prototyping, modification, reuse, re-engineering, maintenance, or any other activities that result in software products.
Chemometrics is the science of extracting information from chemical systems by data-driven means. Chemometrics is inherently interdisciplinary, using methods frequently employed in core data-analytic disciplines such as multivariate statistics, applied mathematics, and computer science, in order to address problems in chemistry, biochemistry, medicine, biology and chemical engineering. In this way, it mirrors other interdisciplinary fields, such as psychometrics and econometrics.
Code review is a software quality assurance activity in which one or more people check a program, mainly by viewing and reading parts of its source code, either after implementation or as an interruption of implementation. At least one of the persons must not have authored the code. The persons performing the checking, excluding the author, are called "reviewers".
In product development and process optimization, a requirement is a singular documented physical or functional need that a particular design, product or process aims to satisfy. It is commonly used in a formal sense in engineering design, including for example in systems engineering, software engineering, or enterprise engineering. It is a broad concept that could speak to any necessary function, attribute, capability, characteristic, or quality of a system for it to have value and utility to a customer, organization, internal user, or other stakeholder. Requirements can come with different levels of specificity; for example, a requirement specification or requirement "spec" refers to an explicit, highly objective/clear requirement to be satisfied by a material, design, product, or service.
In software development, agile practices include requirements, discovery and solutions improvement through the collaborative effort of self-organizing and cross-functional teams with their customer(s)/end user(s). Popularized in the 2001 Manifesto for Agile Software Development, these values and principles were derived from, and underpin, a broad range of software development frameworks, including Scrum and Kanban.
In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.
The Personal Software Process (PSP) is a structured software development process that is designed to help software engineers better understand and improve their performance by bringing discipline to the way they develop software and tracking their predicted and actual development of the code. It clearly shows developers how to manage the quality of their products, how to make a sound plan, and how to make commitments. It also offers them the data to justify their plans. They can evaluate their work and suggest improvement direction by analyzing and reviewing development time, defects, and size data. The PSP was created by Watts Humphrey to apply the underlying principles of the Software Engineering Institute's (SEI) Capability Maturity Model (CMM) to the software development practices of a single developer. It claims to give software engineers the process skills necessary to work on a team software process (TSP) team.
In the context of software engineering, software quality refers to two related but distinct notions:
Reliability engineering is a sub-discipline of systems engineering that emphasizes the ability of equipment to function without failure. Reliability describes the ability of a system or component to function under stated conditions for a specified period of time. Reliability is closely related to availability, which is typically described as the ability of a component or system to function at a specified moment or interval of time.
Lean software development is a translation of lean manufacturing principles and practices to the software development domain. Adapted from the Toyota Production System, it is emerging with the support of a pro-lean subculture within the agile community. Lean offers a solid conceptual framework, values and principles, as well as good practices, derived from experience, that support agile organizations.
Unstructured data is information that either does not have a pre-defined data model or is not organized in a pre-defined manner. Unstructured information is typically text-heavy, but may contain data such as dates, numbers, and facts as well. This results in irregularities and ambiguities that make it difficult to understand using traditional programs as compared to data stored in fielded form in databases or annotated in documents.
Threat modeling is a process by which potential threats, such as structural vulnerabilities or the absence of appropriate safeguards, can be identified and enumerated, and countermeasures prioritized. The purpose of threat modeling is to provide defenders with a systematic analysis of what controls or defenses need to be included, given the nature of the system, the probable attacker's profile, the most likely attack vectors, and the assets most desired by an attacker. Threat modeling answers questions like "Where am I most vulnerable to attack?", "What are the most relevant threats?", and "What do I need to do to safeguard against these threats?".
NeSSI is a global and open initiative sponsored by the Center for Process Analysis and Control (CPAC) at the University of Washington, in Seattle.
Quality engineering is the discipline of engineering concerned with the principles and practice of product and service quality assurance and control. In software development, it is the management, development, operation and maintenance of IT systems and enterprise architectures with a high quality standard.
Software analytics is the analytics specific to the domain of software systems taking into account source code, static and dynamic characteristics as well as related processes of their development and evolution. It aims at describing, monitoring, predicting, and improving the efficiency and effectiveness of software engineering throughout the software lifecycle, in particular during software development and software maintenance. The data collection is typically done by mining software repositories, but can also be achieved by collecting user actions or production data.
Development testing is a software development process that involves synchronized application of a broad spectrum of defect prevention and detection strategies in order to reduce software development risks, time, and costs.
SNAP is the acronym for "Software Non-functional Assessment Process," a measurement of the size of the software derived by quantifying the non-functional user requirements for the software. The SNAP sizing method complements ISO/IEC 20926:2009, which defines a method for the sizing of functional software user requirements. SNAP is a product of the International Function Point Users Group (IFPUG), and is sized using the “Software Non-functional Assessment Process (SNAP) Assessment Practices Manual” (APM) now in version 2.4. Reference “IEEE 2430-2019-IEEE Trial-Use Standard for Non-Functional Sizing Measurements,” published October 19, 2019. Also reference ISO standard “Software engineering — Trial use standard for software non-functional sizing measurements,”, published October 2021. For more information about SNAP please visit YouTube and search for "IFPUG SNAP;" this will provide a series of videos overviewing the SNAP methodology.
Root Cause Analysis Solver Engine is a proprietary algorithm developed from research originally at the Warwick Manufacturing Group (WMG) at Warwick University. RCASE development commenced in 2003 to provide an automated version of root cause analysis, the method of problem solving that tries to identify the root causes of faults or problems. RCASE is now owned by the spin-out company Warwick Analytics where it is being applied to automated predictive analytics software.