ISO/IEC 15504

Last updated

ISO/IEC 15504Information technology – Process assessment, also termed Software Process Improvement and Capability Determination (SPICE), is a set of technical standards documents for the computer software development process and related business management functions. It is one of the joint International Organization for Standardization (ISO) and International Electrotechnical Commission (IEC) standards, which was developed by the ISO and IEC joint subcommittee, ISO/IEC JTC 1/SC 7. [1]

Contents

ISO/IEC 15504 was initially derived from process lifecycle standard ISO/IEC 12207 and from maturity models like Bootstrap, Trillium and the Capability Maturity Model (CMM).

ISO/IEC 15504 has been revised by: ISO/IEC 33001:2015 Information technology – Process assessment – Concepts and terminology as of March, 2015 and is no longer available at ISO.

Overview

ISO/IEC 15504 is the reference model for the maturity models (consisting of capability levels which in turn consist of the process attributes and further consist of generic practices) against which the assessors can place the evidence that they collect during their assessment, so that the assessors can give an overall determination of the organization's capabilities for delivering products (software, systems, and IT services). [2]

History

A working group was formed in 1993 to draft the international standard and used the acronym SPICE. SPICE initially stood for Software Process Improvement and Capability Evaluation, but in consideration of French concerns over the meaning of evaluation, SPICE has now been renamed Software Process Improvement and Capability Determination. SPICE is still used for the user group of the standard, and the title for the annual conference. The first SPICE was held in Limerick, Ireland in 2000, SPICE 2003 was hosted by ESA in the Netherlands, SPICE 2004 was hosted in Portugal, SPICE 2005 in Austria, SPICE 2006 in Luxembourg, SPICE 2007 in South Korea, SPICE 2008 in Nuremberg, Germany and SPICE 2009 in Helsinki, Finland.

The first versions of the standard focused exclusively on software development processes. This was expanded to cover all related processes in a software business, for example project management, configuration management, quality assurance, and so on. The list of processes covered grew to cover six business areas: organizational, management, engineering, acquisition supply, support, and operations.

In a major revision to the draft standard in 2004, the process reference model was removed and is now related to the ISO/IEC 12207 (Software Lifecycle Processes). The issued standard now specifies the measurement framework and can use different process reference models. There are five general and industry models in use.

Part 5 specifies software process assessment and part 6 specifies system process assessment.

The latest work in the ISO standards working group includes creation of a maturity model, which is planned to become ISO/IEC 15504 part 7.

The standard

The Technical Report (TR) document for ISO/IEC TR 15504 was divided into 9 parts. The initial International Standard was recreated in 5 parts. This was proposed from Japan when the TRs were published at 1997.

The International Standard (IS) version of ISO/IEC 15504 now comprises 6 parts. The 7th part is currently in an advanced Final Draft Standard form [3] and work has started on part 8.

Part 1 of ISO/IEC TR 15504 explains the concepts and gives an overview of the framework.

Reference model

ISO/IEC 15504 contains a reference model . The reference model defines a process dimension and a capability dimension.

The process dimension in the reference model is not the subject of part 2 of ISO/IEC 15504, but part 2 refers to external process lifecycle standards including ISO/IEC 12207 and ISO/IEC 15288. [4] The standard defines means to verify conformity of reference models. [5]

Processes

The process dimension defines processes divided into the five process categories of:

  • customer-supplier
  • engineering
  • supporting
  • management
  • organization

With new parts being published, the process categories will expand, particularly for IT service process categories and enterprise process categories.

Capability levels and process attributes

For each process, ISO/IEC 15504 defines a capability level on the following scale: [2]

LevelName
5Optimizing process
4Predictable process
3Established process
2Managed process
1Performed process
0Incomplete process

The capability of processes is measured using process attributes. The international standard defines nine process attributes:

  • 1.1 Process performance
  • 2.1 Performance management
  • 2.2 Work product management
  • 3.1 Process definition
  • 3.2 Process deployment
  • 4.1 Process measurement
  • 4.2 Process control
  • 5.1 Process innovation
  • 5.2 Process optimization

Each process attribute consists of one or more generic practices, which are further elaborated into practice indicators to aid assessment performance.

Rating scale of process attributes

Each process attribute is assessed on a four-point (N-P-L-F) rating scale:

  • Not achieved (0–15%)
  • Partially achieved (>15–50%)
  • Largely achieved (>50–85%)
  • Fully achieved (>85–100%).

The rating is based upon evidence collected against the practice indicators, which demonstrate fulfillment of the process attribute. [6]

Assessments

ISO/IEC 15504 provides a guide for performing an assessment. [7]

This includes:

Assessment process

Performing assessments is the subject of parts 2 and 3 of ISO/IEC 15504. [8] Part 2 is the normative part and part 3 gives a guidance to fulfill the requirements in part 2.

One of the requirements is to use a conformant assessment method for the assessment process. The actual method is not specified in the standard although the standard places requirements on the method, method developers and assessors using the method. [9] The standard provides general guidance to assessors and this must be supplemented by undergoing formal training and detailed guidance during initial assessments.

The assessment process can be generalized as the following steps:

  • initiate an assessment (assessment sponsor)
  • select assessor and assessment team
  • plan the assessment, including processes and organizational unit to be assessed (lead assessor and assessment team)
  • pre-assessment briefing
  • data collection
  • data validation
  • process rating
  • reporting the assessment result

An assessor can collect data on a process by various means, including interviews with persons performing the process, collecting documents and quality records, and collecting statistical process data. The assessor validates this data to ensure it is accurate and completely covers the assessment scope. The assessor assesses this data (using his expert judgment) against a process's base practices and the capability dimension's generic practices in the process rating step. Process rating requires some exercising of expert judgment on the part of the assessor and this is the reason that there are requirements on assessor qualifications and competency. The process rating is then presented as a preliminary finding to the sponsor (and preferably also to the persons assessed) to ensure that they agree that the assessment is accurate. In a few cases, there may be feedback requiring further assessment before a final process rating is made. [10]

Assessment model

The process assessment model (PAM) is the detailed model used for an actual assessment. This is an elaboration of the process reference model (PRM) provided by the process lifecycle standards. [11]

The process assessment model (PAM) in part 5 is based on the process reference model (PRM) for software: ISO/IEC 12207. [12]

The process assessment model in part 6 is based on the process reference model for systems: ISO/IEC 15288. [13]

The standard allows other models to be used instead, if they meet ISO/IEC 15504's criteria, which include a defined community of interest and meeting the requirements for content (i.e. process purpose, process outcomes and assessment indicators).

Tools used in the assessment

There exist several assessment tools. The simplest comprise paper-based tools. In general, they are laid out to incorporate the assessment model indicators, including the base practice indicators and generic practice indicators. Assessors write down the assessment results and notes supporting the assessment judgment.

There are a limited number of computer based tools that present the indicators and allow users to enter the assessment judgment and notes in formatted screens, as well as automate the collated assessment result (i.e. the process attribute ratings) and creating reports.

Assessor qualifications and competency

For a successful assessment, the assessor must have a suitable level of the relevant skills and experience.

These skills include:

The competency of assessors is the subject of part 3 of ISO/IEC 15504.

In summary, the ISO/IEC 15504 specific training and experience for assessors comprise:

Uses

ISO/IEC 15504 can be used in two contexts:

Process improvement

ISO/IEC 15504 can be used to perform process improvement within a technology organization. [14] Process improvement is always difficult, and initiatives often fail, so it is important to understand the initial baseline level (process capability level), and to assess the situation after an improvement project. ISO 15504 provides a standard for assessing the organization's capacity to deliver at each of these stages.

In particular, the reference framework of ISO/IEC 15504 provides a structure for defining objectives, which facilitates specific programs to achieve these objectives.

Process improvement is the subject of part 4 of ISO/IEC 15504. It specifies requirements for improvement programmes and provides guidance on planning and executing improvements, including a description of an eight step improvement programme. Following this improvement programme is not mandatory and several alternative improvement programmes exist. [10]

Capability determination

An organization considering outsourcing software development needs to have a good understanding of the capability of potential suppliers to deliver.

ISO/IEC 15504 (Part 4) can also be used to inform supplier selection decisions. The ISO/IEC 15504 framework provides a framework for assessing proposed suppliers, as assessed either by the organization itself, or by an independent assessor. [15]

The organization can determine a target capability for suppliers, based on the organization's needs, and then assess suppliers against a set of target process profiles that specify this target capability. Part 4 of the ISO/IEC 15504 specifies the high level requirements and an initiative has been started to create an extended part of the standard covering target process profiles. Target process profiles are particularly important in contexts where the organization (for example, a government department) is required to accept the cheapest qualifying vendor. This also enables suppliers to identify gaps between their current capability and the level required by a potential customer, and to undertake improvement to achieve the contract requirements (i.e. become qualified). Work on extending the value of capability determination includes a method called Practical Process Profiles - which uses risk as the determining factor in setting target process profiles. [10] Combining risk and processes promotes improvement with active risk reduction, hence reducing the likelihood of problems occurring.

Acceptance of ISO/IEC 15504

ISO/IEC 15504 has been successful as:

On the other hand, ISO/IEC 15504 has not yet been as successful as the CMMI [ citation needed ]. This has been for several reasons:

Like the CMM, ISO/IEC 15504 was created in a development context, making it difficult to apply in a service management context. But work has started to develop an ISO/IEC 20000-based process reference model (ISO/IEC 20000-4) that can serve as a basis for a process assessment model. This is planned to become part 8 to the standard (ISO/IEC 15504-8). In addition there are methods available that adapt its use to various contexts.

Related Research Articles

The Capability Maturity Model (CMM) is a development model created after a study of data collected from organizations that contracted with the U.S. Department of Defense, who funded the research. The term "maturity" relates to the degree of formality and optimization of processes, from ad hoc practices, to formally defined steps, to managed result metrics, to active optimization of the processes.

ISO/IEC/IEEE 12207Systems and software engineering – Software life cycle processes is an international standard for software lifecycle processes. First introduced in 1995, it aims to be a primary standard that defines all the processes required for developing and maintaining software systems, including the outcomes and/or activities of each process.

TickIT is a certification program for companies in the software development and computer industries, supported primarily by the United Kingdom and Swedish industries through UKAS and SWEDAC respectively. Its general objective is to improve software quality.

Capability Maturity Model Integration (CMMI) is a process level improvement training and appraisal program. Administered by the CMMI Institute, a subsidiary of ISACA, it was developed at Carnegie Mellon University (CMU). It is required by many United States Department of Defense (DoD) and U.S. Government contracts, especially in software development. CMU claims CMMI can be used to guide process improvement across a project, division, or an entire organization. CMMI defines the following maturity levels for processes: Initial, Managed, Defined, Quantitatively Managed, and Optimizing. Version 2.0 was published in 2018. CMMI is registered in the U.S. Patent and Trademark Office by CMU.

Quality management ensures that an organization, product or service is consistent. It has four main components: quality planning, quality assurance, quality control and quality improvement. Quality management is focused not only on product and service quality, but also on the means to achieve it. Quality management, therefore, uses quality assurance and control of processes as well as products to achieve more consistent quality. What a customer wants and is willing to pay for it determines quality. It is a written or unwritten commitment to a known or unknown consumer in the market. Thus, quality can be defined as fitness for intended use or, in other words, how well the product performs its intended function.

Software quality assurance (SQA) is a means of monitoring the software engineering processes and methods used to ensure proper quality. This is accomplished by many and varied approaches. It may include ensuring conformance to one or more standards, such as ISO 9000 or a model such as CMMI.

ISO/IEC 20000 is the first international standard for service management. It was developed in 2005 by ISO/IEC JTC1/SC7 and revised in 2011 and 2018. It was originally based on the earlier BS 15000 that was developed by BSI Group.

Capability Immaturity Model (CIMM) in software engineering is a parody acronym, a semi-serious effort to provide a contrast to the Capability Maturity Model (CMM). The Capability Maturity Model is a five point scale of capability in an organization, ranging from random processes at level 1 to fully defined, managed and optimized processes at level 5. The ability of an organization to carry out its mission on time and within budget is claimed to improve as the CMM level increases.

The ISO/IEC 15288 is a systems engineering standard covering processes and lifecycle stages. Initial planning for the ISO/IEC 15288:2002(E) standard started in 1994 when the need for a common systems engineering process framework was recognized. The previously accepted standard MIL STD 499A (1974) was cancelled after a memo from SECDEF prohibited the use of most United States Military Standards without a waiver. The first edition was issued on 1 November 2002. Stuart Arnold was the editor and Harold Lawson was the architect of the standard. In 2004 this standard was adopted as IEEE 15288. ISO/IEC 15288 has been updated 1 February 2008 as well as on 15 May 2015.

An independent test organization is an organization, person, or company that tests products, materials, software, etc. according to agreed requirements. The test organization can be affiliated with the government or universities or can be an independent testing laboratory. They are independent because they are not affiliated with the producer nor the user of the item being tested: no commercial bias is present. These "contract testing" facilities are sometimes called "third party" testing or evaluation facilities.

Maturity is a measurement of the ability of an organization for continuous improvement in a particular discipline. The higher the maturity, the higher will be the chances that incidents or errors will lead to improvements either in the quality or in the use of the resources of the discipline as implemented by the organization.

ISO/IEC 21827 is an International Standard based on the Systems Security Engineering Capability Maturity Model (SSE-CMM) developed by the International Systems Security Engineering Association (ISSEA). ISO/IEC 21827 specifies the Systems Security Engineering - Capability Maturity Model, which describes the characteristics essential to the success of an organization's security engineering process, and is applicable to all security engineering organizations including government, commercial, and academic. ISO/IEC 21827 does not prescribe a particular process or sequence, but captures practices generally observed in industry. The model is a standard metric for security engineering practices covering the following:

In software engineering, a software development process is the process of dividing software development work into distinct phases to improve design, product management, and project management. It is also known as a software development life cycle (SDLC). The methodology may include the pre-definition of specific deliverables and artifacts that are created and completed by a project team to develop or maintain an application.

The European Software Institute (ESI) is a division of Tecnalia with its headquarters in Spain.

In-Step BLUE project management software by microTOOL

in-STEP BLUE is a project management software program developed and sold by microTOOL GmbH, based in Berlin, Germany. It is designed to assist project managers in developing plans, assigning resources to tasks, tracking progress, managing budgets, requirements, changes and risks as well as analyzing workloads. The tool automatically stores all project results in a central repository shared by all users. Individual project management methods can be supported as well as the agile method Scrum, official methods like the British PRINCE2, the German V-Model XT, the Swiss HERMES method and methods for the automotive industry according to ISO/IEC 15504, also known as SPICE.

ISO/IEC 29110: Systems and Software Life Cycle Profiles and Guidelines for Very Small Entities (VSEs) International Standards (IS) and Technical Reports (TR) are targeted at Very Small Entities (VSEs). A Very Small Entity (VSE) is an enterprise, an organization, a department or a project having up to 25 people. The ISO/IEC 29110 is a series of international standards and guides entitled "Systems and Software Engineering — Lifecycle Profiles for Very Small Entities (VSEs)". The standards and technical reports were developed by working group 24 (WG24) of sub-committee 7 (SC7) of Joint Technical Committee 1 (JTC1) of the International Organization for Standardization and the International Electrotechnical Commission.

ISO/IEC 27001 is an information security standard, part of the ISO/IEC 27000 family of standards, of which the last version was published in 2013, with a few minor updates since then. It is published by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) under the joint ISO and IEC subcommittee, ISO/IEC JTC 1/SC 27.

Tudor IT Process Assessment

Tudor IT Process Assessment (TIPA®) is a methodological framework for process assessment. Its first version was published in 2003 by the Public Research Centre Henri Tudor based in Luxembourg. TIPA is now a registered trademark of the Luxembourg Institute of Science and Technology (LIST). TIPA offers a structured approach to determine process capability compared to recognized best practices. TIPA also supports process improvement by providing a gap analysis and proposing improvement recommendations.

ISO/IEC 33001Information technology -- Process assessment -- Concepts and terminology is a set of technical standards documents for the computer software development process and related business management functions.

References

  1. ISO. "Standards Catalogue: ISO/IEC JTC 1/SC 7" . Retrieved 2014-01-06.
  2. 1 2 ISO/IEC 15504-2 Clause 5
  3. DTR, meaning Draft Technical Report
  4. ISO/IEC 15504-2 Clause 6
  5. ISO/IEC 15504-2 Clause 7
  6. ISO/IEC 15504 part 3
  7. ISO/IEC 15504 parts 2 and 3
  8. ISO/IEC 15504-2 Clause 4 and ISO/IEC 15504-3
  9. 1 2 van Loon, 2007a
  10. 1 2 3 van Loon, 2007b
  11. ISO 15504-2 Clause 6.2
  12. ISO/IEC 15504-2 Clause 6.3 and ISO/IEC 15504-5
  13. ISO/IEC 15504-6
  14. ISO/IEC 15504-4 Clause 6
  15. ISO/IEC 15504-4 Clause 7

Further reading