Software audit review

Last updated July 31, 2024

A software audit review, or software audit, is a type of software review in which one or more auditors who are not members of the software development organization conduct "An independent examination of a software product, software process, or set of software processes to assess compliance with specifications, standards, contractual agreements, or other criteria".^[1]

"Software product" mostly, but not exclusively, refers to some kind of technical document. IEEE Std. 1028^[2] offers a list of 32 "examples of software products subject to audit", including documentary products such as various sorts of plan, contracts, specifications, designs, procedures, standards, and reports, but also non-documentary products such as data, test data, and deliverable media.

Software audits are distinct from software peer reviews and software management reviews in that they are conducted by personnel external to, and independent of, the software development organization, and are concerned with compliance of products or processes, rather than with their technical content, technical quality, or managerial implications.

The term "software audit review" is adopted here to designate the form of software audit described in IEEE Std. 1028.

Objectives and participants

"The purpose of a software audit is to provide an independent evaluation of conformance of software products and processes to applicable regulations, standards, guidelines, plans, and procedures".^[3] The following roles are recommended:

The Initiator (who might be a manager in the audited organization, a customer or user representative of the audited organization, or a third party), decides upon the need for an audit, establishes its purpose and scope, specifies the evaluation criteria, identifies the audit personnel, decides what follow-up actions will be required, and distributes the audit report.
The Lead Auditor (who must be someone "free from bias and influence that could reduce his ability to make independent, objective evaluations") is responsible for administrative tasks such as preparing the audit plan and assembling and managing the audit team, and for ensuring that the audit meets its objectives.
The Recorder documents anomalies, action items, decisions, and recommendations made by the audit team.
The Auditors (who must be, like the Lead Auditor, free from bias) examine products defined in the audit plan, document their observations, and recommend corrective actions. (There may be only a single auditor.)
The Audited Organization provides a liaison to the auditors, and provides all information requested by the auditors. When the audit is completed, the audited organization should implement corrective actions and recommendations.

Principles of a Software Audit

The following principles of an audit should find a reflection:^[4]

Timeliness: Only when the processes and programming is continuous inspected in regard to their potential susceptibility to faults and weaknesses, but as well with regard to the continuation of the analysis of the found strengths, or by comparative functional analysis with similar applications an updated frame can be continued.
Source openness: It requires an explicit reference in the audit of encrypted programs, how the handling of open source has to be understood. E.g. programs, offering an open source application, but not considering the IM server as open source, have to be regarded as critical. An auditor should take an own position to the paradigm of the need of the open source nature within cryptologic applications.
Elaborateness: Audit processes should be oriented to certain minimum standard. The recent audit processes of encrypting software often vary greatly in quality, in the scope and effectiveness and also experience in the media reception often differing perceptions. Because of the need of special knowledge on the one hand and to be able to read programming code and then on the other hand to also have knowledge of encryption procedures, many users even trust the shortest statements of formal confirmation. Individual commitment as an auditor, e.g. for quality, scale and effectiveness, is thus to be assessed reflexively for yourself and to be documented within the audit.

The financial context: Further transparency is needed to clarify whether the software has been developed commercially and whether the audit was funded commercially (paid Audit). It makes a difference whether it is a private hobby / community project or whether a commercial company is behind it.
Scientific referencing of learning perspectives: Each audit should describe the findings in detail within the context and also highlight progress and development needs constructively. An auditor is not the parent of the program, but serves in a role of a mentor if the auditor is regarded as part of a PDCA learning circle (PDCA = Plan-Do-Check-Act). There should be next to the description of the detected vulnerabilities also a description of the innovative opportunities and the development of the potentials.
Literature-inclusion: A reader should not rely solely on the results of one review, but also judge according to a loop of a management system (e.g. PDCA, see above), to ensure, that the development team or the reviewer was and is prepared to carry out further analysis, and also in the development and review process is open to learnings and to consider notes of others. A list of references should be accompanied in each case of an audit.
Inclusion of user manuals & documentation: Further a check should be done, whether there are manuals and technical documentations, and, if these are expanded.
Identify references to innovations: Applications that allow both, messaging to offline and online contacts, so considering chat and e-mail in one application - as it is also the case with GoldBug - should be tested with high priority (criterion of presence chats in addition to the e-mail function). The auditor should also highlight the references to innovations and underpin further research and development needs.

This list of audit principles for crypto applications describes - beyond the methods of technical analysis - particularly core values, that should be taken into account

Tools

Parts of Software audit could be done using static analysis tools that analyze application code and score its conformance with standards, guidelines, best practices. From the List of tools for static code analysis some are covering a very large spectrum from code to architecture review, and could be use for benchmarking.

Related Research Articles

<span class="mw-page-title-main">Software testing</span> Checking software against a standard

Software testing is the act of checking whether software satisfies expectations.

<span class="mw-page-title-main">Configuration management</span> Process for maintaining consistency of a product attributes with its design

Configuration management (CM) is a management process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life. The CM process is widely used by military engineering organizations to manage changes throughout the system lifecycle of complex systems, such as weapon systems, military vehicles, and information systems. Outside the military, the CM process is also used with IT service management as defined by ITIL, and with other domain models in the civil engineering and other industrial engineering segments such as roads, bridges, canals, dams, and buildings.

Code review is a software quality assurance activity in which one or more people check a program, mainly by viewing and reading parts of its source code, either after implementation or as an interruption of implementation. At least one of the persons must not have authored the code. The persons performing the checking, excluding the author, are called "reviewers".

ISO/IEC/IEEE 12207Systems and software engineering – Software life cycle processes is an international standard for software lifecycle processes. First introduced in 1995, it aims to be a primary standard that defines all the processes required for developing and maintaining software systems, including the outcomes and/or activities of each process.

In software project management, software testing, and software engineering, verification and validation is the process of checking that a software engineer system meets specifications and requirements so that it fulfills its intended purpose. It may also be referred to as software quality control. It is normally the responsibility of software testers as part of the software development lifecycle. In simple terms, software verification is: "Assuming we should build X, does our software achieve its goals without any bugs or gaps?" On the other hand, software validation is: "Was X what we should have built? Does X meet the high-level requirements?"

Multiple encryption is the process of encrypting an already encrypted message one or more times, either using the same or a different algorithm. It is also known as cascade encryption, cascade ciphering, multiple encryption, and superencipherment. Superencryption refers to the outer-level encryption of a multiple encryption.

In the context of software engineering, software quality refers to two related but distinct notions:

An information technology audit, or information systems audit, is an examination of the management controls within an Information technology (IT) infrastructure and business applications. The evaluation of evidence obtained determines if the information systems are safeguarding assets, maintaining data integrity, and operating effectively to achieve the organization's goals or objectives. These reviews may be performed in conjunction with a financial statement audit, internal audit, or other form of attestation engagement.

Software quality assurance (SQA) is a means and practice of monitoring all software engineering processes, methods, and work products to ensure compliance against defined standards. It may include ensuring conformance to standards or models, such as ISO/IEC 9126, SPICE or CMMI.

Computer-assisted audit tool (CAATs) or computer-assisted audit tools and techniques (CAATTs) is a growing field within the IT audit profession. CAATs is the practice of using computers to automate the IT audit processes. CAATs normally include using basic office productivity software such as spreadsheets, word processors and text editing programs and more advanced software packages involving use statistical analysis and business intelligence tools. But also more dedicated specialized software are available.

An information security audit is an audit of the level of information security in an organization. It is an independent review and examination of system records, activities, and related documents. These audits are intended to improve the level of information security, avoid improper information security designs, and optimize the efficiency of the security safeguards and security processes.

Software assurance (SwA) is a critical process in software development that ensures the reliability, safety, and security of software products. It involves a variety of activities, including requirements analysis, design reviews, code inspections, testing, and formal verification. One crucial component of software assurance is secure coding practices, which follow industry-accepted standards and best practices, such as those outlined by the Software Engineering Institute (SEI) in their CERT Secure Coding Standards (SCS).

A Software management review is a management study into a project's status and allocation of resources. It is different from both a software engineering peer review, which evaluates the technical quality of software products, and a software audit, which is an externally conducted audit into a project's compliance to specifications, contractual agreements, and other criteria.

In software development, peer review is a type of software review in which a work product is examined by author's colleagues, in order to evaluate the work product's technical content and quality.

A software review is "a process or meeting during which a software product is examined by a project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval".

In software engineering, a walkthrough or walk-through is a form of software peer review "in which a designer or programmer leads members of the development team and other interested parties through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems". The reviews are also performed by assessors, specialists, etc. and are suggested or mandatory as required by norms and standards.

A software technical review is a form of peer review in which "a team of qualified personnel ... examines the suitability of the software product for its intended use and identifies discrepancies from specifications and standards. Technical reviews may also provide recommendations of alternatives and examination of various alternatives".

IEEE 1471 is a superseded IEEE standard for describing the architecture of a "software-intensive system", also known as software architecture.

Verification and validation are independent procedures that are used together for checking that a product, service, or system meets requirements and specifications and that it fulfills its intended purpose. These are critical components of a quality management system such as ISO 9000. The words "verification" and "validation" are sometimes preceded with "independent", indicating that the verification and validation is to be performed by a disinterested third party. "Independent verification and validation" can be abbreviated as "IV&V".

In engineering, technical peer review is a well defined review process for finding and correcting defects conducted by a team of peers with assigned roles. Technical peer reviews are carried out by peers representing areas of life cycle affected by material being reviewed. Technical peer reviews are held within development phases, between milestone reviews, on completed products, or on completed portions of products. A technical peer review may also be called an engineering peer review, a product peer review, a peer review/inspection or an inspection.

References

↑ IEEE Std. 1028-1997, IEEE Standard for Software Reviews, clause 3.2
↑ "IEEE 1028-2008 - IEEE Standard for Software Reviews and Audits". IEEE . Retrieved 2019-03-12.
↑ IEEE Std. 10281997, clause 8.1
↑ References to further core audit principles, in: Adams, David / Maier, Ann-Kathrin (2016): BIG SEVEN Study, open source crypto-messengers to be compared - or: Comprehensive Confidentiality Review & Audit of GoldBug, Encrypting E-Mail-Client & Secure Instant Messenger, Descriptions, tests and analysis reviews of 20 functions of the application GoldBug based on the essential fields and methods of evaluation of the 8 major international audit manuals for IT security investigations including 38 figures and 87 tables., URL: https://sf.net/projects/goldbug/files/bigseven-crypto-audit.pdf - English / German Language, Version 1.1, 305 pages, June 2016 (ISBN: DNB 110368003X - 2016B14779)

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] IEEE Std. 1028-1997, IEEE Standard for Software Reviews, clause 3.2

[2] "IEEE 1028-2008 - IEEE Standard for Software Reviews and Audits". IEEE . Retrieved 2019-03-12.

[3] IEEE Std. 10281997, clause 8.1

[4] References to further core audit principles, in: Adams, David / Maier, Ann-Kathrin (2016): BIG SEVEN Study, open source crypto-messengers to be compared - or: Comprehensive Confidentiality Review & Audit of GoldBug, Encrypting E-Mail-Client & Secure Instant Messenger, Descriptions, tests and analysis reviews of 20 functions of the application GoldBug based on the essential fields and methods of evaluation of the 8 major international audit manuals for IT security investigations including 38 figures and 87 tables., URL: https://sf.net/projects/goldbug/files/bigseven-crypto-audit.pdf - English / German Language, Version 1.1, 305 pages, June 2016 (ISBN: DNB 110368003X - 2016B14779)

[1]

[2]

[3]

[4]