Software audit review

Last updated March 13, 2019

A software audit review, or software audit, is a type of software review in which one or more auditors who are not members of the software development organization conduct "An independent examination of a software product, software process, or set of software processes to assess compliance with specifications, standards, contractual agreements, or other criteria".^[1]

A software review is "A process or meeting during which a software product is examined by a project personnel, managers, users, customers, user representatives, or other interested parties for comment or approval".

Software development is the process of conceiving, specifying, designing, programming, documenting, testing, and bug fixing involved in creating and maintaining applications, frameworks, or other software components. Software development is a process of writing and maintaining the source code, but in a broader sense, it includes all that is involved between the conception of the desired software through to the final manifestation of the software, sometimes in a planned and structured process. Therefore, software development may include research, new development, prototyping, modification, reuse, re-engineering, maintenance, or any other activities that result in software products.

"Software product" mostly, but not exclusively, refers to some kind of technical document. IEEE Std. 1028^[2] offers a list of 32 "examples of software products subject to audit", including documentary products such as various sorts of plan, contracts, specifications, designs, procedures, standards, and reports, but also non-documentary products such as data, test data, and deliverable media.

The Institute of Electrical and Electronics Engineers (IEEE) is a professional association with its corporate office in New York City and its operations center in Piscataway, New Jersey. It was formed in 1963 from the amalgamation of the American Institute of Electrical Engineers and the Institute of Radio Engineers.

Software audits are distinct from software peer reviews and software management reviews in that they are conducted by personnel external to, and independent of, the software development organization, and are concerned with compliance of products or processes, rather than with their technical content, technical quality, or managerial implications.

In software development, peer review is a type of software review in which a work product is examined by its author and one or more colleagues, in order to evaluate its technical content and quality.

A Software management review is a management study into a project's status and allocation of resources. It is different from both a software engineering peer review, which evaluates the technical quality of software products, and a software audit, which is an externally conducted audit into a project's compliance to specifications, contractual agreements, and other criteria.

The term "software audit review" is adopted here to designate the form of software audit described in IEEE Std. 1028.

Objectives and participants

"The purpose of a software audit is to provide an independent evaluation of conformance of software products and processes to applicable regulations, standards, guidelines, plans, and procedures".^[3] The following roles are recommended:

The Initiator (who might be a manager in the audited organization, a customer or user representative of the audited organization, or a third party), decides upon the need for an audit, establishes its purpose and scope, specifies the evaluation criteria, identifies the audit personnel, decides what follow-up actions will be required, and distributes the audit report.
The Lead Auditor (who must be someone "free from bias and influence that could reduce his ability to make independent, objective evaluations") is responsible for administrative tasks such as preparing the audit plan and assembling and managing the audit team, and for ensuring that the audit meets its objectives.
The Recorder documents anomalies, action items, decisions, and recommendations made by the audit team.
The Auditors (who must be, like the Lead Auditor, free from bias) examine products defined in the audit plan, document their observations, and recommend corrective actions. (There may be only a single auditor.)
The Audited Organization provides a liaison to the auditors, and provides all information requested by the auditors. When the audit is completed, the audited organization should implement corrective actions and recommendations.

Principles of a Software Audit

The following principles of an audit should find a reflection:^[4]

Timeliness: Only when the processes and programming is continuous inspected in regard to their potential susceptibility to faults and weaknesses, but as well with regard to the continuation of the analysis of the found strengths, or by comparative functional analysis with similar applications an updated frame can be continued.
Source openness: It requires an explicit reference in the audit of encrypted programs, how the handling of open source has to be understood. E.g. programs, offering an open source application, but not considering the IM server as open source, have to be regarded as critical. An auditor should take an own position to the paradigm of the need of the open source nature within cryptologic applications.
Elaborateness: Audit processes should be oriented to certain minimum standard. The recent audit processes of encrypting software often vary greatly in quality, in the scope and effectiveness and also experience in the media reception often differing perceptions. Because of the need of special knowledge on the one hand and to be able to read programming code and then on the other hand to also have knowledge of encryption procedures, many users even trust the shortest statements of formal confirmation. Individual commitment as an auditor, e.g. for quality, scale and effectiveness, is thus to be assessed reflexively for yourself and to be documented within the audit.

Figure 01 Eight Principles of a Crypto-IT-Audit.png

The financial context: Further transparency is needed to clarify whether the software has been developed commercially and whether the audit was funded commercially (paid Audit). It makes a difference whether it is a private hobby / community project or whether a commercial company is behind it.
Scientific referencing of learning perspectives: Each audit should describe the findings in detail within the context and also highlight progress and development needs constructively. An auditor is not the parent of the program, but at least he or she is in a role of a mentor, if the auditor is regarded as part of a PDCA learning circle (PDCA = Plan-Do-Check-Act). There should be next to the description of the detected vulnerabilities also a description of the innovative opportunities and the development of the potentials.
Literature-inclusion: A reader should not rely solely on the results of one review, but also judge according to a loop of a management system (e.g. PDCA, see above), to ensure, that the development team or the reviewer was and is prepared to carry out further analysis, and also in the development and review process is open to learnings and to consider notes of others. A list of references should be accompanied in each case of an audit.
Inclusion of user manuals & documentation: Further a check should be done, whether there are manuals and technical documentations, and, if these are expanded.
Identify references to innovations: Applications that allow both, messaging to offline and online contacts, so considering chat and e-mail in one application - as it is also the case with GoldBug - should be tested with high priority (criterion of presence chats in addition to the e-mail function). The auditor should also highlight the references to innovations and underpin further research and development needs.

PDCA is an iterative four-step management method used in business for the control and continuous improvement of processes and products. It is also known as the Deming circle/cycle/wheel, the Shewhart cycle, the control circle/cycle, or plan–do–study–act (PDSA). Another version of this PDCA cycle is OPDCA. The added "O" stands for observation or as some versions say: "Observe the current condition." This emphasis on observation and current condition has currency with the literature on lean manufacturing and the Toyota Production System. The PDCA cycle, with Ishikawa’s changes, can be traced back to S. Mizuno of the Tokyo Institute of Technology in 1959.

This list of audit principles for crypto applications describes - beyond the methods of technical analysis - particularly core values, that should be taken into account

Tools

Parts of Software audit could be done using static analysis tools that analyze application code and score its conformance with standards, guidelines, best practices. From the List of tools for static code analysis some are covering a very large spectrum from code to architecture review, and could be use for benchmarking.

Related Research Articles

Software testing is an investigation conducted to provide stakeholders with information about the quality of the software product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include the process of executing a program or application with the intent of finding software bugs, and verifying that the software product is fit for use.

Configuration management process for maintaining consistency of a product attributes with its design

Configuration management (CM) is a systems engineering process for establishing and maintaining consistency of a product's performance, functional, and physical attributes with its requirements, design, and operational information throughout its life. The CM process is widely used by military engineering organizations to manage changes throughout the system lifecycle of complex systems, such as weapon systems, military vehicles, and information systems. Outside the military, the CM process is also used with IT service management as defined by ITIL, and with other domain models in the civil engineering and other industrial engineering segments such as roads, bridges, canals, dams, and buildings.

Audit systematic and independent examination of books, accounts, documents and vouchers of an organization

An audit is a systematic and independent examination of books, accounts, statutory records, documents and vouchers of an organization to ascertain how far the financial statements as well as non-financial disclosures present a true and fair view of the concern. It also attempts to ensure that the books of accounts are properly maintained by the concern as required by law. Auditing has become such a ubiquitous phenomenon in the corporate and the public sector that academics started identifying an "Audit Society". The auditor perceives and recognises the propositions before them for examination, obtains evidence, evaluates the same and formulates an opinion on the basis of his judgement which is communicated through their audit report.

ISO/IEC/IEEE 12207Systems and software engineering – Software life cycle processes is an international standard for software lifecycle processes. First introduced in 1995, it aims to be a primary standard that defines all the processes required for developing and maintaining software systems, including the outcomes and/or activities of each process.

Multiple encryption is the process of encrypting an already encrypted message one or more times, either using the same or a different algorithm. It is also known as cascade encryption, cascade ciphering, multiple encryption, and superencipherment. Superencryption refers to the outer-level encryption of a multiple encryption.

An information technology audit, or information systems audit, is an examination of the management controls within an Information technology (IT) infrastructure. The evaluation of obtained evidence determines if the information systems are safeguarding assets, maintaining data integrity, and operating effectively to achieve the organization's goals or objectives. These reviews may be performed in conjunction with a financial statement audit, internal audit, or other form of attestation engagement.

Computer-assisted audit techniques (CAATs) or computer-assisted audit tools and techniques (CAATTs) is a growing field within the IT audit profession. CAATs is the practice of using computers to automate the IT audit processes. CAATs normally includes using basic office productivity software such as spreadsheet, word processors and text editing programs and more advanced software packages involving use statistical analysis and business intelligence tools. But also more dedicated specialized software are available.

In business and accounting, information technology controls are specific activities performed by persons or systems designed to ensure that business objectives are met. They are a subset of an enterprise's internal control. IT control objectives relate to the confidentiality, integrity, and availability of data and the overall management of the IT function of the business enterprise. IT controls are often described in two categories: IT general controls (ITGC) and IT application controls. ITGC include controls over the Information Technology (IT) environment, computer operations, access to programs and data, program development and program changes. IT application controls refer to transaction processing controls, sometimes called "input-processing-output" controls. Information technology controls have been given increased prominence in corporations listed in the United States by the Sarbanes-Oxley Act. The COBIT Framework is a widely used framework promulgated by the IT Governance Institute, which defines a variety of ITGC and application control objectives and recommended evaluation approaches. IT departments in organizations are often led by a Chief Information Officer (CIO), who is responsible for ensuring effective information technology controls are utilized.

An information security audit is an audit on the level of information security in an organization. Within the broad scope of auditing information security there are multiple types of audits, multiple objectives for different audits, etc. Most commonly the controls being audited can be categorized to technical, physical and administrative. Auditing information security covers topics from auditing the physical security of data centers to auditing the logical security of databases and highlights key components to look for and different methods for auditing these areas.

A computer security audit is a manual or systematic measurable technical assessment of a system or application. Manual assessments include interviewing staff, performing security vulnerability scans, reviewing application and operating system access controls, and analyzing physical access to the systems. Automated assessments, or CAAT's, include system generated audit reports or using software to monitor and report changes to files and settings on a system. Systems can include personal computers, servers, mainframes, network routers, switches.

In software engineering, a walkthrough or walk-through is a form of software peer review "in which a designer or programmer leads members of the development team and other interested parties go through a software product, and the participants ask questions and make comments about possible errors, violation of development standards, and other problems".

A software technical review is a form of peer review in which "a team of qualified personnel ... examines the suitability of the software product for its intended use and identifies discrepancies from specifications and standards. Technical reviews may also provide recommendations of alternatives and examination of various alternatives".

IEEE 1471 is a superseded IEEE Standard for describing the architecture of a "software-intensive system", also known as software architecture.

Internal audit An audit of accounting for audiences within a firm

'Internal auditing is an independent, objective assurance and consulting activity designed to add value and improve an organization's operations. It helps an organization accomplish its objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control and governance processes. Internal auditing is a catalyst for improving an organization's governance, risk management and management controls by providing insight and recommendations based on analyses and assessments of data and business processes. With commitment to integrity and accountability, internal auditing provides value to governing bodies and senior management as an objective source of independent advice. Professionals called internal auditors are employed by organizations to perform the internal auditing activity.

In engineering, technical peer review is a type of engineering review. Technical peer reviews are a well defined review process for finding and fixing defects, conducted by a team of peers with assigned roles. Technical peer reviews are carried out by peers representing areas of life cycle affected by material being reviewed. Technical peer reviews are held within development phases, between milestone reviews, on completed products or completed portions of products.

IT risk management is the application of risk management methods to information technology in order to manage IT risk, i.e.:

References

↑ IEEE Std. 1028-1997, IEEE Standard for Software Reviews, clause 3.2
↑ "IEEE 1028-2008 - IEEE Standard for Software Reviews and Audits". standards.ieee.org. Retrieved 2019-03-12.
↑ IEEE Std. 10281997, clause 8.1
↑ References to further core audit principles, in: Adams, David / Maier, Ann-Kathrin (2016): BIG SEVEN Study, open source crypto-messengers to be compared - or: Comprehensive Confidentiality Review & Audit of GoldBug, Encrypting E-Mail-Client & Secure Instant Messenger, Descriptions, tests and analysis reviews of 20 functions of the application GoldBug based on the essential fields and methods of evaluation of the 8 major international audit manuals for IT security investigations including 38 figures and 87 tables., URL: https://sf.net/projects/goldbug/files/bigseven-crypto-audit.pdf - English / German Language, Version 1.1, 305 pages, June 2016 (ISBN: DNB 110368003X - 2016B14779)

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.

[1] IEEE Std. 1028-1997, IEEE Standard for Software Reviews, clause 3.2

[2] "IEEE 1028-2008 - IEEE Standard for Software Reviews and Audits". standards.ieee.org. Retrieved 2019-03-12.

[3] IEEE Std. 10281997, clause 8.1

[4] References to further core audit principles, in: Adams, David / Maier, Ann-Kathrin (2016): BIG SEVEN Study, open source crypto-messengers to be compared - or: Comprehensive Confidentiality Review & Audit of GoldBug, Encrypting E-Mail-Client & Secure Instant Messenger, Descriptions, tests and analysis reviews of 20 functions of the application GoldBug based on the essential fields and methods of evaluation of the 8 major international audit manuals for IT security investigations including 38 figures and 87 tables., URL: https://sf.net/projects/goldbug/files/bigseven-crypto-audit.pdf - English / German Language, Version 1.1, 305 pages, June 2016 (ISBN: DNB 110368003X - 2016B14779)