Software bug

Last updated

A software bug is a design defect (bug) in computer software. A computer program with many or serious bugs may be described as buggy.

Contents

The effects of a software bug range from minor (such as a misspelled word in the user interface) to severe (such as frequent crashing).

In 2002, a study commissioned by the US Department of Commerce's National Institute of Standards and Technology concluded that "software bugs, or errors, are so prevalent and so detrimental that they cost the US economy an estimated $59 billion annually, or about 0.6 percent of the gross domestic product". [1]

Since the 1950s, some computer systems have been designed to detect or auto-correct various software errors during operations.

History

Terminology

Mistake metamorphism (from Greek meta = "change", morph = "form") refers to the evolution of a defect in the final stage of software deployment. Transformation of a mistake committed by an analyst in the early stages of the software development lifecycle, which leads to a defect in the final stage of the cycle has been called mistake metamorphism. [2]

Different stages of a mistake in the development cycle may be described as mistake, [3] : 31  anomaly, [3] : 10  fault, [3] : 31  failure, [3] : 31  error, [3] : 31  exception, [3] : 31  crash, [3] : 22  glitch, bug, [3] : 14  defect, incident, [3] : 39  or side effect.

Controversy

Sometimes the use of bug to describe the behavior of software is contentious due to perception. Some suggest that the term should be abandoned; replaced with defect or error.

Some contend that bug implies that the defect arose on its own and push to use defect instead since it more clearly connotates caused by a human. [4]

Some contend that bug may be used to coverup an intentional design decision. In 2011, after receiving scrutiny from US Senator Al Franken for recording and storing users' locations in unencrypted files, [5] Apple called the behavior a bug. However, Justin Brookman of the Center for Democracy and Technology directly challenged that portrayal, stating "I'm glad that they are fixing what they call bugs, but I take exception with their strong denial that they track users." [6]

Prevention

Error resulting from a software bug displayed on two screens at La Croix de Berny station in France Software bug displayed on two screens at La Croix de Berny station in France - 2021-10-28.jpg
Error resulting from a software bug displayed on two screens at La Croix de Berny station in France

Preventing bugs as early as possible in the software development process is a target of investment and innovation. [7] [8]

Language support

Newer programming languages tend to be designed to prevent common bugs based on vulnerabilities of existing languages. Lessons learned from older languages such as BASIC and C are used to inform the design of later languages such as C# and Rust.

Languages may include features such as a static type system, restricted namespaces and modular programming. For example, for a typed, compiled language (like C):

float num = "3";

is syntactically correct, but fails type checking since the right side, a string, cannot be assigned to a float variable. Compilation fails – forcing this defect to be fixed before development progress can resume. With an interpreted language, a failure would not occur until later at runtime.

Some languages exclude features that easily lead to bugs, at the expense of slower performance – the principle being that it is usually better to write simpler, slower correct code than complicated, buggy code. For example, the Java does not support pointer arithmetic which is generally fast, but is considered dangerous; relatively easy to cause a major bug.

Some languages include features that add runtime overhead in order to prevent some bugs. For example, many languages include runtime bounds checking and a way to handle out-of-bounds conditions instead of crashing.

A compiled language allows for detecting some typos (such as a misspelled identifier) before runtime which is earlier in the software development process than for an interpreted language.

Techniques

Programming techniques such as programming style and defensive programming are intended to prevent typos.

For example, a bug may be caused by a relatively minor, typographical error (typo) in the code. For example, this code executes function foo only if conditionis true.

if (condition) foo();

But this code always executes foo:

if (condition); foo();

A convention that tends to prevent this particular issue is to require braces for a block even if it has just one line.

if (condition) {   foo(); }

Enforcement of conventions may be manual (i.e. via code review) or via automated tools.

Specification

Some contend that writing a program specification which states the behavior of a program, can prevent bugs.

Some contend that formal specifications are impractical for anything but the shortest programs, because of problems of combinatorial explosion and indeterminacy.

Software testing

One goal of software testing is to find bugs.

Measurements during testing can provide an estimate of the number of likely bugs remaining. This becomes more reliable the longer a product is tested and developed.[ citation needed ]

Agile practices

Agile software development may involve frequent software releases with relatively small changes. Defects are revealed by user feedback.

With test-driven development (TDD), unit tests are written while writing the production code, and the production code is not considered complete until all tests complete successfully.

Static analysis

Tools for static code analysis help developers by inspecting the program text beyond the compiler's capabilities to spot potential problems. Although in general the problem of finding all programming errors given a specification is not solvable (see halting problem), these tools exploit the fact that human programmers tend to make certain kinds of simple mistakes often when writing software.

Instrumentation

Tools to monitor the performance of the software as it is running, either specifically to find problems such as bottlenecks or to give assurance as to correct working, may be embedded in the code explicitly (perhaps as simple as a statement saying PRINT "I AM HERE"), or provided as tools. It is often a surprise to find where most of the time is taken by a piece of code, and this removal of assumptions might cause the code to be rewritten.

Open source

Open source development allows anyone to examine source code. A school of thought popularized by Eric S. Raymond as Linus's law says that popular open-source software has more chance of having few or no bugs than other software, because "given enough eyeballs, all bugs are shallow". [9] This assertion has been disputed, however: computer security specialist Elias Levy wrote that "it is easy to hide vulnerabilities in complex, little understood and undocumented source code," because, "even if people are reviewing the code, that doesn't mean they're qualified to do so." [10] An example of an open-source software bug was the 2008 OpenSSL vulnerability in Debian.

Debugging

Debugging can be a significant part of the software development lifecycle. Maurice Wilkes, an early computing pioneer, described his realization in the late 1940s that “a good part of the remainder of my life was going to be spent in finding errors in my own programs”. [11]

A program known as a debugger can help a programmer find faulty code by examining the inner workings of a program such as executing code line-by-line and viewing variable values.

As an alternative to using a debugger, code may be instrumented with logic to output debug information to trace program execution and view values. Output is typically to console, window, log file or a hardware output (i.e. LED).

Some contend that locating a bug is something of an art.

It is not uncommon for a bug in one section of a program to cause failures in a different section,[ citation needed ] thus making it difficult to track, in an apparently unrelated part of the system. For example, an error in a graphics rendering routine causing a file I/O routine to fail.

Sometimes, the most difficult part of debugging is finding the cause of the bug. Once found, correcting the problem is sometimes easy if not trivial.

Sometimes, a bug is not an isolated flaw, but represents an error of thinking or planning on the part of the programmers. Often, such a logic error requires a section of the program to be overhauled or rewritten.

Some contend that as a part of code review, stepping through the code and imagining or transcribing the execution process may often find errors without ever reproducing the bug as such.

Typically, the first step in locating a bug is to reproduce it reliably. If unable to reproduce the issue, a programmer cannot find the cause of the bug and therefore cannot fix it.

Some bugs are revealed by inputs that may be difficult for the programmer to re-create. One cause of the Therac-25 radiation machine deaths was a bug (specifically, a race condition) that occurred only when the machine operator very rapidly entered a treatment plan; it took days of practice to become able to do this, so the bug did not manifest in testing or when the manufacturer attempted to duplicate it. Other bugs may stop occurring whenever the setup is augmented to help find the bug, such as running the program with a debugger; these are called heisenbugs (humorously named after the Heisenberg uncertainty principle).

Since the 1990s, particularly following the Ariane 5 Flight 501 disaster, interest in automated aids to debugging rose, such as static code analysis by abstract interpretation. [12]

Often, bugs come about during coding, but faulty design documentation may cause a bug. In some cases, changes to the code may eliminate the problem even though the code then no longer matches the documentation.

In an embedded system, the software is often modified to work around a hardware bug since it's cheaper than modifying the hardware.

Management

Example bug history (GNU Classpath project data). A new bug is initially unconfirmed. Once reproducibility is confirmed, it is changed to confirmed. Once the issue is resolved, it is changed to fixed. Classpath bugs.png
Example bug history (GNU Classpath project data). A new bug is initially unconfirmed. Once reproducibility is confirmed, it is changed to confirmed. Once the issue is resolved, it is changed to fixed.

Bugs are managed via activities like documenting, categorizing, assigning, reproducing, correcting and releasing the corrected code.

Tools are often used to track bugs and other issues with software. Typically, different tools are used by the software development team to track their workload than by customer service to track user feedback. [13]

A tracked item is often called bug, defect, ticket, issue, feature, or for agile software development, story or epic. Items are often categorized by aspects such as severity, priority and version number.

In a process sometimes called triage, choices are made for each bug about whether and when to fix it based on information such as the bug's severity and priority and external factors such as development schedules. Triage generally does not include investigation into cause. Triage may occur regularly. Triage generally consists of reviewing new bugs since the previous triage and maybe all open bugs. Attendees may include project manager, development manager, test manager, build manager, and technical experts. [14] [15]

Severity

Severity is a measure of impact the bug has. [16] This impact may be data loss, financial, loss of goodwill and wasted effort. Severity levels are not standardized, but differ by context such as industry and tracking tool. For example, a crash in a video game has a different impact than a crash in a bank server. Severity levels might be crash or hang, no workaround (user cannot accomplish a task), has workaround (user can still accomplish the task), visual defect (a misspelling for example), or documentation error. Another example set of severities: critical, high, low, blocker, trivial. [17] The severity of a bug may be a separate category to its priority for fixing, or the two may be quantified and managed separately.

A bug severe enough to delay the release of the product is called a show stopper. [18] [19]

Priority

Priority describes the importance of resolving the bug in relation to other bugs. Priorities might be numerical, such as 1 through 5, or named, such as critical, high, low, and deferred. The values might be similar or identical to severity ratings, even though priority is a different aspect.

Priority may be a combination of the bug's severity with the level of effort to fix. A bug with low severity but easy to fix may get a higher priority than a bug with moderate severity that requires significantly more effort to fix.

Patch

Bugs of sufficiently high priority may warrant a special release which is sometimes called a patch .

Maintenance release

A software release that emphasizes bug fixes may be called a maintenance release – to differentiate it from a release that emphasizes new features or other changes.

Known issue

It is common practice to release software with known, low-priority bugs or other issues. Possible reasons include but are not limited to:

Implications

The amount and type of damage a software bug may cause affects decision-making, processes and policy regarding software quality. In applications such as human spaceflight, aviation, nuclear power, health care, public transport or automotive safety, since software flaws have the potential to cause human injury or even death, such software will have far more scrutiny and quality control than, for example, an online shopping website. In applications such as banking, where software flaws have the potential to cause serious financial damage to a bank or its customers, quality control is also more important than, say, a photo editing application.

Other than the damage caused by bugs, some of their cost is due to the effort invested in fixing them. In 1978, Lientz et al. showed that the median of projects invest 17 percent of the development effort in bug fixing. [22] In 2020, research on GitHub repositories showed the median is 20%. [23]

Cost

In 1994, NASA's Goddard Space Flight Center managed to reduce their average number of errors from 4.5 per 1000 lines of code (SLOC) down to 1 per 1000 SLOC. [24]

Another study in 1990 reported that exceptionally good software development processes can achieve deployment failure rates as low as 0.1 per 1000 SLOC. [25] This figure is iterated in literature such as Code Complete by Steve McConnell, [26] and the NASA study on Flight Software Complexity. [27] Some projects even attained zero defects: the firmware in the IBM Wheelwriter typewriter which consists of 63,000 SLOC, and the Space Shuttle software with 500,000 SLOC. [25]

Benchmark

To facilitate reproducible research on testing and debugging, researchers use curated benchmarks of bugs:

Types

Some notable types of bugs:

Design error

A bug can be caused by insufficient or incorrect design based on the specification. For example, given that the specification is to alphabetize a list of words, a design bug might occur if the design does not account for symbols; resulting in incorrect alphabetization of words with symbols.

Arithmetic

Numerical operations can result in unexpected output, slow processing, or crashing. [30] Such a bug can be from a lack of awareness of the qualities of the data storage such as a loss of precision due to rounding, numerically unstable algorithms, arithmetic overflow and underflow, or from lack of awareness of how calculations are handled by different software coding languages such as division by zero which in some languages may throw an exception, and in others may return a special value such as NaN or infinity.

Control flow

A control flow bug, a.k.a. logic error, is characterized by code that does not fail with an error, but does not have the expected behavior, such as infinite looping, infinite recursion, incorrect comparison in a conditional such as using the wrong comparison operator, and the off-by-one error.

Interfacing

Concurrency

Resourcing

Syntax

Teamwork

In politics

"Bugs in the System" report

The Open Technology Institute, run by the group, New America, [35] released a report "Bugs in the System" in August 2016 stating that U.S. policymakers should make reforms to help researchers identify and address software bugs. The report "highlights the need for reform in the field of software vulnerability discovery and disclosure." [36] One of the report's authors said that Congress has not done enough to address cyber software vulnerability, even though Congress has passed a number of bills to combat the larger issue of cyber security. [36]

Government researchers, companies, and cyber security experts are the people who typically discover software flaws. The report calls for reforming computer crime and copyright laws. [36]

The Computer Fraud and Abuse Act, the Digital Millennium Copyright Act and the Electronic Communications Privacy Act criminalize and create civil penalties for actions that security researchers routinely engage in while conducting legitimate security research, the report said. [36]

See also

Related Research Articles

Computer programming or coding is the composition of sequences of instructions, called programs, that computers can follow to perform tasks. It involves designing and implementing algorithms, step-by-step specifications of procedures, by writing code in one or more programming languages. Programmers typically use high-level programming languages that are more easily intelligible to humans than machine code, which is directly executed by the central processing unit. Proficient programming usually requires expertise in several different subjects, including knowledge of the application domain, details of programming languages and generic code libraries, specialized algorithms, and formal logic.

In computing, source code, or simply code or source, is a plain text computer program written in a programming language. A programmer writes the human readable source code to control the behavior of a computer.

<span class="mw-page-title-main">Software testing</span> Checking software against a standard

Software testing is the act of checking whether software satisfies expectations.

Regression testing is re-running functional and non-functional tests to ensure that previously developed and tested software still performs as expected after a change. If not, that would be called a regression.

<span class="mw-page-title-main">Debugger</span> Computer program used to test and debug other programs

A debugger is a computer program used to test and debug other programs. Common features of debuggers include the ability to run or halt the target program using breakpoints, step through code line by line, and display or modify the contents of memory, CPU registers, and stack frames.

Unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source code is tested to validate expected behavior.

In computer programming, specifically when using the imperative programming paradigm, an assertion is a predicate connected to a point in the program, that always should evaluate to true at that point in code execution. Assertions can help a programmer read the code, help a compiler compile it, or help the program detect its own defects.

Software development is the process of designing and implementing a software solution to satisfy a user. The process is more encompassing than programming, writing code, in that it includes conceiving the goal, evaluating feasibility, analyzing requirements, design, testing and release. The process is part of software engineering which also includes organizational management, project management, configuration management and other aspects.

Source lines of code (SLOC), also known as lines of code (LOC), is a software metric used to measure the size of a computer program by counting the number of lines in the text of the program's source code. SLOC is typically used to predict the amount of effort that will be required to develop a program, as well as to estimate programming productivity or maintainability once the software is produced.

Test-driven development (TDD) is a way of writing code that involves writing an automated unit-level test case that fails, then writing just enough code to make the test pass, then refactoring both the test code and the production code, then repeating with another new test case.

A programming tool or software development tool is a computer program that is used to develop another program. A tool provides a command line interface (CLI), a graphical user interface (GUI), or both. A CLI allows a tool to be used for automation such as for build or test.

A patch is data that is intended to be used to modify an existing software resource such as a program or a file, often to fix bugs and security vulnerabilities. A patch may be created to improve functionality, usability, or performance. A patch is typically provided by a vendor for updating the software that they provide. A patch may be created manually, but commonly it is created via a tool that compares two versions of the resource and generates data that can be used to transform one to the other.

A tracking system or defect tracking system is a software application that keeps track of reported software bugs in software development projects. It may be regarded as a type of issue tracking system.

An instruction set simulator (ISS) is a simulation model, usually coded in a high-level programming language, which mimics the behavior of a mainframe or microprocessor by "reading" instructions and maintaining internal variables which represent the processor's registers.

Game testing, also called quality assurance (QA) testing within the video game industry, is a software testing process for quality control of video games. The primary function of game testing is the discovery and documentation of software defects. Interactive entertainment software testing is a highly technical field requiring computing expertise, analytic competence, critical evaluation skills, and endurance. In recent years the field of game testing has come under fire for being extremely strenuous and unrewarding, both financially and emotionally.

In computer programming jargon, a heisenbug is a software bug that seems to disappear or alter its behavior when one attempts to study it. The term is a pun on the name of Werner Heisenberg, the physicist who first asserted the observer effect of quantum mechanics, which states that the act of observing a system inevitably alters its state. In electronics, the traditional term is probe effect, where attaching a test probe to a device changes its behavior.

Extreme programming (XP) is an agile software development methodology used to implement software systems. This article details the practices used in this methodology. Extreme programming has 12 practices, grouped into four areas, derived from the best practices of software engineering.

In engineering, debugging is the process of finding the root cause, workarounds and possible fixes for bugs.

Debug code is computer code introduced to a computer program to test for errors or to help determine the cause of an error. It can be as simple as an echo command to print the value of a variable at certain points of a program. Modern integrated development environments sometimes render this unnecessary by allowing the placement of stop points at specific places in the program, and providing the ability to view the value of variables through the IDE rather than program output.

Software construction is a software engineering discipline. It is the detailed creation of working meaningful software through a combination of coding, verification, unit testing, integration testing, and debugging. It is linked to all the other software engineering disciplines, most strongly to software design and software testing.

References

  1. ↑ "Software bugs cost US economy dear". June 10, 2009. Archived from the original on June 10, 2009. Retrieved September 24, 2012.
  2. ↑ "Testing experience : te : the magazine for professional testers". Testing Experience. Germany: testingexperience: 42. March 2012. ISSN   1866-5705.(subscription required)
  3. 1 2 3 4 5 6 7 8 9 610.12-1990: IEEE Standard Glossary of Software Engineering Terminology. IEEE. December 31, 1990. doi:10.1109/IEEESTD.1990.101064. ISBN   978-0-7381-0391-4.
  4. ↑ "News at SEI September 1999". SEI Interactive. 2 (3). Carnegie Mellon University: Software Engineering Institute. September 1, 1999.
  5. ↑ Gregg Keizer (April 21, 2011). "Apple faces questions from Congress about iPhone tracking". Computerworld .
  6. ↑ Gregg Keizer (April 27, 2011). "Apple denies tracking iPhone users, but promises changes". Computerworld .
  7. ↑ Dorota Huizinga; Adam Kolawa (September 2007). Automated Defect Prevention: Best Practices in Software Management. Wiley-IEEE Computer Society Press. ISBN   978-0-470-04212-0.
  8. ↑ McDonald, Marc; Musson, Robert; Smith, Ross (2007). The Practical Guide to Defect Prevention . Microsoft Press. p.  480. ISBN   978-0-7356-2253-1.
  9. ↑ "Release Early, Release Often" Archived May 14, 2011, at the Wayback Machine , Eric S. Raymond, The Cathedral and the Bazaar
  10. ↑ "Wide Open Source" Archived September 29, 2007, at the Wayback Machine , Elias Levy, SecurityFocus, April 17, 2000
  11. ↑ "Maurice Wilkes Quotes". QuoteFancy. Retrieved April 28, 2024.
  12. ↑ "PolySpace Technologies history". christele.faure.pagesperso-orange.fr. Retrieved August 1, 2019.
  13. ↑ Allen, Mitch (May–June 2002). "Bug Tracking Basics: A beginner's guide to reporting and tracking defects". Software Testing & Quality Engineering Magazine. Vol. 4, no. 3. pp. 20–24. Retrieved December 19, 2017.
  14. ↑ Rex Black (2002). Managing The Testing Process (2nd ed.). Wiley India Pvt. Limited. p. 139. ISBN   978-8126503131 . Retrieved June 19, 2021.
  15. ↑ Chris Vander Mey (2012). Shipping Greatness - Practical Lessons on Building and Launching Outstanding Software, Learned on the Job at Google and Amazon. O'Reilly Media. pp. 79–81. ISBN   978-1449336608.
  16. ↑ Soleimani Neysiani, Behzad; Babamir, Seyed Morteza; Aritsugi, Masayoshi (October 1, 2020). "Efficient feature extraction model for validation performance improvement of duplicate bug report detection in software bug triage systems". Information and Software Technology. 126: 106344. doi:10.1016/j.infsof.2020.106344. S2CID   219733047.
  17. ↑ "5.3. Anatomy of a Bug". bugzilla.org. Archived from the original on May 23, 2013.
  18. ↑ Jones, Wilbur D. Jr., ed. (1989). "Show stopper". Glossary: defense acquisition acronyms and terms (4 ed.). Fort Belvoir, Virginia: Department of Defense, Defense Systems Management College. p. 123. hdl:2027/mdp.39015061290758 – via Hathitrust.
  19. ↑ Zachary, G. Pascal (1994). Show-stopper!: the breakneck race to create Windows NT and the next generation at Microsoft . New York: The Free Press. p. 158. ISBN   0029356717 – via archive.org.
  20. ↑ "The Next Generation 1996 Lexicon A to Z: Slipstream Release". Next Generation . No. 15. March 1996. p. 41.
  21. ↑ Carr, Nicholas (2018). "'It's Not a Bug, It's a Feature.' Trite – or Just Right?". wired.com.
  22. ↑ Lientz, B. P.; Swanson, E. B.; Tompkins, G. E. (1978). "Characteristics of Application Software Maintenance". Communications of the ACM. 21 (6): 466–471. doi: 10.1145/359511.359522 . S2CID   14950091.
  23. ↑ Amit, Idan; Feitelson, Dror G. (2020). "The Corrective Commit Probability Code Quality Metric". arXiv: 2007.10912 [cs.SE].
  24. ↑ "An Overview of the Software Engineering Laboratory" (PDF). Software Engineering Laboratory Series (SEL-94-005). December 1994.
  25. 1 2 Cobb, Richard H.; Mills, Harlan D. (1990). "Engineering software under statistical quality control". IEEE Software . 7 (6): 46. doi:10.1109/52.60601. ISSN   1937-4194. S2CID   538311 – via University of Tennessee – Harlan D. Mills Collection.
  26. ↑ McConnell, Steven C. (1993). Code Complete . Redmond, Washington: Microsoft Press. p. 611. ISBN   978-1556154843 – via archive.org. (Cobb and Mills 1990)
  27. ↑ Gerard Holzmann (March 5, 2009). "Appendix D – Software Complexity" (PDF). Final Report: NASA Study on Flight Software Complexity (Daniel L. Dvorak (Ed.)). NASA Office of Chief Engineer Technical Excellence Program.
  28. ↑ Le Goues, Claire; Holtschulte, Neal; Smith, Edward K.; Brun, Yuriy; Devanbu, Premkumar; Forrest, Stephanie; Weimer, Westley (2015). "The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs". IEEE Transactions on Software Engineering. 41 (12): 1236–1256. doi: 10.1109/TSE.2015.2454513 . ISSN   0098-5589.
  29. ↑ Just, René; Jalali, Darioush; Ernst, Michael D. (2014). "Defects4J: a database of existing faults to enable controlled testing studies for Java programs". Proceedings of the 2014 International Symposium on Software Testing and Analysis – ISSTA 2014. pp. 437–440. CiteSeerX   10.1.1.646.3086 . doi:10.1145/2610384.2628055. ISBN   9781450326452. S2CID   12796895.
  30. ↑ Anthony Di Franco; Hui Guo; Cindy Rubio-González (November 23, 2017). A comprehensive study of real-world numerical bug characteristics. 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE. doi:10.1109/ASE.2017.8115662.
  31. ↑ Kimbler, K. (1998). Feature Interactions in Telecommunications and Software Systems V. IOS Press. p. 8. ISBN   978-90-5199-431-5.
  32. ↑ Syed, Mahbubur Rahman (2001). Multimedia Networking: Technology, Management and Applications: Technology, Management and Applications. Idea Group Inc (IGI). p. 398. ISBN   978-1-59140-005-9.
  33. ↑ Wu, Chwan-Hwa (John); Irwin, J. David (2016). Introduction to Computer Networks and Cybersecurity. CRC Press. p. 500. ISBN   978-1-4665-7214-0.
  34. ↑ RFC 1263: "TCP Extensions Considered Harmful" quote: "the time to distribute the new version of the protocol to all hosts can be quite long (forever in fact). ... If there is the slightest incompatibly between old and new versions, chaos can result."
  35. ↑ Wilson, Andi; Schulman, Ross; Bankston, Kevin; Herr, Trey. "Bugs in the System" (PDF). Open Policy Institute. Archived (PDF) from the original on September 21, 2016. Retrieved August 22, 2016.
  36. 1 2 3 4 Rozens, Tracy (August 12, 2016). "Cyber reforms needed to strengthen software bug discovery and disclosure: New America report – Homeland Preparedness News" . Retrieved August 23, 2016.
  37. ↑ Ullman, Ellen (2004). The Bug. Picador. ISBN   978-1-250-00249-5.
This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.