DevOps

Last updated

DevOps is the integration and automation of the software development and information technology operations [a] . DevOps encompasses necessary tasks of software development and can lead to shortening development time and improving the development life cycle. [1] According to Neal Ford, DevOps, particularly through continuous delivery, employs the "Bring the pain forward" principle, tackling tough tasks early, fostering automation and swift issue detection. [2] Software programmers and architects should use fitness function to keep their software in check. [3]

Contents

Although debated, [b] [c] [d] [e] DevOps is characterized by key principles: shared ownership, workflow automation, and rapid feedback. From an academic perspective, Len Bass, Ingo Weber, and Liming Zhu—three computer science researchers from the CSIRO and the Software Engineering Institute—suggested defining DevOps as "a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality". [7] However, the term is used in multiple contexts. At its most successful, DevOps is a combination of specific practices, culture change, and tools. [8]

History

Proposals to combine software development methodologies with deployment and operations concepts began to appear in the late 80s and early 90s. [9]

Around 2007 and 2008, concerns were raised by those within the software development and IT communities that the separation between the two industries, where one wrote and created software entirely separate from those that deploy and support the software was creating a fatal level of dysfunction within the industry. [10]

In 2009, the first conference named DevOps Days was held in Ghent, Belgium. The conference was founded by Belgian consultant, project manager and agile practitioner Patrick Debois. [11] [12] The conference has now spread to other countries. [13]

In 2012, a report called "State of DevOps" was first published by Alanna Brown at Puppet Labs. [14] [15]

As of 2014, the annual State of DevOps report was published by Nicole Forsgren, Gene Kim, Jez Humble and others. They stated that the adoption of DevOps was accelerating. [16] [17] Also in 2014, Lisa Crispin and Janet Gregory wrote the book More Agile Testing, containing a chapter on testing and DevOps. [18] [19]

In 2016, the DORA metrics for throughput (deployment frequency, lead time for changes), and stability (mean time to recover, change failure rate) were published in the State of DevOps report. [14] However, the research methodology and metrics were criticized by experts. [20] [21] [22] [23] In response to these criticisms, the 2023 State of DevOps report [24] published changes that updated the stability metric "mean time to recover" to "failed deployment recovery time" acknowledging the confusion the former metric has caused. [25]

Relevant metrics

DORA metrics are a set of key metrics developed by DevOps Research and Assessment (DORA) which can help to measure software development efficiency and reliability. These metrics include: [26]

These metrics, when applied appropriately and within relevant context, facilitate insights into DevOps performance, enabling teams to optimize deployment speed, reliability and quality, thereby informing data-driven decisions to enhance software development processes. [26]

Relationship to other approaches

Many of the ideas fundamental to DevOps practices are inspired by, or mirror, other well known practices such as Lean and Deming's Plan-Do-Check-Act cycle, through to The Toyota Way and the Agile approach of breaking down components and batch sizes. [28] Contrary to the "top-down" prescriptive approach and rigid framework of ITIL in the 1990s, DevOps is "bottom-up" and flexible, having been created by software engineers for their own needs. [29]

Agile

The motivations for what has become modern DevOps and several standard DevOps practices such as automated build and test, continuous integration, and continuous delivery originated in the Agile world, which dates (informally) to the 1990s, and formally to 2001. Agile development teams using methods such as extreme programming couldn't "satisfy the customer through early and continuous delivery of valuable software" [30] unless they took responsibility for operations and infrastructure for their applications, automating much of that work. Because Scrum emerged as the dominant Agile framework in the early 2000s and it omitted the engineering practices that were part of many Agile teams, the movement to automate operations and infrastructure functions splintered from Agile and expanded into what has become modern DevOps. Today, DevOps focuses on the deployment of developed software, whether it is developed using Agile oriented methodologies or other methodologies.

ArchOps

ArchOps presents an extension for DevOps practice, starting from software architecture artifacts, instead of source code, for operation deployment. [31] ArchOps states that architectural models are first-class entities in software development, deployment, and operations.

Continuous Integration and Delivery (CI/CD)

Automation is a core principle for achieving DevOps success and CI/CD is a critical component. [32] Plus, improved collaboration and communication between and within teams helps achieve faster time to market, with reduced risks. [33]

Mobile DevOps

Mobile DevOps is a set of practices that applies the principles of DevOps specifically to the development of mobile applications. Traditional DevOps focuses on streamlining the software development process in general, but mobile development has its own unique challenges that require a tailored approach. [34] Mobile DevOps is not simply as a branch of DevOps specific to mobile app development, instead an extension and reinterpretation of the DevOps philosophy due to very specific requirements of the mobile world.

Site-reliability engineering

In 2003, Google developed site reliability engineering (SRE), an approach for releasing new features continuously into large-scale high-availability systems while maintaining high-quality end-user experience. [35] While SRE predates the development of DevOps, they are generally viewed as being related to each other. Some of the original authors of the discipline consider SRE as an implementation of DevOps. [36]

Toyota production system, lean thinking, kaizen

Toyota production system, also known under the acronym TPS, was the inspiration for lean thinking with its focus on continuous improvement, kaizen, flow and small batches. The andon cord principle to create fast feedback, swarm and solve problems stems from TPS. [37] [38]

DevSecOps, shifting security left

DevSecOps is an augmentation of DevOps to allow for security practices to be integrated into the DevOps approach. Contrary to a traditional centralized security team model, each delivery team is empowered to factor in the correct security controls into their software delivery. Security practices and testing are performed earlier in the development lifecycle, hence the term "shift left". Security is tested in three main areas: static, software composition, and dynamic.

Checking software statically via static application security testing (SAST) is white-box testing with special focus on security. Depending on the programming language, different tools are needed to do such static code analysis. The software composition is analyzed, especially libraries, and the version of each component is checked against vulnerability lists published by CERT and other expert groups. When giving software to clients, library licenses and their match to the license of the software distributed are in focus, especially copyleft licenses.

In dynamic testing, also called black-box testing, software is tested without knowing its inner functions. In DevSecOps this practice may be referred to as dynamic application security testing (DAST) or penetration testing. The goal is early detection of defects including cross-site scripting and SQL injection vulnerabilities. Threat types are published by the open web application security project, e.g. its TOP10, [39] and by other bodies.

DevSecOps has also been described as a cultural shift involving a holistic approach to producing secure software by integrating security education, security by design, and security automation. [40]

Cultural change

DevOps initiatives can create cultural changes in companies [41] by transforming the way operations, developers, and testers collaborate during the development and delivery processes. [42] Getting these groups to work cohesively is a critical challenge in enterprise DevOps adoption. [43] [44] DevOps is as much about culture as it is about the toolchain. [45]

Microservices

Although in principle it is possible to practice DevOps with any architectural style, the microservices architectural style is becoming the standard for building continuously deployed systems. Small size service allows the architecture of an individual service to emerge through continuous refactoring. [46]

DevOps automation

It also supports consistency, reliability, and efficiency within the organization, and is usually enabled by a shared code repository or version control. As DevOps researcher Ravi Teja Yarlagadda hypothesizes, "Through DevOps, there is an assumption that all functions can be carried out, controlled, and managed in a central place using a simple code." [47]

Automation with version control

Many organizations use version control to power DevOps automation technologies like virtual machines, containerization (or OS-level virtualization), and CI/CD. The paper "DevOps: development of a toolchain in the banking domain" notes that with teams of developers working on the same project, "All developers need to make changes to the same codebase and sometimes edit even the same files. For efficient working, there has to be a system that helps engineers avoid conflicts and retain the codebase history," [48] with the Git version control system and the GitHub platform referenced as examples.

GitOps

GitOps evolved from DevOps. The specific state of deployment configuration is version-controlled. Because the most popular version-control is Git, GitOps' approach has been named after Git. Changes to configuration can be managed using code review practices, and can be rolled back using version-controlling. Essentially, all of the changes to a code are tracked, bookmarked, and making any updates to the history can be made easier. As explained by Red Hat, "visibility to change means the ability to trace and reproduce issues quickly, improving overall security." [49]

Best practices for cloud systems

The following practices can enhance productivity of DevOps pipelines, especially in systems hosted in the cloud: [50] [51] [52]

See also

Notes

  1. ↑ a portmanteau of development operations
  2. ↑ Dyck et al. (2015) "To our knowledge, there is no uniform definition for the terms release engineering and DevOps. As a consequence, many people use their own definitions or rely on others, which results in confusion about those terms." [4]
  3. ↑ Jabbari et al. (2016) "The research results of this study showed the need for a definition as individual studies do not consistently define DevOps." [5]
  4. ↑ Erich et al. (2017) "We noticed that there are various gaps in the study of DevOps: There is no consensus of what concepts DevOps covers, nor how DevOps is defined." [6]
  5. ↑ Erich et al. (2017) "We discovered that there exists little agreement about the characteristics of DevOps in the academic literature." [6]

Related Research Articles

<span class="mw-page-title-main">Continuous integration</span> Software development practice of building and testing frequently

Continuous integration (CI) is the practice of integrating source code changes frequently and ensuring that the integrated codebase is in a workable state.

Release engineering, frequently abbreviated as RE or as the clipped compound Releng, is a sub-discipline in software engineering concerned with the compilation, assembly, and delivery of source code into finished products or other software components. Associated with the software release life cycle, it was said by Boris Debic of Google Inc. that release engineering is to software engineering as manufacturing is to an industrial process:

Release engineering is the difference between manufacturing software in small teams or startups and manufacturing software in an industrial way that is repeatable, gives predictable results, and scales well. These industrial style practices not only contribute to the growth of a company but also are key factors in enabling growth.

Build automation is the practice of building software systems in a relatively unattended fashion. The build is configured to run with minimized or no software developer interaction and without using a developer's personal computer. Build automation encompasses the act of configuring the build system as well the resulting system itself.

AnthillPro is a software tool originally developed and released as one of the first continuous integration servers. AnthillPro automates the process of building code into software projects and testing it to verify that project quality has been maintained. Software developers are able to identify bugs and errors earlier by using AnthillPro to track, collate, and test changes in real time to a collectively maintained body of computer code.

<span class="mw-page-title-main">Release management</span> Process of software building

Release management is the process of managing, planning, scheduling and controlling a software build through different stages and environments; it includes testing and deploying software releases.

Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. Continuous testing was originally proposed as a way of reducing waiting time for feedback to developers by introducing development environment-triggered tests as well as more traditional developer/tester-triggered tests.

Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time. It aims at building, testing, and releasing software with greater speed and frequency. The approach helps reduce the cost, time, and risk of delivering changes by allowing for more incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery.

Application-release automation (ARA) refers to the process of packaging and deploying an application or update of an application from development, across various environments, and ultimately to production. ARA solutions must combine the capabilities of deployment automation, environment management and modeling, and release coordination.

CloudForge is a Global DevOps and Cloud solutions provider specializing in infrastructure automation, Kubernetes management, and cloud cost optimization. The company delivers verified DevOps expertise within 48 hours, focusing on building reliable, cost-efficient systems through automation and proven methodologies.

<span class="mw-page-title-main">BuildMaster</span>

BuildMaster is an application release automation tool, designed by the software development team Inedo. It combines build management and ARA capabilities to manage and automate processes primarily related to continuous integration, database change scripts, and production deployments, overall releasing applications reliably. The tool is browser-based and able to be used "out-of-the-box". Its feature set and scope puts it in line with the DevOps movement, and is marketed as "more than a release automatigs together the people, processes, and practices that allow teams to deliver software rapidly, reliably, and responsibly.” It's a tool that embodies incremental DevOps adoption.

Infrastructure as code (IaC) is the process of managing and provisioning computer data center resources through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. The IT infrastructure managed by this process comprises both physical equipment, such as bare-metal servers, as well as virtual machines, and associated configuration resources. The definitions may be in a version control system, rather than maintaining the code through manual processes. The code in the definition files may use either scripts or declarative definitions, but IaC more often employs declarative approaches.

<span class="mw-page-title-main">DevOps toolchain</span> DevOps toolchain release package.

A DevOps toolchain is a set or combination of tools that aid in the delivery, development, and management of software applications throughout the systems development life cycle, as coordinated by an organisation that uses DevOps practices.

Continuous configuration automation (CCA) is the methodology or process of automating the deployment and configuration of settings and software for both physical and virtual data center equipment.

In software engineering, CI/CD or CICD is the combined practices of continuous integration (CI) and continuous delivery (CD) or, less often, continuous deployment. They are sometimes referred to collectively as continuous development or continuous software development.

DataOps is a set of practices, processes and technologies that combines an integrated and process-oriented perspective on data with automation and methods from agile software engineering to improve quality, speed, and collaboration and promote a culture of continuous improvement in the area of data analytics. While DataOps began as a set of best practices, it has now matured to become a new and independent approach to data analytics. DataOps applies to the entire data lifecycle from data preparation to reporting, and recognizes the interconnected nature of the data analytics team and information technology operations.

Nicole Forsgren is an American technology executive, IT impact expert, and author. In 2020, she was named vice president of Research & Strategy at Microsoft's GitHub and, more recently, Partner at Microsoft Research. She coauthored Accelerate: The Science of Lean Software and DevOps, which won the Shingo Research and Professional Publication Award in 2019.

TestOps refers to the discipline of managing the operational aspects of testing within the software delivery lifecycle.

Buildkite is a continuous integration (CI) and continuous delivery (CD) platform used in DevOps and company was founded in September 2013. Companies using Buildkite include Wayfair, Shopify, Slack, Twilio, Canva, Pinterest, Envato, MYOB and Lyft.

Mobile DevOps is a set of practices that applies the principles of DevOps specifically to the development of mobile applications. Traditional DevOps focuses on streamlining the software development process in general, but mobile development has its own unique challenges that require a tailored approach. Mobile DevOps is not simply as a branch of DevOps specific to mobile app development, instead an extension and reinterpretation of the DevOps philosophy due to very specific requirements of the mobile world.

<i>DevOps Research and Assessment</i> Research team in Google Cloud

DevOps Research and Assessment is a team that is part of Google Cloud that engages in opinion polling of software engineers to conduct research for the DevOps movement.

References

  1. ↑ Courtemanche, Meredith; Mell, Emily; Gills, Alexander S. "What Is DevOps? The Ultimate Guide". TechTarget. Retrieved 2023-01-22.
  2. ↑ Building Evolutionary Architectures: Automated Software Governance. ISBN   978-1492097549.
  3. ↑ Fundamentals of Software Architecture: An Engineering Approach. O'Reilly Media. 2020. ISBN   978-1492043454.
  4. ↑ Dyck, Andrej; Penners, Ralf; Lichter, Horst (2015-05-19). "Towards Definitions for Release Engineering and DevOps". 2015 IEEE/ACM 3rd International Workshop on Release Engineering. IEEE. p. 3. doi:10.1109/RELENG.2015.10. ISBN   978-1-4673-7070-7. S2CID   4659735.
  5. ↑ Jabbari, Ramtin; bin Ali, Nauman; Petersen, Kai; Tanveer, Binish (May 2016). "What is DevOps?: A Systematic Mapping Study on Definitions and Practices". Proceedings of the 2016 Scientific Workshop. Association for Computing Machinery.
  6. 1 2 Erich, F.M.A.; Amrit, C.; Daneva, M. (June 2017). "A Qualitative Study of DevOps Usage in Practice" (PDF). Journal of Software: Evolution and Process. 29 (6): e1885. doi:10.1002/smr.1885. S2CID   35914007.
  7. ↑ Bass, Len; Weber, Ingo; Zhu, Liming (2015). DevOps: A Software Architect's Perspective. Addison-Wesley. ISBN   978-0134049847.
  8. ↑ Muñoz, Mirna; Negrete Rodríguez, Mario (April 2021). "A guidance to implement or reinforce a DevOps approach in organizations: A case study".{{cite journal}}: Cite journal requires |journal= (help)
  9. ↑ Chapman, M., Gatti, N: A model of a service life cycle, Proceedings of TINA '93, pp. I-205–I-215, Sep., 1993.
  10. ↑ Atlassian. "History of DevOps". Atlassian. Retrieved 2023-02-23.
  11. ↑ Mezak, Steve (25 January 2018). "The Origins of DevOps: What's in a Name?". devops.com. Retrieved 6 May 2019.
  12. ↑ Debois, Patrick (9 October 2008). "Agile 2008 Toronto". Just Enough Documented Information. Retrieved 12 March 2015.
  13. ↑ Debois, Patrick. "DevOps Days". DevOps Days. Retrieved 31 March 2011.
  14. 1 2 Alana Brown; Nicole Forsgren; Jez Humble; Nigel Kersten; Gene Kim (2016). "2016 State of DevOps Report" (PDF). Puppet Labs, DORA (DevOps Research. Retrieved 2024-04-24.
  15. ↑ "Puppet - Alanna Brown". Puppet Labs. Retrieved 2019-04-27.
  16. ↑ Nicole Forsgren; Gene Kim; Nigel Kersten; Jez Humble (2014). "2014 State of DevOps Report" (PDF). Puppet Labs, IT Revolution Press and ThoughtWorks. Retrieved 2024-04-24.
  17. ↑ "2015 State of DevOps Report" (PDF). Puppet Labs, Pwc, IT Revolution Press. 2015. Retrieved 2024-04-24.
  18. ↑ "More Agile Testing" (PDF). October 2014. Retrieved 2019-05-06.
  19. ↑ Crispin, Lisa; Gregory, Janet (October 2014). More Agile Testing. Addison-Wesley. ISBN   9780133749571 . Retrieved 2019-05-06.
  20. ↑ Turner, Graham (20 November 2023). "Report: Software Engineers Face Backlash for Reporting Wrongdoing". DIGIT. Retrieved 5 January 2024.
  21. ↑ Saran, Cliff. "Software engineers worry about speaking out - Computer Weekly". ComputerWeekly.com. Retrieved 5 January 2024.
  22. ↑ "75% of software engineers faced retaliation the last time they reported wrongdoing - ETHRWorldSEA". ETHRWorld.com.
  23. ↑ Cummins, Holly. "Holly Cummins on X". X.com. Retrieved 5 January 2024.
  24. ↑ DeBellis, Derek; Lewis, Amanda; Villalba, Daniella; Farley, Dave. "2023 State of DevOps Report". Google Cloud DevOps Research and Assessment. Retrieved 2024-04-24.
  25. ↑ DeBellis, Derek; Harvey, Nathan. "2023 State of DevOps Report: Culture is everything". Google Cloud Blog. Retrieved 2024-04-24.
  26. 1 2 Forsgren, Nicole. Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations. ISBN   978-1942788331.
  27. ↑ "DORA Accelerate State of DevOps 2021". DevOps Research and Assessment (2021). 2021.
  28. ↑ Klein, Brandon Thorin (2021-05-01). "The DevOps: A Concise Understanding to the DevOps Philosophy and Science". Osti.gov. doi:10.2172/1785164. OSTI   1785164. S2CID   236606284.
  29. ↑ "The History and Evolution of DevOps | Tom Geraghty". 5 July 2020. Retrieved 2020-11-29.
  30. ↑ "Principles behind the Agile Manifesto". agilemanifesto.org. Retrieved 2020-12-06.
  31. ↑ Castellanos, Camilo; Correal, Dario (15 September 2018). "Executing Architectural Models for Big Data Analytics". Software Architecture. Lecture Notes in Computer Science. Vol. 11048. pp. 364–371. doi:10.1007/978-3-030-00761-4_24. ISBN   978-3-030-00760-7.
  32. ↑ Humble, Jez; Farley, David (2011). Continuous Delivery: reliable software releases through build, test, and deployment automation. Pearson Education Inc. ISBN   978-0-321-60191-9.
  33. ↑ Chen, Lianping (2015). "Continuous Delivery: Huge Benefits, but Challenges Too". IEEE Software. 32 (2): 50–54. doi:10.1109/MS.2015.27. S2CID   1241241.
  34. ↑ Tak, Rohin; Modi, Jhalak (2018). Mobile DevOps: Deliver continuous integration and deployment within your mobile applications. Packt Publishing. pp. 12–18. ISBN   9781788296243.
  35. ↑ Beyer, Betsy; Jones, Chris; Petoff, Jennifer; Murphy, Niall Richard (April 2016). Site Reliability Engineering. O'Reilly Media. ISBN   978-1-4919-2909-4.
  36. ↑ Dave Harrison (9 Oct 2018). "Interview with Betsy Beyer, Stephen Thorne of Google" . Retrieved 24 July 2024.
  37. ↑ Analyzing the DNA of DevOps, Brent Aaron Reed, Willy Schaub, 2018-11-14.
  38. ↑ Gene Kim; Patrick Debois; John Willis; Jezz Humble (2016). The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations.
  39. ↑ "OWASP TOP10". Archived from the original on June 8, 2023. Retrieved June 8, 2023.
  40. ↑ Wilson, Glenn (December 2020). 'DevSecOps: A leader's guide to producing secure software with compromising flow, feedback and continuous improvement'. Rethink Press. ISBN   978-1781335024.
  41. ↑ Emerging Technology Analysis: DevOps a Culture Shift, Not a Technology (Report). Gartner.
  42. ↑ Loukides, Mike (7 June 2012). "What is DevOps?". O'Reilly Media.
  43. ↑ "Gartner IT Glossary – devops". Gartner. Retrieved 30 October 2015.
  44. ↑ Jones, Stephen; Noppen, Joost; Lettice, Fiona (21 July 2016). Proceedings of the 2nd International Workshop on Quality-Aware DevOps - QUDOS 2016 (PDF). pp. 7–11. doi:10.1145/2945408.2945410. ISBN   9781450344111. S2CID   515140.
  45. ↑ Mandi Walls (25 September 2015). "Building a DevOps culture". O'Reilly.
  46. ↑ Chen, Lianping; Ali Babar, Muhammad (2014). "2014 IEEE/IFIP Conference on Software Architecture". The 11th Working IEEE/IFIP Conference on Software Architecture(WICSA 2014). IEEE. pp. 195–204. doi:10.1109/WICSA.2014.45. ISBN   978-1-4799-3412-6.
  47. ↑ Teja Yarlagadda, Ravi (9 March 2021). "DevOps and Its Practices". SSRN   3798877.
  48. ↑ Morisio, Maurizio (16 April 2021). DevOps: development of a toolchain in the banking domain. Politecnico di Torino (laurea thesis). Retrieved 16 August 2021.
  49. ↑ "What is GitOps?". www.redhat.com. Retrieved 2023-03-30.
  50. ↑ Serverless Architectures on AWS. Manning. ISBN   978-1617295423.
  51. ↑ Pipeline as Code Continuous Delivery with Jenkins, Kubernetes, and Terraform. Manning. ISBN   9781638350378.
  52. ↑ Continuous Delivery Reliable Software Releases Through Build, Test, and Deployment Automation. ISBN   9780321670229.

Further reading

This page is based on this Wikipedia article
Text is available under the CC BY-SA 4.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.