 | This article needs additional citations for verification .(February 2011) |
| Part of a series on |
| Software development |
|---|
Test automation management tools are specific tools that provide a collaborative environment that is intended to make test automation efficient, traceable and clear for stakeholders. Test automation is becoming a cross-discipline (i.e. a mix of both testing and development practices.)
Test automation systems usually need more reporting, analysis and meaningful information about project status. Test management systems target manual effort and do not give all the required information. [1]
Test automation management systems leverage automation efforts towards efficient and continuous processes of delivering test execution and new working tests by:
Test automation management tools fit Agile Systems Development Life Cycle methodologies. In most cases, test automation covers continuous changes to minimize manual regression testing. Changes are usually noted by monitoring test log diffs. For example, differences in the number of failures signal probable changes either in AUT or in test code (broken test code base, instabilities) or in both. Quick notice of changes and unified workflow of results analysis reduces testing costs and increases project quality.
Test-driven development utilizes test automation as the primary driver to rapid and high-quality software production. Concepts of green line and thoughtful design are supported with tests before actual coding, assuming there are special tools to track and analyze within TDD process.
Another test automation practice [2] is continuous integration, which explicitly supposes automated test suites as a final stage upon building, deployment and distributing new versions of software. Based on acceptance of test results, a build is declared either as qualified for further testing or rejected. [3] Dashboards provide relevant information on all stages of software development including test results. However, dashboards do not support comprehensive operations and views for an automation engineer. This is another reason for dedicated management tools that can supply high-level data to other project management tools such as test management, issue management and change management.
Software testing is the act of examining the artifacts and the behavior of the software under test by validation and verification. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not necessarily limited to:
In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Test automation is critical for continuous delivery and continuous testing.
In software engineering, continuous integration (CI) is the practice of merging all developers' working copies to a shared mainline several times a day. Nowadays it is typically implemented in such a way that it triggers an automated build with testing. Grady Booch first proposed the term CI in his 1991 method, although he did not advocate integrating several times a day. Extreme programming (XP) adopted the concept of CI and did advocate integrating more than once per day – perhaps as many as tens of times per day.
Requirements management is the process of documenting, analyzing, tracing, prioritizing and agreeing on requirements and then controlling change and communicating to relevant stakeholders. It is a continuous process throughout a project. A requirement is a capability to which a project outcome should conform.
Keyword-driven testing, also known as action word based testing, is a software testing methodology suitable for both manual and automated testing. This method separates the documentation of test cases – including both the data and functionality to use – from the prescription of the way the test cases are executed. As a result, it separates the test creation process into two distinct stages: a design and development stage, and an execution stage. The design substage covers the requirement analysis and assessment and the data analysis, definition, and population.
Build automation is the process of automating the creation of a software build and the associated processes including: compiling computer source code into binary code, packaging binary code, and running automated tests.
Quality engineering is the discipline of engineering concerned with the principles and practice of product and service quality assurance and control. In software development, it is the management, development, operation and maintenance of IT systems and enterprise architectures with a high quality standard.
Azure DevOps Server is a Microsoft product that provides version control, reporting, requirements management, project management, automated builds, testing and release management capabilities. It covers the entire application lifecycle and enables DevOps capabilities. Azure DevOps can be used as a back-end to numerous integrated development environments (IDEs) but is tailored for Microsoft Visual Studio and Eclipse on all platforms.
AnthillPro is a software tool originally developed and released as one of the first continuous integration servers. AnthillPro automates the process of building code into software projects and testing it to verify that project quality has been maintained. Software developers are able to identify bugs and errors earlier by using AnthillPro to track, collate, and test changes in real time to a collectively maintained body of computer code.
Parasoft is an independent software vendor specializing in automated software testing and application security with headquarters in Monrovia, California. It was founded in 1987 by four graduates of the California Institute of Technology who planned to commercialize the parallel computing software tools they had been working on for the Caltech Cosmic Cube, which was the first working hypercube computer built.
Business process management (BPM) is the discipline in which people use various methods to discover, model, analyze, measure, improve, optimize, and automate business processes. Any combination of methods used to manage a company's business processes is BPM. Processes can be structured and repeatable or unstructured and variable. Though not required, enabling technologies are often used with BPM.
Test management tools are used to store information on how testing is to be done, plan testing activities and report the status of quality assurance activities. The tools have different approaches to testing and thus have different sets of features. Generally they are used to maintain and plan manual testing, run or gather execution data from automated tests, manage multiple environments and to enter information about found defects. Test management tools offer the prospect of streamlining the testing process and allow quick access to data analysis, collaborative tools and easy communication across multiple project teams. Many test management tools incorporate requirements management capabilities to streamline test case design from the requirements. Tracking of defects and project tasks are done within one application to further simplify the testing.
Micro Focus Application Lifecycle Management (ALM) is a set of software tools developed and marketed by Micro Focus (previously Hewlett-Packard and Hewlett Packard Enterprise) for application development and testing. It includes tools for requirements management, test planning and functional testing, performance testing (when used with Performance Center), developer management (through integration with developer environments such as Collabnet, TeamForge and Microsoft Visual Studio), and defect management.
Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate. Continuous testing was originally proposed as a way of reducing waiting time for feedback to developers by introducing development environment-triggered tests as well as more traditional developer/tester-triggered tests.
Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time and, following a pipeline through a "production-like environment", without doing so manually. It aims at building, testing, and releasing software with greater speed and frequency. The approach helps reduce the cost, time, and risk of delivering changes by allowing for more incremental updates to applications in production. A straightforward and repeatable deployment process is important for continuous delivery.
Development testing is a software development process that involves synchronized application of a broad spectrum of defect prevention and detection strategies in order to reduce software development risks, time, and costs.
Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. The IT infrastructure managed by this process comprises both physical equipment, such as bare-metal servers, as well as virtual machines, and associated configuration resources. The definitions may be in a version control system. The code in the definition files may use either scripts or declarative definitions, rather than maintaining the code through manual processes, but IaC more often employs declarative approaches.
The cTuning Foundation is a global non-profit organization developing open-source tools and a common methodology to enable sustainable, collaborative and reproducible research in Computer science, perform collaborative optimization of realistic workloads across devices provided by volunteers, enable self-optimizing computer systems, and automate artifact evaluation at machine learning and systems conferences and journals.
TestOps refers to the discipline of managing the operational aspects of testing within the software delivery lifecycle.