API testing

API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. ^[1] Since APIs lack a GUI, API testing is performed at the message layer. ^[2] API testing is now considered critical for automating testing because APIs serve as the primary interface to application logic and because GUI tests are difficult to maintain with the short release cycles and frequent changes commonly used with Agile software development and DevOps. ^{[3] [4]}

API testing overview

API testing involves testing APIs directly (in isolation) and as part of the end-to-end transactions exercised during integration testing. ^[1] Beyond RESTful APIs, these transactions include multiple types of endpoints such as web services, ESBs, databases, mainframes, web UIs, and ERPs. API testing is performed on APIs that the development team produces as well as APIs that the team consumes within their application (including third-party APIs). ^[5]

API testing is used to determine whether APIs return the correct response (in the expected format) for a broad range of feasible requests, react properly to edge cases such as failures and unexpected/extreme inputs, deliver responses in an acceptable amount of time, and respond securely to potential security attacks. ^{[1] [4]} Service virtualization is used in conjunction with API testing to isolate the services under test as well as expand test environment access by simulating APIs/services that are not accessible for testing. ^[6]

API testing commonly includes testing REST APIs or SOAP web services with JSON or XML message payloads being sent over HTTP, HTTPS, JMS, and MQ. ^{[2] [7]} It can also include message formats such as SWIFT, FIX, EDI and similar fixed-length formats, CSV, ISO 8583 and Protocol Buffers being sent over transports/protocols such as TCP/IP, ISO 8583, MQTT, FIX, RMI, SMTP, TIBCO Rendezvous, and FIX. ^{[8] [9]}

API testing, GUI testing, and test automation

API Testing is recognised as being more suitable for test automation and continuous testing (especially the automation used with Agile software development and DevOps) than GUI testing. ^{[3] [4]} Reasons cited include:

• System complexity: GUI tests can't sufficiently verify functional paths and back-end APIs/services associated with multitier architectures. APIs are considered the most stable interface to the system under test.
• Short release cycles with fast feedback loops: Agile and DevOps teams working with short iterations and fast feedback loops find that GUI tests require considerable rework to keep pace with frequent change. Tests at the API layer are less brittle and easier to maintain.

For these reasons, it is recommended that teams increase their level of API testing while decreasing their reliance on GUI testing. API testing is recommended for the vast majority of test automation efforts and as much edge testing as possible. GUI testing is then reserved for validating typical use cases at the system level, mobile testing, and usability testing. ^{[3] [4] [10]}

Types of API testing

There are several types of tests that can be performed on APIs. Some of these include smoke testing, functional testing, security testing, penetration testing, and validation testing.

• Artificial intelligence (AI) used in API testing improves the efficiency and accuracy of the testing process. It can automatically generate test cases, identify potential issues, and analyze test results through machine learning to identify patterns and anomalies. ^[11]
• Smoke test - This is a preliminary test that checks if the most crucial functions of an API are working correctly and identifies any major issues before further testing.
• Functional testing - This type of testing validates a software system against its functional requirements by providing input and verifying the output. It mainly involves black box testing and is not concerned with the source code.
• Black box testing - This is a type of testing where the tester interacts with the API without knowing its internal workings. The tester provides input and observes the output generated by the API to identify how it responds to expected and unexpected user actions.
• Unit testing - This tests the smallest parts of an application, called units, for proper operation. In API testing, this includes testing single endpoints with a single request.
• Interoperability testing - This test checks if an API can interact with other software components and systems without compatibility issues. This applies to SOAP APIs.
• Reliability testing - This tests APIs to determine if they meet expectations for functionality, reliability, performance, and security. It aims to ensure that the API consistently performs as expected.
• Validation testing - This confirms the software matches business requirements and if API tests match expected results. It is closely related to User Acceptance Testing.
• Runtime error detection - This evaluates the actual running of an API and focuses on monitoring, execution errors, resource leaks, and error detection. Detected errors are fixed to prevent runtime breakdowns.
• Fuzzing - This test transmits random, invalid or unexpected input to an API to find unknown bugs and defects. An API fuzzer generates test inputs and request sequences to the API and records the response to see if any bugs or security vulnerabilities are uncovered.
• Load test - This type of testing simulates real-world workloads to see how a system or application performs. The goal is to find bottlenecks and determine the maximum number of users or transactions the system can handle.
• Performance testing - This type of testing evaluates how an API performs under certain conditions to assess the API's ability to handle high loads and maintain high-performance levels. There are two main types of API performance testing: functional testing and load testing.
• Security Testing - This checks for vulnerabilities in APIs to find and fix security gaps. It involves mimicking hacker actions to find bugs and prevent attackers from accessing or disrupting the API or its data.
• Penetration Testing - Ethical hacking is used to assess the security of an API design. An external pentester finds vulnerabilities in API integrations due to incorrect business logic or programming issues to identify security vulnerabilities that attackers could exploit.
• WS-* compliance testing - This testing applies to SOAP APIs and ensures proper implementation of standards such as WS-Addressing, WS-Discovery, WS-Federation, WS-Policy, WS-Security, and WS-Trust.
• Web UI testing - Checks if the visual elements of a web application's user interface work correctly and are user-friendly. It is different from API testing, which tests the communication between software components.

Software

Name	Vendor
SoapSonar	Crosscheck Networks
SoapUI	SmartBear Software
Postman API Platform	Postman (software)
SOAtest	Parasoft
Swagger	SmartBear Software
Katalon Studio	Katalon
Step CI	Step CI
Insomnia REST	Kong Inc.

See also

• Automated testing
• Service virtualization
• Software testing
• Web API security

References

1. Testing APIs protects applications and reputations, by Amy Reichert, SearchSoftwareQuality March 2015
2. All About API Testing: An Interview with Jonathan Cooper, by Cameron Philipp-Edmonds, Stickyminds August 19, 2014
3. The Forrester Wave Evaluation Of Functional Test Automation (FTA) Is Out And It's All About Going Beyond GUI Testing Archived 2015-05-28 at the Wayback Machine , by Diego Lo Giudice, Forrester April 23, 2015
4. Produce Better Software by Using a Layered Testing Strategy, by SEAN Kenefick, Gartner January 7, 2014
5. Onus for third-party APIs is on enterprise developers Archived 2019-07-31 at the Wayback Machine , by Amy Reichert, SearchSoftwareQuality July 2014
6. Accelerate Development with Automated Testing ^{[ dead link ‍]}, by Nathan Wilson, Gartner December 30, 2013
7. A Guidance Framework for Designing a Great Web API ^{[ dead link ‍]}, by Eric Knipp and Gary Olliffe, Gartner August 20, 2014
8. The Fight Against Brittle Scripts and Software Defects, by Adrian Bridgwater, Dr. Dobb's Journal October 26, 2011
9. How Do We Learn Composite App Testing-Speak?, by Adrian Bridgwater, Dr. Dobb's Journal February 14, 2012
10. Cohn, Mike (2009). Succeeding with Agile: Software Development Using Scrum . Addison-Wesley Professional. p.  312. ISBN   978-0321579362.
11. J. Gao, C. Tao, D. Jie ĺ, S. Lu (2019). What is AI Software Testing? and Why. IEEE.