API Testing Tutorial

This section will help you get a basic understanding of Web Services and Web API, which, in turn, will be
helpful in understanding the major concepts in the upcoming tutorials in this API Testing series.

API (Application Programming Interface) is a set of all procedures and functions that allow us to create
an application by accessing the data or features of the operating system or platforms. Testing of such
procedures is known as API Testing.

Shift Left Testing

One of the important types of testing that is being asked nowadays in API Testing Interviews is Shift Left
Testing. This type of testing is practiced in almost all projects that follow an Agile Methodology.

Before Shift Left Testing was introduced, software testing came into picture only after the coding was
complete and code was delivered to the testers. This practice led to the last minute hustle to meet the
deadline and it also hampered the product quality to a great extent.

Apart from that, the efforts made (when defects were reported at the last phase before production)
were huge as developers had to go through both the design and coding phase all over again.

Software Development Life Cycle (SDLC) Before Shift Left Testing

Tradional SDLC flow was: Requirement –> Design –> Coding –> Testing.

Disadvantages of Traditional Testing

• Testing is at extreme right. A lot of costs are incurred when a bug is identified at the last minute.

• Time consumed in resolving the bug and retesting it before promoting it to production is huge.
Hence, a new idea popped up to shift the testing phase to the left which thereby led to Shift Left Testing.

Suggested Read => Shift Left Testing: A Secret Mantra For Software Success

Phases Of Left Shift Testing

Left Shift Testing led to a successful migration from Defect Detection to Defect Prevention. It also helped
the software to fail fast and fix all the failures at the earliest.

Also Read =>> What Are Vulkan Runtime Libraries

Web API

In general terms, a Web API can be defined as something that takes the request from a client system to a
web server and sends back the response from a web server to a client machine.

How Does an API Work?

Let’s take a very common scenario of booking a flight on www.makemytrip.com, which is an online travel
service that aggregates information from multiple airlines. When you go for a flight booking, you enter
information like journey date/return date, class, etc. and click on search.

This will show you the price of multiple airlines and their availability. In this case, the application
interacts with APIs of multiple airlines and thereby gives access to the airline’s data.

Another example is www.trivago.com which compares and lists down the price, availability, etc. of
different hotels from a particular city. This website communicates with APIs of multiple hotels to access
the database and lists down the prices and availability from their website.
Thus, a Web API can be defined as “an interface which facilitates the communication between a client
machine and the webserver”.

Web Services

Web Services are (like Web API) the services that serve from one machine to another. But the major
difference that arises between API and Web Services is that the Web Services uses a network.

It Is safe to say that all Web Services are Web APIs but all Web APIs are not Web Services (explained in
the latter part of the article). Thus, Web Services are a subset of Web API. Refer to the below diagram to
know more about Web API and Web Services.

Web API vs Web Services

Web Services vs Web API

Both Web API and Web Services are used to facilitate the communication between the client and the
server. The major difference comes only in the way they communicate.

Each of them requires a request body that is acceptable in a specific language, their differences in
providing a secure connection, their speed of communicating to the server and responding back to the
client, etc.

Differences Between Web Services and Web API is listed below for your reference.

Web Service

• Web Services generally use XML (Extensible Markup Language), which means they are more secure.
• Web Services is more secure as both Web Services and APIs provide SSL (Secure Socket Layer) during
data transmission, but it also provides WSS (Web Services Security).

• Web Service is a subset of Web API. For Example, Web Services are based only on three styles of use
i.e. SOAP, REST and XML-RPC.

• Web Services always need a network to operate.

• Web Services support “One Code different applications”. This means a more generic code is written
across different applications.

Web API

• A Web API generally uses JSON (JavaScript Object Notation), which means Web API is faster.

• Web API is faster as JSON is light-weighted, unlike XML.

• Web APIs are the superset of Web Services. For Example, All three styles of Web Services are present in
the Web API as well, but apart from that, it uses other styles like JSON – RPC.

• Web API does not necessarily require a network to operate.

• Web API may or may not support interoperability depending upon the nature of the system or
application.

Introducing API Testing In Your Organization

In our day to day life, all of us are so used to interacting with the Apps with APIs and yet we do not even
think about the back-end processes that drive the underlying functionality.
For Example, Let us consider that you are browsing through the products on Amazon.com and you see a
product/deal that you really like and you wish to share it with your Facebook network.

The moment you click on the Facebook icon on the share section of the page and enter your Facebook
account credentials to share, you are interacting with an API that is seamlessly connecting the Amazon
website to Facebook.

Focus Shift to API Testing

Before discussing more on API testing, let’s discuss the reasons for which the API based applications have
gained popularity in recent times.

There are several reasons for which organizations are transitioning to API based products and
applications. And few of them are enlisted below for your reference.

#1) API based applications are more scalable when compared to traditional applications/software. The
rate of code development is faster and the same API can service more requests without any major code
or infrastructural changes.

#2) Development teams don’t need to start coding from scratch every time they start working on
developing a feature or application. APIs most often reuse existing, repeatable functions, libraries, stored
procedures, etc. and hence this process can make them more productive overall.

For Example, If you are a developer working on an e-commerce website and you want to add Amazon as
a payment processor – then you do not have to write the code from scratch.

All you need to do is to set up integration between your website and Amazon API using Integration keys
and call Amazon API for processing payments during checkout.

#3) APIs allow easy integration with the other systems both for supported standalone applications as
well as with API based software products.
For Example, Let us consider that you want to send a shipment from Toronto to New York. You go online,
navigate to a well know Freight or Logistics website and enter the required information.

After providing the mandatory information, when you click on Get Rates button – in the back end, this
logistics website may be connecting with several carrier and service provider APIs and applications to get
the dynamic rates for the origin to destination combination of locations.

Full Spectrum of API Testing

Testing of APIs is not restricted to sending a request to API and analyzing the response for correctness
alone. The APIs need to be tested for their performance under different loads for vulnerabilities.

Let’s discuss this in detail.

(i) Functional Testing

Functional testing can be a challenging task due to the lack of a GUI interface.

Let’s see how the functional testing approach for APIs is different from GUI based application and we will
also discuss some examples around it.

a) The most obvious difference is that there is no GUI to interact with. Testers who usually do GUI
based functional testing find it a little harder to transition into non-GUI application testing when
compared to someone who is already familiar with it.

Initially, even before you start testing the API, you will need to test and verify the Authentication process
itself. The authentication method will vary from one API to another API and would involve some sort of
key or token for authentication.

If you are unable to connect to the API successfully, then further testing cannot proceed. This process
can be considered comparable to user authentication in the standard applications where you need valid
credentials to log in and use the application.
 b) Testing field validations or input data validation is very important during testing APIs. If an actual
form-based (GUI) interface was available, then field validations could be implemented in the
front end or back end, thereby ensuring that a user is not allowed to enter invalid field values.

For Example, If an application needs the date format to be DD/MM/YYYY, then we can apply this
validation on the form collecting information to ensure that the application is receiving and processing a
valid date.

This, however, is not the same for API applications. We need to ensure that the API is well written and is
able to enforce all these validations, distinguish between valid and invalid data and return the status
code and validation error message to the end-user through a response.

c) Testing the correctness of the responses from API for valid and invalid response is crucial indeed.
If a status code of 200 (meaning all Okay) is received as a response from test API, but if the
response text says an error has been encountered, then this is a defect.

Additionally, if the error message itself is incorrect, then that can be very misleading to the end customer
who is trying to integrate with this API.

In the screenshot below, the user has entered invalid weight, which is more than the acceptable 2267
Kgs. The API responds with the error status code and error message. However, the error message
incorrectly mentions the weight units as lbs instead of KG. This is a defect that can confuse the end
customer.

(ii) Load and Performance Testing

APIs are meant to be scalable by design.

This, in turn, makes Load and Performance Testing essential, especially if the system being designed is
expected to be servicing thousands of requests per minute or hour, depending on the requirement.
Routinely performing Load and Performance Tests on the API can help benchmark the performance,
peak loads and breaking point.
This data is useful while planning to scale up an application. Having this information available will help to
support decisions and planning especially if the organization is planning to add more customers, which
would mean more incoming requests.

For Example, let’s say that based on the requirements provided, we know that the API that is designed
needs to service at least 500 requests per hour and maintain the average response time of less than .01
seconds.

Based on our load and performance tests we found out that as long as API receives less than 500
requests per hour, it is able to maintain SLA for average response time. However, if it receives another
200 requests, then the average response time increases and the breaking point is reached when the
incoming request exceeds 1200 per hour.

Usually, it is seen that during the initial design phases, the emphasis is often on the functional aspects of
the API. As time goes by, a product starts supporting multiple live clients, that is when the testing for API
performance and Load testing comes into the picture in a more routine manner.

(iii) Security Testing

Application Programming Interfaces or APIs are vulnerable and are the easiest access point for malicious
hackers who want access to data or gain control of an application.

This can lead any company into legal trouble, where due to a security breach unintended people and/or
organizations are able to access client’s data through a venerable API.

Security testing is a specialized branch of testing and should be handled by specialists. The security
testing resources can be from within the organization or independent consultants.

Also read =>> What is Pact Contract Testing

How to Introduce API Testing in Your Organization

The process for introducing API testing in any organization is similar to the process used for
implementing or rolling out any other testing tool and framework.

The table below summarizes the main steps along with the expected outcome of each step.

Phase Step Expected Outcome

Tool Selection Gather requirements and identify constraints Understand the requirements for
researching market for appropriate API test tool.

E.g.

What kind of API is being tested – SOAP or REST?

Do we need to hire tester for this role or train existing tester?

What kind of tests will be performed – functional, performance tests etc.

What is the budget for implementation?

Evaluate available tools Compare available tools and shortlist 1 or 2 tools that best meet the
requirements.

Proof of Concept Implement a subset of tests with the shortlisted tool.

Present findings to stakeholders.

Finalize the tool to be implemented.

ImplementationGetting started Depending on your choice f tool, you would wither need to install the
required tool on a PC, Virtual machine or server.

If tool of choice is subscription based, create required team accounts.

Train the team if required.

Get going Create tests

Execute tests

Report defects

Common Challenges And Ways to Mitigate Them

Let us discuss some of the common challenges that QA teams face while trying to implement an API
testing framework in an organization.

#1) Choosing the Right Tool

Selecting the correct tool for the job is the most common challenge. There are several API test tools that
are available in the market.

It may seem very appealing to implement the latest, most expensive tool available in the market- but if it
does not bring the desired results, then that tool is of no use.

Hence, always choose the tool that addresses the ‘must-have’ requirements based on your
organizational needs.

Here is a sample tool evaluation matrix for the available API Tools
Tool Pricing Notes

Soap UI Free Version available for SoapUI Open Source (Functional testing) * REST, SOAP and other
popular API and IoT protocols.

 Included in Free version

SOAP and REST ad-hoc testing

Message Assertion

Drag & Drop Test Creation

Test Logs

Test Configuration

Test from Recordings

Unit Reporting.

 Complete list of features can be found in their website.

Postman Free Postman App available * Most-used for REST.

  Features can be found in their website.

Parasoft It’s a paid tool, requires purchasing a license and then requires installation before the
tool can be used. * Comprehensive API testing: functional, load, security testing, test data
management

vREST Based on Number of users * Automated REST API Testing.

 Record and replay.

 Removes dependency from frontend and backend using mock APIs.

 Powerful Response Validation.

 Works for test applications deployed on localhost/intranet/internet.

 JIRA Integration, Jenkins Integration Imports from Swagger, Postman.

HttpMaster Express Edition: Free to download * Helps in Website testing as well as API testing.

Professional version: Based on Number of users * Other features include an ability to define
global parameters, provides the user with an ability to create checks for data response validation by
using the large set of validation types that it supports.

Runscope Based on the number of users and plan types * For monitoring and testing API’s.

 Can be used for data validation to ensure correct data are returned.
  Contains feature of tracking and notifying in the case of any API transaction failure ( if your
application requires payment validation, then this tool can prove to be a good choice ).

LoadFocus Based on the number of users and the plan types * Can be used for API load
testing – allows running few tests to find out the number of users an API can support.

 Simple to use – allows running tests within the browser.

PingAPI Free for 1 project (1,000 request) * Beneficial for Automated API Testing and Monitoring.

#2) Missing Test Specifications

As testers, we need to know the expected results to effectively test an application. This is often a
challenge, as in order to know the expected results, we need to have clear precise requirements – which
is not the case.

For Example, consider the requirements provided below:

“The application should only accept a valid shipping date and all invalid requirements should be
rejected”

These requirements are missing key details and are very ambiguous – how are we defining a valid date?
What about the format? Are we returning any rejection message to the end-user, etc.?

Example of Clear Requirements:

1) The application should only accept a valid shipping date.

The shipping date is considered valid if it is

• Not in the past

• Greater or equal to today’s date

• Is in acceptable format: DD/MM/YYYY

2) <For all valid shipping dates >

Response Status code = 200

Message: OK

3) The shipping date that does not meet the above criteria should be considered invalid. If a
customer sends an invalid shipping date, then it must respond with the following error
message(s):

3.1 <If date format is incorrect>

Response Status code NOT 200

Error: The shipping date provided is invalid; please ensure that the date is in DD/MM/YYYY format

3.2 <if date is in past>

Response Status code NOT 200

Error: Provided shipping date is in the past

#3) Learning Curve

As mentioned previously, the approach for API testing is different when compared to the approach
followed while testing GUI based applications.

If you are hiring specialists either in-house or consultants for API testing, then the learning curve of the
API test approach or the API test tool may be minimal. Any learning curve, in this case, would be
associated with acquiring the product or application knowledge.

If an existing team member is assigned to learn API testing, then depending on the tool of choice, the
learning curve may be medium to high, along with changing the test approach. The learning curve for
the product or application itself may be low-medium depending on whether this tester has tested that
application before or not.

#4) Existing Skill Set

This ties in directly with the previous point about the learning curve.

If a tester was transitioning over from GUI based testing, then the tester would need to change the
testing approach and learn the new tool or framework as required. E.g. If the API accepts the requests in
JSON format, then the tester would need to learn what JSON is, in order to start creating the tests.

Case Study

Task

In order to scale up an existing application, a company wanted to offer a product in API as well as a
standard GUI application. QA Team was asked to provide a Test Coverage Plan to ensure that they are
ready to accommodate API testing beyond the regular GUI based tests.

Challenges
• None of the other software products had API based architecture, hence to accommodate testing
around this task, the team needs to establish the API test process from scratch. This means that the tools
were to be evaluated, shortlisted, finalized and the team had to be trained for the tests.

• There was no additional budget allocated for acquiring and implementing the tool. This means that the
team had to choose a free or open-source API testing tool and someone from the existing team had to
be trained to take this task.

• There were no requirements for API fields and data validation. Requirements were “should work the
same as the corresponding GUI application”.

The approach followed by the team to mitigate risks and work around the challenges

• QA team worked with the project team to identify the following requirements:

• API type (REST/SOAP ): REST

• Tests required (Functional, Load, Security): Functional testing only

• Automated Tests required (Yes/No): Optional for now

• Test reports (Yes/No): Required

• QA team did tool evaluation on the available API testing tools based on the must-have requirements.
Postman API Tool was finalized as a tool of their choice as it was free, and easy to use as well, thus
minimizing the learning curve, and had the potential to automate tests, and came with good inbuilt
reports.

• The same tester who tested the application was trained for using Postman to create initial tests
thereby eliminating any product knowledge gaps.
• To deal with the missing requirements, the project team built the high-level field-level documentation
using Swagger. This however left some gaps in terms of acceptable data formats and this was taken up
with the project team and the expected formats were agreed on and documented.

Conclusion

API based applications have gained popularity in recent times. These applications are more scalable
compared to the traditional applications/software and allow easier integration with the other APIs or
applications.

This API Testing tutorial explained all about API Testing, Shift Left Testing, Web Services, and Web API in
detail. We also explored the differences between Web Services vs Web API with examples.

In the second part of the tutorial, we discussed the full spectrum of API Testing, how to introduce API
Testing in your organization and some common challenges in this process along with solutions for them.

Check out our upcoming tutorial to know more about Web Services along with examples!!

NEXT Tutorial