SOA Testing

Ahamad Patan
8th Sept 2009
ahamad.patan@oracle.com
Agenda
 SOA Overview
 SOA Testing – An End to End Approach
Testing Web Service API’s
API Testing Strategies
Exercise
Testing Web Service Applications
 SOA Testing Challenges
 SOA Test Automation Approach
 QA Session
SOA Overview
What is SOA?
 Service-Oriented Architecture (SOA) is an architectural
style.

 Applications built using an SOA style deliver functionality as

services that can be used or reused when building
applications.
SOA Continued…
 SOA in other terms
 A service-oriented architecture is essentially a collection of
services
 Services are based on heterogeneous technologies.
 Application functions are modularized and presented as
services.
 A service is an implementation of a well-defined business
functionality.
 Service interface is independent of the implementation.
SOA Continued…
 Services are platform independent, self describing
interfaces (XML)
 Messages are formally defined
 Services can be provided on any platform
SOA and Webservices
 SOA is not Webservices, but web services are the
preferred standards-based way to realize SOA .

 Web services are software systems designed to

support interoperable machine-to-machine
interaction over a network. This interoperability
is gained through a set of XML-based open
standards, such as WSDL, SOAP, and UDDI.
These standards provide a common approach for
defining, publishing, and using web services.
SOA and Webservices
continued…
 Interoperability is the most important
principle of SOA. This can be realized
through the use of web services, as one of the
key benefits of web services is
interoperability, which allows different
distributed web services to run on a variety of
software platforms and hardware
architectures.
Webservices basics
Points to
UDDI
UDDI description
WSDL
Registry
Registry
Points to
service Describes
Finds Service
Service

SOAP Web
WebService
Service
Web
WebService
Service (J2EE, PL/SQL,
(J2EE, PL/SQL,
Client
Client .NET,C/C++,
.NET,C/C++,
(J2EE,
(J2EE,.NET,
.NET, Legacy
Legacy…)
…)
PL/SQL
PL/SQL…)…)
SOA Testing

 Testing Web Service API’s

 Testing Web Service Applications
Testing Web Service API’s

Following types of testing needs to be performed

Functionality
Performance
Re-usability
Security
Interoperability
Compliance
Backward compatibility
Testing web service API’s

Functional Testing
 API Testing
 Detect errors at an early stage, right from the
requirements level.
 Ensure that real-world test data is used
 Automate test case execution to improve productivity
Testing web service API’s

Performance Testing
 Web Services have high usage and high reuse
opportunities. Example: A service has been
identified as serving 10000 concurrent users with 5
sec of response time. Typically, services are server
side components that do not have a front end. It is
therefore useful to have a sample GUI interface that
can be used to load test the service from a web
client.
Testing web service API’s

Reusability Testing
 The more a service gets reused, the greater
the impact associated with its downtime will
be.
 Make sure that the APIs would work in all
possible business scenarios, thus ensuring
consumability
Testing web service API’s

Security Testing
 Web Services are exposed to the outside world, this
fact opens up a host of vulnerabilities, such as DOS
(denial-of-service) attacks, penetration, high
volumes of spam data, etc. Typical security policies
have to be enforced at the network level to create
bullet proof SOA. Specific test cases aimed at
targeting these policies need to be designed to fully
test SOA security.
Testing web service API’s

Security Testing
 A denial-of-service attack (DoS attack) is an
attempt to make a computer resource unavailable to
its intended users.
 Prevent an Internet site or service from functioning
efficiently, temporarily or indefinitely.
 Consumption of computational resources, such as
bandwidth, disk space, or processor time
 Crash the operating system itself.
Testing web service API’s

Security Testing
 Testing - Simulate denial-of-service attacks

 Popular denial-of-service mitigation products

from Arbor Networks
Testing web service API’s

Interoperability Testing
 The ability of two or more systems or components
to exchange information and to use the information
that has been exchanged.
 Involves different technology platforms and
development tools. Developers use many different
tools to generate the contracts (WSDL) and modify
them in the process.
Testing web service API’s

Interoperability Testing
 Testing – should be tested on different
technology platforms.
Testing web service API’s

Compliance Testing
 Testing to determine whether we meet some
specified standards
 Validating WSDL definitions

 Validating SOAP request/response messages

Testing web service API’s

Backward Compatibility Testing

As businesses evolve, changes may required in the Web Services, but

some consumers may want to continue the old services. The challenge is
to make sure that old service consumers are still supported.
SOA Testing – Challenges
SOA Testing Challenges
Major Challenges:
 No user interface for the Web Services
 Difficult to create Real time applications
 Test Data should be close to the end user environment
 Infinite consumers possible
 SOA Testing Challenges

 Test Automation
 Client Simulators (Message simulation)
 Service Simulators
 Automatic test data creation
 Choosing the right test automation tool
 Maintaining pool of testing resources with SOA domain knowledge
SOA Test Automation Approach
Traditional Testing Tools
GUI Testing Approach
In House Test Tools
Web Service Testing Tools
SOA Test Automation Approach

Traditional Testing Tools

 Web Services SOAP requests needs to be generated manually or using a
supported third party vendor testing tool
 Integrating traditional functionality testing tool and other such third party tool is not
feasible due to its nature and limitations
 A mix of manual and automated process is an additional overhead
 It’s a time consuming and expensive effort
 SOA Test Automation Approach
GUI Testing Approach

 Web Service is UI less application.

 GUI application should be developed

 Web service interface changes demands often GUI changes

 Web Services security testing that requires encrypted & singed data is not
possible

 Overall cost involved is very high for testing,development & maintenance

efforts
SOA Test Automation Approach

In House Test Tools

 Focus is drawn towards developing & maintaining the test tools than the actual
testing
 In-house test tool itself need to undergo testing to ensure it is coded to the
specifications
 Dependency on in-house tools will impact test schedules & quality of the deliverable
in demanding situations

 Time Consuming and expensive approach

SOA Test Automation Approach
Web Services Testing Tools
 Provides robust, reliable, scalable and interoperable solution
 Best fit for UI less testing
 Provides a solution for load testing
 Test data generation for any WSDL version is possible and easily
maintainable.
 Depending on existing legacy systems & back-end applications, web services test
tools can be tweaked by developing / modifying test scripts to provide an E2E
solution for backward compatibility
 Unlike any other traditional testing tool, the cost involved is very minimal.
Example

 A Web Service maps user input IP

addresses to countries
 String findCountry(String ip)
Example

Test Plan
Description Input Value Expected Result

Valid ip address 10.177.223.253 India

Valid Ipv6 address 3ffe:1900:4545:3: US

200:f8ff:fe21:67c
f

Invalid address 12.54.12.32.77.88 Error, Not a valid input

Example – Strategy 1

Suppose this API belongs to the class IPHelper.

IPHelper iph = new IPHelper();

String result1 = Iph.findCountry(“10.177.223.253”);
If (result1.equals(“India”))
System.out.println(“Test1 Passed”);
Else
System.out.println(“Test1 Failed”);
 Example – Strategy 2
Store the data in a xml file
<Testsuite>
<testcase1>
<input>10.177.223.253</input>
<expectedresult>India</expectedresult>
</testcase1>
<testcase2>
<input>3ffe:1900:4545:3:200:f8ff:fe21:67cf </input>
<expectedresult>US</expectedresult>
</testcase2>
<testcase3>
<input>12.54.12.32.77.88</input>
<expectedresult>Invalid Input</expectedresult>
</testcase3>
</Testsuite>
 Example – Strategy 2
Read the xml file as test data input.
Parse the xml file into an Object(xmlObject)

IPHelper iph = new IPHelper();

For (int I =0; I<xmlObject.length; I++){
String result1 = Iph.findCountry(xmlObject.input);
If (result1.equals(xmlObject.expectedresult))
System.out.println(“Test” + I + “Passed”);
Else
System.out.println(“Test” + I + “Failed”);
}
Exercise

 Web Service API

 Gives the current temparature of the city in
required units
 Float getTemparature(String City, Char Unit)
 Unit can be F(Farenheit) or C(Celsius)
 Write a test plan – Better come up with
unique test cases.
Description input1 input2 Expected
Result
Exercise

 Some Unique Test Cases

 Case Sensitivity – bangalore, f, c, F, C
Exercise

 Some Unique Test Cases

 Multiple names for a same city – Mumbai,
Bombay
Exercise

 Some Unique Test Cases

 City name contains space – New Delhi
Exercise

 Some Unique Test Cases

 Same city name in many countries –
Hyderabad(Ind), Hyderabad(Pakistan)
Exercise

 Some Unique Test Cases

 City Name contains Single Quote – Gizzard’s
Cove
Exercise

 Some Unique Test Cases

 City Name contains Same string repeated –
Zu Zu
Exercise

 Some Unique Test Cases

 The City is in North/South Pole – should
return negative value for the expected result
Exercise

 Some Unique Test Cases

 City Name is very Long string - Guangzhou in
southern China, Name of the city –
LLANFAIRWLLGWYNGLLGOGERYYCHWY
RNDROBWILLLLANTYSILIOGOGOGOCH
 In New Zealand, Name of the city -
Taumatawhakatangihangakouuauotamateatur
ipukakapikimaungahoronukupokaiwhenuakita
natahu
Testing web service Applications

 Functional Testing
 Services specifically designed to function as independent
entities may work perfectly well in isolation, but might not
function as expected when integrated into applications due to
underlying platform and network specific issues.
 Consider a sequence of web service calls made to various web
services – This should not affect the functionality.
 Since SOA involves a lot of asynchronous communication
across systems, it is possible that messages will not reach their
destinations in an orderly fashion because of delays in
processing at the service provider. Some systems will respond
quickly, and some slowly, depending on the load at any given
point in time.
Testing web service Applications

 Integration Testing
 Testing the composite application that is built using

sub-systems or modules that are independent of the

rest of the system.
Testing web service Applications

 Performance
 When a composite application is being tested for

load, stress, it is important to observe the throughput

of the web service to see if a bottleneck occurs.
Questions or Comments?

Questions
or
Comments?

Thanks