PERSPECTIVE

Need for a Comprehensive

Test Maturity Model

- Reghunath Balaraman, Harish Krishnankutty

Abstract
Constant change and ever growing complexity of business has necessitated that IT organizations
and specifically Test/QA organizations make thorough and periodic introspection of their
processes and delivery capabilities. This is necessary to ensure that at all possible times the Test/QA
organization, and its systems and processes, are relevant and available to support business needs.
While there are multiple maturity models in the marketplace to help this process, there are yet not
comprehensive enough and fail to provide today’s dynamic businesses the much needed flexibility
and power of customization. The need of the hour is a comprehensive Test/QA maturity assessment
model, which not only answers the requirements of customization and flexibility, but also
ensures relevance in today’s complex delivery structures of multi-vendor scenarios, multi-location
engagements, global delivery models, etc.
 The global
economic crisis and revolutionary technology trends organization’s ability to assure and control
have changed the role of IT organizations quality of its IT systems and processes
in supporting business growth. Though largely determines the success or failure
the recession created a scarcity of capital of the business in capturing, servicing and
for IT investments, the demands and expanding its client base. When Quality
expectations on the ability of IT to quickly and reliability play a very significant role
adapt and support business, has only in determining the current and future
increased multifold. In addition, rapid/ course of business outcomes delivered, it
revolutionary changes in technologies are is imperative that the Quality Assurance
forcing companies to recast their entire IT function itself is evaluated periodically
landscape. All these factors together have for the relevance, effectiveness and
created a complex environment where the efficiency of the processes, practices and
demand for change from the business is systems. An objective self-introspection
high, the capital for investment is scarce is the ideal first step. However, most often
and time-to-market is critical to success. than not, QA organizations fall short of
using this process to unearth gaps in their
The constant change and ever growing
current systems and practices. Also, many
complexity of the business environment,
organizations may have lost touch with the
and the risks associated, have necessitated
ever-evolving world of QA to be aware of
that organizations make thorough and
the leading practices and systems available
periodic introspection of their processes
today. This necessitates an independent
and delivery capabilities to ensure
assessment of the organization’s QA
operation in an efficient, effective and
practices to benchmark it against the
agile manner. A key requirement, amongst
practices prevalent in the industry and
all, is the ability to ensure that there are
to get that all-important question “where
adequate controls in place to ensure
do we stand in comparison with Industry
quality in the processes and the outcomes,
standards?” answered. Also, the assessment
no matter the extent of change being
of maturity in testing processes becomes
introduced in business or technology
critical in laying out the blue-print for a QA/
landscape of the organization.
testing transformation program that would
This is where the Test/QA organization’s establish the function as a fit-for-purpose
capability comes under the scanner. An one, and often world-leading.

External Document © 2017 Infosys Limited

1 Limitations of traditional approaches to
Test/QA Maturity Assessment
There are several models, proprietary delivery structures created through
and others, available for assessing the multi-vendor scenarios, multi-location
maturity of the IT processes and systems, engagements focusing on selective parts
including quality assurance. Most of of the software development lifecycle,
these are developed and promoted as etc. This significantly reduces the overall
models helping an organization to certify effectiveness of the output provided
capabilities in one or more areas of the by the existing maturity model and its
software development lifecycle. Like all applicability to the client situation.
other models and frameworks that lead to
certification, these maturity models too
Focused on comprehensive
have a fixed framework for an organization
to operate within, and provide very little certification rather than required
flexibility to address specific assessment capabilities
needs. Further, these models fail to help Most conventional models are “certification
organizations assess overall process focused” and can help organizations in
maturity due to the following limitations: assessing their IT process capabilities and
getting certified. They are exhaustive in

Inability to accommodate and the coverage of process areas and answer

the question, “how comprehensive are
account for heterogeneous
the processes and practices to service a
delivery structures
diverse sets of users of the QA services?”.
Over the last decade or so, most Such a certification is often a much
organizations have evolved into a needed qualification for IT service provider
heterogeneous composition of internal organizations to highlight their process
staff and service providers, delivering capability and maturity to diverse clients
services through global delivery models and prospects. However, most non-IT
with diverse talent, disparate processes, businesses maintain IT divisions to support
etc. All this has made assessing an their business and are more interested
organization’s process maturity increasingly in selectively developing the required
difficult. The existing maturity models in capabilities of their respective IT groups,
the marketplace are not flexible enough leading to efficient business processes and
to accommodate for these complex better business outcomes. Hence

External Document © 2017 Infosys Limited

 the focus of maturity assessments in approach to certification, such maturity
these organizations is not certification, models do not present organizations
but the ability to deliver specific business with a good view of where their current
outcomes. Since the traditional assessment capabilities stand with respect to what is
models are often certification-focused, needed by the organization.
most non-IT businesses find it an
overhead to go through an exhaustive
Lack of focus on QA
assessment process that does not help
them answer the question, “how effective The existing maturity models primarily
are my organization’s QA processes and focus on software development, and
practices to ensure quality of my business treat testing as a phase in the Software
outcomes? Development Lifecycle. However, today,
testing has evolved as a mature and
specialized discipline in the software
Staged Vs Continuous model for
industry and hence the ability of the
growth in maturity traditional models to assess the QA/testing
Majority of certification models follow processes and practices to the required
a staged approach, which means that level of detail is very limited. They fall
the organization has to satisfy all the short of organizations that have realized
requirements of a particular level and get the need/ importance for an independent
certified in the same, before becoming testing team and want to manage the
eligible for progress to the next level. QA maturity mapping process as an
But, most organizations are selective in independent entity. Hence, the various
their focus and want to develop those dimensions of the test organization should
areas that are relevant and necessary to be given adequate focus in the maturity
their business, rather than meeting all assessment approach covering the Process,
the requirements just to get certified at People and Technology aspects of testing.
a particular level. Because of the staged

External Document © 2017 Infosys Limited

2 A comprehensive model to assessing an organization’s
test capabilities and the ability to handle transformational
programs

Now that we have looked at the which depends on one or a set of service
shortcomings of the traditional models of providers for their key IT services may
QA assessment, it is time to answer that want strong governance and gating
all-important question, “what should a mechanisms. While, another organization
comprehensive model for assessing QA/ that does testing in-house, and leverages
Test maturity be like?”. The key attributes vendors for development, will have a much
of a comprehensive QA/Test assessment wider focus on maturity in processes and
framework/model can be summed up as practices. Basically, the model should
follows: be flexible enough to account for the
intent of assessment, as outlined by the
organization.
Provide business-comprehensible
decision-aiding results
Detailed and comprehensive view
The model should allow for selective
of areas of improvement and
assessment of the relevant parameters for
maturity, in the context of business. The strengths
results of the assessment should help the The model should also be one that helps
business identify and plot the possibility of determine the maturity of the testing
immaturity in their systems and processes, organization in a detailed manner. The
using lead indicators that have a negative methods and the systems of the model
impact on the business. These indicators should provide a robust mechanism of
should help the senior management objectively calculating the maturity level
to decide whether to go for a detailed of the testing organization, based on the
assessment of maturity, before any adverse behaviors exhibited by the organization.
effect on business is felt. It should provide the members of the
QA organization with a detailed view of
the areas of strength (and hence to be
Choice of business-relevant factors
retained) and the areas of improvement.
and focus areas
The model should enable the testing
The model should be flexible enough to organization to understand the measures
provide the right level of focus on the that should be implemented at the
various factors, business deems relevant, granular level, rather than at a high-level
that contribute to the overall maturity and thereby help the organization to focus
index. For example, an organization on their key QA dimensions,

External Document © 2017 Infosys Limited

 and strengthen the maturity of these effectiveness. It should provide a reference
dimensions. framework for selective improvement of
capabilities, keeping in mind the business
context and organizational objectives.
A frame of reference for
This will help the organization design a
improvement initiatives roadmap for improvement and devise ways
The comprehensive maturity model should to implement the same effectively.
provide the organization with a roadmap
to move its QA/Testing processes and
practices to a higher level of maturity and

3 Conclusion
So, in order to meet the needs of a this assessment, should move away from
dynamic business environment and certification-based models with “generic”
rapidly evolving technology space, IT and “hard-to-customize” stages, to a
organizations need to respond quickly model that is adaptable to the context in
and efficiently with high-quality, high- which business operates. It needs to be a
reliability and cost-effective processes model that evaluates factors that influence
and systems. This calls for a robust and maturity and quality of processes at a
scalable QA organization that can guard detailed level and helps the organization to
and ensure the quality of solutions that embed quality and maturity in processes,
are put into operation, and assess itself on governance and development of key
its capabilities and maturity, periodically competencies. This would help ensure
to ensure business-relevance and the maturity of operations and promote
effectiveness. continuous improvement and innovation
throughout the organization.
Hence a comprehensive QA maturity
model, which assists organizations in

External Document © 2017 Infosys Limited

About the Authors

Reghunath Balaraman
(Reghunath_Balaraman@infosys.com)

Reghunath Balaraman is a Principal Consultant, and has over 16 years of experience. A post graduate in Engineering and Management, he
has beenworking closely with several large organizations to assess the maturity of their test and QA organizations and to help them build
mature and scalable QA organizations. Raghunath is also well versed with several industry models for assessing maturity of software testing.

Harish Krishnankutty
(Harish_T@infosys.com)

Harish Krishnankutty is a Industry Principal in Infosys’ Independent Validation Solutions unit and has over 14 years of experience in the IT
industry. His area of specialization is QA consulting and program management and has extensive expertise in the design and implementation
of Testing Centers of Excellence/ Managed QA services. Currently, he focuses on the development of tools, IP and services in different areas of
specialized testing, including Test Automation, SOA Testing, Security Testing, User Experience Testing, Data Warehouse Testing and Test Data
Management.

External Document © 2017 Infosys Limited

For more information, contact askus@infosys.com

© 2017 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys
acknowledges the proprietary rights of other companies to the trademarks, product names, and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this
documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording, or otherwise, without the
prior permission of Infosys Limited and/or any named intellectual property rights holders under this document.

Infosys.com | NYSE: INFY Stay Connected