See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/236031163

The Theory of Software Testing

Article  in  Intelligent Transportation Systems Journal · June 2012

CITATIONS READS

4 14,604

1 author:

Adtha Lawanna
Assumption University of Thailand
27 PUBLICATIONS   31 CITATIONS   

SEE PROFILE

All content following this page was uploaded by Adtha Lawanna on 04 June 2014.

The user has requested enhancement of the downloaded file.

AU J.T. 16(1): 35-40 (Jul. 2012)

The Theory of Software Testing

Adtha Lawanna
Department of Information Technology, Faculty of Science and Technology
Assumption University, Bangkok, Thailand
E-mail: <adtha@scitech.au.edu>

Abstract
Software testing is the process of testing bugs in lines of code of a program that
can be performed by manual or automation testing. The theory of software testing
involves problem definitions of testing such as test team, failure after testing, manual
testing, uncertainty principle, participation, and incorrect test case selection. This
article shows the details of a critical part of software testing, which is how to test the
performance of new software and the entire system. The outcome of this article is the
whole picture of three phases for software testing as follows: preliminary testing,
testing and user acceptance testing.
Keywords: Automation testing, verification, validation, user acceptance testing.

to be discouraging (Pan 1999). In response to

1. Introduction this, verification can be used to test or check
Software testing in the Software items, including code, for consistency and
Development Life Cycle (SDLC) is the process conformance by evaluating the results against
of executing and evaluating a program with the pre-specified requirements. Debugging implies
intent of finding faults (Myers 1979). In that testing should intentionally aim to make
general, it focuses on any activity aimed at things go wrong to find if things occur when
evaluating an attribute or capability of a they should not or things do not occur when
program or system and finding that it meets its they should. The validation involves the system
specification requirements (Hetzel 1988). In correctness e.g., it is the process of checking
fact, a software system is not unlike other whether the requirement that should be
physical systems where inputs are received and specified is what the user really wants. This
outputs are produced (Pan 1999). However, means that the validation process checks
software does not suffer from physical changes. whether the invented system is what the
Generally, it will not change until customer wants and needs to be included in it,
modifications, upgrades, or user requirement while the verification process checks whether
changes take place (Pan 1999). Therefore, once that system is invented correctly (Fischer
the software is released, the design faults or 1977). Therefore, both verification and
bugs will be buried in and remain latent until validation are necessary as distinct elements of
activation (Pan 1999). All the possible values any testing activity. This article shows three
are required to be verified and tested, but phases of software testing in order to consider
perfect testing is impossible to be made possible requirements which can affect the
(Humphrey 1995). If a testing failure occurs abilities of the whole system.
during a preliminary step and the code is
released, the program may now work for a test 2. The Theory of Software Testing
case that it may not have been intended to work
for previously (Pan 1999). But its behavior on 2.1 Problems in Software Testing
pre-debugging test cases that it dealt with This article is concerned with five
before can no longer be trusted (Pan 1999). To problems found in software testing as follows.
respond to this possibility, the testing has to be The first problem is concerned with the
restarted. The expense of this step often tends limitations of the testing team. Insufficiency
may be the result of limited resources, lack of

Review Article 35
AU J.T. 16(1): 35-40 (Jul. 2012)

training of the individual team members, or 1995). There are several typical processes of
issues with the leadership. Also, suitable black-box testing. Equivalence Partitioning can
testing means may not be available to the team. remove the number of test cases and divides a
The second problem is failure of the test software unit into partitions of data from which
maintenance. This is because the specification test cases can be determined (Spillner et al.
changes of the requirements result in 2007). Boundary Value Analysis concerns the
abnormally long reverse times. The third testing at boundary values such as minimum,
problem is with manual testing. The software maximum, just inside and outside boundaries,
testing team is busy making manual testing error values and typical values. Cause-Effect
instead of building new test specifications, or Graph creates a relation between the causes and
modifying old ones to fit a new or changed effects (Nursimulu and Probert 1995). Fuzz
requirement. The forth problem is about the Testing determines implementation bugs and
uncertainty principle. Sometimes, the uses malformed data injection in an automated
uncertainty is in the exact testing conditions as session (Miller et al. 1990). There are also
well as how to view the condition for other processes of black-box testing.
replication. The fifth problem is in selecting the Regression Testing reruns some of the selected
right tests. The software testing cannot address test cases to ensure that the modified software
some of the essential aspects of the software system still has the functionality as required
testing application or system when testing only (Ball 1998). Pattern Testing is a new type of
some part of the required functions or choosing automated testing which can verify the good
only the expected interactions when executing application for its design or architecture and
a fault-tolerant application. patterns (Glaser and Strauss 1967). Orthogonal
Array Testing is a systematic, statistical way of
2.2 Processes in Software Testing software testing which can be applied in user
The processes of verification and interface testing, system testing, regression
validation are discussed below. Briefly, testing, configuration testing and performance
verification makes the product right, but testing. Matrix Testing states the status report
validation makes the right product. Many of the project. With Manual Testing, a tester
testers use white-box testing in these processes. defines manual test operations to test software
It deals with the investigation of internal logic without the aim of test
and structure of the code. Some important automation. Accordingly, this testing is a
processes of white-box testing are briefly laborious activity that uses a tester possessing a
described. Data Flow Testing is the process certain set of qualities e.g., to be smart, hard
that can define and use program variables working, observant, creative, speculative,
(Horgan and London 1991). Loop Testing innovative, open-minded, resourceful, and
exclusively concerns the validity of loop skillful. Automated Testing runs the program
constructs. Branch Testing tests true and false being tested, using the proper input and
values given to compound conditions evaluating the output against the expectation
appearing in different branch statements before testing. Automated Testing needs a test
(Jorgensen 2002). Control Flow Testing is a suite which is generated by test case generator,
structural testing that applies the program’s no human intervention is required. Examples of
control flow as a model and selects a set of test Automated Testing Tools are regression
paths through the program (Rapps and testing, unit testing, automated functional
Weyuker 1985). Basis Path Testing allows the testing and test management. Regression
test suite designer to build a logical complexity Testing refers to retesting the unchanged parts
measure of procedural design and then applies of the application. In the test suite, test cases
this measure for determining a basic set of could be re-executed in order to check whether
execution paths (Clarke et al. 1989). Besides new changes have not produced any new bugs
this, some testers use black-box testing instead and the previous functionality of an application
of white-box testing for the examination of the is still preserved. This test can be constructed
fundamental aspects of a system with little in a new build when there are significant
regard to its internal logical structure (Beizer
Review Article 36
AU J.T. 16(1): 35-40 (Jul. 2012)

changes in original functionalities or even a approach, individual modules of the programs

single debugging. This article focuses on three are not integrated until determination of
main techniques which are described as modules is made. This approach is not suitable
follows. Retest-All Testing is one of the mostly for inexperienced programmers who
techniques for regression testing in which all rely on running tests (Sage and Palmer 1990).
the tests in the existing functionality or test With Bottom-Up Integration Approach, each
suite could be re-executed. A drawback of this subclass is invented and tested separately and
technique is that it is very expensive as it needs after this the entire program is tested. A
huge resources and time (Smith and Robson subclass may consist of many modules through
1992). With Regression Test Selection, instead well-defined interfaces. The primary objective
of retesting the entire test suite, it is better to of this approach is that each subclass needs to
choose some test cases to be run. One of the test the interfaces among different modules
advantages of this technique is that reusable forming the subsystem. The test cases must be
test cases can be used in succeeding regression carefully selected to exercise the interfaces in
cycles. Prioritization of Test Case produces all possible manners. Top-Down Integration
scheduling over test cases in an order that Approach is an incremental integration testing
improves the ability of regression testing that begins by testing the top level module.
(Elbaum et al. 2000). On the other hand, for After this, it adds in lower level modules one
Manual Testing, testers use Unit Testing/ by one. It uses stubs to simulate lower level
Module Testing instead of Automated Testing. modules. A Stub is a special code arrangement
A unit is the smallest testable component in the that can stimulate the behavior of a well-
software system. Unit Testing is introduced to designed and existing module which is not yet
verify that the lowest independent function in constructed or developed (Cleve and Zeller
the software is working fine. The testable unit 2005). Mixed Integration Approach
is tested to determine whether it works (sandwiched approach or hybrid approach)
correctly when isolated from the remainding combines top-down and bottom-up testing
code (Rothermel and Kinneer 2004). In approaches. In this approach, drivers (calling
addition, Integration Testing can: provide the programs) and stubs (called programs) are used
intended functionality when modules are built by testers wherever the programs are
to interact with each other; and validate such incomplete.
modules. While individual classes can be
invented and tested correctly, bugs may occur 2.4 Efficiency Testing
due to their faulty interaction. Unit Testing Testers concern performance testing
concerns Integration Testing which involves which provides information about their
the consideration of: the states of multiple application regarding stability, speed, and
modules concerned at the same time in an scalability to stakeholders. The performance
interaction; and the state of a single object of a testing will find whether or not their software
module (Meyers 1979). The last strategy is meets stability, speed, and scalability
grey box testing, it combines the concepts of requirements. Next, they concern recovery
white-box and black-box testing to test an testing which is defined as how well the system
application having partial knowledge of the should recover from system crashes, hardware
internal structure and at the same time having failures, and other catastrophic problems. Many
knowledge of fundamental aspects of the computer-based systems must recover from
system (Barnett et al. 2003). bugs and resume operation within a pre-
specified time. A system can be fault tolerant,
2.3. Integration Test Approaches which means that processing faults must not
A mixture of the following approaches cause the overall system function to cease.
can be applied to develop the integration test However, a system failure needs to be
plan. Big-Bang Integration Approach is a type corrected within a specified period. The
of integration testing in which software program is verified under test stores of data
elements and hardware elements are combined files in the correct directories. Accordingly,
all at once rather than in stages. In this sufficient space is handled and unexpected
Review Article 37
AU J.T. 16(1): 35-40 (Jul. 2012)

terminations resulting from lack of storage are Review

Requirements
avoided. This is external storage as opposed to Specification
internal storage. After this, procedure testing is
used to provide detailed instructions for Prepare
executing one or more test cases. The last Test Plan

testing, User Acceptance Testing (UAT) is the

formal testing proposed to find whether a Prepare
software system meets its acceptance criteria. It Software Tool
helps the buyer to find whether to reject the
system or not. Moreover, acceptance testing is Prepare
proposed to find whether software is suitable Environment
for use or not. Beside this, it involves the
factors related to business environment. Alpha Prepare
Testing is conducted when any type of new Test Case
software or version of old software (called
alpha version) is released. In addition, alpha Prepare
versions are tested by some specific groups of Test Automation Tool
users, those who are chosen by the software
developer. They test and check whether all the Determine
features that are provided work properly or not. Acceptance Test Tool
Beta Testing is conducted after alpha testing.
Versions of the software refer to beta versions Fig.1. Preliminary testing.
that are released to limited groups of people. It
can ensure the product has a few bugs or faults. Review Requirements Specification: This
Sometimes, beta versions are produced to the step is important because many customers
public to increase the feedback area and to initially provide insufficient information (e.g.,
increase the number of future users. size and cost of the project). Therefore, the
Unfortunately, software testing is one of basic document that is needed is called a
the most difficult processes in SDLC. Testers requirements specification. It is a description of
can test software with different testing what the system should do.
techniques. Therefore, this article proposes Prepare Test Plan: This step provides a
software testing phases as an alternative option document describing the resources, schedule of
for testers. testing activities, scope, and testing approach
(e.g., items, features, tasks, environment, skills
of tester independence, design techniques
3. Design of Software Testing Phases criteria, and risks)
Prepare Software Tool: This step must be
This section is the result of studying the set up when the different modules in the
theory of software testing. Software testing can product, hardware, and operating system are
be divided into three main phases: preliminary known.
testing, testing and user acceptance testing. Prepare Test Environment: This step
identifies production environment elements
that must be tested and creates a test
3.1 Preliminary Testing Phase environment relevant to these elements (e.g.,
Preliminary testing phase is conducted business applications, administrative tools,
especially for testers to clarify the specification computer hardware, databases, services,
requirements of the customer. According to security system, applications, and network
this, software testing can fulfill the needs of systems).
both testers and customer. Fig. 1 describes the Prepare Test Case: This step provides a
preliminary testing phase as follows. document that explains an input and an
expected response of an application feature. A

Review Article 38
AU J.T. 16(1): 35-40 (Jul. 2012)

test case must contain particulars such as name Other techniques useful for validation are
of test case, input data requirements, test case prototyping and early release.
identifier, objective, test setup, steps and
expected output. 3.3 User Acceptance Testing (UAT) Phase
Prepare Test Automation Tool: This step It would be most important to complete
provides planning of a test technique on how to the UAT as shown in Fig. 3 to ensure that the
automate software testing. For instance, test system, which is to be implemented, is working
cases are executed for regression testing. correctly.
Determine Acceptance Test Tool: This Check
Integration
step can provide acceptance test tool for Testing
software testing to meet the requirements
specification. Check
Test Strategy
3.2. Testing Phase Document
The testing phase is a separate phase
which is conducted by a different test team Check
Integration Testing
after the implementation is completed. The Signoff
testing technique is selected based on the
perspective of the test team. In this article, the
testing phase is divided into three steps as Repair
shown in Fig. 2: independent verification,
independent validation and testing.
Coordinate
Verification Release

Validation
Fig. 3. User acceptance testing.

Check Integration Testing: When placing

Testing several units together, one has to conduct
integration testing to ensure that the unit
integration has not produced any errors.
Fig. 2. Testing phase. Check Test Strategy Document: This
Verification: The verification activities document is prepared at the planning stage
include technical reviews, software inspections, when the user requirements are determined.
walkthroughs, and efforts to: check the software The strategy document is communicated to the
requirements (e.g., they are traceable to the test team for testing the software release.
requirement specifications), check the design Check Integration Testing Signoff: This
components (they are traceable to the step covers the acceptance for the whole of the
software), check conduct requirements, perform system testing. An agreement for the defects is
unit testing, perform integration testing, reached. This is a form, signed by the project
perform system testing, check the acceptance manager, which indicates that the
testing, and audit. One can say that it is to documentation, system testing, and training
determine the right thing, which concerns the materials have satisfied all tests within
testing of the implementation of right process, acceptable margins.
e.g., to determine whether the software is Repair: This step is conducted to have
properly made. stable performance based on technical and
Validation: It checks whether the developed functional specifications.
software adheres to the user requirements. Coordinate Release: This step is the last
Testing: It is a useful technique for both step before the release of new software to the
verification and validation. Obviously, other market.
techniques useful for verification are: static The three phases are designed to describe
analysis, reviews, inspections and walkthroughs. all activities when dealing with software

Review Article 39
 AU J.T. 16(1): 35-40 (Jul. 2012)

testing. According to this, software testers can Fischer, K.F. 1977. A test case selection
take the benefit of the design to implement the method for the validation of software
ideas of software testing and address the maintenance modifications. Proc. IEEE Int.
critical problems described in Section 2. Computer Software and Application
Conference (COMPSAC), Chicago, IL,
4. Conclusion USA, 8-11 November 1977, pp. 421-6.
Glaser, B.G.; and Strauss A.L. 1967. The
In summary, software testing is not only Discovery of Grounded Theory: Strategies
the process of a team tester to: determine the for Qualitative Research. Aldine, Chicago,
bugs and report the bugs to software IL, USA.
developers; and fix the bugs to develop new Hetzel, W.C. 1988. The Complete Guide to
quality software. The critical problems of Software Testing. 2nd ed. QED Information
testing must also be considered, including the Sciences, Inc., Wellesley, MA, USA.
structure of source codes and the whole Horgan, J.R.; and London, S. 1991. Data flow
system. However, there is no one to guarantee coverage and the C language. Proc. 4th
the use of best methods in software testing. ACM Symp. on Testing, Analysis, and
Therefore, many researchers are still working Verification (TAV 4), Victoria, BC, Canada,
on it. In future research, the focus will be on 8-9 October 1991, pp. 87-97.
software maintenance. This is because software Humphrey, W. S. 1995. A Discipline for
testing may result in high costs before feeding Software Engineering. Addison Wesley,
into the maintenance system. New York, NY, USA.
Jorgensen, P.C. 2002. Software Testing: A
5. References Craftsman’s Approach. 2nd ed. CRC Press,
New York, NY, USA. Chapter 6.
Ball, T. 1998. On the limit of control flow Miller, B.P.; Fredriksen, L.; and Bryan, S.
analysis for regression test selection. Proc. 1990. An empirical study of the reliability of
ACM SIGSOFT Int. Symp. on Software UNIX utilities. Commun. ACM, 33(12): 32-
Testing and Analysis (ISSTA), Clearwater 44.
Beach, FL, USA, 2-5 March 1998, pp. 134-42. Myers, G.J. 1979. The Art of Software Testing.
Barnett, M.; Grieskamp, W.; Kerer, C.; John Wiley & Sons, New York, NY, USA.
Schulte, W.; Szyperski, C.; Tilmann, N.; and Nursimulu, K.; and Probert, R.L. 1995. Cause-
Watson, A. 2003. Serious specification for effect graphing analysis and validation of
composing components. Proc. 6th ICSE requirements. Proc. Conf. of the Centre for
Workshop on Component-based Software Advanced Studies on Collaborative
Engineering: Automated Reasoning and Research (CASCON), IBM Press, Toronto,
Prediction, Portland, OR, USA, 3-4 May Ontario, Canada, 7-9 November 1995. p. 46.
2003. 6 pages. Pan, J. 1999. Software testing. Student Report.
Clarke, L.A.; Podgurski, A.; Richardson, D.J.; Available:<http://www.ece.cmu.edu/~koop
and Zeil, S.J. 1989. A formal evaluation of man/des_s99/sw_testing>.
data flow path selection criteria. IEEE Rapps, S.; and Weyuker, E.J. 1985. Selecting
Trans. Software Eng. 15(11): 1,318-32. software test data using data flow
Cleve, H.; and Zeller, A. 2005. Locating causes information. IEEE Trans. Software Eng.
of program failures. Proc. 27th ACM Int. 11(4): 367-75.
Conf. Software Eng. (ICSE), St. Louis, MO, Sage, A.P.; and Palmer, J.D. 1990. Software
USA, 15-21 May 2005, pp. 342-51. Systems Engineering. John Wiley & Sons,
Elbaum, S.; Malishevsky, A.G.; and Rothermel, New York, NY, USA.
G. 2000. Prioritizing test cases for Spillner, A.; Linz T.; and Schaefer, H. 2007.
regression testing. Proc. ACM SIGSOFT Software Testing Foundations. A Study
Int. Symp. on Software Testing and Guide for the Certified Tester Exam.
Analysis (ISSTA), Portland, OR, USA, 22- Foundation Level, ISTQB compliant. Rocky
25 August 2000, pp. 102-12. Nook Inc., Santa Barbara, CA, USA.

Review Article 40

View publication stats