Unit Testing - JUNIT
ALIN ZAMFIROIU
[email protected]
Contents
White box testing & Black box testing
Test Driven Development
Unit Testing
Junit – Assertions
Riht-BICEP & CORRECT
FIRST
TestSuite
Double Test
White box testing & Black box testing
WhiteBox Testing
Has many names such as:
Clear Box Testing;
Open Box Testing;
Glass Box Testing;
Transparent Box Testing;
Code-Based Testing;
Structural Testing.
It is a method of testing by developers or by testers who know the internal
structure of the tested application.
WhiteBox Testing Advantages
Testing can be commenced at an earlier stage. It is not necessarily to wait
for the GUI to be available.
Testing is more thorough, with the possibility of covering most paths.
WhiteBox Testing Disadvantages
Since tests can be very complex, highly skilled resources are required, with
thorough knowledge of programming and implementation.
Test script maintenance can be a burden if the implementation changes
too frequently.
Since this method of testing it closely tied with the application being
testing, tools to cater to every kind of implementation/platform may not
be readily available.
BlackBox Testing
BlackBox or Behavioral Testing, is a method used to test the software
application by persons who don’t know the internal arhitecture of the
tested application.
The tester know only the input and the output of the application.
BlackBox Testing Advantages
The tests are made from the user’s point of view .
Tester do not need to know programming languages or the code structure
of the application.
Tests are done independent from the developers and have a objective
perspective.
Test cases are designed after the specifications are complete, because
are based on these specifications.
BlackBox Testing Disadvantages
The tests will have a small number of inputs;
Test cases will be difficult to be designed because the tester don’t have
the specification of the application;
Tests can be redundant with the another test made by developer.
Causes of software errors
Programming errors – 15%
Design errors – 30%
Errors of specification – 55%
Test Driven Development
Development based on tests.
It is exactly how people think about
how to do the methods.
What is Unit Testing?
A way of testing the code, by programmers still from the software product
development stage.
UNIT TEST – code sequence used to test a well-defined unit of the software application
code. Usually the unit is a method.
Unit testing is performed in a well-defined context in test specifications.
Reasons to use Unit Tests
Easy to write
Tests can be written ad-hoc when you need them
Although they are simple, they can define collections of tests – Test Suites
They can be played automatically every time that need it (write once, use many
times)
Reasons to use Unit Tests
There are many frameworks and tools that simplify the process of writing
and running
Reduce the lost time for debugging and for finding bugs
Reduce the number of bugs in the delivered or integrated code
Increases the rate of bugs identified during the code writing phase
Reasons to not use Unit Tests
Why do I have to test my code?
The code written by me is correct !!!
I do not have time for tests. I have to implement functionalities, not tests.
There is no writed in the specification that we need to do tests.
Frameworks used for Unit Testing
Framework Programming language / scripting
JUnit Java
PHPUnit PHP
PyUnit Python
CPPUnit C++
VBUnit Visual Basic
DUnit Delphi
cfcUnit ColdFusion
HTMLUnit Html și JavaScript
Jsunit JavaScript
dotUnit .NET
NUnit C#, ASP.NET
Ruby Ruby
XMLUnit XML
ASPUnit ASP
xUnit C#
JUnit
It is a framework that allows you to create and run tests for different
methods from the developed projects.
The most common framework for unit code testing in JAVA.
It is an adaptation from xUnit.
Junit - history
Kent Beck has developed in the 90's the first automated test tool, xUnit, for Smalltalk
Beck and Gamma (Gang of Four) developed JUnit during a flight from Zurich to
Washington DC.
Junit has become the standard tool for TDD-type development processes in Java
Junit is the standard component in multiple Java IDEs (Eclipse, BlueJ, Jbuilder, DrJava,
InteliJ)
Xunit tools have been developed for other languages as well (Perl, C++, Python, Visual
Basic, C#, …)
JUnit
JUnit works according to two design patterns: Composite and Command.
A TestCase class is a Command object and a TestSuite class is composed
of several TestCases or TestSuite instances.
Junit concepts
Fixture – set of objects used in the test
Test Case – a class that defines the fixture set to run multiple tests
Setup – a method / step of defining the set of objects (fixture) used for the test.
Teardown – a method / stage of destroying the fixture after completion of the tests
Test Suite – collection of test cases
Test Runner – a tool used to run the tests (test suites) and to display the results.
JUnit- Assertions
assertEquals(expected, actual) assertEquals(message, expected, actual)
assertEquals(expected, actual, delta) assertEquals(message, expected, actual, delta)
assertSame(expected, actual) assertSame(message, expected, actual)
assertNotSame(expected, actual) assertNotSame(message, expected, actual)
assertNull(object) assertNull(message, object)
assertNotNull(object) assertNotNull(message, object)
assertTrue(condition) assertTrue(message, condition)
assertFalse(condition) assertFalse(message, condition)
fail(message) fail(message)
Junit și NUnit
JUnit NUnit
assertEquals Assert.AreEqual
assertEquals Assert.AreNotEqual
assertSame Assert.AreSame
assertNotSame Assert.AreNotSame
assertNull Assert.IsNull
assertNotNull Assert.IsNotNull
assertTrue Assert.IsTrue
assertFalse Assert.IsFalse
JUnit3 și JUnit4 – differences
JUnit3 needs a version of JDK newer than JDK1.2 while JUnit4 needs a version newer than
JDK5;
in JUnit3 the test classes must be derived from the TestCase class, and in JUnit 4 it is not
necessary to inherit the TestCase class;
in JUnit3 the name of test methods is built on the testAAA format; so all test methods
contain in their name the test word; in JUnit4 methods name is not important, but the
methods that are run as tests have the @Test annotation;
in JUnit4 it is possible to use the adnotation @Ignore, if a test should be avoided; in JUnit3 that
test should be deleted or commented.
Test Case
Unit testing - Junit skeleton
Unit testing - Junit skeleton
Unit testing - Junit skeleton
Annotations used in JUnit4 for automated methods:
@BeforeClass for setUpBeforeClass() method;
@AfterClass for tearDownAfterClass() method;
@Before for setUp() method;
@After for tearDown() method.
Because in JUnit3 ,are no annotations name of the methods setup () respectively
teardown () are required.
Unit testing - Junit skeleton
JUnit4 JUnit5 - Jupiter
@BeforeClass @BeforeAll
@AfterClass @AfterAll
@Before @BeforeEach
@After @AfterEach
Unit testing - Assertions
Create class Utils in Eclipse.
Create unit tests for the methods of this class, using JUnit4.
Unit testing
Unit testing - Assertions
assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
Test the “sum” method. assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
Unit testing - Assertions - assertEquals
Unit testing - Assertions
assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
Test the “division” method.
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
Unit testing - Assertions - assertEquals
That method can fail. So we have to test that, also.
Unit testing - Assertions - fail
Unit testing - Assertions - fail
Because the test fail, we have to correct the method.
Now the test should pass.
Unit testing - Assertions
assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
assertNotSame(expected, actual)
Test the “isEven” method.
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(conditin)
fail(message)
Assertions – assertTrue / assertFalse
Unit testing - Assertions
assertEquals(expected, actual)
assertEquals(expected, actual, delta)
assertSame(expected, actual)
Test the “nEvenNumbers” method. assertNotSame(expected, actual)
assertNull(object)
assertNotNull(object)
assertTrue(condition)
assertFalse(condition)
fail(message)
Assertions – assertNull / assertNotNull
Assertions – assertNull / assertNotNull
This method also should be modified, because for n=0, we will have an
empty list, not a null value.
Assertions
These assertions are only a small part of the all assertions that exists, but are
the most used.
Right-BICEP principle
Right-BICEP
Now that we know how to do tests, all we have to do is to know what tests we have
to do.
People with a lot of experience know directly (from their own experience) what they
have to test.
People with less experience need to follow certain directions or certain principles.
The best known test principle is Right-BICEP.
Unit testing - Right-BICEP
RIGHT – are the results right?
B – are all the Boundary conditions correct?
I – check Inverse relationship;
C – Cross-Check results using other means;
E – force Error condition to happen;
P – are Performance characteristics verified?
Unit testing - Right
Every time when we test a method, the first thing that should be veryfied is
if that method delivers the right results.
That is why the first direction is to check the correctness of the results.
This verification is made in accordance with the specifications of the
developed project.
For the Utils.java class it is necessary to check :
if the sum is correct calculated.
if the division is correct determinate.
Unit testing - Right
Unit testing - Boundary
Problems usually appear at the edges, so we need to be careful to test
the methods for the boundaries of the ranges.
For each method, we have to determine the interval in which input
parameter values can be, as well as the range of the result provided by
the method.
Unit testing - Boundary
Once these limits have been determined, exact tests are performed for
those values.
Boundary tests do not require testing outside these values but checking
the correctness for these values - limit values.
There are usually lower limits and higher limits. Tests are done for both
situations.
Unit testing - Boundary
For the Utils.java class it is necessary to check
The division to 1 it will be the numerator value?.
The division of a number to that number itself, returns 1?
Unit testing - Boundary
Unit testing - Boundary
In order to identify the extreme limits more easily, we must use the
CORRECT principle :
C – Conformance;
O – Ordering;
R – Range;
R – References;
E – Existence;
C – Cardinality;
T – Time.
Unit testing - Inverse relationship
Certain methods can be tested by applying the inverse rule: starting from
the result it must reach the same input from which it initially started..
Not applicable for all methods. It usually applies to mathematical
methods.
Unit testing - Inverse relationship
Also for the databases it can be checked whether an insert was made by
the reverse operation: select.
For the Utils.java class to check
Check the sum in inverse way – by difference;
Unit testing - Inverse relationship
Unit testing - Cross-Check
For each method, try testing it by using another method.
Usually there are several ways to solve a problem. Thus, another method
for solving the same problems for testing the newly implemented method
can be used
Unit testing - Cross-Check
This situation is possible when the implemented method was designed to
increase productivity, or if the old method was consuming resources or
time.
Testing the new method is done through the old method, even if it
consumes more resources.
Unit testing - Cross-Check
For the Utils.java class to check:
The division method can be verified by / operator.
Unit testing - Cross-Check
Unit testing - Error conditions
Probably the ugliest scenario for an application is to crash. That's why
when we test each method unitally, we also need to test the situations in
which the application might crash.
If we have studied the extreme boundaries for input value ranges or
resulting values, testing for error provisioning should use values outside
these ranges.
Unit testing - Error conditions
Testing for forcing errors is done on all methods. All methods have at least
one situation where they will provide error. Testing is done for these
situations and it is verified that the method treats that case and throws or
delivers an exception.
Unit testing - Error conditions
For the Utils.java class to check:
What happening if we try the division method with denominator=0?
What is happening if the parameter of “nEvenNumbers” method is -2?
Unit testing - Error conditions
Unit testing - Error conditions
Unit testing - Performance
For different methods, it is possible to test how well the method is
performing.
Besides testing the correctness of the results of the methods, it is very
important to check and perform the results with which they are obtained.
Unit testing - Performance
Performance verification is done both in terms of consumed resources and
as time required to achieve the results.
Performance testing is performed when the input or result of a method is
represented by a list or a very large number of elements, and these values
can increase a lot.
Unit testing - Performance
For JUnit4 to test the time it is used the next adnotation: @Test(timeout=100)
For the Utils.java class to check:
It is enough 1 second to obtain the first 1.000.000 even numbers?
but for 1.000.000.000 of numbers?
Unit testing - Performance
What else to test?
CORRECT principle
CORRECT
C – Conformance;
O – Ordering;
R – Range;
R – Reference;
E – Existence;
C – Cardinality;
T – Time.
CORRECT
Each subprinciple has a question that should be in the mind of the tester.
This principle is used also to helps the testers for the boundery conditions
Conformance
It is also known as
Type testing
Compliance testing
Conformity assessment
Conformance
It is applied a lot of domains where something should meet some specific
standards.
Usually for any input and for any output it must be verified that it conforms
to a format or to a standard.
Conformance
Tests can be done to check what happens if the input data does not
conform to the format or to see if the result obtained conforms to the
project specification format.
Conformance
For the Person class we have to test if the CNP atribute has 13 characters.
Conformance
Ordering
Order tests are specific to lists but not only.
In the case of lists, check if the order of the items is the one that you want.
You can also test the behavior of the method if it receives some
parameters in another order or a list of items in a different order than
expected.
Ordering
We can check if the result of the nEvenNumbers method is a list in the
correct order, if the second one is bigger than the first one; the third one is
bigger than the second and so on.
Ordering
Range
For both input and output values, certain intervals are set. These intervals
need to be checked.
Several intervals are set for certain methods. This will test for all these
intervals.
Range
All functions that have an index must be tested for range because that
index has a well-established range.
Usually it is necessary to check:
The start and end value for the index are the same;
The first element is larger or smaller than the last element;
What happens if the index is negative;
What happens if the index is higher than the upper limit;
The number of items is not the same as the one you want;
Etc.
Range
For the nEvenNumbers () method, it can be checked if length of the returned
list is required number.
Range
Reference
Certain methods depend on external things or external objects to these
methods. These elements need to be verified and controlled.
For example:
A web application requires the user to be logged in;
A pop stack works if there are elements in the stack;
Etc.
Reference
These elements are also called preconditions.
Preconditions for that the method to work normally.
These tests are performed using test doubles (stub, fake, dummy, mock).
Existence
“does some given thing exist?”
We need to ask what happens to the method if a parameter does not
exist, if it is null or if it is 0.
Also for software systems that work with files or with internet connection, it
is necessary to check the existence of these files or the availability of
internet connection. Otherwise, the application does not need to crack
but must behave normally.
Existence
Make sure your method can stand up to
nothing.
It is similar to the error conditions from
Right-BICEP.
Cardinality
0-1-n Rule
It is simlar to the Existence tests and Range tests.
We have to verify if the method has 0 elements, 1 element or n elements.
If it works for 2, 3 or 4 elements is considered that it will work for more
elements.
Cardinality
For the nEvenNumbers() method we have to check the situations:
0 even numbers;
1 even number;
n even numbers, where n in {2,3,4….}.
Time
It is similar to the Right-BICEP performance test.
It can also be tested if the call template is respected. Similar to the
Template design pattern.
For example, to call the logout method(), you must first call the login
method() and other examples.
F.I.R.S.T.
F.I.R.S.T
“ For unit tests to be useful and effective for your programming team you
need to remember to make them FIRST.”
F.I.R.S.T
Fast
Isolated/Independent
Repeatable
Self-Validating
Timely
Fast
The developed test should be fast because if we have too many tests we
don’t have to wait to much time when we run them.
Isolated
Single responsibility (SOLID)
“Each unit test should have a single reason to fail.”
Isolated
When a test fail, the developer should not debug to identify what is wrong
and where.
The test should be isolated and to say exactly where is the problem and
what problem.
Repeatable
The obtained results should be the same indifferent of the number of test runs.
Tests should run repeatedly without any other interventions.
Self-Validating
The confidence in the implemented tests.
If the tests pass, the developer should have high confidence that the code
is correct without errors.
If a test fail the developer must have high confidence that the code
should be improved.
Timely
When we have to implement the tests for our method?
When do we consider that we have done all tests?
TestSuite
Unit testing – TestSuite
Why do we need suites of tests?
In this moment we have a lot of testCases and we have to run only few
test from these testCases.
With suites we can group some tests from different testCases and to run
only these tests.
Unit testing – TestSuite
Custom TestSuite – JUnit 4
The CustomSuite class or interface is being implemented.
For each test that you want to be part of that suite, add the annotation :
@Category(CustomSuite.class)
Custom TestSuite – JUnit 4
To create a custom suite, all the desired categories are included.
A single category is included in this case. Tests in this category are found in
two TestCases UtilsTest and PersonTest :
@RunWith(Categories.class)
@IncludeCategory(CustomSuite.class)
@SuiteClasses({ UtilsTest.class, PersonTest.class })
public class NewSuiteTests {
}
Custom TestSuite – JUnit 4
We can also exclude some categories from our suite.
In this way we can create custom suites, based only on these categories.
Double test
Double test
Every time when we test a method, we try to test the correctness of this
method.
There is a problem for situations where our method uses external objects,
external links, or external method calls.
These objects, links, or methods can influence the results obtained using
the method that we want to test.
Double test
This is the way to test the method, but with a control over the external
references.
REFERENCE (CORRECT)
For this, test doubles are used.
Double test
In automated testing, it is common to use objects that look and behave
like their production equivalents, but they are actually simplified. This
reduces complexity, allows verification of code independent of the rest of
the system, and sometimes it is even necessary to perform self-validation
tests. A generic term used for these objects is a double test.
A double test is simply another object that fits the necessary collaborator
interface and can be replaced in its place. There are several types of test
doubles.
Dummy object
Dummy object – an object that respects the interface but the methods do nothing or
return 0 or null.
When we have to use the real object, actually we use a dummy object.
These doubles are used when we don’t have to call the methods from that object.
Because they are doing nothing.
Dummy object
Stub
Stub – in contradistinction to Dummies, methods from a Stub will return
conserved / hardcoded responses.
In this way we can use these objects with real calls.
Stub
https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
Stub
Fake
Fake – is an object that behaves like a real one but has a simplified version.
Usually for a fake we can set what the value should it return. It will not be a
hardcoded value.
Fake
https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
Spy
Spy– is a Stub or Fake that manages and counts the number of calls.
Mock
Mock – different from all the others, but works similar.
https://blog.pragmatists.com/test-doubles-fakes-mocks-and-stubs-1a7491dfa3da
Mock testing
Mock testing – It is used when we want the tested method not to be influenced by
external references.
Mock object – an object that simulates the behavior of a real object, but in a controlled
manner.
Frameworks used like Mockito, EasyMock, etc.
Mockito
Download the jar and add to the current project.
If you do not find it on google:
https://mvnrepository.com/artifact/org.mockito/mockito-all/1.10.19
Mockito
A mock object is created based on a real class :
Set behavior for desired methods:
Mockito
Methods:
doReturn()
doAnswer()
when()
thenReturn()
thenAnswer()
thenThrow()
doThrow()
doCallRealMethod()
Mockito
Test data files
There will be a single TestCase with the assert method called in a loop.
For every set of test data in the file, call the assert method.
What you've improved on this code?
Test data files
The file, or stream is considered a fixture and therefore must be opened in
setUp and closed in tearDown.
Integration tests and Unit tests
Integration tests and Unit tests
References
https://github.com/ghsukumar/SFDC_Best_Prac
tices/wiki/F.I.R.S.T-Principles-of-Unit-Testing
https://pragprog.com/magazines/2012-01/unit-
tests-are-first
http://agileinaflash.blogspot.com/2009/02/first.
html
