Unity: A Lightweight C Test Harness For Embedded Systems: Niall Cooling Feabhas Limited
Unity: A Lightweight C Test Harness For Embedded Systems: Niall Cooling Feabhas Limited
One of the key benefits on the Agile movement is moving the test activity from test-after-construction (TAC) to
test-before-construction (TBC). However almost all current test frameworks are based around either Java or
C++. This paper introduces Unity, an open source lightweight test harness that can be used for in-target testing
of an embedded C application. This paper explains how Unity works, how to integrate it into your environment
and how to use it.
Niall Cooling
Feabhas Limited
5 Lowesden Works
Lambourn Woodlands
skype: feabhas
twitter: @feabhas
blog.feabhas.com
linkedin: niallcooling
Unity: a lightweight c test harness for embedded systems
1 INTRODUCTION
The last decade of software development has seen the growth of Agile software development. Acceptance of
Agile software development has widened within application development and is now becoming accepted in
Embedded Development. However there still can be confusion as to what ‘Agile’ actually entails.
aThere are ongoing debates about the merits and shortcomings of Agile processes, but one of the major
benefits has been around the concept of Test-Driven Design (TDD) (1). TDD advocates writing test code before
application code, thus bringing all the concepts of structured testing to the average programmer (rather than
1
being seen as a separate activity) ; an alternative and less political term for this is “Test-Before-Construction”
(TBC).
1.2 TOOLS
1.2.1 XUNIT
Programmers being programmers have developed tools to automate and simplify the process of TBC. A major
part of the TBC process is the use of a test harness/framework for the running and reporting of tests.
Testing tools have existed for many years but have typically been based on the more traditional model of
“Test-After-Construction” (TAC). What makes the TBC approach different is that it is a programmatical model
developed by the programmer (i.e. they actually write the software in the same programming language they’re
testing in) as opposed to a separate toolchain and possibly a separate programming paradigm (e.g. TSL (2)).
There are now many code-driven test frameworks for all manner of languages, which collectively fall under the
umbrella term of “XUnit” (3). Their origins begin with Smalltalk but their popularity grew with the
development of the JUnit (4) framework for testing Java. Most modern languages (Python, Ruby, etc.) now
have their own test framework closely based around the ideas and concepts from JUnit.
With the popularity of Agile development and XUnit test frameworks, programmers have looked to use this
approach with older languages, most notably C++. There now exist over 20 different C++ testing frameworks
based around the XUnit model, including CppUnit (5), Google’s own C++ Testing Framework (6) and CppUTest
(7) (specifically targeted at embedded systems testing).
1.2.2 ISSUES
1
Again, there are debates around the reality of TDD in non-trivial, non-software-only systems, but we can save
that for a later date
However, as C is still the dominant programming language for embedded systems (8) our interest is in using a
framework to test embedded C applications. The majority of leading frameworks prove problematic when
looking at testing in the context of an embedded application. Of course it is possible to test a C program using
a C++ test framework, but this isn’t always suitable due to the change of programming paradigm or due to a
lack of compiler support.
Another problem for the embedded C programmer is that many, if not most, frameworks have been
developed by the open source community. This is not a criticism of open source, but it means these tools are
heavily dependent on the Linux operating system and the GCC toolchain. A knock-on issue is that many
frameworks are library based, which means running them in a non-GCC environment (especially with a cross-
complier) can be a frustrating, and sometimes futile, porting activity.
1.2.3 GOALS
So rather than re-inventing the wheel we were looking for a test framework with some clear goals:
o C - not C++
o Simple - ideally not library based
o Embeddable – does not require stdout, but supports redirection of output over serial port
o Small footprint – systems with small amounts (kB) of SRAM
o Not compiler-specific - works with commercial cross compilers
o Works on any OS – especially Windows
o Single toolchian – do not have to work outside the regular IDE
2 UNITY
Looking around, there are a number of commercial and open source options for C-based test frameworks, with
CUnit (9) being the best known of these. However, CUnit is a classic “Linux-centric” toolset that requires the
building of a library. There are ports for Visual Studio, but no obvious support for embedded compilers.
I cannot say I have extensively looked at all the C-based test frameworks (this is not meant to be a comparison
of test frameworks), but upon discovering Unity (10) it appeared to tick all the boxes, so we started to use it in
anger.
2.2 UNITY
2.2.1 ATTRACTIONS
First and foremost, the immediate attraction of Unity is its simplicity. Embedded programmers are busy people
with constant deadlines, so I believe a crucial point for acceptance of new ideas or concepts is how easyily it
fits with the way people develop code today. Being code-based testing it means C programmers are writing
their tests in C; a point that shouldn’t be underestimated. Admittedly, an OO/C++ framework can offer a richer
feature set, but a major goal is for the programmer to embrace testing. Writing the tests in the same language
and tool environment that the development is developed in eliminates the barrier of having to ‘context switch’
to another tool and way of thinking in order to test their code.
Secondly, the code of Unity is in simple source code form, consisting of two header files and one C file. To use
Unity involves including one of the headers and setting up the compilation/build paths to link to the Unity files
(more on this later). The code base is also compiler independent, meaning no porting activity is required to
work with any standard C compiler. Note, however, much of the automation around Unity is based on Ruby
scripts. Ruby isn’t required, but as you shall see makes Unity a really powerful test framework.
Finally, and probably its greatest attraction, is that Unity was developed by embedded C programmers (11).
Before getting in to the details of Unity, is should be noted that the XUnit frameworks and their derivates are
all focused on unit testing (12). The focus of the testing is on the functionality of the application rather than
performance, stress, etc. This means invoking C function(s) via their interface (declaration) and looking for
conditions to be true or false up on executing the function(s).
o Setup
o Test – run the test
o Analyze – report on the test outcome
o Teardown
The Setup and Teardown phases are responsible for putting the test environment in to a well-known state
prior to running one or more tests so that test results are repeatable – known as the test context, or more
commonly in the Agile world, the test fixture. Depending on the framework, tests may share the same fixture,
or each test may have its own fixture. In Unity all tests share the same fixture setup and teardown.
Finally we need a Test runner, a part of the framework that executes the tests.
2.2.3 REQUIREMENTS
To explain setting up and using Unity 2.0 the examples are based on using IAR EW (13) under Windows 7.
Download the Unity zip file from Sourceforge and unzip it to a known location. I recommend, and will base my
examples on C:\unity, but the files can reside anywhere. I have found that when setting up path names and
upgrading unity the simple path makes life easier.
Once unzipped, our initial files of interest can be found under C:\unity\src. Here we will find:
o unity.c
o unity.h
o unity_internals.h
You will also find testHarness.c, but ignore that for now. We are now ready to try out Unity.
Create a new C project (e.g. unity1)) and import the Unity C file into the project.
If we build the project at this point the compilation will fail as the project does not know the path for the unity
header file. This involves setting the C preprocessor include path so it can see the unity header files.
[IAR] To add the correct path, open the project's options (ALT+F7), select "C/C++ Compiler" and select the tab
"Preprocessor". Add the unity path to the "Additional include directories:" text box (e.g. C:\unity\src
C: )
Next we need to build a simple test. Create a new C file (e.g. myTests.c)) and add this to the project. Enter
the following simple test code to the test file. This is using the simple Unity test to compare two integers. All
Unity test functions must be of the form:
Note we need to create (empty) setup and teardown functions as the unity framework expects these functions
function
(without them it will fail to link). Save the code and return to main.c.
We now need to build the simple test runner part of the Unity test harness in main.c . Unity uses
setjmp/longjmp to manage the tests, so we need to include setjmp.h from the C standard library.
libr By default
the reporting mechanism of Unity uses the function putchar from stdio.h (this can be redirected). Note that
this requirement for setjmp/longjmp means that on small 88-bit processors
ors with a hardware stack, Unity is
unlikely to work (check your compiler documentation).
We declare a function
nction prototype for our test function from our test file (i.e. testWillAlwaysPass from
myTest.c). ). We could define a header for myTest.c but here it is just as simple to using an extern declaration
(I know the extern isn't needed but it is good practice if the prototype is from a function in another file to
include it).
We are now in a position to run the Unity test. The output from the Unity test will now be displayed in the
Terminal I/O window.
INSTALLING RUBY
Installing Ruby for Windows is very straightforward thanks to the Ruby Installer executable that sets up the
PATH environment as well.
Once installed, open a command window and enter gem install rake at command line. Rake is Ruby’s
make and a requirement of Unity.
Rather then manually editing the main (test runner) file each time we add a new test, Unity uses a Ruby script
to parse the test code and automatically generate the test runner main file. To automatically generate a test
runner.c file, open a command window and navigate to the project containing the unity test file (e.g.
myTest.c from previous example).
If this runs successfully it will create a new file of the form: <test_file_name>_Runner.c
for example, from myTest.c we get myTest_Runner.c
We now need to include this in our project. If we build now we will get a link time error (duplicate definitions
for "main") as both main.c and myTest_Runner.c both define a main function. We need to
remove/exclude main.c from our build. The project should now rebuild without and errors; run the code as
before.
The auto-generated Runner file should not be edited (as it will be overwritten when new tests are generated
and this file needs regenerating). If you look at the file there shouldn’t be anything of surprise in there. The
RUN_TEST is configured as a macro, and there are some additional items set up for reporting purposes; but all
in all it is very similar to the main.c we developed earlier.
One difference is the second parameter to the RUN_TEST macro. In our previous code we used __LINE__ just
as filler. However, the actual number supplied is the line number of the test function in myTest.c (e.g.
testWillAlwaysPass is at line 11 in myTest.c).
Ruby c:\unity\auto\generate_test_runner.rb
generate_test_runner.rb myTest.c
Revisiting the test runner file, myTest_Runner.c the automated changes are the inclusion of the extern
function prototype testBinaryMatchPass
Depending on your environment, one further improvement can be made. Remembering to switch to the
command line and auto-generate the revised test runner file each time ccan
an be tedious and error-prone.
error Many
environments have a “pre-build” command line option. For example
example, in IAR EW there is the option for pre-pre
build actions. Here we can run the R
Ruby script (though please note I have had some problems regarding
confliction between IAR and Windows UAC whic
which causes the pre-build
build to fail; I have yet to get a satisfactory
workaround).
Overall, un-optimised Unity adds about 1kB of code and about 90 bytes of data plus the standard library
requirements. In terms of running Unity in a target system, it is pretty straightforward.
Many modern debug environments using JTAG connections support automatic redirection of standard I/O
back to the debug IDE (often referred to as semi-hosting).
In the target environment either semimi-hosting may not be available or we want to eliminate, for size reasons,
reason
stdio.h. Unity allows redirection of the test summary messages (typically to a serial port)) by implementing a
function and mapping the macro UNITY_OUTPUT_CHAR to it. This is managed in unity_internals.h
unity_
(which is included in unity.h) so define the macro before including the unity.h header.
Unity supplies a wide range of macros for basic unit testing. Most are quite straightforward,
straightforward but still eliminate
reinventing the wheel. The list can be found in unity.h, and include examples such as
An example for a simple Unity target test is shown, where a seven segment display is being driven off GPIO
lines 16 through 19 on Port 1 of an NXP LPC2129 ARM7 system. The test code checks that the appropriate bits
in the Direction register are set high after the device has been initialsed.
3 BEYOND UNITY
Until now we considered modules in isolation; and the units under test only have a single interface which the
test harness can exploit. In practice, modules typically communicate with other modules. The public interface
of a module iss therefore the combination of:
o The Provided Interface. This is the services the module provides to its clients
o The Required Interface. This is the set of calls the module will make to its peers or subordinates in
order to fulfil its function.
One of the problems with unit testing is that it can be difficult to isolate the piece of code required for testing
(referred to as the Unit Under Test or the UUT).
For testing purposes subordinate or peer code can be replaced by stub code. The stub has the same interface
in
as the actual system code but provides a simulation of its behaviour. Typically, a stub produces a fixed
response or a narrow range of responses; as long as the responses are adequate to test the behaviour of the
unit under test. The use of stubs to support higher
higher-level
level code leads to them sometimes being referred to as
scaffold code. Stubs can be replaced by working code as development continues.
What happens if the unit-under-test test is a control object (doesn’t return values)? In this
his example the Closed
Loop Control element provides a closed
closed-loop
loop control mechanism (for example, for a motor position controller).
The Closed Loop Control unit requires the services of a Motor unit (for drive) a Sensor unit (for feedback) and a
Filter unit
nit (for calculating drive signals). The four units form a hierarchical composite.
We could test Closed Loop Control with stubs for the Sensor, Motor and Filter, but what tests could we run?
Closed Loop Control is an example of where data
data-oriented
oriented testing can be of limited value.
sd Closed-Loop control [ go to
pos ]
loop GetPosition
[current != newPos]
currentPos
Calc_Drive(error)
driveSignal
Drive (driveSignal)
That is, the Closed Loop Control object is functioning correctly if it makes particular function calls, in a
particular order, and with particular values. If any of these is incorrect, then the behaviour of the Closed Loop
Control object must be incorrect. This form of testing is referred to as Behavioural Testing.
In this case we cannot substitute traditional stub objects for the Sensor, Motor and Filter objects.
For a control-like object we must verify its behaviour from the point of view of its subordinates. That is, did
the unit under test make the correct calls to its subordinates, with the right data, at the right time?
Mock objects are an extension of traditional function stubs. A Mock object ‘stands in’ for a real module. The
Mock object has the same interface as the actual system code; and can return representative values.
However, its interface is extended with additional functions for testing purposes.
When performing behavioural testing using Mock objects, it is the Mock objects that confirm the behaviour of
the unit under test, not the test context code. The Test Context is responsible for configuring the Mock objects
and invoking the behaviour of the unit under test. The Mock objects tell the Test Context whether the unit
under test is working as expected; the Test Context merely collates the results.
A mock object can be set expectations. An expectation is a function call the mock object is expected to receive.
Expectations are set in sequential order. Expected calls can also be given expected parameters; and can even
be given a particular response to return to the caller. If a mock object has all its expectations fulfilled it passes
the test; if not all expectations have been fulfilled the unit under test must have failed.
3.3 CMOCK
Unity has mocking support through CMock (14) - a collection of files, including Ruby scripts to generate the
mocks. CMock is a nice little extension to Unity which takes your header files and creates a Mock interface for
it so that you can easily unit test modules that rely on other modules.
First download CMock zip file from SourceForge. Unzip this to a well known location, e.g. c:\cmock. In the
well-known
docs folder you will find a brief introduction to CMock and configuration details. The lib directory contains
contains cmock.c and cmock.h..
all of the interesting Ruby scripts and the src directory contain
As with Unity, the cmock.c needs adding to the project and the project path needs to be setup to see
cmock.h.
To test the Go_To function, we need to create mocks for get_position, calc_drive and drive functions. These
are contained in the headers sensor.h
sensor.h, filter.h and motor.h respectively.
CREATING MOCKS
Use the supplied Ruby scripts to create the mock files for the required headers by typing: cmock.rb
<header files>
e.g.
c:\cmock\lib\cmock.rb
cmock.rb sensor.h filter.h motor.h
In the mocks subdirectory you will find a Mock<header>.c and Mock<header>.h file for each supplied
header
USING MOCKS
Create a group in the project and add the (generated) mock .c files to the project. Modify the preprocessor
settings so the build knows where the mocks are ($PROJ_DIR$\mocks) and also add the project directory
($PROJ_DIR$) so the mock files can an find the original headers (e.g. sensor.h)
To use the mocks we have to include the key header files required for the test in our test harness file
(controller_test_harness.c)) e.g.
If we build and run our test with the mocks, the output is:
The test reported that the function “get_position”, declared in sensor.h has been called (as expected), but
for this test to pass we have to ‘program’ the mock framework to say that we expect this function (and
calc_drive() and drive())) to be called during this test.
For each function declared in sensor.h a mock (stub) function is created (e.g. the stub of float
get_position(void) is automatically created). In addition, the mock framework creates another function
that allows us to inform it that we expect this function to be called by the test. The mock-expectation
mock function
created, takes the form
void get_position_ExpectAndReturn(
get_position_ExpectAndReturn(float cmock_retval)
The ‘cmock_retval’ is an argument we supply, that the stub of get_position will actually return. So here is an
example of testing
ing ‘Go_To’ using the CMock fram
framework:
Given the earlier code, passing the Go_To function and argument of 1.0, and the said expectation, the test will
now pass.
Note that if we state we expect a function to be called and it is not, or it is called more times than expected,
both these cases will result in test failure. For example, if we wanted to iterate around the loop a couple of
times checking the algorithmic behaviour, we could write our test code thus:
As you can imagine, a test could be set up using real test vectors stored in a file (if available).
Mocks can do a lot more than covered here (e.g. calling user
user-defined code as part of the stub call – a concept
known as a wrapper).). One word of warning: mock frameworks (including CMock) k) are dependent on
dynamically-allocated
allocated memory, so may be not be useable in smaller embedded systems. That said, mocks are
more useful in a host environment to simulate (and stub) the behaviour of low-level
level software, or even
hardware devices themselves. In such cases, dynamic memory allocation may not be an issue.
4 SUMMARY
o C - not C++
o Simple - ideally not library based
o Embeddable – does not require stdout
stdout, but supports redirection of output over serial port
o Small footprint – systems with small amounts (kB) of SRAM
o Not compiler-specific - work
works with commercial cross compilers
o Works on any OS – especially Windows
o Single toolchian – do not have to work outside the regular IDE
With the addition of CMock (and not to mention the other projects of CException and Ceedling)
Ceed we have a very
useful and usable testing framework ffor the embedded C programmer. I would encourage you to download
these tools and give them a try. For more information about TDD and Embedded C then I would recommend
James Grenning’s Book ‘Test-Driven
Driven De
Development for Embedded C’ (15).
BIBLIOGRAPHY
1. Beck, Kent. Test Driven Development: By Example. s.l. : Addison-Wesley Professional, 2002. 0321146530.
13. IAR Embedded Workbench® for ARM. IAR Systems. [Online] http://www.iar.com/website1/1.0.1.0/68/1/.
15. Test Driven development for Embedded C. The Pragmatic Bookshelf. [Online] 2011.
http://pragprog.com/book/jgade/test-driven-development-for-embedded-c. ISBN: 978-1-93435-662-3.