Lecture 11

Debugging techniques for
embedded systems
Mariusz Janiak
p. 331 C-3, 71 320 26 44
c 2015 Mariusz Janiak

All Rights Reserved
General debugging techniques
Debugging is certainly an art. But like most art, success is a

combination of talent, experience, and mastery of the tools of the
trade.
Shunryu Suzuki
In the beginner’s mind there are many possibilities; in
the expert’s mind there are few
The secrets of debugging are not mysterious. Know your tools. Avoid
the common, dangerous mistakes if you can, then check to make
sure you’ve succeeded. Be proactive. Save data to help you compare
the past with the present, even though ”nothing has changed.” Keep
your eyes and your mind open. And accept the responsibility of
excellence.
1
How to debug embedded systems, http://www.edn.com
What’s so special about embedded?

Limited input/output options – one analog voltage pin, one led
Limited computing resources – smaller CPU, smaller memory,
lower speed
Hardware access – timing-sensitive, can break and burn out
things, catatonia
Real-time behaviour – response deadlines, dynamic control
algorithms
Avoiding bugs
Coding style: use clear, consistent style and useful naming
standards
Document everything, from architecture and interface
specification documents to comments on code lines
Hold code reviews
Program defensively
Check function return values
Be suspicious of cut/paste
Consider using an integrated development environment (IDE)
with dynamic syntax checking
Debugging and testing (inseparable pair)

Testing finds errors; debugging localizes and repairs them
Together these form the “testing/debugging cycle”: we test,
then debug, then repeat
Any debugging should be followed by a reapplication of all
relevant tests, particularly regression tests. This avoids
(reduces) the introduction of new bugs when debugging
Testing and debugging need not be done by the same people
Why debugging is hard?

There may be no obvious relationship between the external
manifestation of an error and its internal cause
Symptom and cause may be in remote parts of the program.
Changes in program may mask or modify bugs.
Symptom may be due to human mistake or misunderstanding
that is difficult to trace
Bug may be triggered by rare or difficult to reproduce input
sequence, program timing (threads) or other external causes
Bug may depend on other software/system state, things others
did to you systems weeks/months ago
Designing for Debug/Test

When you write code think about how you are going to
test/debug it
lack of thought always translates into bugs
Write test cases when you write your code
If something should be true assert() it
Create functions to help visualize your data
Design for testing/debugging from the start
Test early, test often
Test at abstraction boundaries
Types of bugs
Compile time: syntax, spelling, static type mismatch – usually
caught with compiler
Design: flawed algorithm – incorrect outputs
Program logic (if/else, loop termination, select case, etc) –
incorrect outputs
Memory nonsense: null pointers, array bounds, bad types, leaks
– runtime exceptions
Interface errors between modules, threads, programs (in
particular, with shared resources: sockets, files, memory, etc) –
runtime exceptions
Off-nominal conditions: failure of some part of software of
underlying machinery (network, etc) – incomplete functionality
Deadlocks: multiple processes fighting for a resource – freeze
ups, never ending processes
The ideal debugging process

Identify test case that reliably show existence of fault (when
possible)
Isolate problem to small fragment of program
Correlate incorrect behavior with program logic/code error
Change the program (and check for other parts of program
where same or similar program logic may also occur)
Regression test to verify that the error has really been removed
– without inserting new errors
Update documentation when appropriate
General advice
Try to understand as much of what is happening as possible
“it compiles” is NOT the same as “it works”
When in doubt, ask
Then test the answer
Error messages are generally just a vague hint and can be
misleading
Don’t always trust the “comments/documents”, they can be
out-of-date
What is a debugger?
A debugger is not an IDE – though the two can be integrated,
they are separate entities
A debugger loads in a program (compiled executable, or
interpreted source code) and allows the user to trace through
the execution.
Debuggers typically can do disassembly, stack traces, expression
watches, and more
No need for precognition of what the error might be
Allows for “live” error checking – no need to re-write and
re-compile when you realize a certain type of error may be
occurring
Debugging techniques
Execution tracing
running the program
print
trace utilities
single stepping in debugger
hand simulation
Interface checking
check procedure parameter number/type (if not enforced by
compiler) and value
defensive programming: check inputs/results from other modules
documents assumptions about caller/callee relationships in
modules, communication protocols, etc
Assertions – include range constraints or other information with data
Skipping code – comment out suspect code, then check if error
remains
Debugging hints
Understand the problem before you fix it
Debug on the host as much as possible
Most common “Heisenbugs” are memory or thread related.
Find memory problems early
Reproduce and isolate the problem – find the smallest system
which demonstrates the bug
Insert checks for invariants and have the program stop
everything when one is violated
Verify each layer with small, simple tests
Keep a log of events and assumptions (git)
Tools
Source-level debugger
Simple printf statement
In-circuit emulators (ICE) and JTAG debuggers
Data monitors
Operating system monitors
Profilers
Memory testers
Execution tracers
Coverage testers
Summary
Testing for memory leaks and corruption
Scanning for performance bottlenecks
Collecting a variable trace of key sequences
Ensuring sufficient CPU-bandwidth margin
Evaluating test coverage effectiveness
Tracing operating-system resource usage
Recording execution sequences (semaphores, process
interactions) of key activities
Checking for error returns from common application
programming interfaces
The End
Thank you for your kind attention.

Lecture 11

Uploaded by

Lecture 11

Uploaded by

Debugging techniques for

c 2015 Mariusz Janiak

Debugging is certainly an art. But like most art, success is a

What’s so special about embedded?

Debugging and testing (inseparable pair)

Why debugging is hard?

Designing for Debug/Test

The ideal debugging process

Thank you for your kind attention.

You might also like