Testing - Overview

• Motivation
– fault models
– testing methods
• Automatic Test Pattern Generation (ATPG)
algorithms
– D-algorithm
– PODEM
– Socrates
– SAT
1
 Testing: Why?
• Testing is manufacture verification
“Is what I manufactured what I designed?”
• Incorrect Operation (faults) occurs due to
physical defects
– Logical faults
• shorts, missing transistors, …
– Parametric faults
• process variations, die anomalies, …
• Faults may be intermittent or permanent
• Permanent faults may be created during life of
the circuit
– physical/thermal stress
– radiation 2
 Testing: Why?
• No manufacturing process can guarantee 100% defect free
IC’s
• Larger the circuit, greater the probability of defects
occuring
• Economics: Cost of detecting a faulty component is lowest
before it is packaged and embedded in a system and shipped.

Detection (either during manufacture or during operation) of

intermittent and permanent faults
 reliable circuits

3
 Fault Modeling and Testing
A fault model is a model of how a physical or parametric fault
manifests itself in the circuit. Operation. Fault tests are
derived based on these models
Faults Tests
• Logical Fault Parametric (electrical) tests
– Single/multiple stuck-at also detect stuck-on faults
(most used)
– CMOS stuck-open Logical tests detect stuck-at
– CMOS stuck-on faults
– Bridging faults
• Parametric faults Transition tests detect stuck-
– low/high voltage/current open faults
levels
– gate or path delay faults
Timed transition tests detect
delay faults
4
Stuck-at fault test generation
s-a-0
PI justify y propagate PO

To generate a test for y stuck-at 0, we need to find

an vector of primary inputs which sets signal y to 1
(justify) and such that some primary output
differs between the good circuit and the faulty
circuit (propagate)

5
 Fault Reductions
s-a-0 s-a-0 f1: s-a-1 f2: s-a-1

fault equivalence fault dominance

Dominated fault: If every test for fault f1 detects

f2 , then f1 dominates f2 .
 only have to generate test for f1

Set of faults required for testing is a minimal set

with respect to fault equivalence and dominance

6
 Fault Reductions
Stuck fault checkpoints: [Davidson] [Kohavi, Kohavi]

Theorem: In a combinational circuit, any set of tests

which detects all single (multiple) stuck faults on
1. all primary inputs and
2. all branches of fanout points (including primary inputs),
detects all single (multiple) stuck faults.

The set of primary inputs and branches of fanout points

are called the checkpoints of the circuit (i.e.
checkpoints are the set of faults that are tested)

7
 Checkpoints & fault
collapsing
Fault collapsing is the process of reducing the size
of the set of faults to be tested, using fault
equivalence and fault dominance on the
checkpoints of the circuit
• x shows checkpoints (s-a-0 and
s-a-1 on each checkpoint)
• f1 s-a-0 is equivalent to f2 s-a-0.
Similarly for other faults.
c Exercise: find minimal set of
f2 collapsed faults for this circuit.
a Have to test all sites for both s-
b a-0 and s-a-1.
f1 8
 Test Effectiveness
Undetectable fault: No test exists for fault
Redundant fault: Undetectable fault but whose occurrence
does not affect circuit operation
Testability = (#detectable faults) / #faults
Effective faults = faults\redundant faults
(These are the ones we must detect if we want to completely test the
chip. Since redundant faults cause no harm, they should not be
counted against us.)

|detectable faults|
fault coverage =
(This is a better measure of how well a piece of logic is tested by a set of
test vectors.) |effective faults|

9
 Test Effectiveness
Test set size = # of test vectors

Goal:
 100% fault coverage (not 100% testability) with a
minimum sized test set

10
 Logic Level Test Generation
Test generation in combinational circuits
– Satisfiability check on Boolean network [Larrabee]
[Stephan et al.]
– “Best” solution is SAT based approach (Berkeley
experience - not all test community agrees)

Test generation in sequential circuits

– understood only for stuck-at faults
– tests are sequences of vectors
– still impractical on large circuits ( > 30 latches)

11
 Logic Level Test Generation
Convert testing of sequential circuits to testing on
combinational circuits by allowing access to
memory elements ( “scan design” )
– 10-20% area overhead
– 10% performance penalty

All combinational test generation algorithms reduce

to SAT
 solve SAT problem efficiently

12
 Combinational test pattern
generation - Heuristic
Structural search methods: Symbolic and algebraic methods:
perform search on Boolean Abstract formulation
space using topology of the (conceptually elegant but
circuit as guide. practically infeasible)
– D-algorithm [Roth ‘66] – ENF [Armstrong ‘66]
– PODEM [Goel ‘81] – Boolean differences [Sellers
– FAN [Fujiwara, Shimono ‘83] et al. ‘68]
– Socrates [Schulz et al. ‘88]

SAT based methods:

– [Larrabee 89]
– [Stephan 91]

13
 D-Algorithm
Use 5-valued logic for logic value in good (true) and
faulty circuit. Implies only one circuit is
necessary.
– 0  0 in true circuit, 0 in faulty circuit
– 1  1 in true circuit, 1 in faulty circuit
– D  1 in true circuit, 0 in faulty circuit
– D  0 in true circuit, 1 in faulty circuit
– X  unknown value in either true or faulty circuit

Goal: Find an assignment to primary inputs that

causes a D or D at some primary output
14
 D-Algorithm Calculus
B=~A C=A&B
A B A\B 0 1 X D ~D
0 1 0 0 0 0 0 0
1 0 1 0 1 X D ~D
X X X 0 X X X X
D ~D D 0 D X D 0
~D D ~D 0 ~D X 0 ~D

15
 D-Algorithm example
d s-a-0
1 0
1
c D
D
X j
Xa X 0
X i
b 0
e 0

Test is abcde = xx110

16
 D-Algorithm
• At fault location, determine what value must exist
(e.g. for a s-a-0 fault, a D is required)
• Assign each internal line of the circuit a value
(0,1,D,D,X) which is consistent under some
primary input vector. A test exists if such a
vector is found (with at least one D or D at an
output), otherwise the fault cannot be tested
(redundant).
Note: Given n (internal) lines, 2n values need to
enumerated in the worst case

17
End of lecture 16

18
 D-Algorithm: Decision Tree
start
k=1

l =1 “backtracking”

i=1,j=0
i=0,j=1
g=1,h=1
g=0,h=0
c=0,d=0
c=1,d=1

a=0,b=1 a=1,b=0

Search performed level by level from PO’s to PI’s

19
 D-algorithm Problem
e m
f n
a i
b k
c j
d

g
l
h

Note that k and l are complementary signals.

Assume k =1, l =1 is chosen as assignment by D-algorithm.
It takes several other assignments before D-algorithm determines
this is inconsistent.

Solution: Backtrack only on PI values to determine consistency of signals

20
 Implicit Search
Enumeration: PODEM
Actual space of consistent assignments is only 2n,
where n is the number of primary inputs

Hence, search space can be greatly reduced

(compared to D-algorithm) by enumerating over
primary inputs only

PODEM (Path oriented decision making) is such an

algorithm

21
 PODEM Decision Tree
start (All PI’s initially unassigned)

PI1=0 PI1=1
(unused alternative assignment)
PI2=0 PI2=1
(unused alternative (no remaining alternative)
assignment)
PI3=1 PI3=0
(no remaining alternative)
PI4=0 PI4=1 PI4=1
PI4=0
(conflict: (conflict:
no test) no test) PI5=1 PI5=0
(conflict: (conflict:
no test) no test)

indicates no remaining alternative at node

22
 PODEM: Algorithm
1. Start with given fault, empty decision tree, all PI’s
set to X
2. 3 types of operations performed
a) check if current PI assignment is consistent. If so, choose
an unassigned PI and set it to 0 or 1
b) If inconsistent and if alternative value of currently assigned
PI has not been tried, try it and mark this PI as having no
remaining alternative
c) If no remaining alternative on this PI, backup to previous PI
that was assigned, deleting the decision tree below
Algorithm complete: either terminates with a test (all
PI’s assigned) or proves fault is redundant

23
 PODEM: Heuristics
Choosing which PI to assign next
– This depends on how the fault could propagate to a
primary output
– Choose “closest” PO to which fault can propagate and
determine which PI affects the propagation “the most”
– This is done by computing approximate node
controllabilities and observabilities

Heuristic is quite ad-hoc.

PODEM is ineffective on large networks with a lot of
reconvergence
24
 Socrates
Provides improvements to PODEM
– implications
– static and dynamic learning

Basic Idea
– When a value is set on a node (due to current partial PI
assignment) what conclusions can be made?
• Values on other nodes/PI’s
This allows detection of inconsistencies, thereby causing early
backtracking

25
 Implications
b d
a f
e
c

a = 1  (d = 1) & (e = 1)  f = 1
Hence, f = 0  a = 0

Implications are computed in pre-processing phase and stored

for use during backtracking and assignment phase of
algorithm

26
Static and Dynamic Learning
b d
a g (dominator)

c
If a has a D value and it must propagate through g, d must be set to 1.
If it can’t be, then D on a can’t propagate.
– This is an implication learned from the topology of the network
a=D d=1
• Static learning: implications learned in pre-processing phase
• Dynamic learning: implications learned under partial PI assignments

27
 Socrates: Algorithm
1. Perform implication of each single lead value and store
information
2. Given a fault, determine implied values by static learning
3. Use PODEM and determine implied values using dynamic
learning
Heuristic used to determine when to “learn”
1. (e.g. don’t learn implications which are trivial or already known
from previous steps)

Socrates completely subsumed by SAT procedure (P.R. Stephan,

R.K. Brayton and A. Sangiovanni-Vincentelli, “Combinational Test
Generation Using Satisfiability,” IEEE TCAD, Nov. 1992 )

28
Boolean Network Satisfiability
Problem: Given a Boolean network, find a satisfying assignment
to the primary inputs that makes at least primary output
have value 1.
Applications:
Test pattern generation
– Combinational Image computation
– Sequential Low power
– Delay faults
Timing analysis
Hazard detection
In general, SAT is a good alternative to BDD’s if
• only one solution is needed or
• a canonical form is not useful

29
 SAT Outline
• Method of Boolean Differences
• Formulation as SAT problem
• Solving SAT problems
• Greedy, Dynamic Variable Ordering
• Experimental Results
• SAT conclusions

30
 Combinational Test
Generation
• Structural search methods (most widely used)
– D-algorithm [Roth 66]
– PODEM [Goel 81]
– FAN [Fujiwara, Shimono 83]
– Socrates [Schulz et al. 88]
• Symbolic & algebraic methods (not practical)
– Poage’s Method, [Poage 63]
– ENF [Armstrong 66] (equivalent normal form)
– Boolean differences [Sellers et al. 68]
• Hybrid Approach
– SAT-based, [Larrabee 89], [Stephan 92]
31
 Boolean Differences
General Case: Let F(X) be the global function of a network with one primary
output, X = {x1,…,xn } the primary inputs, and let Fz ( X ) be the output of
the network modified by a fault z. Then
Tz = Fz  F
characterizes all tests for fault z. In some cases, we can build this as a
BDD.
Stuck-at Faults: Let g(X) be the global function at any intermediate node.
Rewrite F as F(X,g). Then
T0 = [F(X,0)  F(X,1)] g(X)
~
T1 = [F(X,0)  F(X,1)] g(X)
~ ~
characterizes all tests for g s-a-0, g s-a-1 respectively.
F(X,0)  F(X,1) ~ ~
is the Boolean Difference of F with respect to g.
Solve T0 ,T1 algebraically~[Sellers et
~ al.]
~

32
 Conjunctive Normal Form
Consider T0 in a circuit representation:

F
T0
Fz
Express network T0 in CNF using characteristic function of each gate.
• Example: a =x y
– F = (x + y + a )(x + a )(y + a )
– Conjunction over all gates describes T0
Add clause with only one literal ( T0 ) to the CNF for the circuit. This
says that we want to find an input where T0 = 1, i.e. a test for the
fault. Since the entire formula is in CNF, problem is
SATISFIABLITY.

33
 Satisfiability
The original NP-complete problem
Instance: A set of variables U and a collection of clauses,
C, over U.
Question: Is there a satisfying truth assignment for C ?

Cook’s Theorem (1971): SAT is NP-complete (See Garey &

Johnson, 1979)
Still NP-complete if:
c  C |c | = 3 (3-SAT)
u  U at most 3 clauses contain u or ~u
Polynomial (linear) if:
c  C |c|  2 (2-SAT)

34
 SAT vs. Tautology
Duality:

SAT non-Tautology
CNF, product of sums DNF, sum of products
Is expression ever true? Is expression ever false?
null clause: always false universal cube: always true
assign variable x cofactor with respect to x
backtrack if null clause backtrack if universal cube
unate: easy to satisfy unate: trivial tautology test

Why not just use Espresso?

Unate recursive paradigm for tautology is inefficient for
formulas derived from gate-level characteristic functions.
• Extremely binate cubes, few literals
• Experiments with Espresso confirm this
35
 Implication Graph
• View 2-clauses as pair of implications
– (a + ~b)  (~a  ~b)  (b  a)
– forms implication graph
• Strongly-connected components (SCCs) are equivalent
variables (inverters, buffers)
~a b
a b
a ~b

• More complex equivalences not detected.

– Example: symmetry vs. SCC
~a
a c d ~c ~e
d ~b

e a
b ~d c e
b

• (~a+~b+c)(a+~c)(b+~c)(~a+~b+e)(a+~e)(b+~e)(~c+~d)(c+d)
36
 Non-local Implications
a x
b f
y
c
(~a + ~b + x)(a + ~x)(b + ~x)
(~b+ ~c + y)(b + ~y)(c + ~y)
(x + y + ~f)(~x + f)(~y + f)
1. Find non-local implications for b:
– Try asserting (~b)
– (b + ~x)  (~x), and (b + ~y)  (~y)
– (x + y + ~f)  (~f)
– Thus, (~b)  (~f), so deduce (f)  (b)
2. If contradiction, (e.g. f  ~f) fix to other constant (e.g. f=0)
3. Repeat for every formula variable
Crucial for hard faults (esp. redundancies, where no test exists)

37
 Active Clauses
Problem: Good/faulty circuits related only at I/Os, slow to find
contradictions
Solution: active clauses define relationships for internal nodes (Larrabee
1990)

• Active variable xa is true if net x differs in good and faulty network.

Here, xg refers to signal x in good circuit and xf to x in the faulty
circuit:
(~xa + xg + xf )(~xa + ~xg + ~xf )
• If gate is active, we require that some fanout must be active

y
x (~xa + ya + za)
z
38
 Example Formula
a
s-a-0 g
x z
b

c y h
Good circuit (32 literals):
(a + x)(b + x)(~x + ~a ~b)(~a + g)(~x + g)(~g + a + x)(x + z)
(~x + ~z)(~z + h)(~y + h)(~h + z +y)(b + ~y)(c + ~y)(y +~b + ~c)
Faulty circuit (18 literals):
(~a + gf)(~xf + gf)(~gf + a + xf)(xf + zf)(~xf + ~zf)(~zf + hf)
(~yf + hf)(~hf + zf +yf)
Active clause (29 literals):
(~xa + x +xf)(~xa + ~x + ~xf)(~za + z + zf)(~za + ~z + ~zf)
(~ga + g + gf)(~ga + ~g + ~gf)(~ha + h + hf)(~ha + ~h + ~hf)
(~xa + za + ga)(~za + ha)
Fault site (3 literals) & Goal (2 literals)
(x)(~xf)(xa)(ga + ha) 39
 Search Strategies
• Use a basic branch-and-bound approach

identified non-
solution areas
unidentified non-
solution areas
solution areas

• four basic parameters

2. variable order

3. Dynamic processing
at each branch point
1. initial assignment 4. How long to search?

40
 Orthogonal Strategies
• In theory, any complete algorithm will find a test
for every fault, if one exists.
• In practice, we cannot afford to wait for the worst
case.
Compromise: try several strategies in succession for a
short time (backtrack limit)
– improves average performance
– increases robustness
– has difficulty with “hard” redundancies
• Two strategies which complement each other are
called orthogonal
41
 Static Variable Ordering
Larrabee’s heuristics (LSAT)
– Add clauses for structural heuristics (dominators,
critical paths, etc.)
– static variable ordering
– three search strategies
– static non-local implications (after all faults have been
tried without them)
• No results reported without random patterns

Implemented as  command in 

42
 Benchmark Networks
Network PI PO Gates Faults
C432 36 7 159 413
C499 41 32 370 850
• MCNC Test Generation C880 60 26 320 703
C1355 41 32 506 1058
Benchmarks C1908 33 25 441 1102
C2670 233 140 734 1678
C3540 50 22 961 2177
• Full scan assumed for C5315 178 123 1473 3798
sequential networks C6288
C7552
32
207
32 2384
108 2164
4835
4901
S1494 14 25 577 1220
S5378 214 213 1149 2850
S9234 247 250 2264 465
S13207 700 790 3287 7068
S15850 611 684 4057 8340
S35932 1763 2048 12236 28222
S38417 1664 1742 10232 21113
S38584 1464 1730 11774 26008
Total 121029 43
 Larrabee’s Heurictics
Ckt. LSAT Socrates
C499 72 0 4.9 0
C880 49 0 5.2 0
C1355 99 0 13.9 0
With random tests
C1908 184 0 33.8 0
(seconds | # aborts)
C2670 479 0 57.5 0
C3540 607 0 56.6 0
C5315 464 0 31.3 0
C6288 227 0 87.0 0
C7552 2052 0 248.0 0

Ckt. LSAT Socrates

C499 5240 3 29 0
C880 1730 0 10 0
Without random tests C1355 9540 1 44 0
(seconds | # aborts) C1908 31000 53 81 0
C2670 8600 3 80 0
C3540 110000 64 159 0
C5315 56000 145 102 0
C6288 980000 4506 139 0
C7552 270000 197 418 0
44
 Greedy Heuristics
• Heuristics must be evaluated without using random tests
• Static variable ordering is not effective
• Dynamic ordering can require too much computation at
each branch of the search
Solution: greedy, dynamic orderings.
At each branch point, select:
– 1st literal in 1st unsatisfied clause
– last literal in 1st unsatisfied clause
– 1st literal in last unsatisfied clause
– last literal in last unsatisfied clause
Results: improved performance and robustness

45
 Algorithm
For each uncaught fault:
extract CNF formula
try 4 greedy strategies
if all fail then
find static NLI (non local implications)
repeat 4 strategies
endif
if satisfied then
fault simulate test (to see what other faults are caught)
else
flag possible redundant fault
endif

Simple algorithm:
– no testability measures
– no 5,9-valued algebras
– no multiple backtracking
Standalone program TEGUS
(also in sis 1.1, but slower by 1-2 orders of magnitude)

46
 Experiments
• Theoretical worst case performance is same for
all complete  algorithms
• Heuristics must be evaluated by experiment
• To compare heuristics:
– use the 10 ISCAS’85 benchmark networks and 8 larger
ISCAS’89 networks (assuming full scan)
– Run on same model computer as other reported results
with same options
– Compare CPU time (not backtracks)
– Try with and without random tests

47
 TEGUS Base Results
• Robust: 0 aborted faults in ISCAS’85/’89
networks without fault simulation
• Efficient: for 18 ISCAS networks
– no fault simulation: 10 min. total CPU*
75% extract, 25% SAT
– with random: 1 min. total CPU*
55% extract, 20% fault sim, 15% SAT, 10% I/O
10 MB peak memory
• Simple: 3k lines of code
– 300 to extract CNF formula
– 800 for SAT package

*(DEC 3000/500) 48
 Robustness
Available results with no fault simulation for
ISCAS’85/’89 networks
# of Aborted Normalized
Algorithm Circuits Faults CPU Time
CHE88 7 798 1.6
CHN89 9 1208 2.2
JAC89 8 693 6.6
GIR90 10 0 16.0
ABR90 6 408 30.0
RAJ90 10 1268 24.0
GIR91 10 0 12.0
TEGUS 18 0 1.0
• Algorithms with backtrack limits are incomplete
• Heuristics which abort on fewer faults are more
robust 49
 Efficiency
Available results with random tests for ISCAS’85/’89 networks
# of Aborted Normalized
Algorithm Circuits Faults CPU Time
SIM89 6 0 30.0
SCH89 10 0 1.7
MIN89 9 16 22.0
LAR89 10 0 31.0
JAC89 10 0 6.8
SCH90 18 0 36.0
WAI90 18 0 1.0
MAH90 15 0 9.4
CHK91 1 0 2900.0
GIR91 10 0 25.0
MAT92 18 0 0.9
TEGUS 18 0 1.0
• See Tech Report UCB/ERL M92/112 or TCAD Sept. 96, Vol
15, N0.9, 1167-1175
• Random tests mask effectiveness of deterministic algorithm
50
 SAT Conclusions
• Hybrid test generation algorithm extracts CNF
formula for structural fault, and applies generic
SAT algorithm to generate a test
• Greedy, dynamic ordering heuristics are as
efficient as best published results for structural
algorithms
• Heuristics must be carefully compared using real
implementations and without random tests
• Other fault models and test restrictions can be
accommodated without modifying the generic SAT
solver
51