ECE 250 Algorithms and Data Structures

Backtracking algorithms

Douglas Wilhelm Harder, M.Math. LEL

Department of Electrical and Computer Engineering
University of Waterloo
Waterloo, Ontario, Canada

ece.uwaterloo.ca
dwharder@alumni.uwaterloo.ca

© 2006-2013 by Douglas Wilhelm Harder. Some rights reserved.

 Backtracking algorithms
2

Outline

In this topic, we will cover:

– Traversals of trees and graphs
– Backtracking
 Backtracking algorithms
3

Backtracking

Suppose a solution can be made as a result of a series of choices

– Each choice forms a partial solution
– These choices may form either a tree or DAG
• Separate branches may recombine or diverge
 Backtracking algorithms
4

Backtracking

With Dijkstra’s algorithm, we keep track of all current best paths

– There are at most |V| – 1 paths we could extend at any one time
– These can be tracked with a relatively small table

Suppose we cannot evaluate the relative fitness of solutions

– There may just be too many to record efficiently
– Are we left with a brute-force search?

Suppose there are just too many partial solutions at any one time to
keep track of
– At any point in time in a game of chess or Go ( 围棋 or 碁 ), there are a
plethora of moves, each valid, but the usefulness of each will vary
 Backtracking algorithms
5

Sudoku

In the first case, consider the game Sudoku:

– The search space is 953

5 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
9 5 8 1
1 2 3
8 2 7

http://www.sudokuoftheday.com/
 Backtracking algorithms
6

Sudoku

At least for the first entry in the first square, only 1, 3, 7 fit
5 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
9 5 8 1
1 2 3
8 2 7

5 1 4 6 5 3 4 6 5 7 4 6
6 3 1 6 3 1 6 3 1
9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1
8 4 8 4 8 4
2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7
 Backtracking algorithms
7

Sudoku

If the first entry has a 1, the 2nd entry in that square could be 7 or 8
5 1 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
9 5 8 1
1 2 3
8 2 7

5 1 4 6 5 1 4 6
7 6 3 1 8 6 3 1
9 2 5 4 9 2 5 4
6 8 1 6 8 1
8 4 8 4
2 6 1 2 6 1
9 5 8 1 9 5 8 1
1 2 3 1 2 3
8 2 7 8 2 7
 Backtracking algorithms
8

Sudoku

If the first entry has a 3, the 2nd entry in that square could be 7 or 8
5 3 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
9 5 8 1
1 2 3
8 2 7

5 3 4 6 5 3 4 6
7 6 3 1 8 6 3 1
9 2 5 4 9 2 5 4
6 8 1 6 8 1
8 4 8 4
2 6 1 2 6 1
9 5 8 1 9 5 8 1
1 2 3 1 2 3
8 2 7 8 2 7
 Backtracking algorithms
9

Sudoku

If the first entry has a 7, the 2nd entry in that square could be 8
5 7 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
9 5 8 1
1 2 3
8 2 7

5 7 4 6
8 6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
9 5 8 1
1 2 3
8 2 7
 Backtracking algorithms
10

Sudoku
5 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
In the next child, there are no 1
9 5
2
8 1
3

options available for the

8 2 7

next entry 5 1 4
6 3 1
6 5 3 4
6 3 1
6 5 7 4
6 3 1
6

9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1
8 4 8 4 8 4
2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7

5 1 4 6 5 1 4 6 5 3 4 6 5 3 4 6 5 7 4 6 5 7 4 6
7 6 3 1 8 6 3 1 7 6 3 1 8 6 3 1 1 6 3 1 8 6 3 1
9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1 6 8 1 6 8 1 6 8 1
8 4 8 4 8 4 8 4 8 4 8 4
2 6 1 2 6 1 2 6 1 2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7 8 2 7 8 2 7 8 2 7

5 1 4 6
7 6 × 3 1 Any candidate solution built from this
9 2 5 4
6 8 1
partial is infeasible
8 4 – We can ignore this branch
2 6 1
9 5 8 1
1 2 3
8 2 7
 Backtracking algorithms
11

Sudoku
5 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
Three other branches lead to 1
9 5
2
8 1
3

similar dead ends

8 2 7

5 1 4 6 5 3 4 6 5 7 4 6
6 3 1 6 3 1 6 3 1
9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1
8 4 8 4 8 4
2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7

5 1 4 6 5 1 4 6 5 3 4 6 5 3 4 6 5 7 4 6
7 6 3 1 8 6 3 1 7 6 3 1 8 6 3 1 8 6 3 1
9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1 6 8 1 6 8 1
8 4 8 4 8 4 8 4 8 4
2 6 1 2 6 1 2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7 8 2 7 8 2 7

5 1 4 6 5 3 4 6 5 7 4 6
7 6 × 3 1 7 6 × 3 1 8 6 × 3 1
9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1
8 4 8 4 8 4
2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7
 Backtracking algorithms
12

Sudoku
5 4 6
6 3 1
9 2 5 4
6 8 1
8 4
2 6 1
With the other two, there is only 1
9 5
2
8 1
3

one candidate for each of the

8 2 7

last two 5 1 4
6 3 1
6 5 3 4
6 3 1
6 5 7 4
6 3 1
6

9 2 5 4 9 2 5 4 9 2 5 4
entries 6 8
8
1
4
6 8
8
1
4
6 8
8
1
4
2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7

5 1 4 6 5 1 4 6 5 3 4 6 5 3 4 6 5 7 4 6
7 6 3 1 8 6 3 1 7 6 3 1 8 6 3 1 8 6 3 1
9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4 9 2 5 4
6 8 1 6 8 1 6 8 1 6 8 1 6 8 1
8 4 8 4 8 4 8 4 8 4
2 6 1 2 6 1 2 6 1 2 6 1 2 6 1
9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1 9 5 8 1
1 2 3 1 2 3 1 2 3 1 2 3 1 2 3
8 2 7 8 2 7 8 2 7 8 2 7 8 2 7

5 1 4 6 5 3 4 6
8 6 7 3 1 8 6 7 3 1
3 9 2 5 4 1 9 2 5 4
6 8 1 6 8 1
8 4 8 4
2 6 1 2 6 1
9 5 8 1 9 5 8 1
1 2 3 1 2 3
8 2 7 8 2 7
 Backtracking algorithms
13

Sudoku

It may seem that this is a reasonably straight-forward method;

however, the decision tree continues to branch quick once we start
filling the second square
 Backtracking algorithms
14

Sudoku

A binary tree of this height would have around 254 – 1 nodes

– Fortunately, as we get deeper into the tree, more get cut
 Backtracking algorithms
15

Implementation

Our straight-forward implementation takes a 9 × 9 matrix

– Default entries are values from 1 to 9, empty cells are 0
– Two helper functions:
bool next_location( int[9][9], int &i, int &j )
• Finds the next location empty location returning false if none is found
bool is_valid( int[9][9], int i, int j, int value )
• Checks if there are any conflicts created if matrix[i][j] is assigned
value
– The backtracing function:
• Finds the next unoccupied cell
• For each value from 1 to 9, it checks if it is valid to insert that it there
– If so, backtracking is called recursively on the matrix with that entry set

The main function creates the initial matrix and calls backtrack
 Backtracking algorithms
16

Implementation
// Find the next empty location in 'matrix'
// If one is found, assign 'i' and 'j' the indexes of that
entry
// Otherwise, return false
// - In this case, the values of 'i' and 'j' are undefined
bool next_location( int matrix[9][9], int &i, int &j ) {
// If 'value' already appears in
for ( int i1 = 0; i1 < 3; ++i1 ) {
// - the row 'm'
for ( int j1 = 0; j1 < 3; ++j1 ) {
// - the column 'n'
for ( int i2 = 0; i2 < 3; ++i2 ) {
// - the 3x3 square of entries it (m, n) appears in
for ( int j2 = 0; j2 < 3; ++j2 ) {
// return true, otherwise return false
i = 3*i1 + i2;
bool is_valid( int matrix[9][9], int m, int n, int value ) {
j = 3*j1 + j2;
// Check if 'value' already appears in either a row or
column
// return 'true' if we find an for ( int i = 0; i < 9; ++i ) {
// unoccupied entry
if ( (matrix[m][i] == value) || ( matrix[i][n] ==
if ( matrix[i][j] == 0 ) value ) )
return true; return false;
} }
}
} // Check if 'value' already appears in either a row or
} column
int ioff = 3*(m/3);
return false; // all the entries are occupied int joff = 3*(n/3);
}
for ( int i = 0; i < 3; ++i ) {
for ( int j = 0; j < 3; ++j ) {
if ( matrix[ioff + i][joff + j] == value )
return false; // 'value' already in the 3x3
square
}
}
 Backtracking algorithms
17

Implementation
bool backtrack( int matrix[9][9] ) {
int i, j;

// If the matrix is full, we are done

if ( !next_location( matrix, i, j ) ) {
return true;
}

for ( int value = 1; value <= 9; ++value ) {

if ( is_valid( matrix, i, j, value ) ) {
// Assume this entry is part of the
// solution--recursively call backtrack
matrix[i][j] = value;

// If we found a solution, return

// otherwise, reset the entry to 0
if ( backtrack( matrix ) ) {
return true;
} else {
matrix[i][j] = 0;
}
}
}

// No solution found--reset the entry to 0

return false;
}
 Backtracking algorithms
18

Implementation
int main() {
int matrix[9][9] = {
{5, 0, 4, 0, 0, 0, 0, 6, 0},
{0, 6, 0, 3, 0, 0, 1, 0, 0},
{0, 9, 2, 5, 4, 0, 0, 0, 0},
{6, 0, 8, 0, 0, 1, 0, 0, 0},
{0, 0, 0, 8, 0, 4, 0, 0, 0},
{0, 0, 0, 2, 0, 0, 6, 0, 1},
{0, 0, 0, 0, 9, 5, 8, 1, 0},
{0, 0, 1, 0, 0, 2, 0, 3, 0},
{0, 8, 0, 0, 0, 0, 2, 0, 7}
};

// If found, print out the resulting matrix

if ( backtrack( matrix ) ) {
for ( int i = 0; i < 9; ++i ) {
for ( int j = 0; j < 9; ++j ) {
std::cout << matrix[i][j] << " ";
}

std::cout << std::endl;

}
}

return 0;
}
 Backtracking algorithms
19

Implementation

In this case, the traversal:

– Recursively calls backtrack 874 times
• The last one determines that there are no unoccupied entries
– Checks if a placement is valid 7658 times

5 3 4 1 7 8 9 6 2
8 6 7 3 2 9 1 4 5
1 9 2 5 4 6 3 7 8
6 7 8 9 3 1 5 2 4
2 1 5 8 6 4 7 9 3
3 4 9 2 5 7 6 8 1
4 2 3 7 9 5 8 1 6
7 5 1 6 8 2 4 3 9
9 8 6 4 1 3 2 5 7
 Backtracking algorithms
20

Backtracking

This should give us an idea, however:

– Perform a traversal
– Do not continue traversing if a current node indicates all descendants are
infeasible solutions
 Backtracking algorithms
21

Classical applications

Classic applications of this algorithm technique include:

– Eight queens puzzle
– Knight’s tour
– Logic programming languages
– Crossword puzzles
 Backtracking algorithms
22

Eight queens puzzle

Arrange eight queens on

a chess board so that no
queen can take another

♕
 Backtracking algorithms
23

Knight’s tour

Have a knight visit all the

squares of a chess board
either as a path or a cycle

♘
 Backtracking algorithms
24

Logic programming languages

Consider the Prolog programming language where we state facts:

female(juliana). parent(willem, catharina). parent(beatrix, willem).

male(bernhard). parent(maxima, catharina). parent(claus, willem).
female(beatrix). parent(willem, alexia). parent(beatrix, friso).
male(claus). parent(maxima, alexia). parent(claus, friso).
male(firso). parent(willem, ariane). parent(beatrix, constantijn).
female(mabel). parent(maxima, ariane). parent(claus, constantijn).
male(constantijn).
female(laurentien).parent(firso, luana). parent(juliana, beatrix).
parent(mabel, luana). parent(bernhard, beatrix).
female(catharina). parent(firso, zaria). parent(juliana, irene).
female(alexia). parent(mabel, zaria). parent(bernhard, irene).
female(ariane). parent(juliana, margriet).
parent(constantijn, eloise). parent(bernhard, margriet).
female(luana). parent(laurentien, eloise). parent(juliana, christina).
female(zaria). parent(constantijn, parent(bernhard, christina).
claus_ii).
female(eloise). parent(laurentien, spouses(willem, maxima).
male(claus_ii). claus_ii). spouses(firso, mabel).
female(leonore). parent(constantijn, spouses(constantinjn,
leonore). laurentien).
parent(laurentien, leonore). spouses(beatrix, claus).
 Backtracking algorithms
25

Logic programming languages

You can now define relationships between individuals

% Relationships
mother(M, X) :- parent(M, X), female(M).
father(F, X) :- parent(F, X), male(F).
sister(S, X) :- sibling(S, X), female(S), \+ (S = X).
brother(B, X) :- sibling(B, X), male(B), \+ (S = X).
grandparent(G, X) :- parent(G, P), parent(P, X).

% Symmetric
spouses(X, Y) :- spouses(Y, X);
sibling(X, Y) :- parent(P, X), parent(P, Y), \+ (X = Y).
cousin(X, Y) :- parent(A, X), parent(B, Y), sibling(A, B).

% Antisymmetric
uncle(U, X) :- male(U), sibling(U, Y), parent(Y, X).
uncle(U, X) :- male(U), spouse(U, Z), sibling(Z, Y),
parent(Y, X).
aunt(A, X) :- female(A), parent(Y, X), sibling(A, Y).
aunt(A, X) :- female(A), spouse(A, Z), sibling(Z, Y),
parent(Y, X).
 Backtracking algorithms
26

Logic programming languages

Given these relationships, you can now make queries:

cousin(zaria, alexia).
uncle(constantijn, alexia).
aunt(laurentien, alexia).

Backtracking can be used to determine whether the above

relationships hold given the stated facts
 Backtracking algorithms
27

Parsing

Question: how do we define a programming language?

– Why are any of the following never valid?
a + < b
c[3)
d?e;
54f
""";
g$ = "Hello world!";

– Programming languages are defined by grammars

– The C++ programming language grammar is available here:
http://www.nongnu.org/hcb/
 Backtracking algorithms
28

Parsing and grammars

Consider just the conditional statements from the pre-processor

– Square brackets is used to indicate something is optional

<group> ::= <group-part>

<group> <group-part>
<group-part> ::= <if-section>
<control-line>
<if-section> ::= <if-group> [<elif-groups>] [<else-group>] <endif-
line>
<if-group> ::= #if <constant-expression> <new-line> [<group>]
#ifdef <identifier> <new-line> [<group>]
#ifndef <identifier> <new-line> [<group>]
<elif-groups> ::= <elif-group>
<elif-groups> <elif-group>
<elif-group> ::= #elif <constant-expression> <new-line> [<group>]
<else-group> ::= #else <new-line> [<group>]
<endif-line> ::= #endif <new-line>
 Backtracking algorithms
29

Parsing and grammars

We cannot work with a full grammar for C++

– Instead, we will consider some vastly oversimplified versions
<digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
<pos> ::= <digit> | <digit><pos>
<int> ::= <pos> | +<pos> | -<pos>
<std> ::= <int>.<pos> | .<pos> | -.<pos> | +.<pos>
<sci> ::= <int>e<int> | <int>E<int> | <std>e<int> |
<std>E<int>
<float> ::= <std> | <sci>

<nondigit> ::= A | B | … | Z | a | b | … | z | _
<identifier> ::= <nondigit> | <identifier><nondigit>
| <identifier><digit>

<declaration> ::= int <identifier> = <identifier>;

| int <identifier> = <int>;
| double <identifier> = <int>
| double <identifier> = <float>
 Backtracking algorithms
30

Parsing and grammars

As you can see, each of these defines a tree

– Some of these trees are recursively defined
<digit> ::= 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
<pos> ::= <digit> | <digit><pos>
<int> ::= <pos> | +<pos> | -<pos>
<std> ::= <int>.<pos> | .<pos> | -.<pos> | +.<pos>
<sci> ::= <int>e<int> | <int>E<int> | <std>e<int> |
<std>E<int>
<float> ::= <std> | <sci>

<nondigit> ::= A | B | … | Z | a | b | … | z | _
<identifier> ::= <nondigit> | <identifier><nondigit>
| <identifier><digit>

<declaration> ::= int <identifier> = <identifier>;

| int <identifier> = <int>;
| double <identifier> = <int>
| double <identifier> = <float>
 Backtracking algorithms
31

Parsing and grammars

Suppose we are trying to parse the string

int var0 = 3532700;

double var1 = 3.5e-27;
double var2 = -44.203;

What if we’re parsing garbage?

double var0 = 3.5g-27;

int 1var = 44203;
double var2 = 0.0
double var3 = 1.0;

Note that in C/C++, 052 defines the octal number equal to 4210
 Backtracking algorithms
32

Backjumping

In some cases, the following may occur:

– Determining that one leaf does not constitute a solution may
simultaneously determine that the corresponding sub-tree does not
contain a solution, either
– In this case, return to the closest ancestor such that it has not yet been
determined that all descendants have been ruled out
– This is described as backjumping
 Backtracking algorithms
33

Searching a maze

In trying to find your way through a maze, one simple rule works
quite nicely:
– The right-hand rule:
Touch a wall with your right hand, and continue forward always
keeping your right hand touching a wall until you get out.
 Backtracking algorithms
34

Searching a maze

This works well in finding your way through a maze

It doesn’t work if you’re trying to get into the maze or out of a maze
 Backtracking algorithms
35

Searching a maze

Consider the following algorithm:

– If the goal is reached, we are done
– If there is only one move into a previously unoccupied cell, move
to it and flag it as occupied
– If there is more than one move into a previously unoccupied cell,
push that position onto a stack, and take the right-most available
path
– If there are no more moves, check the stack:
• If the stack is empty,
there is no path to the goal
• If the stack is not empty, pop to top position
and continue the algorithm from that point
 Backtracking algorithms
36

Searching a maze

In each example, the solution is always found

– In the normal maze, less work is required due to backjumping
 Backtracking algorithms
37

References

Wikipedia, http://en.wikipedia.org/wiki/Backtracking

[1] Cormen, Leiserson, and Rivest, Introduction to Algorithms, McGraw Hill, 1990, §11.1, p.200.
[2] Weiss, Data Structures and Algorithm Analysis in C++, 3 rd Ed., Addison Wesley, §9.2, p.342-5.

These slides are provided for the ECE 250 Algorithms and Data Structures course. The
material in it reflects Douglas W. Harder’s best judgment in light of the information available to
him at the time of preparation. Any reliance on these course slides by any party for any other
purpose are the responsibility of such parties. Douglas W. Harder accepts no responsibility for
damages, if any, suffered by any party as a result of decisions made or actions based on these
course slides for any other purpose than that for which it was intended.