Complexity Theory

Complexity Theory
Course Notes
Sebastiaan A. Terwijn
Radboud University Nijmegen
Department of Mathematics
P.O. Box 9010
6500 GL Nijmegen
the Netherlands
terwijn@math.ru.nl
Copyright c 2010 by Sebastiaan A. Terwijn
Revised 2011
iii
Contents
1 Introduction 1
1.1 Complexity theory . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Big O and small o . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.4 Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.5 Number theory . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2 Basics 6
2.1 Time and space bounds . . . . . . . . . . . . . . . . . . . . . . 6
2.2 Inclusions between classes . . . . . . . . . . . . . . . . . . . . 7
2.3 Hierarchy theorems . . . . . . . . . . . . . . . . . . . . . . . . 8
2.4 Central complexity classes . . . . . . . . . . . . . . . . . . . . 9
2.5 Problems from logic, algebra, and graph theory . . . . . . . . 10
2.6 The Immerman-Szelepcsenyi Theorem . . . . . . . . . . . . . 12
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3 Reductions and completeness 15
3.1 Many-one reductions . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 NP-complete problems . . . . . . . . . . . . . . . . . . . . . . 17
3.3 More decision problems from logic . . . . . . . . . . . . . . . . 18
3.4 Completeness of Hamilton path and TSP . . . . . . . . . . . . 20
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4 Relativized computation and the polynomial hierarchy 26
4.1 Relativized computation . . . . . . . . . . . . . . . . . . . . . 26
4.2 The Polynomial Hierarchy . . . . . . . . . . . . . . . . . . . . 27
4.3 Relativization . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
5 Diagonalization 32
5.1 The Halting Problem . . . . . . . . . . . . . . . . . . . . . . . 32
5.2 Intermediate sets . . . . . . . . . . . . . . . . . . . . . . . . . 32
iv
5.3 Oracle separations . . . . . . . . . . . . . . . . . . . . . . . . 34
5.4 Many-one versus Turing reductions . . . . . . . . . . . . . . . 35
5.5 The Gap Theorem . . . . . . . . . . . . . . . . . . . . . . . . 37
5.6 The Speed-Up Theorem . . . . . . . . . . . . . . . . . . . . . 37
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
6 Randomized computation 41
6.1 Probabilistic classes . . . . . . . . . . . . . . . . . . . . . . . . 41
6.2 More about BPP . . . . . . . . . . . . . . . . . . . . . . . . . 44
6.3 The classes RP and ZPP . . . . . . . . . . . . . . . . . . . . . 46
6.4 Primes again . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
7 Circuit complexity 50
7.1 Boolean functions . . . . . . . . . . . . . . . . . . . . . . . . . 50
7.2 Circuits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
7.3 Relation with advice classes . . . . . . . . . . . . . . . . . . . 52
7.4 Small circuits for NP? . . . . . . . . . . . . . . . . . . . . . . 53
7.5 Sparse T-complete sets . . . . . . . . . . . . . . . . . . . . . . 54
7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
8 Cryptography 57
8.1 Public-key cryptography . . . . . . . . . . . . . . . . . . . . . 57
8.2 Signed messages . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.3 One-way functions . . . . . . . . . . . . . . . . . . . . . . . . 59
8.4 The class UP . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
8.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
9 Interactive proofs 62
9.1 Interactive protocols and the class IP . . . . . . . . . . . . . . 62
9.2 IP = PSPACE . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
9.3 Zero-knowledge proofs . . . . . . . . . . . . . . . . . . . . . . 65
9.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
10 Approximations 67
10.1 Approximating the traveling salesman . . . . . . . . . . . . . . 67
10.2 Approximation algorithms . . . . . . . . . . . . . . . . . . . . 68
10.3 Probabilistically checkable proofs . . . . . . . . . . . . . . . . 69
10.4 PCP theory and nonapproximability . . . . . . . . . . . . . . 70
10.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
v
11 Proof complexity 73
11.1 Propositional proof systems and Cooks program . . . . . . . . 73
11.2 The pigeonhole principle . . . . . . . . . . . . . . . . . . . . . 74
11.3 Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
11.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
Further reading 79
Bibliography 80
Index 83
vi
Chapter 1
Introduction
1.1 Complexity theory
Complexity theory is concerned with the resources, such as time and space,
needed to solve computational problems. After the success of the general
theory of computability, that gave us a precise denition of the notion of
algorithm and fundamental insights into the notion of mathematical proof
and undecidability, it was only natural to also consider the notion of ecient
computation, and ask why some problems that are computable in principle
resist all attempts to compute them in a reasonable amount of time. There
is an abundance of problems from mathematics and computer science that
are trivially solvable by brute force search of an exponential number of in-
stances, but for which currently no ecient algorithm is known. Complexity
theory is the appropriate setting for the study of such problems. It is also the
home of one of the most fundamental open problems in mathematics, namely
the famous NP versus P problem. Some 40 years after the discovery of this
problem, complexity theory has matured into an extremely rich and fasci-
nating subject, despite the lack of progress on some of its most fundamental
problems.
1.2 Preliminaries
In the following sections we list some preliminaries and notation. Prelimi-
naries about Turing machines can be found in [28]. (Note however that in
complexity theory Turing machines are allowed to have any nite number of
tapes.) Most of the material treated here can be found in textbooks such as
[4], [22], [27], and [2]. Given a Turing machine M and an input x, we use the
notation M(x) to denote that the computation of M on x halts in a nite
number of steps, and we write M(x) if this is not the case. A set or a func-
1
2 Chapter 1. Introduction
tion is computable if there is a Turing machine computing it.
1
For notational
convenience, we often identify machines with the sets they compute. A set
is computably enumerable (c.e.) if it is empty or the range of a computable
function (cf. also Exercise 2.7.11.)
The notation used below is mostly standard. Given a set , the set of all
nite strings of elements from is denoted by
. (This set is also referred to

as the set of all words over the alphabet .) The -operator is called Kleene
star. We usually work over the set of nite binary strings {0, 1}
, that can be
interpreted as binary representations of the natural numbers N. The length of
a string x is denoted by |x|, and {0, 1}
n
denotes the set of strings of length n.
The concatenation of x and y is denoted by xy or simply by xy. The i-fold
concatenation of x with itself is denoted by x
i
. Whether x
i
denotes the i-fold
concatenation of the string x or the i-th power of the natural number x is
always clear from the context. Throughout, , denotes a bijective pairing
function from N N to N (or equivalently from pairs of strings to strings)
dened by
x, y =
1
2
(
(x + y + 1)
2
(x + y + 1)
)
+ x =
1
2
(
(x + y)
2
+ 3x + y
)
.
Objects such as formulas and Turing machines that can be coded using strings
from {0, 1}
are routinely handled as such. Subsets A {0, 1}
are variously
called set, problem (in case we think of it as a decision problem, namely to
decide for a given element x whether x A), or language. We often identify
a set A with its characteristic sequence, i.e. we write A(x) = 1 if x A
and A(x) = 0 otherwise. For a given set A, A denotes the complement of
A, and A
x denotes the nite set A {0 . . . x}, or equivalently, the nite

string A(0)A(1) . . . A(x). For a given class C of sets, co-C denotes the class
{
A : A C
}
of all sets whose complement is in C.
1.3 Big O and small o
We use the following common notation to compare the asymptotic behavior
of functions on N. The phrase for almost all n N means for all n except
perhaps nitely many.
f O(g) if there is a constant c N such that for almost all n we have
f(n) cg(n).
f o(g) if
lim
n
f(n)
g(n)
= 0.
1
In computability theory it is essential to work with partial computable functions, i.e.
functions that need not be dened on all arguments. Partial functions that are everywhere
dened are called total, and a computable function is total by denition. In complexity
theory, because of the presence of time and space bounds, the distinction between total
and partial functions is less important.
1.4. Logic 3
We can rephrase this as follows: for every real constant r > 0 it holds that
for almost every n, f(n) rg(n).
The Big-O notation is often used to introduce constants without having to
name them: f O(1) means that f is bounded by a constant. For example,
g(n) n
O(1)
means that g is at most of polynomial growth.
Big O and small o also have a dual notation, denoted with omegas:
f (g) if for every constant c N there are innitely many n such that
f(n) cg(n).
f (g) if there is a real constant r > 0 such that for innitely many n,
f(n) rg(n).
2
We also use the notation
x(x) to denote that (x) holds for almost

all x. Similarly,
x(x) denotes that there are innitely many x such that

(x) holds.
1.4 Logic
We use the following terminology from logic. We assume the reader is familiar
with the basics of propositional logic. Propositional formulas are also called
Boolean formulas. A literal is a Boolean variable or the negation thereof.
For a literal x its negation is denoted by x. A clause is a disjunction of
literals. Clauses are often denoted using set notation, so the set of literals
{x
1
, . . . , x
k
} denotes the clause x
1
. . . x
k
. denotes the empty clause, and
stands for false. A Boolean formula is in conjunctive normal form (CNF) if
it is a conjunction of clauses. A formula is in disjunctive normal form (DNF)
if it is a disjunction of conjunctions of literals.
1.5 Number theory
In this section we summarize some preliminaries from number theory that we
will use later. The reader can skip this section and refer back to it later when
needed.
For any integer n, Z/nZ is the additive group of integers modulo n.
(Z/nZ)
denotes the multiplicative group of integers x with gcd(x, n) = 1.

Theorem 1.5.1. (Chinese remainder theorem) For all n and m such that
gcd(n, m) = 1 we have (Z/nmZ)

= (Z/nZ) (Z/mZ).
Proof. Dene the isomorphism by x mod nm (x mod n, x mod m).
The order (cardinality) of (Z/nZ)
is denoted by (n). The function is

called Eulers function. For p prime we have (p) = p 1 by the following:
2
This is the Hardy-Littlewood denition of , see also Exercise 1.6.1. Knuth uses the
version with almost every n instead of innitely many n, which is not equivalent.
4 Chapter 1. Introduction
Proposition 1.5.2. (n) = n
p|n
(
1
1
p
)
. Here the product is taken over
all primes dividing n.
Theorem 1.5.3. (Euler) x
(n)
1 mod n for all x and n with gcd(x, n) = 1.
Proof. By denition, (n) is the order of (Z/nZ)
. By Lagranges theorem,
the order of every element in the group divides the order of the group.
A special case of Theorem 1.5.3 is Fermats little theorem: if p is prime then
a
p1
1 mod p for every a with gcd(a, p) = 1.
We will apply the above results later in section 8.1.
Denition 1.5.4. Given a prime p > 2, an element r (Z/pZ)
is a primitive
root modulo p if it has order p 1, i.e.
(i) r
p1
1 mod p, and
(ii) r
i
1 mod p for every 1 < i < p 1.
Note that we can replace (ii) in the above denition by
(ii)
r
p1/q
1 mod p for every prime factor q of p 1.
Namely, if i is least such that r
i
1 and r
p1
1 mod p then i | p 1, and
also r
ai
1 for every multiple ai of i. So if there exists i < p 1 with
r
i
1 mod p then there is a prime factor q of p 1 with r
p1/q
1 mod p.
Theorem 1.5.5. (Fermat) A number p > 2 is prime precisely when there is
a primitive root modulo p.
Proof. Suppose that (ii)
above holds. We prove that p is prime. If r

p1
1 mod p then the order of r divides p 1. By (ii)
it cannot be a proper
divisor, so the order of r is p 1. Hence (Z/pZ)
has order p 1, and p is

prime.
Conversely, suppose that n is not prime, and suppose that r (Z/nZ)
satises r
n1
1 mod n. By Theorem 1.5.3 we also have r
(n)
1 mod n,
and since n is not prime, (n) < n 1. Since the order of r divides both
(n) and n 1 it is strictly smaller than n 1, hence (ii)
does not hold.

There exist also primitive roots modulo numbers n that are not prime: r is a
primitive root modulo n if r
i
1 mod n for every i | (n). We will later use
the fact that for every p > 2 prime, there is a primitive root modulo p
2
, cf.
Exercise 1.6.3.
1.6. Exercises 5
1.6 Exercises
Exercise 1.6.1. (a) Show that f (g) if and only if f / O(g).
(b) Show that f (g) if and only if f / o(g).
Exercise 1.6.2. Prove by formula induction that every propositional formula
can be put into conjunctive normal form.
Exercise 1.6.3. Suppose that p > 2 is prime. In this exercise we show that
there exists a primitive root modulo p
2
. Let r be a primitive root modulo p.
(a) Show that r is a primitive root modulo p
2
if both r
p
1 mod p
2
and
r
p1
1 mod p
2
.
(b) Show that r
p
1 mod p
2
.
(c) Show that (r +p)
p
r
p
mod p
2
. Conclude that also (r +p)
p
1 mod p
2
.
(d) Suppose that r is not a primitive root modulo p
2
. Show that in that case
r + p is a primitive root modulo p
2
.
Chapter 2
Basics
2.1 Time and space bounds
In the following we will consider computations with bounded resources. Usu-
ally, the complexity of computations is measured in the size of the input,
which is usually its length when represented as a binary string. This means
in particular that the size of a number n N is of order log n. We only want
to consider resource bounds that are reasonable in the following sense.
t : N N is time constructible if there is a Turing machine which on
every input of length n halts in precisely t(n) steps.
s : N N is space constructible if there is a Turing machine that on
every input of length n halts in a conguration with exactly s(n) non-
blank tape cells and that used no other cells during the computation.
1
For such reasonable resource bounds it is possible to construct an eective
enumeration of all Turing machines that are clocked with the given bound.
Thus we can avoid decidability problems arising when considering arbitrary
Turing machines. From now on we assume that all complexity bounds are
time or space constructible. What may happen in absence of this assumption
is discussed in section 5.5.
Given such constructible resource bounds we dene the following classes:
TIME(t) is the class of all sets that are accepted by a Turing machine
within running time t(n).
1
Note that an input of length n automatically uses n tape cells of the input tape. In
order to let sublinear space bounds such as log n make sense, it is customary in complexity
theory to equip Turing machines with a separate input tape, and to not count the input
tape when counting the space used in a computation.
6
2.2. Inclusions between classes 7
NTIME(t) is dened as TIME(t), but now with nondeterministic Turing
machines. By denition a nondeterministic machine accepts an input
when some computation path accepts it.
SPACE(s) is the class of all sets that are accepted by a Turing machine
using space at most s(n) for all inputs of length n.
NSPACE(s) is dened as SPACE(s), but now with nondeterministic
Turing machines.
2.2 Inclusions between classes
We have the following inclusions between classes:
TIME(t) NTIME(t), SPACE(s) NSPACE(s).
TIME(t) SPACE(t), NTIME(t) NSPACE(t). This holds because
a machine cannot use more space than time.
SPACE(c s) SPACE(s) for any c N. This result is called the
tape compression theorem. The idea of the proof is to increase the tape
alphabet by introducing a symbol for every block of length c. Idem for
NSPACE. The result means that we can work with O(s) rather than s
in the future.
TIME(c t) TIME(t) for any c N and t with n o(t), i.e. such
that r > 0
n(n < r t(n)). This result is called the linear speed-

up theorem. The proof is slightly more complicated than that of the
previous item, but the idea is similar: again use symbols for c-blocks,
and simulate several steps with one by monitoring the actions of the
tape head within a given region.
NTIME(t) SPACE(t). Reuse space! We only need to keep a counter
of length t for the current path.
NSPACE(s) TIME(2
O(s)
). This holds because there are only 2
s(n)
congurations of length s(n). In particular we have NTIME(t)
TIME(2
O(t)
) by the previous item.
NSPACE(s) SPACE(s
2
). This is Savitchs Theorem proved below.
Savitchs Theorem can be proved using that the graph reachability problem
is solvable in space log
2
n, cf. Theorem 2.2.1 and Exercise 2.7.1 below. The
problem PATH (also known as reachability or connectivity) is dened by
Input: a directed graph G with n nodes, x, y G,
Question: is there a path from x to y?
8 Chapter 2. Basics
Theorem 2.2.1. PATH SPACE(log
2
n).
Proof. Let Path(x, y, i) express that there is a path from x to y of length 2
i
.
Since there are n nodes, we will try to determine whether Path(x, y, log n).
If i = 0 we can check Path(x, y, i) directly from the input.
If i > 0, check for all z G whether Path(x, z, i 1) and Path(z, y, i 1).
For the second, we can reuse the space of the rst!
We have to keep a stack of answers to the recursive calls. The depth of the
recursion is log n, and the current value is of size 3 log n (because we work
with triples (x, y, i)), so the cost is at most 3 log
2
n.
Note that in the path problem, the path from x to y can have length n, which
does not t in space log
2
n, so that the algorithm from Theorem 2.2.1 can
really only give a yes/no answer.
2.3 Hierarchy theorems
The following hierarchy theorems were proved by Hartmanis and Stearns in
1965, and can be seen as the start of the eld of complexity theory.
Theorem 2.3.1. (Space Hierarchy Theorem) SPACE(s) SPACE(s
) for
every s
(s), i.e. such that c > 0
n (s
(n) > c s(n)).

Theorem 2.3.2. (Time Hierarchy Theorem) TIME(t) TIME(t
) for every
t
(t log t). (The extra log t is needed for the simulation of many tapes.)
Proof of the hierarchy theorems. We only sketch the proof of the time hier-
archy theorem, since the space version is similar. Given time bounds t and t
as above, consider an eective enumeration {M

i
}
iN
of all Turing machines
working in time t. (Note that we use the time-constructibility of t here.) On
input w = 1
i
0u run M
i
(w) for t
(|w|) steps. If M
i
halts within this time, let
M(w) = 1 M
i
(w), and let M(w) = 0 otherwise. Then M = M
i
for every i,
because t
is innitely often large enough to complete a computation of M

i
.
The proof just given is of course a variant of Cantors famous diagonal ar-
gument showing that the set of reals is uncountable. The diagonalization
method has been very fruitful in computability and complexity theory. We
will come back to it in Chapter 5.
There are also hierarchy theorems for the nondeterministic classes. The
problem is how to do the diagonalization step M(w) = 1 M
i
(w) from the
proof above. In case of nondeterministic space, this can be arranged in a nice
way with an appeal to Theorem 2.6.1. The following hierarchy theorem for
nondeterministic time was proven by Cook [10].
Theorem 2.3.3. NTIME(n
2
) NTIME(n).
2.4. Central complexity classes 9
Proof. Let {M
i
}
iN
be an enumeration of all nondeterministic Turing ma-
chines working in linear time. Such an enumeration can certainly be con-
structed in time n
2
, that is, there is a nondeterministic Turing machine work-
ing in time n
2
that, given a code i and input x, simulates M
i
(x). We want
to construct a nondeterministic machine M working in time n
2
such that for
every i the requirement
R
i
: x
(
M(x) = M
i
(x)
)
is satised. Given i, it seems problematic to satisfy R
i
directly, as in the proof
of Theorem 2.3.2. Instead we pick a fresh candidate witness x and guarantee
the following:
(i) M(y) = M
i
(y + 1) for all y [x, 2
x
),
(ii) M(2
x
) = M
i
(x).
Item (ii) can be easily achieved by direct diagonalization, since in time 2
n
there is enough time to simulate all computation paths of M
i
(x). To see
that this suces, suppose that M
i
(x) = M(x) for all x. Then in particular
M
i
(y) = M
i
(y + 1) for every y [x, 2
x
) by (i), and hence M
i
(x) = M
i
(2
x
).
But by (ii) we have M
i
(2
x
) = M
i
(x), a contradiction.
To satisfy all the requirements R
i
in one construction, all we need to do is
pick the witnesses x in such a way that the intervals [x, 2
x
] are disjoint.
Using a more careful analysis, a somewhat tighter version of the previous
hierarchy theorem can be obtained. Of course, the proof generalizes also to
larger time bounds.
2.4 Central complexity classes
There are many important complexity classes, but any list of them would
certainly include the following.
P =
i
TIME(n
i
) is the class of polynomial time computable sets.
NP =
i
NTIME(n
i
) is the class of nondeterministic polynomial time
computable sets.
PSPACE =
i
SPACE(n
i
) is the class of polynomial space computable
sets.
EXP =
i
TIME(2
n
i
) is the class of exponential time computable sets.
Note that by the results of the previous section we have
P NP PSPACE EXP (2.1)
10 Chapter 2. Basics
n
c
accept/reject
Figure 2.1: A nondeterministic computation
and that by Savitchs Theorem it holds that PSPACE = NPSPACE (since
(n
i
)
2
is still polynomial). Note also that by the time hierarchy theorem,
P EXP, hence some of the inclusions in (2.1) must be strict! Unfortunately,
we have no clue which ones, but the conjecture is that in fact all the inclusions
are strict.
One of the central themes in complexity theory is the dierence between
determinism and nondeterminism, and the tradeo between time and space.
This translates into the most prominent questions P
?
= NP and P
?
= PSPACE.
P is often identied with the class of problems that are solvable in reason-
able time, although it may of course depend on the context what one nds
reasonable. In contrast, NP corresponds to the class of problems of which it
may be hard to nd solutions, but of which one can eciently check whether
a candidate solution really is one. Because it does seem not very useful if
one cannot even check whether a solution is correct, one could with a bit
of exaggeration say that NP is the largest class of problems that one would
be interested in knowing the answers to. Given a nondeterministic Turing
machine M working in time n
c
, one can view a computation of M as a binary
tree of depth n
c
, so that every binary string of length n
c
corresponds with
a computation path, and acceptance means that there is an accepting path,
cf. Figure 2.1. (Cf. also Exercise 2.7.4.) In discussing NP-problems one of-
ten says: guess a path and then perform the polynomial computation. An
accepting path is sometimes referred to as a certicate for the acceptance of
the computation.
2.5 Problems from logic, algebra, and graph theory
The satisability problem SAT is dened by
Input: a Boolean formula ,
2.5. Problems from logic, algebra, and graph theory 11
Question: is there a satisfying assignment of ?
It is easy to check that SAT NP: Given of length n, has at most n
propositional variables. Nondeterministically guess an assignment, and check
that it satises . If so, accept, if not, reject.
The problem PRIMES is dened by:
Input: n N,
Question: is n prime ?
Obviously, PRIMES is in co-NP (guess a decomposition). It was a longstand-
ing open problem whether PRIMES is in P, until it was nally solved in [1].
The proof is too long to include here, but we prove instead the weaker result
that PRIMES is in NP. Note that NPco-NP is not known to be equal to P.
In section 6.4 we will discuss a randomized algorithm for PRIMES.
Theorem 2.5.1. (Pratt) PRIMES is in NP.
Proof. We use Theorem 1.5.5, namely that p > 2 is prime precisely when
there is a primitive root modulo p, that is, an element 1 < x < p of order
p 1:
(i) x
p1
1 mod p, and
(ii) x
p1/q
1 mod p for every prime factor q of p 1.
Now given p, guess x and verify (i) and (ii). (i) poses no problem. To check
(ii), guess a prime factorization of p 1 and recursively check that all the
factors q are prime and that (ii) holds.
Since graphs can be used to model so many discrete problems, it should not
come as a surprise that numerous problems in complexity theory come from
graph theory. Here are two examples.
The HAMILTON PATH problem is dened by:
Input: a graph G,
Question: does G have a Hamilton path, i.e. a path that visits every node
exactly once?
HAMILTON PATH is in NP: simply guess a path of length |G| and check
that it is Hamilton.
The traveling salesman problem TSP is dened by:
Input: Given a complete weighted graph G, that is, every two nodes i and
j have a distance d
i,j
(with d
i,j
= d
j,i
), and k N,
Question: is there a tour of size k ? (A tour is a path without repetitions
visiting all nodes.)
2
TSP is in NP: Guess a tour and check.
2
Note that the parameter k is used to phrase the problem as a decision problem. For-
mally, all our complexity classes are classes of sets, that can be thought of as coding deci-
sion problems with yes/no answers. One can also formulate complexity classes of functions
rather than sets, but we will not do this here.
2.6 The Immerman-Szelepcsenyi Theorem
The following theorem was proven independently in 1987 by Immerman [16]
and Szelepcsenyi [25]. It says that the nondeterministic space classes are
closed under complementation. Note that if A NSPACE(f), an application
of Savitchs Theorem would only give that A NSPACE(f
2
).
Theorem 2.6.1. Let f be a space constructible function such that f(n)
log n. Then NSPACE(f) = co-NSPACE(f).
Proof. Suppose that machine M accepts A in nondeterministic space f(n).
Note that on a given input x of length n there are 2
f(n)
possible congurations.
We dene a nondeterministic machine M that accepts the complement A,
i.e. x / A if and only if M has a path accepting x. The idea is to cycle
through all possible congurations , for each one checking whether it is
a nal conguration that is reachable from the initial conguration by a
computation of M. In case no such accepting conguration is found we know
x / A so we let M accept x. The problem is that to see if is reachable we
can only guess a path of M.
3
If a wrong path was guessed we may falsely think
that was unreachable. (Note that we will never think that an unreachable
is reachable.) Hence, in doing so for every we may underestimate the
number n(x) of reachable s. If we knew the number n(x) in advance, we
could match the number of s found in the above procedure against it, and
let M reject x in case the numbers did not match. In case the numbers
matched we would know we had seen all reachable congurations, and if none
of them were accepting we could safely declare x / A. If indeed x / A, for
some path of M all the guessed paths for the would be correct and we
would correctly declare x / A.
So it remains to be shown that we can compute the number n(x) of con-
gurations that are reachable by M from the initial conguration. This
is done by inductive counting. Let m(x, t) be the number of congura-
tions that are reachable in at most t steps. We can deterministically compute
m(x, 1). Suppose that we know m(x, t). Conguration is reachable in t +1
steps if there is a reachable in t steps such that is reachable from in
one more step. As above, cycle through all possible , each time guessing a
path to reach . For every found to be reachable, and for each reachable
from in one step, increase the counter m(x, t + 1) by one. If at the end
the number of s found does not match m(x, t), reject x (since we know
a mistake was made). If there is a match with m(x, t) we know that also
m(x, t +1) is correct. Finally, n(x) is equal to m(x, t) for the rst t such that
m(x, t) = m(x, t + 1).
3
Note that trying all paths is too expensive: with 2
f(n)
possible congurations there
are simply too many paths.
2.7. Exercises 13
Theorem 2.6.1 has an important consequence for the theory of formal lan-
guages, namely that the class of languages that can be generated with a
context-sensitive grammar is closed under complements. This had been an
open problem for about 25 years.
Corollary 2.6.2. The class of context-sensitive languages is closed under
complements.
Proof. It is a result of formal language theory that the class of context-
sensitive languages is precisely the class of sets accepted by linearly bounded
automata, that is, NSPACE(n), so the result follows from Theorem 2.6.1.
Note that while Theorem 2.6.1 shows the closure of the nondeterministic space
classes under complementation, the same result for the nondeterministic time
classes remains one of the fundamental open problems in complexity theory.
It seems unlikely that a similar result holds in this case, e.g. that NP = co-NP
would hold.
2.7 Exercises
Exercise 2.7.1. Derive Savitchs Theorem from Theorem 2.2.1. (Hint: Given
a nondeterministic machine we can apply Theorem 2.2.1 to the conguration
graph of the machine on a given input by asking whether an accepting con-
guration is reachable.)
Exercise 2.7.2. The following problem is related to PATH. Suppose we are
given the adjacency matrix A of a binary relation on n points. Show that
it is possible to construct the adjacency matrix of the transitive closure of
in polynomial time. (Hint: A
k
contains the information whether from node i
we can reach node j in k steps.)
Exercise 2.7.3.
In this exercise we prove that the class of regular sets equals

SPACE(k), the class of sets recognizable in constant space. By denition, the
regular sets are recognizable by one-way Turing machines (i.e. tape heads can
only move right) with a read-only input. Note that SPACE(k) = SPACE(1)
by the tape compression theorem.
(a) Show that regular sets can be recognized using constant space. (Recall
from page 6 that the space taken by the input does not count.)
(b) Given L, dene an equivalence relation on
by x
L
y precisely
when for all z, xz L yz L. Prove the Myhill-Nerode Theorem: L is
regular if and only if
L
has only nitely many equivalence classes.
(c) (Shepherdson) Prove that two-way machines with a read-only input
and working in constant space can be simulated by one-way machines. (Hint:
For the equivalence dened in (b) we have x
L
y if and only if x and y
induce the same mapping of states of M as follows. Dene mappings f
x
and
g
x
by f
x
(q) = p if whenever M enters x at the right in state q it leaves x at
the left in state p. Similar for g
x
but now from right to left. Acceptance of
xz depends only on f
x
and g
x
and not on x, and there are only nitely many
such functions since the number of states is nite.)
Exercise 2.7.4. In general the computation tree of a nondeterministic com-
putation in NTIME(n
c
) is k-splitting, where k is the number of states in the
Turing machine. Argue that we may think of the tree as a binary tree, where
every path has length log kn
c
.
Exercise 2.7.5. Give a more formal proof of PRIMES co-NP.
Exercise 2.7.6. Deciding whether gcd(x, y) = 1 is in P. (Hint: The Eu-
clidean algorithm uses at most a logarithmic number of divisions x = ny +r,
since each time the remainder r is
1
2
x.)
Exercise 2.7.7. TSP was dened using tours, i.e. paths without repetitions.
(a) What is the total number of tours in TSP?
(b)
Note that, despite the bound from (a), TSP is solvable in time 2
O(n)
.
Exercise 2.7.8. Show that NP is closed under Kleene star.
Exercise 2.7.9.
Show that P is closed under Kleene star. This is much

harder than the previous exercise! (Hint: Given L P and a string w let
w[i, j] denote the substring from bit i to j. Dene a binary relation on
pairs (i, j) by (i, j) (i
, j
) if i
= j + 1 and w[i, j] and w[i
, j
] are both in
L. Use Exercise 2.7.2 to compute the transitive closure of and express the
problem of membership in L
in terms of it.)
Exercise 2.7.10. Is every subset of a set in NP automatically in NP ?
Exercise 2.7.11. A function g : {0, 1}
{0, 1}
is called honest if there is

a polynomial p such that p(|g(x)|) |x|, that is, g cannot shrink its input
by more than a polynomial factor. This exercise indicates the interest in the
notion of honesty for complexity theory.
(i) By denition, a set is computably enumerable (c.e.) if it is empty or
the range of a computable function. Show that a set A is c.e. if and
only if it is the range of a polynomial time computable function g, i.e.
A =
{
y : x g(x) = y
}
.
(ii) Show that a set A is in NP if and only if it is the range of an honest
polynomial time computable function g.
Chapter 3
Reductions and completeness
One of the goals of complexity theory is to classify problems according to their
complexity. The main tool for doing this is to consider eective reductions
between problems. A key insight is that classes such as NP contain hardest
problems. Reductions also allow us to substantiate the idea that various
problems, though dierently formulated, are actually the same.
3.1 Many-one reductions
Denition 3.1.1. Given sets A and B, A is many-one reducible or simply
m-reducible to B, written A
m
B, if for some computable function f,
x A f(x) B
for every x. If f is in addition polynomial time computable then we say that
A is p-m-reducible to B, written A
p
m
B. We write A
p
m
B if both A
p
m
B
and B
p
m
A. The set
{
B : A
p
m
B
}
is called the p-m-degree of A.
The join A B of two sets A and B is dened as
A B =
{
2x : x A
}
{
2x + 1 : x B
}
.
The set AB contains precisely all the information from A and B zipped
together.
Proposition 3.1.2. (i)
p
m
is reexive and transitive.
(ii) A
p
m
B precisely when A
p
m
B.
(iii) If A P then A
p
m
B for every set B that is not empty or equal to
{0, 1}
.
(iv) If both A
p
m
C and B
p
m
C then A B
p
m
C. Since clearly A,
B
p
m
AB this shows that on the p-m-degrees gives the least upper
bound of A and B.
15
16 Chapter 3. Reductions and completeness
Proof. Exercise 3.5.1.
The graph problem 3COLORING is dened by
Input: a graph G,
Question: can G be colored with 3 colors, that is, does there exist a 3-
coloring of the vertices such that no pair of adjacent vertices have the same
color?
Proposition 3.1.3. 3COLORING
p
m
SAT.
Proof. Suppose the given graph G has k vertices. Introduce variables x
i,j
,
i = 1, . . . , k and j = 1, 2, 3, meaning vertex i has color j. We dene an
instance of SAT by the following set of clauses:
C(i) =
{
x
i,1
, x
i,2
, x
i,3
}
, i = 1 . . . k, every vertex has a color
T(i) =
{
x
i,1
, x
i,2
}
, i = 1 . . . k, i not both color 1 and 2
U(i) =
{
x
i,1
, x
i,3
}
,
V (i) =
{
x
i,2
, x
i,3
}
.
Together these clauses express that every vertex has exactly one color. Next,
for every edge e from u to v and every color j dene
D(e, j) =
{
x
u,j
, x
v,j
}
that is, u and v should not both have color j. We can write down these
clauses in linear time, and the set of clauses is satisable precisely when G is
3-colorable.
Proposition 3.1.3 is an example of the expressivity of SAT.
Denition 3.1.4. A set A is called hard for a class C if C
p
m
A for every
C C. A is complete for C if in addition A C.
Note that if A is hard and A
p
m
B then also B is hard. Also note that all
the classes P, NP, co-NP, PSPACE are downwards closed under
p
m
, that is,
if A
p
m
B and B is in the class then also A belongs to it.
The notion of completeness is one of the central notions of our subject.
It allows us to study complexity classes by focussing on certain individual
elements in them. Before we can do so, we rst have to show that such
complete elements exist.
3.2. NP-complete problems 17
3.2 NP-complete problems
One can easily dene an NP-complete problem as follows. Let
K =
{
M, x, 1
t
: M is a nondeterministic TM that accepts xin t steps
}
.
It is debatable whether K is a natural NP-problem. Of course, K is nat-
ural in our given context of complexity theory. It is the direct analogue of
the halting problem in computability theory. However, sometimes the term
natural is reserved for problems that arose in some other context and were
not dened directly using the notions at hand.
Theorem 3.2.1. K is NP-complete.
Proof. K NP: Given input M, x, 1
t
, guess a computation path of M of
length t and check whether it accepts x. This takes time O(n
2
).
K is NP-complete: Let A be an arbitrary set in NP. We show that
A
p
m
K. Let M be a nondeterministic machine accepting A in time p(n).
Let f(x) = M, x, 1
p(|x|
. Then x A f(x) K.
The following problem has perhaps a better claim to the label natural. It
is surely the most famous NP-complete problem, and it was also the rst one.
Theorem 3.2.2. (Cook)
1
SAT is NP-complete.
Proof. Let L(M) be a language in NP accepted by machine M working in
time p. We can eectively dene the following Boolean formulas. Let and
denote vectors of Boolean variables.
1. Initial(, x) expresses that codes an initial conguration of M on input x.
2. Next(, ) expresses that is the conguration following after one com-
putation step of M.
3. Accept() expresses that codes an accepting conguration.
Now combine these formulas to form
Accepted(x) = Initial(
1
, x)
p(n)1
i=1
Next(
i
,
i+1
) Accept(
p(n)
).
Note that we can write this formula down in polynomial time, and that
it is satisable precisely when there are assignments
i
encoding a com-
plete accepting computation of M on x. Hence x L(M) if and only if
Accepted(x) SAT.
1
Because of similar work by Levin this is sometimes called the Cook-Levin theorem.
3.3 More decision problems from logic
We already encountered the set SAT of all satisable Boolean formulas. Con-
sider the following variants of SAT.
SAT-CNF is the subset of SAT consisting of all satisable formulas that
are in conjunctive normal form.
nSAT is the subset of SAT-CNF where every clause has exactly n vari-
ables.
QBF is the set of satisable quantied Boolean formulas. The set of
q.b.f.s is dened by
(i) 0,1,x (variable) are q.b.f.s,
(ii) the set of q.b.f.s is closed under , , ,
(iii) if F is a q.b.f. then also xF and xF are q.b.f.s.
xF is true if F with x = 0 is true or F with x = 1 is true. xF is true
if both F with x = 0 is true and F with x = 1 is true. As for SAT,
a q.b.f. is satisable if there is an assignment of the free variables that
makes the formula true. Note that it also makes sense to talk about the
truth of closed q.b.f.s, i.e. those without free variables. E.g. x(xx)
is true.
Note that putting a Boolean formula in CNF using the distributive law
( ) ( ) ( )
can be expensive: it may result in an exponential blow-up of the formula
because of the repetition of on the right hand side. For QBF, however, we
have the following result:
Proposition 3.3.1. For every Boolean formula F of size m and containing
variables x
1
. . . x
m
there is an equivalent q.b.f. y
1
. . . y
k
F
with variables
x
1
. . . x
m
, y
1
. . . y
k
such that F
is in CNF and of size at most c m

2
, where c
is a constant independent of F.
Proof. Without loss of generality occurs only directly in front of variables
in F. The proof is by induction on F:
The base case where F is a literal or a variable is clearly o.k.
F = F
1
F
2
. By induction F
1
and F
2
have the required form. Now just
put their s in front, after a possible renaming of variables.
F = F
1
F
2
. By induction F
1
= yF
1
and F
2
= zF
2
. Introduce a new
variable x to indicate whether F
1
or F
2
is true: Convert every clause C in F
1
3.3. More decision problems from logic 19
to x C and every C in F
2
to x C and let F
be the conjunction of all

these new clauses. Then xyzF
is equivalent to F. Note that the -step

adds a constant to the size of every clause, hence at most c m to the formula.
Since there are at most m such steps, the size of the new formula is bounded
by c m
2
.
Corollary 3.3.2. SAT
p
m
SAT-CNF. Hence SAT-CNF is NP-complete.
Proof. Given F, Proposition 3.3.1 gives an equivalent q.b.f. yF
with F
in
CNF. Now just drop the s: F SAT if and only if F
SAT.
The problem CLIQUE is dened by
Input: a graph G and k N,
Question: does G have a complete subgraph of size k ?
Theorem 3.3.3. CLIQUE is NP-complete.
Proof. That the problem is in NP is obvious: simply guess a subgraph of
size k. We show that SAT-CNF
p
m
CLIQUE. Given a formula F with literals
x
1
, . . . , x
r
and clauses C
1
, . . . , C
s
, let
V =
{
(x
i
, C
j
) : x
i
C
j
}
E =
{(
(x
i
, C
j
), (x
m
, C
n
)
)
: j = n x
i
= x
m
}
k = s
We leave it as Exercise 3.5.12 to check that this is a p-m-reduction.
Theorem 3.3.4. 3SAT is NP-complete.
Proof. We show that SAT-CNF
p
m
3SAT, so the result follows from Corol-
lary 3.3.2. Given an instance of SAT-CNF, replace the clauses with more
than 3 literals as follows: Replace
{
x
1
, x
2
, . . . , x
k
}
by the k 2 new clauses
{
x
1
, x
2
, z
1
}
,
{
x
3
, z
1
, z
2
}
,
{
x
4
, z
2
, z
3
}
, . . .
{
x
k1
, x
k
, z
k3
}
. (3.1)
If an assignment of x, z satises (3.1) then x alone satises
{
x
1
, x
2
, . . . , x
k
}
.
Namely, suppose that all x
i
are false. Since in (3.1) they occur only positively
also
{
z
1
}
,
{
z
1
, z
2
}
,
{
z
2
, z
3
}
, . . .
{
z
k4
, z
k3
}
,
{
z
k3
}
are satised, contradiction. Conversely, if x
i
is true in
{
x
1
, x
2
, . . . , x
k
}
, then
assign z
j
= true for j i 2 and false for j > i 2 to satisfy (3.1).
In contrast to Theorem 3.3.4 we have
Theorem 3.3.5. 2SAT is in P.
The proof of the following theorem has a certain similarity with the proof of
Savitchs theorem.
Theorem 3.3.6. QBF is PSPACE-complete.
Proof. First note that QBF SPACE(n): try all (linear size) assignments,
using a (linear sized) counter.
For the completeness, let M be a given machine with space bound p. A
computation of M on an input of size n has at most 2
p(n)
congurations, and
is accepting if an accepting conguration is reachable in at most 2
p(n)
steps
from the initial conguration. So the theorem follows if we can eciently
construct a q.b.f. Access
2
m(, ) expressing that and code congurations
of M and is reachable from in 2
m
steps. We already saw in the proof of
Theorem 3.2.2 that there are Boolean formulas expressing that is an (initial
or accepting) conguration of M.
m = 0: We can use the formula Next(, ) from the proof of Theorem 3.2.2
to express that is reachable from in at most one step.
m > 0: Given Access
2
m1(, ), a bad attempt would be to write
Access
2
m(, ) =
(
Access
2
m1(, ) Access
2
m1(, )
)
because then the length of the formula doubles with every step. Instead write
Access
2
m(, ) =
(
(
= ) (
= )
Access
2
m1(
)
)
.
Note that the extra quantication is used so that we have to write the previous
formula Access
2
m1 only once instead of twice. This makes the recursion a
linear aair.
3.4 Completeness of Hamilton path and TSP
We can view reductions as a translation from the language of the easier prob-
lem to that of the harder one. As we have seen, these reductions are a useful
tool in proving that certain problems are complete, using problems that we
already know to be complete. The reductions we have seen so far were rather
easy to describe. In this section we give here an example of a more elaborate
reduction, showing completeness of HAMILTON PATH and TSP.
Theorem 3.4.1. HAMILTON PATH is NP-complete.
3.4. Completeness of Hamilton path and TSP 21
Figure 3.1: The truth module

Figure 3.2: The clause module
a
0
a
1

b
0
b
1

Figure 3.3: The XOR module

Proof. We follow the proof in [22]. That the problem is in NP was already
discussed on page 11. We prove completeness by showing that 3SAT
p
m
HAMILTON PATH. Suppose that is an instance of 3SAT, with variables
x
1
, . . . , x
n
and clauses C
1
, . . . , C
m
, each with three variables. We construct an
instance f() of HAMILTON PATH by piecing together some basic modules.
Truth or falsity of a variable is coded by the truth module from Figure 3.1.
Any Hamilton path has to choose between the left and the right edge. We
let the left one stand for true and the right one for false. We represent
the clauses by triangles, as in Figure 3.2, where each side of the triangle
represents a literal from the clause. We will arrange things such that only
sides for which the corresponding literal is false have to be traversed. Note
that by denition no Hamilton path can traverse all sides of a triangle. Hence
if all literals are false there will be no Hamilton path.
Next consider the module on the left hand side of Figure 3.3. Any Hamil-
ton path that does not start or end in a node of this module has only two
possible ways of routing through it: Either entering in a
0
and exiting in a
1
or
entering in b
0
and exiting in b
1
. (Cf. Exercise 3.5.16.) Thus the module acts
as a kind of XOR gate forcing a choice between the upper or the lower level.
We abbreviate this module by the picture on the right of Figure 3.3.
We put all the pieces together as follows. The graph f() has n truth
modules, one for each variable x
1
, . . . , x
n
, connected in series, as on the left
side of Figure 3.4. Call the rst node 1 and the last one 2. It also has m
triangles, one for each clause. Every side of a triangle represents a literal in
the clause. If the literal is x
i
, we connect the side with a XOR module to the
true side of the truth module of x
i
, and if the literal is x
i
we connect it
to the false side. Also, we add an extra node 3. We pairwise connect all
3m nodes of the triangles plus nodes 2 and 3. Finally, we add a node 4 only
connected to node 3. An example of the nal graph thus obtained is pictured
in Figure 3.4.
We verify that the construction works. Suppose that the graph f()
possesses a Hamilton path. Then its two ends must be 1 and 4. Without
loss of generality suppose it starts in 1. The only way to get to node 2 is to
traverse the series of truth modules. This denes a valuation v of the variables
x
i
. We claim that v satises . Since no Hamilton path can traverse all edges
of a triangle, every triangle has an untraversed edge, which by use of the
XOR module means that the corresponding literal is true under v. Hence
is satised.
Conversely, suppose that there is a valuation v satisfying . Then f()
has a Hamilton path as follows. First traverse the truth modules according
to v. Since every clause has a literal that is true under v, for no triangle we
have to traverse all edges. Since all edges between the nodes of the triangles
and nodes 2 and 3 are present, we easily see from this that we can reach 4
visiting all nodes.
Corollary 3.4.2. TSP is NP-complete.
Proof. We show that HAMILTON PATH
p
m
TSP. Given a graph G with n
nodes, construct an instance f(G) of TSP by letting the cities be the nodes
of G. Let the distance between two cities be 1 if the nodes are connected in
G, and 2 otherwise. Then G has a Hamilton path if and only if there is a
tour of length at most (n 1) 1 + 2 = n + 1.
3.5 Exercises
Exercise 3.5.1. Prove Proposition 3.1.2.
Exercise 3.5.2. (a) Show that for every L in PSPACE there is an L
in
SPACE(n) such that L p-m-reduces to L
. (Hint: dene L
by prexing the
elements of L with strings of polynomial length. This technique is called
padding.)
(b) Suppose that C is closed downwards under p-m-reducibility and that
SPACE(n) C PSPACE. Show that C = PSPACE.
(c) Show that P = SPACE(n) and that NP = SPACE(n). (Hint: use (b)
and the space hierarchy theorem.)
Exercise 3.5.3. Use the time hierarchy theorem to show that TIME(n
k
) is
not closed downwards under
p
m
.
3.5. Exercises 23
Exercise 3.5.4. Dene analogous versions of the set K that are p-m-complete
for co-NP, PSPACE, and EXP.
Exercise 3.5.5. Think about the details of Theorem 3.2.2.
Exercise 3.5.6. Show that P = NP if and only if SAT P.
Exercise 3.5.7. In a 1956 letter to von Neumann (reprinted in [24]) Godel
mentioned the following problem: Given a predicate formula and n N,
does have a proof (in a xed proof system for predicate logic) of length at
most n ? Godel explicitly mentions the possibility that the problem could
be solvable in quadratic time, as opposed to the exponential bound resulting
from brute force search. Show that the problem is in NP, and that it is
NP-complete by reducing SAT to it.
Exercise 3.5.8. A set A is called self-reducible if A
p
m
A via a reduction f
such that f(x) = x for all x. Show that SAT is self-reducible.
Exercise 3.5.9. There are two exponential time classes in the literature:
E =
c
TIME(2
cn
)
EXP =
c
TIME(2
n
c
).
These are often abbreviated to E = 2
linear
and EXP = 2
polynomial
. Show that
EXP is equal to the downward closure of E under
p
m
.
Exercise 3.5.10. Let LIN = TIME(n), and let NLIN = NTIME(n). It is
known that LIN = NLIN, see e.g. [4] for a proof. In this exercise we prove
that P = NLIN.
(i) If NLIN P then P = NP.
(ii) If P NLIN then P = NP.
(iii) P = NLIN.
Exercise 3.5.11. Positive reductions are a generalization of m-reductions
dened as follows: A
p
pos
B if A = M
B
for a deterministic Turing machine
M working in polynomial time such that
C D = M
C
M
D
.
Show that NP is downwards closed under
p
pos
.
Exercise 3.5.12. Finish the proof of Theorem 3.3.3.
Exercise 3.5.13. Prove Theorem 3.3.5, namely that 2SAT is in P. (Hint:
Given , draw a directed graph of all literals, with an arrow from x to y if
{ x, y} is a clause of . Argue that is satisable if and only if there is no
literal x for which there is both a path from x to x and from x to x. Then
use Exercise 2.7.2.)
Exercise 3.5.14. Let kDNF be the set of all satisable formulas in disjunc-
tive normal form (i.e. written as a disjunction of conjunctions) where every
conjunct has at most k literals. Show that kDNF is in P for every k.
Exercise 3.5.15. HORNSAT is the subset of SAT-CNF consisting of the
satisable Horn formulas, i.e. those where every clause contains at most one
positive literal. Show that HORNSAT is in P. (Hint: Given , dene the
minimal truth assignment T satisfying . Hence is satisable if and only
if T satises .)
Exercise 3.5.16. Check the claim from Theorem 3.4.1 that there are only two
possible ways for a Hamilton path to traverse the module of Figure 3.3, given
that the path enters in either a
0
or b
0
and the internal nodes of the module
(i.e. those dierent from the end points of the module) are not connected to
any nodes outside the module.
Exercise 3.5.17. Dene the problem HAMILTON CIRCUIT by:
Input: a graph G,
Question: does G have a cycle, starting and returning in the same node
and visiting every other node exactly once?
Dene the variant H by:
Input: graph G and two nodes x
0
, x
1
G,
Question: does G have a Hamilton path starting in x
0
and ending in x
1
?
Show that the problems HAMILTON PATH, HAMILTON CIRCUIT, and H
all have the same p-m-degree.
3.5. Exercises 25
1 4
x
1
3
x
2

x
3
.
.
.
.
.
.
.
.
.
.
.
/

.
.
.
.
.
Figure 3.4: Graph for the formula
(
x
1
x
2
x
3
)
(
x
1
x
2
x
3
)
. The
upper triangle is for the rst clause, the lower for the second. Also, every
pair of black dots is connected.
Chapter 4
Relativized computation and the
polynomial hierarchy
4.1 Relativized computation
One of the central notions in computability theory is Turings notion of rela-
tive computability, dened using oracle Turing machines. Such a machine has
an extra tape on which the (innitely many) bits of a oracle set can be writ-
ten, and that can be used during the computation. The information coded
by this oracle set can be thought of as given for free, and the sets computable
in this fashion as being computable relative to the oracle set. The resource
bounded version of this notion is dened as follows. Given an oracle machine
M, let M
B
denote the function computed by M using oracle B. Then a set
A Turing reduces to B in polynomial time, denoted A
p
T
B, if A = M
B
for
some machine M working in polynomial time. Here the queries to the oracle
B count as a single computation step.
Note that we have that
A
p
m
B = A
p
T
B.
Turing reducibility is much more liberal than many-one reducibility: To an-
swer questions of the form x A? we not only can query the oracle B more
than once, but the queries can also be adaptive, i.e. depend on the answer
to previous queries. By diagonalization (similar to the proof of the hierarchy
theorems) one can show that
p
T
diers from
p
m
on the computable sets, or
even EXP. It is open if the two notions are dierent on NP. Of course, if
P = NP they are the same on NP. By Exercise 4.4.1, NP is closed under
p
T
if and only if NP = co-NP. Hence if NP = co-NP then
p
T
and
p
m
dier on
NP.
The notion of reducibility
p
T
satises the same properties as those of
p
m
listed in Proposition 3.1.2, except that now we have in addition
A
p
T
B = A
p
T
B.
26
4.2. The Polynomial Hierarchy 27
Again, A B is a least upper bound operator. We also have an equivalence
relation
p
T
induced by
p
T
, and
{
B : A
p
T
}
is called the p-T-degree of A.
We employ the following notation in the presence of oracles:
P
A
= P(A) =
{
B : B
p
T
A
}
,
NP
A
= NP(A) =
{
B : B = M
A
for some nondeterministic M
}
,
P
C
= P(C) =
{
P
C
: C C
}
.
Proposition 4.1.1. NP(P
C
) = NP(C).
Proof. () Obvious since C P
C
.
() Suppose B = M
A
0
, where M
0
is a nondeterministic machine, and A = M
C
1
for a C C and a deterministic machine M
1
. Nondeterministically compute
B as follows: guess a computation path of M
0
. For every query to A, insert
the equivalent computation of M
C
1
.
4.2 The Polynomial Hierarchy
We dene a hierarchy, the polynomial time hierarchy, or simply the polynomial
hierarchy, that is the direct analogue of Kleenes arithmetical hierarchy from
computability theory. In the analogy P corresponds to the computable sets
and NP to the computably enumerable sets. The analogy only goes so far.
For starters, we do not know whether P = NP. Also, a set A is computable
precisely if both A and its complement A are c.e. In contrast with this, it is
believed that NP co-NP = P.
Denition 4.2.1. (The Polynomial Hierarchy) We inductively dene a hier-
archy as follows.

p
0
=
p
0
= P.

p
n+1
= NP(
p
n
).

p
n+1
= co-
p
n+1
.

p
n+1
= P(
p
n
).
PH =
n0

p
n
.
Figure 4.1 is a picture of the levels of this hierarchy and their inclusions.
Proposition 4.2.2. (i)
p
1
= P.
(ii)
p
n+1
= co-NP(
p
n
).
(iii)
p
n+1
= NP(
p
n
).
28 Chapter 4. Relativized computation and the polynomial hierarchy
.
.
.
p
n+1
p
n
p
n
p
n
.
.
.
p
3
p
2
p
2
p
2
NP =
p
1
p
1
= co-NP
Figure 4.1: The polynomial hierarchy

(iv)
p
n+1
= P(
p
n
).
(v)
p
n+1
= NP(
p
n+1
).
The levels of the arithmetical hierarchy can be dened using quantication
over previous levels. This is also true for the polynomial time hierarchy. The
analogue of classical quantication is polynomial time bounded quantication.
Denition 4.2.3.
p(n)
xR(x) denotes that there is a string x with |x| p(n)
with property R. If no confusion can arise we will simply write
p
.
p(n)
xR(x)
of course means that for all x with |x| p(n) property R holds. For a given
class C, C denotes the class of sets of the form
{
x :
p(|x|)
y (x, y) B
}
4.2. The Polynomial Hierarchy 29
where p can be any polynomial and B C. The class C is dened analo-
gously.
Proposition 4.2.4. (i) P = NP, P = co-NP.
(ii)
p
n
=
p
n
,
p
n
=
p
n
.
(iii)
p
n
=
p
n+1
,
p
n
=
p
n+1
.
It follows from Proposition 4.2.4 that a set is in
p
n
precisely if it can be
dened by a formula with n quantier alternations, starting with . Similarly
for
p
n
, but now the rst quantier is .
In contrast to the situation in computability theory, we do not know
whether the polynomial time hierarchy is proper, since we do not even know
whether P = NP. Of course, when P = NP then the whole hierarchy collapses
to P, as we invite the reader to check. The conjecture is that PH does not
collapse, i.e. that
p
n
=
p
n+1
for all n. However, it is also possible that
P = NP and that the hierarchy collapses to some higher level. Whatever is
in the hierarchy, it takes place within PSPACE:
Proposition 4.2.5. PH PSPACE.
Proof. The proof is by induction on the levels
p
n
of PH.
n = 0: By denition,
p
0
= P PSPACE.
n+1: By induction hypothesis,
p
n+1
= NP(
p
n
) is included in NP(PSPACE).
But NP(PSPACE) PSPACE(PSPACE) = PSPACE.
By Proposition 4.2.5 we see that PH = PSPACE expresses that PH is as large
as possible. Funnily enough, this implies its collapse, i.e. if PH is large then
it is already large at one of its nite levels:
Proposition 4.2.6. If PH = PSPACE then PH collapses.
Proof. Consider the PSPACE-complete set QBF from Theorem 3.3.6. If
PH = PSPACE then QBF
p
n
for some level n. By completeness of QBF
and
p
n+1
PSPACE this implies that all sets
p
n+1
reduce to a set in
p
n
.
Since
p
n
is closed under p-m-reductions we obtain
p
n+1

p
n
.
There is a world of dierences between the two extremes of p-m-reducibility
and p-T-reducibility. There are situations in which
p
m
is too restrictive, and
p
T
too liberal. There is a whole spectrum of intermediate notions such as 1-1,
truth-table, and bounded truth-table reducibility, that we do not discuss here
for lack of space, but that can be found in many of the standard textbooks.
30 Chapter 4. Relativized computation and the polynomial hierarchy
4.3 Relativization
We have seen that the classes P and NP have relativized versions P
A
and
NP
A
for any oracle A. In general we can relativize any class C that is dened
in terms of Turing machine computations, simply by writing A on the oracle
tape. Given an oracle A, the relativized class is denoted by C
A
.
Often a result about complexity classes holds in fact for all the relativized
versions of the classes as well. In this case we say that the result relativizes.
For example, the strict inclusion P EXP relativizes, i.e. P
A
EXP
A
for
any A. In fact, we can obtain this result by using the same proof as before,
just adding the oracle A in the appropriate places. In this case we say that
the proof relativizes.
Sometimes we need to be more specic about what we mean by rela-
tivization. For example, Theorem 3.2.1 relativizes to: K
A
is NP
A
-complete
for every A. But what do we mean by NP
A
-complete? It could be taken to
mean complete under the unrelativized
p
m
, or complete under m-reductions
computed by functions in P
A
. In this case the stronger version, using unrel-
ativized reductions, holds, but in general one has to be specic about this.
We may not know whether P = NP, but we know something about a
possible proof: Whether P = NP or P = NP, this cannot be proven using a
proof that relativizes. This will follow from the results in section 5.3.
4.4 Exercises
Exercise 4.4.1. NP is closed under
p
T
if and only if NP = co-NP.
Exercise 4.4.2. Show that a class C is countable if and only if there is a set
A that is p-T-hard for C.
Exercise 4.4.3.
Show that the class COMP of all computable sets does not
have p-m-complete sets. Show the same result for p-T-completeness. (You
can either prove this by direct diagonalization, or by using the fact that there
is no universal computable set.)
Exercise 4.4.4. Explain why we did not dene a polynomial space hierarchy.
Exercise 4.4.6. Prove Proposition 4.2.4. (Hint: Item (ii) follows by taking
two quantiers of the same kind together using the pairing function , . For
(iii), prove
p
n+1

p
n
by induction. Separate the positive and the negative
queries.)
Exercise 4.4.7. Let VALID be the set of all valid propositional formulas.
(i) Show that SAT
p
m
VALID.
4.4. Exercises 31
(ii) For which level of PH is VALID complete?
Exercise 4.4.8. Show that the following results relativize:
(i) All of section 2.2,
(ii) All of section 2.3.
Exercise 4.4.9. Show that K
A
is complete for NP
A
under unrelativized p-
m-reductions.
Chapter 5
Diagonalization
Diagonalization is a technique for constructing sets in innitely many stages.
The name derives from Cantors famous diagonal argument showing that the
reals are uncountable. Since Cantor the method has been extended and re-
ned in numerous ways in the eld of mathematical logic. In this chapter
we will see some incarnations of this method in the resource bounded set-
ting. Note that we have already made use of the method in the proofs of the
hierarchy theorems in section 2.3.
5.1 The Halting Problem
As a warm-up, let us consider the proof that the halting problem is undecid-
able. Let {M
e
}
eN
be an eective enumeration of all Turing machines. The
halting set is dened as
H =
{
e, x : M
e
(x)
}
.
Now suppose for a contradiction that H is computable. Then we can dene
a machine M such that for all e, M(e) if and only if M
e
(e). Now let d be a
code of M, i.e. M = M
d
. Then we have M
d
(d) if and only if M
d
(d). From
this contradiction we conclude that H must be noncomputable.
5.2 Intermediate sets
Are there any sets that are neither in P nor NP-complete? Call such sets of
intermediate degree. Of course, in case P = NP then the class of NP-complete
sets coincides with P, so then there are no intermediate sets. A priori it
could be possible that P = NP and that still there are no intermediate sets.
In that case NP would split into the sets in P and the NP-complete sets.
The next result shows that this cannot happen. The result uses a form of the
diagonalization method introduced by Ladner, called delayed diagonalization,
also known as the looking back technique or wait-and-see arguments.
32
5.2. Intermediate sets 33
Theorem 5.2.1. (Ladner [18]) If P = NP then there is a set B NP that is
neither in P nor NP-complete.
Proof. We will in fact ensure that B is not p-T-complete, so that in particular
it is not p-m-complete. Fix a set A that is p-m-complete for NP, and suppose
that A / P (i.e. that P = NP). Fix a polynomial time enumeration {M
e
}
eN
of all polynomial time machines. We construct B in innitely many stages,
while trying to satisfy for every e the requirements
R
2e
: y
(
A(y) = M
B
e
(y)
)
,
R
2e+1
: y
(
B(y) = M
e
(y)
)
.
Note that the even requirements ensure that B is not complete, and the odd
requirements that B is not in P. Call y as in R
d
a witness for the requirement.
The problem is that to see if y is a witness for R
2e
we need to compute A,
which is expensive. We know however that we will succeed if we let B look
like a nite set for long enough. If we wait until we are dealing with larger
inputs (and hence we have more computing time) we can simply look back
and see a point where we succeeded. We deal with R
2e+1
in a similar way,
now letting B look like A to keep it out of P.
At stage x we have dened B up to input x. We dene B(x) as follows.
We use |x| computation steps to compute for as many y = 0, 1, 2, . . . as we
can whether y is a witness for some R
d
with d < x. (We say that y is a
witness for R
2e
at stage x if A(y) = M
B
x
e
(y).) We do this by dovetailing all
pairs y, d in their natural order. Note that since A is hard to compute, this
may not get us very far, but that is o.k.
1
Let d be the least number such that
no witness for R
d
is found. If d = 2e is even that means we are still working
on R
2e
, so we let B(x) = 0. If d = 2e + 1 is odd that means we are still
working on R
2e+1
, so we let B(x) = A(x). This ends the construction of B.
Clearly B NP by construction. We verify that the construction succeeds
in satisfying all requirements. Suppose for a contradiction that some require-
ment is not satised, and let R
d
be the least such. Note that if a witness
for R
d
is never found, it does not exist. If d = 2e is even then B(x) = 0 for
almost every x, and hence, because A(y) = M
B
e
(y) for every y, that A P,
contrary to assumption. If d = 2e + 1 is odd then B(x) = A(x) for almost
every x, and B(y) = M
e
(y) for every y, hence we have again that A P,
contrary to assumption.
Since we do not know that P = NP, of course we cannot prove at the moment
of any particular set A in NP that it has intermediate degree. However, there
are natural candidates for such sets. The most famous one is the problem
1
This is reminiscent of Sternes Tristram Shandy, who needs a full year to describe just
one day of his life. (Tristram Shandy, Vol. 2, Chapter XIII). Slow as this may be, every
day would eventually be described, if he were to live forever.
34 Chapter 5. Diagonalization
GRAPH ISOMORPHISM dened by:
Input: two graphs G
0
and G
1
,
Question: are G
0
and G
1
isomorphic?
Obviously, this problem is in NP (guess an isomorphism). Up to now it
has resisted any attempt to prove that it is in P or that it is NP-complete.
Further evidence for the fact that this problem is not NP-complete is the
result of Sch oning that the set A representing GRAPH ISOMORPHISM is
low
2
, meaning that
p,A
2
=
p
2
. Hence if A were NP-complete we would have
p
3
=
p
2
, hence the polynomial time hierarchy would collapse to the second
level.
According to Theorem 5.2.1 there are at least three p-m-degrees inside
NP, provided of course that P = NP. From Exercise 5.7.1 we may conclude
that in this case there are in fact innitely many p-m-degrees inside NP.
Moreover, by Exercise 5.7.3 these degrees are not linearly ordered. Much
more is known about this world of intermediate degrees that falls outside of
the scope of these notes. In a precise sense, the structure of intermediate
degrees is as complicated as possible.
5.3 Oracle separations
A central theme in computability theory is the study of relativized computa-
tion, that is, the way sets behave in the presence of oracles. We will use the
following basic fact about oracle computations.
Proposition 5.3.1. (Use Principle) Suppose that in the oracle computation
M
A
(x) only queries smaller than n are used, and that B
n = A
n. Then
M
A
(x) = M
B
(x).
Proof. Since B
n = A
n and only queries smaller than n are made, the

oracle B gives the same answers as oracle A, hence the computations are the
same.
Theorem 5.3.2. (Baker, Gill, and Solovay [3]) There exists a computable
set A such that P
A
= NP
A
.
Proof. Let A be any set that is p-m-complete for PSPACE, such as QBF.
Then we have
P
A
NP
A
PSPACE
A
PSPACE P
A
.
Theorem 5.3.3. (Baker, Gill, and Solovay [3]) There exists a computable
set A such that P
A
= NP
A
.
5.4. Many-one versus Turing reductions 35
Proof. For any set A the set
B =
{
0
n
: x {0, 1}
n
(x A)
}
is in NP
A
. We want to construct A such that B / P
A
. As in a typical
diagonalization construction, we split this global requirement into innitely
many subrequirements R
e
. We construct A in stages s. At stage s we have
dened a nite part A
s
of the set A =
sN
A
s
. Let M
e
be an enumeration
of all polynomial time p-T-reductions, and suppose that M
e
works in time p
e
.
We want to satisfy for every e the requirement
R
e
: M
A
e
= B.
Requirement R
e
guarantees that the eth p-T-reduction to A does not compute
B, hence all the requirements together guarantee that B is not in P
A
. We
satisfy R
e
by ensuring that for some n, M
A
e
(0
n
) = B(0
n
). We use the fact that
the polynomial time computation M
A
e
has only time to make a polynomial
number of queries to the oracle A. Changing the oracle outside the set of
queries does not change the computation by the Use Principle 5.3.1, and this
gives us an opportunity to diagonalize. At stage s = e, pick a fresh number
n, that is, a number larger than any number used so far in the construction,
and also ensure that 2
n
> p
e
(n).
Case 1. Suppose that M
A
s
e
(0
n
) = 1. Then we can satisfy R
e
by keeping
A {0, 1}
n
= , so that 0
n
/ B.
Case 2. Otherwise we have M
A
s
e
(0
n
) = 0. Now we want to put 0
n
B,
that is, we have to put something of length n into A. Now there are 2
n
strings
of length n, and the computation M
A
s
e
(0
n
) can make at most p
e
(n) queries
to A
s
, hence there must be a string x {0, 1}
n
that is not queried. Hence,
by the Use Principle, adding x to A
s
does not change the computation. Also,
since n was fresh, adding x does not change any computation from a previous
stage. So in this case, setting
A
s+1
= A
s
{x}
satises R
e
, without destroying the satisfaction of previous requirements R
d
with d < e.
As pointed out in section 4.3, Theorems 5.3.2 and 5.3.3 put methodological
limits on the possible ways we can prove P = NP or P = NP.
5.4 Many-one versus Turing reductions
The denition of p-T-reduction seems much more liberal than that of p-m-
reduction. Let us prove that they are indeed dierent.
Theorem 5.4.1. (Ladner, Lynch, and Selman [19]) The relations
p
m
and
p
T
dier on EXP.
Proof. This is in fact an easier diagonalization, exploiting the fact that a p-
m-reduction is only allowed a single query. We want to construct sets A and
B such that A
p
T
B but A
p
m
B. Clearly the former is satised if we let
n A if and only if either 2n B or 2n + 1 B. Let f
e
be an ecient
enumeration of all polynomial time p-m-reductions. We want to satisfy for
every e the requirement
R
e
: n A(n) = B(f
e
(n)),
that is, A does not p-m-reduce to B via f
e
, as witnessed by the counterex-
ample n. We construct A =
s
A
s
and B =
s
B
s
in stages. At stage s = e
of the construction we are given nite sets A
s
and B
s
, and we have already
satised R
d
for all d < e. To satisfy R
e
we pick a fresh n, i.e. an n bigger
than any number used so far in the construction, and we compute f
e
(n). If
f
e
(n) B
s
then we let n / A. In this case we should also have 2n / B and
2n + 1 / B, which is indeed the case since n is fresh, so B
s
does not contain
any elements bigger than n. If f
e
(n) / B
s
then we let n A
s+1
, and we also
choose one of 2n and 2n+1 that is dierent from f
e
(n) and put it into B
s+1
.
This clearly satises R
e
, without aecting the satisfaction of earlier require-
ments since n was picked fresh. A and B thus constructed are in EXP, as is
easily checked.
Do these two fundamental reduction notions dier on NP? To prove this, it
seems one needs an assumption stronger than just P = NP. E.g. the answer
is yes if there is a sparse set in NP P. (Cf. for example [21, p223,254].)
Denition 5.4.2. A set A is sparse if there is a polynomial p such that for
all n, |A {0, 1}
n
| p(n). Here {0, 1}
n
denotes the set of all strings of
length at most n. A set A is tally if A {0}
.
Denition 5.4.3. A polynomial time computable function f : {0, 1}

{0, 1}
is a p-isomorphism if f is bijective and has a polynomial time com-

putable inverse. Two sets A and B are p-isomorphic if there is a p-isomor-
phism f such that f(A) = B.
Berman-Hartmanis Conjecture All sets that are p-m-complete for NP
are p-isomorphic.
This conjecture implies that all NP-complete sets have the same density, a
fact which is indeed provable:
Theorem 5.4.4. (Mahaney) If there is a sparse NP-complete set then P =
NP.
For a proof of this theorem see e.g. [21, p206]. We will return to sparse
complete sets in section 7.5.
5.5. The Gap Theorem 37
5.5 The Gap Theorem
By the time hierarchy theorem, for time constructible functions f we have
TIME(f) = TIME(2
f
). The next result shows that the assumption that f is
time constructible is essential for this result.
Theorem 5.5.1. (Gap Theorem, Trakhtenbrot, Borodin) There exists a
computable function f such that TIME(f(n)) = TIME(2
f(n)
).
Proof. We construct f in stages, satisfying for every e the requirement
R
e
: x
(
|x| = n e M
e
(x) in t steps = t / [f(n), 2
f(n)
]
)
.
This suces to prove the theorem: Suppose that A TIME(2
f(n)
) as wit-
nessed by M
e
. Then for every x, M
e
(x) within 2
f(n)
steps, hence by R
e
,
for every x of length e, M
e
(x) within f(n) steps. Since the nitely many
exceptions do not matter, we have A TIME(f(n)).
The construction of f is as follows. At stage e of the construction we
dene f(e). At stage e we consider M
0
, . . . , M
e
and all inputs x of length
e. Dene the sequence k
0
= e, k
l+1
= 2
k
l
. Call k
l
wrong for input x if
M
e
(x) in t steps and t [k
l
, 2
k
l
]. Note that we can compute whether k
l
is
wrong for x by running M
e
(x) for 2
k
l
steps. Note further that there can be
at most one k
l
wrong for x. Since there are only nitely many x of length
e it follows that there is a k
l
that is not wrong for any of these x. Dene
f(e) = k
l
for the least such k
l
. Since M
e
is considered in the construction of
f for every stage e, this clearly satises R
e
.
5.6 The Speed-Up Theorem
We often talk about the complexity of a computational problem. We al-
ready know that this use of language is not entirely appropriate because of
the linear speed-up theorem: Any program computing a function can be sped
up by any linear factor on almost all of its arguments. We can greet this
situation by interpreting it as that we can be sloppy about constants, as we
in fact did. But the situation is in fact much worse than this. In this section
we show that there are computable problems A such that any program for
A can be sped up exponentially. This result challenges the idea that every
problem has a well-dened level of complexity.
The result in fact holds for any measure of complexity, but we will prove it
here for time complexity. Also, the result holds in more generality: Instead of
an exponential speed-up one can use any fast growing computable function,
cf. Exercise 5.7.12.
As before in Chapter 2, x an enumeration {M
e
}
eN
of all deterministic
Turing machines. In the following we will identify M
e
with the language
computed by it. We call e an index for the language M
e
. Let
e
(x) denote
the number of computation steps of M
e
(x) if this computation converges,
and let
e
(x) be undened otherwise. Note that in general we cannot decide
whether M
e
(x) converges, hence neither whether
e
(x) is dened, but given
t we can decide whether
e
(x) < t by running M
e
for t steps. We will use
this fact in the proof.
Theorem 5.6.1. (Speed-Up Theorem, M. Blum [6]) There exists a com-
putable set A such that for every index e for A there is another index i for A
such that
x
(
i
(x) log
e
(x)
)
. (5.1)
That is, the program M
i
computes A exponentially faster than M
e
.
Proof. Let g(x) = 2
x
, and dene g
(1)
(x) = g(x) and g
(n+1)
(x) = g(g
(n)
(x)).
So g
(n)
(x) is an iterated stack of exponentials of height n. Also, let g
(n)
(x) = 0
if n 0. First we dene a scale of functions as follows. Let
h
e
(x) = g
(xe)
(0).
Then g(h
e+1
(x)) = h
e
(x), so the functions h
e
form a decreasing family of
functions, with an exponential gap between every two of them.
We construct a set A such that
(I) e
(
M
e
= A =
x h
e
(x)
e
(x)
)
,
(II) ei
(
M
i
= A
x
i
(x) h
e+1
(x)
)
.
Note that (I) and (II) together imply (5.1):
i
(x) h
e+1
(x) log h
e
(x) log
e
(x).
Construction. To obtain (I), we satisfy for every e the requirement
R
e
:
x
(
e
(x) < h
e
(x)
)
= M
e
= A.
At stage x of the construction we dene A(x) as follows. Say that e requires
attention at stage x if e x, e is not yet cancelled, and
e
(x) < h
e
(x). At
stage x, choose the least e that requires attention, dene A(x) = 1 M
e
(x),
and declare e cancelled. If there is no such e we give A(x) some default value,
say A(x) = 0. This ends the construction of A.
We verify that the construction satises all requirements. Suppose that
the premiss of R
e
holds, that is, there are innitely many x such that
e
(x) <
h
e
(x). Since there are only nitely many d < e that can require attention
at a stage x, and every time this happens some d < e is cancelled, there are
only nitely many stages at which e requires attention but is not the least
such number. At the rst stage following this where e requires attention
5.7. Exercises 39
(which exists by assumption), e is the least such number and it is ensured
that A = M
e
. Hence R
e
is satised.
It remains to verify (II). Clearly the construction above gives an algorithm
for computing A. We want to compute A in a more ecient way. Fix u N.
Suppose we know the nite set
F
u
=
{
(e, x, A(x)) : e < u e cancelled at stage x
}
.
Then we can compute the set A as follows. We run the above construction,
but instead of considering all e x at stage x we only consider e with
u e x. The full information of what happened at stage x in the original
construction can be recovered from the nite set F
u
. The point of this is that
we save having to run the functions h
e
(x) for e < u. Instead, we directly
nd out which, if any, of these e required attention by consulting F
u
. To see
which e with u e x requires attention we only have to compute h
e
(x) for
e u. Hence the time to compute A(x) is bounded by
x h
u
(x) + O(1) h
u1
(x) a.e. x.
This proves (II), because we may take u as large as we want.
5.7 Exercises
Exercise 5.7.1.
Use delayed diagonalization to prove that the p-m-degrees

of the computable sets are dense, that is, if A and C are computable and
C <
p
m
A then there is a computable B such that C <
p
m
B <
p
m
A.
Exercise 5.7.2.
Same as Exercise 5.7.1, but now for

p
T
instead of
p
m
.
Exercise 5.7.3.
Assuming that NP = P, show that there are incomparable

p-m-degrees in NP, that is, show that there are sets A and B in NP such
that neither A
p
T
B nor B
p
T
A. Note that it automatically follows that
A and B are intermediate sets. This result is the analogue of the Friedberg-
Muchnik theorem in computability theory that states that there are Turing-
incomparable c.e. sets.
Exercise 5.7.4. For every sparse set S there is a tally set T such that S
p
T
T.
Hint: consider
T =
{
0
n,j,i
: the i-th bit of the j-th string in S
=n
is 1
}
Exercise 5.7.5. Show that the Berman-Hartmanis conjecture implies that
P = NP.
Exercise 5.7.6.
There are no sparse p-m-complete sets for EXP.

Exercise 5.7.7. Use diagonalization to explicitly construct a computable
function that is not time constructible.
Exercise 5.7.8. Show that the function f from Theorem 5.5.1 is bounded
by a time constructible function.
Exercise 5.7.9. Prove the analog of the Gap Theorem for space complexity.
Exercise 5.7.10. Prove that the gap in Theorem 5.5.1 can be made arbi-
trarily large by using, instead of 2
n
, an arbitrary computable function.
Exercise 5.7.11. Given any nondecreasing computable function g, with at
least g(x) x, construct a g-scale, that is, a descending family of functions
{h
e
}
eN
such that
g(h
e+1
(x)) h
e
(x)
for every x.
Exercise 5.7.12. Let g be an arbitrary fast growing computable function
with g(x) x. Formulate and prove a general version of the speed-up theo-
rem, using g instead of 2
x
.
Chapter 6
Randomized computation
6.1 Probabilistic classes
In this section we consider variants of NP where the acceptance criterion is
given by a probabilistic condition over all paths. Consider nondeterministic
Turing machines M whose computation trees are full binary trees. Given a
set L and a machine M that is supposed to compute L, and an input x, the
error probability is the ratio of the number of paths giving the wrong answer
and the total number of paths. When thinking of Turing machines in this
way we speak also of probabilistic Turing machines.
Denition 6.1.1. The class PP, for probabilistic polynomial time, is the class
of sets L that are computed in polynomial time by a probabilistic machine
such that
x L the fraction of accepting paths is >
1
2
,
x / L the fraction of rejecting paths is
1
2
.
By Exercise 6.5.1, we can replace the second clause in the denition by
x / L the fraction of rejecting paths is >
1
2
.
Example 6.1.2. Let MAJ be the subset of SAT consisting of the Boolean
formulas that are satised by more than
1
2
of all assignments. Then MAJ is
in PP.
Proposition 6.1.3. NP PP PSPACE.
Proof. PP PSPACE: Try all paths, reusing the same space and keep a
counter for the paths used.
NP PP: We can convert any NP-computation into a PP-computation
as in Figure 6.1, where on the right is the original NP-computation and on
the left we add a tree of the same size where all paths accept.
41
42 Chapter 6. Randomized computation
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
1111111111111111
Figure 6.1: Converting an NP into a PP computation
It follows from Exercise 6.5.1 that PP is closed under complementation, hence
also co-NP PP. The closure of PP under unions and intersections was open
for a long time before it was solved in [5].
1
By Exercise 6.5.2, PP is closed
under p-m-reductions.
Consider the set
SA =
{
, i : is satised by > i assignments
}
.
Theorem 6.1.4. The sets MAJ and SA are PP-complete.
Proof. We already saw in Example 6.1.2 that MAJ is in PP. SA PP: We
show that SA
p
m
MAJ. The result then follows from the fact that PP is
closed under
p
m
(Exercise 6.5.2). Let , i be an instance of SA, where
has m variables. Let be a formula on the same m variables as with
exactly 2
m
i satisfying assignments. (One can eectively construct such a
formula, cf. Exercise 6.5.3.) Now dene
f(, i) = (y ) (y ).
Then has > i satisfying assignments if and only if f(, i) has > i+2
m
i =
2
m
satisfying assignments. Note that 2
m
=
1
2
2
m+1
, and that m + 1 is the
number of variables in f(, i). Hence f is a p-m-reduction from SA to
MAJ.
To prove completeness of SA, use the same reduction x
x
as for the
NP-completeness of SAT (Theorem 3.2.2). Given A PP computed in time
p(n), x is accepted if and only if 2
p(n)1
+1 paths accept x. Now A
p
m
SA
via x
x
, 2
p(n)1
.
1
Key to the proof is to approximate the sign of the function #acc #rej, the dierence
of the number of accepting and rejecting paths, by suitable polynomials. This can only be
done for each length separately.
6.1. Probabilistic classes 43
Denition 6.1.5. The class BPP, for bounded probabilistic polynomial time,
is the class of sets L that are recognized in polynomial time by probabilistic
machines with error probability bounded away from
1
2
, i.e. such that for some
> 0
x L the fraction of accepting paths is >
1
2
+ ,
x / L the fraction of rejecting paths is >
1
2
+ .
Obviously P BPP PP. Note that the relation between BPP and NP
is not clear. The interest in the class BPP comes from the following result,
which implies that the answers given by BPP-machines can be boosted to
any precision arbitrarily close to complete correctness!
Theorem 6.1.6. A BPP if and only if for all polynomials p there is a
probabilistic Turing machine M recognizing A in polynomial time with error
probability
1
2
p(n)
.
Proof. We simply iterate the computations of M a polynomial number of
times. A straightforward computation shows that this will suce to obtain
correct answers with a high probability. Suppose M recognizes A with error
probability <
1
2
. Let = 1 , and x a polynomial p. Let q(n) = cp(n),
where c is a constant such that
(
4
)
c
<
1
2
. (6.1)
(Note that <
1
4
because (
1
2
+ )(
1
2
) <
1
4
.) Now, given an input x of
length n, iterate the nondeterministic computation for M(x) m = 2q(n) + 1
times, and accept only when at least q(n) + 1 of the computations accept.
The probability that this procedure gives a wrong answer is the probability
that at most q(n) computations are correct, which is
q(n)
j=0
(
m
j
)
mj
. (6.2)
Since > and j
m
2
, we have
j
mj

m
2

m
2
so the expression (6.2) is
less or equal to
m
2
m
2
q(n)
j=0
(
m
j
)
<
m
2
m
2
2
m
(by the binomial formula 2
m
=
j
(
m
j
)
)
=
(
4
)m
2
(
4
)
cp(n)
(
m
2
= cp(n) +
1
2
)
<
1
2
p(n)
(by choice of c)
Proposition 6.1.7. BPP is closed under complementation, unions, and in-
tersections.
6.2 More about BPP
In this section we prove two fundamental results about BPP. We rst prove
that BPP has short advice strings, a notion that is related to circuit com-
plexity, cf. Chapter 7. Then we prove that BPP is included in
p
2
, the second
level of the polynomial time hierarchy.
Denition 6.2.1. Fix B P. Given a set A and a length n, y is correct
advice for A at length n if
x A x, y B (6.3)
for every x of length n.
One can think of the advice string y as coding a circuit to determine member-
ship in A for all x of a given length. We now prove that the sets in BPP have
(many) short advice strings for each length. Using the circuit terminology
one can phrase this by saying that BPP has small circuits, cf. section 7.3.
Theorem 6.2.2. (Adleman) The following are equivalent.
(i) A BPP,
(ii) For any polynomial q there are B P and a polynomial p such that for
all n, among all words of length p(n) there are at least
2
p(n)
(
1
1
2
q(n)
)
.
correct advice strings y for A at length n, as in (6.3).
Proof. (ii)(i) is straightforward: Given B and p and n, randomly pick an
advice string y of length p(n). Then with high probability, A(x) = B(x, y).
(i)(ii). Fix A and q. By Theorem 6.1.6 we may assume that there is a
machine M accepting A with error probability at most
(
1
2
)
q(n)+n
. Let p be
the running time of M. Given x of length n, at most 2
p(n)q(n)n
paths are
incorrect. Therefore at most 2
n
2
p(n)q(n)n
paths are incorrect for some x,
and hence 2
p(n)
2
p(n)q(n)
paths are correct for all x, and we can use any of
these as a correct advice string.
Theorem 6.2.3. (Sipser) BPP
p
2
.
6.2. More about BPP 45
Proof. Let L BPP. By Theorem 6.1.6 we may assume that there is a
machine M accepting L with error probability at most
1
2
|x|
. Let l be the
length of the paths in M. Given a path r {0, 1}
l
, denote the outcome of
M along that path by M(x, r), and dene
A(x) =
{
r {0, 1}
l
: M(x, r) accepts
}
.
Dene A(x) t =
{
r t : r A(x)
}
, where denotes bitwise exclusive or
(not to be confused with the notation for the join operator we used before).
Thus every t denes a 1-1 map, that we can think of as a translation. If A(x)
is large then there is a small set of translations of it covering the whole space
{0, 1}
l
:
Claim 1. If |A(x)| > 2
l
(
1
1
2
|x|
)
then there exist t
1
. . . t
l
, all of length l, such
that
i
A(x) t
i
= {0, 1}
l
.
To prove Claim 1, pick t
1
. . . t
l
at random. (That is, we use the probabilistic
method to prove the existence of the ts.) Let S =
i
A(x) t
i
. For every
r {0, 1}
l
we have
Pr
[
r / S
]
=
i
Pr
[
r / A(x) t
i
]
(
1
2
|x|
)
l
.
(Here Pr refers to the choice of the t
i
s.) Hence
Pr [r r / S]
r
2
|x|l
= 2
l|x|l
1
2
|x|
,
hence Pr
[
S = {0, 1}
l
]
1
1
2
|x|
. In particular there exist t
1
. . . t
l
such that
S = {0, 1}
l
. This proves Claim 1.
Claim 2. If |A(x)| 2
l 1
2
|x|
then there do not exist t
1
. . . t
l
such that
i
A(x)
t
i
= {0, 1}
l
.
To prove Claim 2, note that |
i
A(x) t
i
| 2
l|x|
for every i and that
l 2
l|x|
< 2
l
. This proves Claim 2.
Now to nish the proof of the theorem, note that we can write
x L t
1
. . . t
l
r {0, 1}
l
1il
M(x, r t
i
) accepts.
Note that it follows immediately from Theorem 6.2.3 that BPP
p
2

p
2
because BPP is closed under complementation (Proposition 6.1.7). In con-
trast with PP, there are no problems known to be complete for BPP. It has
NP
NPco-NP
PH PSPACE
co-NP
.
.
.
.
.
.
.
.
.
.
.
.
.
BPP
PP
Figure 6.2: Relations between complexity classes

been conjectured that P = BPP, and of course this statement is equivalent
to the statement that all the problems in P are complete for BPP.
Figure 6.2 summarizes some of the relations between complexity classes.
The relation between PP and PH is not clear, but the following is known.
In Section 6.1 we discussed closure properties of PP. The ultimate closure
property would be P
PP
= PP, i.e. the closure of PP under
p
T
. If this is the
case then PP in fact contains PH by the following result revealing some of
the strength of randomization.
Theorem 6.2.4. (Toda) PH P
PP
.
Proof. This most important theorem can be proved using counting classes,
which is a very interesting topic in itself. A proof can be found in [24].
6.3 The classes RP and ZPP
Dene the following probabilistic classes:
Denition 6.3.1. The class RP, for randomized polynomial time, is the class
of sets L that are recognized in polynomial time by probabilistic machines
with one-sided error probability
1
2
, i.e. such that
x L the fraction of accepting paths is
1
2
,
x / L the fraction of rejecting paths is 1.
By Exercise 6.5.7 the class RP remains the same if we replace the
1
2
in its
denition by
2
3
. RP-algorithms are also known as Monte Carlo algorithms.
Note that we have P RP NP (Exercise 6.5.6).
The class ZPP, for zero-error probabilistic polynomial time, is the class
of sets L that are recognized in polynomial time by probabilistic machines
with three possible outcomes: accept, reject, and ? for dont know. The
6.4. Primes again 47
requirement is that no path gives a wrong answer, and that the fraction
of paths that outputs ? is at most
1
3
, so that at least
2
3
in fact gives the
correct answer. ZPP-algorithms are also known as Las Vegas algorithms. By
Exercise 6.5.8 we have ZPP = RP co-RP.
6.4 Primes again
We saw in section 2.5 that PRIMES is in NP co-NP. In this section we
show that PRIMES is in co-RP, that is, there is a Monte Carlo algorithm for
testing compositeness. Given a number n, if n is prime this algorithm will
with certainty output the verdict n is prime, and if n is composite it will
with high probability output the verdict n is composite.
We start with some number-theoretic preliminaries. From now on, p is a
prime larger than 2.
Denition 6.4.1. A number 0 < a < p is called a quadratic residue modulo
p if the equation x
2
a mod p has a solution. The Legendre symbol is dened
as
(
a
p
)
=
{
1 if a is a quadratic residue modulo p,
1 otherwise.
Theorem 6.4.2. (Eulers criterion)
(
a
p
)
a
p1
2
mod p.
Proof. Note that since (Z/pZ) is a eld, the equation x
2
a can have at
most two solutions modulo p. We claim that there are either two or zero
solutions. By Theorem 1.5.5, pick a primitive root r modulo p. In particular
there is i < p 1 such that r
i
a mod p. If i = 2j is even then a
p1
2
=
r
j(p1)
1 mod p, and x
2
a has the solutions r
j
and r
j+
p1
2
. Note that if i
varies over the even numbers, we obtain half of all possible residues, and since
each has two square roots, no such roots are left for the odd i. So if i = 2j +1
is odd, r
i
has no square roots, and a
p1
2
r
j(p1)
r
p1
2
r
p1
2
mod p. Since
the latter is a square root of 1 unequal to 1, it must be 1.
Theorem 6.4.2 allows us to compute the Legendre symbol eciently. Note
that we can eciently compute exponentials by repeated squaring, cf. [7] for
more details.
Theorem 6.4.3. (Law of Quadratic Reciprocity)
(
p
q
)
(
q
p
)
= (1)
p1
2
q1
2
.
Denition 6.4.4. The Jacobi symbol generalizes the Legendre symbol as
follows. If n is odd and had prime factorization n =
i
q
i
then
(
m
n
)
=
i
(
m
q
i
)
.
Lemma 6.4.5. (i)
(
m
1
m
2
n
)
=
(
m
1
n
) (
m
2
n
)
.
(ii)
(
m+n
n
)
=
(
m
n
)
.
(iii)
(
n
m
)
(
m
n
)
= (1)
m1
2
n1
2
.
Proof. The rst two items easily follow from the denitions and the third
follows from Theorem 6.4.3.
It follows from Lemma 6.4.5 that also the Jacobi symbol is eectively com-
putable, in a manner similar to that of Euclids algorithm for computing the
gcd.
Lemma 6.4.6. If n is odd and
(
m
n
)
m
n1
2
mod n for all m (Z/nZ)
then n is prime.
Proof. Suppose that the premiss holds but that n is composite. Suppose rst
that n = p
1
. . . p
k
is the product of distinct primes. Let r (Z/p
1
Z)
be such
that
(
r
p
1
)
= 1. By the Chinese remainder theorem let m (Z/nZ)
be such
that m r mod p
1
and m 1 mod p
j
for j = 2 . . . k. Then m
n1
2

(
m
n
)

1 mod n, hence m
n1
2
1 mod p
2
, contradicting that m 1 mod p
2
.
From the previous contradiction we conclude that n = p
2
m for some p > 2
prime. By Exercise 1.6.3, let r be a primitive root modulo p
2
. By assumption
we have r
n1
2

(
r
n
)
{1, 1} mod n, hence r
n1
1 mod n. The latter
implies that n 1 is a multiple of (p
2
) = p(p 1), hence p divides n 1.
But p also divides n, a contradiction.
Lemma 6.4.7. If n is odd and composite, then at least half of the elements
m (Z/nZ)
satisfy
(
m
n
)
m
n1
2
mod n.
Proof. Consider the set
X =
{
x (Z/nZ)
:
(
x
n
)
x
n1
2
mod n
}
.
By Lemma 6.4.5 the elements of X form a subgroup of (Z/nZ)
, and by
Lemma 6.4.6 it is not the whole group. If a (Z/nZ)
X then the left

coset aX has the same cardinality as X, hence X contains at most half the
elements of (Z/nZ)
.
Theorem 6.4.8. (Solovay and Strassen [26]) PRIMES is in co-RP.
Proof. Given n odd, the algorithm runs as follows: Randomly pick a number
m with 2 m n 1. If gcd(m, n) = 1 then n is composite. (Note that
we can eectively compute the gcd using Euclids algorithm.) Otherwise,
m (Z/nZ)
. See if
(
m
n
)
m
n1
2
mod n. If not, declare n to be composite.
If yes, declare n to be probably prime. Note that if p is prime then by
Theorem 6.4.2,
(
m
p
)
m
p1
2
mod p so in the rst case we are sure the answer
is correct. If n is composite then by Lemma 6.4.7 the answer is correct with
probability at least
1
2
.
6.5. Exercises 49
6.5 Exercises
Exercise 6.5.1. Show that if L PP then there is a probabilistic machine
M such that x / L the fraction of accepting paths is <
1
2
.
Exercise 6.5.2. PP is closed under
p
m
.
Exercise 6.5.3. Show that given a set of n Boolean variables {x
1
, . . . , x
n
}
and i 2
n
there is a formula having exactly i satisfying assignments. (Hint:
First do this when i is a power of 2. Then for the general case write i in
binary as a sum of powers of 2 and combine the formulas of the components
by disjunctions, making sure that the assignments do not overlap.)
Exercise 6.5.5. Use Theorem 6.1.6 to prove that in Denition 6.1.5 we can
allow to be a function of the form 1/p(n), where p is a polynomial and n is
the size of the input.
Exercise 6.5.6. Prove that P RP NP.
Exercise 6.5.7. Prove that we obtain an equivalent denition of RP if we
replace the
1
2
in its denition by
2
3
. (Hint: Because the error is one-sided,
we can achieve this by iteration. We cannot use the same argument to prove
that PP BPP.) Conclude from this that RP BPP.
Exercise 6.5.8. Prove that ZPP = RP co-RP.
Exercise 6.5.9. Show that P ZPP NP co-NP.
Exercise 6.5.10. Prove that if NP BPP then NP = RP. (Hint: Given
a BPP-machine for an NP-language L, the only problem are the paths that
accept an input x / L. Use the fact that since we are dealing with an NP-
problem allows us to check whether an accepting path is correct.)
Chapter 7
Circuit complexity
7.1 Boolean functions
We already discussed in Exercise 1.6.2 that every propositional formula can be
put into conjunctive normal form. As mentioned before (on page 18), putting
a formula in CNF may result in an exponential blow-up. In this chapter we
will consider various ways in which Boolean functions and formulas may be
represented succinctly.
Denition 7.1.1. An n-ary Boolean function is a function f : {0, 1}
n
{0, 1}.
First note that every propositional formula (x
1
, . . . , x
n
) of n variables de-
nes an n-ary Boolean function. Conversely, every Boolean function can be
represented by a propositional formula:
Proposition 7.1.2. For every n-ary Boolean function f there is a proposi-
tional formula such that the function dened by is equal to f.
Proof. Dene
F =
{
z {0, 1}
n
: f(z) = 1
}
.
For every vector z = (z
1
, . . . , z
n
) {0, 1}
n
dene the formula
z
as the
conjunction of all variables x
i
such that z
i
= 1 and all negated variables x
i
such that z
i
= 0. Finally dene the formula by
=
zF
z
.
Then clearly is true under an assignment z of its variables precisely when
f(z) = 1.
Note however that the size of the formula in Proposition 7.1.2 can be
exponential in n.
50
7.2. Circuits 51
x
1
x
2
x
3

x
4
?>=< 89:;
?>=< 89:;
?>=< 89:;
?>=< 89:;
?>=< 89:;
?>=< 89:;
z
Figure 7.1
7.2 Circuits
A circuit is a directed acyclic graph (dag for short), the nodes of which are
called gates. The in-degree, or fan-in, of a node is the number of arrows
leading to it, and the out-degree, or fan-out, the number of arrows leaving
it. There are input gates x
1
, . . . , x
n
, output gates z
1
, . . . , z
m
, and Boolean
gates of type not, and, and or. The and-gates and or-gates have in-degree 2
and the not-gates in-degree 1. The fan-out of any gate can be arbitrary. The
input gates act as function inputs that can be 0 or 1. Thus a circuit computes
a function from {0, 1}
n
to {0, 1}
m
. In principle n and m can be arbitrary,
but we will mostly consider circuits with one output gate, i.e. with m = 1.
Figure 7.1 pictures an example circuit. The Boolean formula corresponding
to this circuit is
(
(x
1
x
3
) x
2
)
(
x
2
x
4
)
.
The size of a circuit is the total number of and-gates and or-gates. Tradi-
tion has it that the not-gates are not counted. Thus we can take as a measure
of complexity of a Boolean function f : {0, 1}
n
{0, 1} the minimal size of
a circuit computing it.
To extend the notion of circuit complexity to arbitrary languages L
{0, 1}
we consider circuit families, with one circuit for every input length,
as follows.
Denition 7.2.1. Suppose that {C
n
}
nN
is a family of circuits such that
for every n, C
n
computes a Boolean function of n inputs. We say that such
a family computes a language L {0, 1}
if for every length n and every

x {0, 1}
n
,
x L C
n
(x) = 1.
For a given function s : N N, SIZE(s(n)) denotes the class of languages
L that are computed by a circuit family such that the size of C
n
is bounded
52 Chapter 7. Circuit complexity
by s(n).
It is important to note that circuit size complexity as dened here is a nonuni-
form measure of complexity, in the sense that in the denition of SIZE(s(n))
there is no computational restriction on the sequence of circuits {C
n
}
nN
.
(See also Exercise 7.6.3.) In some contexts it makes sense to impose such ex-
tra computational restrictions, e.g. by requiring that the sequence {C
n
}
nN
is uniformly computable in polynomial time. As for now, we stick to the
nonuniform denition.
The following result shows that we do not need to consider circuits of
more than exponential size.
Proposition 7.2.2. Any language L {0, 1}
is in SIZE(O(2
n
)).
Proof. Given L and a length n, we can consider the characteristic function of
L{0, 1}
n
. We can describe this Boolean function with a formula in DNF of
size O(2
n
) (cf. the proof of Proposition 7.1.2). Finally, we can easily translate
this formula into a circuit of roughly the same size.
Proposition 7.2.3. There are languages that do not have subexponential cir-
cuit size: For any s such that s(n) = 2
o(n)
there is a language L / SIZE(s(n)).
Proof. This follows by counting the circuits of size s. Assuming that s n,
we arrive at an upper bound as follows. There are s internal gates and n
input gates. To describe the circuit, we need to indicate for every gate the
type (one of three) and where its two inputs are coming from. In total this
costs no more than
s (2 log(n + s) + 3) s (2 log 2s + 3) = s (2 log s + 5)
bits. Thus the total number of circuits of size s is bounded by 2
s(2 log s+5)
. On
the other hand, there are 2
2
n
Boolean functions f : {0, 1}
n
{0, 1}. Now if
s(n) = 2
o(n)
then s(n)(2 log s(n) +5) o(2
n
), so there are not enough circuits
of size s(n) to compute all Boolean functions, if n is large.
7.3 Relation with advice classes
Recall the denition of advice strings (Denition 6.2.1).
Denition 7.3.1. Let C be a complexity class and let F be a class of advice
functions of the form s : N {0, 1}
. We think of s as giving an advice

string for every length n. Then C/F is the class of all sets of the form
{
x : x, s(|x|) B
}
for some B C and some s F. Let poly be the class of polynomially
bounded advice functions, i.e. those s such that |s(n)| n
c
for some con-
stant c. In particular P/poly is the class of sets that have polynomial advice.
7.4. Small circuits for NP? 53
Theorem 7.3.2. P/poly = SIZE(n
O(1)
).
Proof. The direction SIZE(n
O(1)
) P/poly follows because we can evaluate
a circuit in polynomial time. This means that the set
B =
{
x, C : C circuit C(x) = 1
}
is in P. So if A SIZE(n
O(1)
) and for every length n, C
n
is a polynomial size
circuit for A {0, 1}
n
, then x A x, C
|x|
B, hence A P/poly.
For the converse direction P/poly SIZE(n
O(1)
) we have to do a bit more
work. We have to show that a computation of a polynomial time Turing
machine on inputs of a xed length n can be converted into a polynomial size
circuit. We leave this as Exercise 7.6.5.
We can now state the following corollary to Theorem 6.2.2:
Theorem 7.3.3. BPP SIZE(n
O(1)
).
7.4 Small circuits for NP?
In this section we prove that not all sets in NP have polynomial size circuits,
unless the polynomial hierarchy collapses to the second level, that is, if
p
2

p
2
. For this we need to prove that polynomially bounded -statements can
be written as -statements. This uses an old proof-theoretic trick called
Skolemization. The idea is that if the formula
yz (y, z)
holds in N (or in {0, 1}
) then there is an implicit Skolem function f : N N

such that y (y, f(y)). That is, f picks for every y a witness z = f(y) such
that (y, f(y)). Now the statement that such an f exists, i.e.
fy (y, f(y))
has the required
p
2
-form, provided that we have a polynomially bounded
way to describe f. Here we use the assumption that SAT has small circuits.
The next lemma shows that in this case we actually have small circuits that
produce satisfying assignments, instead of just deciding satisability.
Lemma 7.4.1. If NP SIZE(n
O(1)
) then there is a family of polynomial size
circuits that, given (the binary encoding of ) a formula as input, compute a
satisfying assignment for that formula, if such an assignment exists.
Proof. Suppose that {C
n
}
nN
is a family of polysize circuits computing SAT,
such that for every formula (x
1
, . . . , x
n
) on n variables, C
n
() = 1 if and
only if SAT. Given such a formula , compute a satisfying assignment as
follows. First see if there is a satisfying assignment starting with 1 by checking
if C
n1
((1, x
2
, . . . , x
n
)) = 1. Continue inductively as follows: Given the rst
i 1 bits b
1
, . . . , b
i1
of a satisfying assignment, see if we can extend it with
b
i
= 1 by checking if
C
ni
((b
1
, . . . , b
i1
, 1, x
i+1
, . . . , x
n
)) = 1.
By putting these n circuit computations together in series we obtain one big
(but still polynomial) circuit C such that SAT if and only if C() satises
, for every on n variables.
1
Theorem 7.4.2. (Karp-Lipton-Sipser [17]) If NP SIZE(n
O(1)
) then
p
2

p
2
.
Proof. Suppose that L
p
2
. By the NP-completeness of SAT there is a
polynomial time computable function f such that for all x,
x L
p
y f(x, y) SAT.
Without loss of generality the formulas f(x, y) all have the same polynomial
size. By Lemma 7.4.1 we have a polysize circuit C such that f(x, y) SAT
if and only if C(f(x, y)) satises f(x, y). Hence there is a polynomial q such
that
x L
q
C
p
y
(
C(f(x, y)) satises f(x, y)
)
which shows that L
p
2
.
7.5 Sparse T-complete sets
Theorem 7.5.1. P/poly =
S sparse
P(S).
Proof. Suppose A P/poly, say B P and f is a polynomially bounded
advice function such that
x A x, f(|x|) B. (7.1)
We code the advice function into a sparse set S as follows:
S =
{
0
n
, x : x f(n)
}
.
S is indeed sparse: the strings of a given length m are of the form 0
n
, x with
n m, and for a given n there are at most m such x, so in total there are no
more than m
2
strings of length m in S. Clearly A
p
T
S: Given x, rst use S
to nd f(|x|), and then apply (7.1).
1
The reader may compare this proof to the self-reducibility of SAT encountered earlier.
7.6. Exercises 55
For the converse, suppose that S is sparse and A
p
T
S. Suppose that
A = M
S
and that M runs in time p. Then on input x, M can only query
elements up to length p(|x|). We can code the polynomial many elements
in S {0, 1}
p(|x|)
into an advice string f(|x|) of polynomial length. In the
computation M
S
(x), instead of using the oracle S we can equivalently use
the advice string f(|x|).
We saw in Exercise 5.7.6 that there are no sparse p-m-complete sets for EXP.
As a consequence of the previous theorem we have:
Corollary 7.5.2. If there is a sparse p-T-hard set for EXP then
(i) PH collapses, and
(ii) P = NP.
Proof. (i) From the assumption and Theorem 7.5.1 we have EXP P/poly.
By Exercise 7.6.8 we then have EXP
p
2
. In particular, since PH EXP,
we have PH =
p
2
.
(ii) If P = NP then P =
p
2
, hence by the proof of (i) we have EXP = P,
contradiction.
7.6 Exercises
Exercise 7.6.1. The Boolean XOR operator is dened as follows: xXORy
is 1 when exactly one of x and y is 1, and it is 0 otherwise.
(a) Show that xXORy is equal to x + y mod 2.
(b) Dene XOR in terms of the usual propositional connectives.
(c) Dene a circuit computing XOR.
(d) Show that adding XOR-gates to the denition of circuits only changes
the notion of circuit size by a constant factor.
Exercise 7.6.2. The n-ary parity function parity
n
: {0, 1}
n
{0, 1} outputs
1 if the input string contains an odd number of 1s, and 0 otherwise. Note that
the 2-ary parity function is equal to the XOR operator from Exercise 7.6.1.
Show that there is a circuit of size O(n) that computes parity
n
.
Exercise 7.6.3. As noted above, circuit size is a nonuniform measure of
complexity. Show that indeed the class SIZE(s(n)) contains noncomputable
sets, even if s is constant.
Exercise 7.6.4. Improve on Exercise 7.6.3 by showing that SIZE(1) is un-
countable.
Exercise 7.6.5.
Finish the proof of Theorem 7.3.2.

Exercise 7.6.6. Show that P/poly is closed downwards under
p
T
.
Exercise 7.6.7. Prove that P/poly =
S tally
P(S).
Exercise 7.6.8. (Meyer, cf. [17]) Prove that if EXP P/poly then EXP
p
2
.
Exercise 7.6.9. There is a sparse p-T-hard set for EXP if and only if EXP
P/poly.
Chapter 8
Cryptography
We tend to think of complexity as something negative entailing that we cannot
do something. In this chapter we will see that sometimes complexity can be
something positive that we can use to our advantage. Cryptography is the
prime example of such a topic. This is in fact one of the oldest elds of
applied mathematics, that has stirred the imagination since ancient history.
The main problem is to devise a secure and ecient method to transfer secret
messages. To complicate the task, we require that the method should also
work in case the parties that want to exchange messages have never met
before, so that they have not had the chance of agreeing in private on some
key to be used. We will see that such a method indeed exists, based on a
complexity-theoretic assumption.
8.1 Public-key cryptography
In public-key cryptography, every participant has a public key known to ev-
erybody and a private key only known to himself. If participant Alice wants
to send participant Bob a secret message over a, presumably unsafe, channel,
she encrypts the message using Bobs public key. The protocol is devised in
such a way that the only way to recover the message is to use Bobs private
key. This ensures that only Bob can decrypt the message, and that a possible
eavesdropper cannot decipher it.
RSA (which stands for the names of its inventors Rivest, Shamir, and
Adleman) is such a public-key encryption protocol. In 1977 the authors of
RSA published a challenge to decipher a message that was encrypted using
their protocol. It took until 1994 until the message was deciphered, in a large
eort using massive parallel computing, coordinated by Arjen Lenstra and
others. The message turned out to be The Magic Words are Squeamish
Ossifrage.
While it is easy to multiply numbers, there does not seem to be an ecient
way of factoring a large number into its prime factors. RSA is based on the
57
58 Chapter 8. Cryptography
assumption that there is no polynomial time algorithm for factoring. (In
contrast, note that we can decide in polynomial time whether a number is
prime, see p 11.) The RSA protocol works as follows. As before, denotes
Eulers function.
Choose two large prime numbers p and q.
Choose an integer e coprime with (pq).
Let d be a multiplicative inverse of e modulo (pq) (which exist by the
previous item), that is, ed 1 mod (pq). Note that d can be easily
computed using the Euclidean algorithm, provided we know p and q,
and hence (pq) = (p 1)(q 1).
Public key: pq and e
Private key: p, q, and d
Encryption algorithm: E(e, x) = x
e
mod pq
Decryption algorithm: D(d, y) = y
d
mod pq
Proposition 8.1.1. For any integer x,
x
ed
x mod pq. (8.1)
Proof. If gcd(x, p) = 1 then by Fermats little theorem (cf. section 1.5), for
any k we have
x
1+k(pq)
x
1+k(p1)(q1)
x mod p. (8.2)
On the other hand, (8.2) clearly also holds if x 0 mod p, hence it holds for
any x. Similarly, x
1+k(pq)
x mod q for any x. It follows by the Chinese
remainder theorem that
x
1+k(pq)
x mod pq (8.3)
for any integer x. Since ed 1 mod (pq), (8.1) follows from this.
From Proposition 8.1.1 we see that the decryption algorithm D is indeed the
inverse of the encryption algorithm E. In particular, since E has an inverse,
it is a bijection on Z/pqZ.
Note further that anybody can encrypt a message, since pq and e are
public, but that one needs d for decryption, where d is obtained from p and q.
Since pq is public, it is essential that there is no ecient way to obtain p and
q from their product pq. It is currently widely believed, but unproven, that
such a method indeed does not exist.
Note using the above protocol we can encrypt messages of length at
most pq. Longer messages need to be chopped up in pieces of this length. In
the original RSA challenge, the primes p and q were 64 and 65 digits long.
8.2. Signed messages 59
8.2 Signed messages
Suppose Alice wants to send Bob a secret message, and that in addition she
wants to prove to Bob that she is the sender. She can use her own pair of
public and private keys e
A
and d
A
to do so in the following way. Let e
B
and d
B
denote Bobs pair of keys. Instead of sending the encrypted message
x
e
B
as before, Alice now sends the message x
d
A
e
B
to Bob. Bob decodes the
message to x
d
A
using his private key d
B
. Then he uses Alices public key e
A
to further decode the message to x. At this point Bob not only knows the
message x, but also that the message had to come from Alice, since Alice is
the only person that knows d
A
.
The dynamics of the interaction between Alice and Bob in this protocol
reveals some of the power of interactive cryptographic protocols that we will
encounter again later in Chapter 9. Note that the introduction of interaction
between various parties that perform computations adds a whole new layer
of complexity to the theory of computation. The computational as well as
the logical aspects of this are highly interesting.
8.3 One-way functions
Informally, a one-way (or trapdoor) function is a function that is easy to
compute, but hard to invert. Such functions may serve as the basis for a
cryptographic scheme.
Note rst that some easily computable functions may be hard to invert
for trivial reasons. For example, if the function is not injective then there is
no inverse at all. We may counter this by requiring that an inverse should
compute some inverse value, that need not be unique. A second trivial reason
that a function may not be easily invertible is that it might map large strings
to small ones. Consider for example f(x) = log x. Since we have f(2
n
) = n, to
write down the inverse f
1
(n) we need to write 2
n
, which is of length n, which
is exponential in log n, the input size of f
1
(n). So f
1
is not polynomial
time computable. Functions that do not map large strings to small ones,
i.e. such that the length of inputs and outputs are polynomially related, are
called honest. So in studying invertability, we should restrict our attention
to such honest functions.
We have the following formal denition. A function f : {0, 1}
{0, 1}
is a one-way function if
(i) f is injective. (As indicated above, this requirement is not always im-
posed.)
(ii) f is honest: There is a constant k such that for all x,
|x|
1
k
|f(x)| |x|
k
.
60 Chapter 8. Cryptography
(iii) f is polynomial time computable.
(iv) f
1
is not polynomial time computable.
Note that the existence of one-way functions implies that P = NP. (Because
of the honesty, computing the unique inverse of f(x) is a typical NP
-task: we can guess an inverse and use f to check its correctness.) Even if
we assume that P = NP, however, the existence of one-way functions is not
known. Their existence is tied to a special complexity class that we discuss
in the next section
The RSA protocol is based on the assumption that multiplication, which
has factoring as an inverse, is a trapdoor function. There are many other such
functions. An example is the discrete exponential function, and its inverse the
discrete logarithm. Given a prime p and an integer x, the discrete exponential
is 2
x
mod p. This function can be computed in time O(n
3
), where n is the
length of p. Currently no polynomial time algorithm is known for computing
the inverse of this function.
8.4 The class UP
Denition 8.4.1. Call a nondeterministic Turing machine unambiguous if
for every input there is at most one accepting computation. UP is the class
of sets that are accepted by some unambiguous machine in polynomial time.
Obviously we have the inclusions
P UP NP.
As usual, nothing more than this is known. The following result ties the class
UP to the existence of one-way functions.
Theorem 8.4.2. One-way functions exist precisely when P = UP.
Proof. Suppose that f is a one-way function. Dene the set
L =
{
(x, y) : z x(f(z) = y)
}
.
Since f is injective, L UP. Also L / P: If L P then we could nd an
inverse of f in polynomial time by binary search.
Conversely, if L is a set in UPP then we can dene a one-way function
f as follows. Suppose M is an unambiguous machine accepting L. Given a
computation path y of M on input x, we let f map y to 1x if y is accepting,
and to 0y if y is not accepting. Note that x can be eectively retrieved from
y. Since accepting paths of M are unique, f is injective. Also, f is honest
since the computations of M are polynomial. So to prove that f is one-way
it suces to show that its inverse cannot be computed in polynomial time.
Suppose we could do this. Then obviously L would be in P, contrary to
assumption. Hence f is one-way.
8.5. Exercises 61
8.5 Exercises
Exercise 8.5.1. Strictly speaking, in the RSA protocol above we only need
to know (pq) and not p and q to compute d. Show however that if we know
both pq and (pq) then we also know p and q, so obtaining (pq) from pq is
just as hard as to obtain p and q.
Exercise 8.5.2. Show that UP is downwards closed under p-m-reductions.
Chapter 9
Interactive proofs
9.1 Interactive protocols and the class IP
We can view NP as the class of languages with short certicates. Think of this
as an unbounded Prover P convincing a polynomial time bounded Verier V .
It is not clear how to describe e.g. co-NP in this way. However, the idea still
works if we add rounds between P and V and make V probabilistic.
Example: Graph Nonisomorphism. Recall the problem GRAPH ISO-
MORPHISM of deciding whether two given graphs are isomorphic. This is a
famous example of an NP-problem not known to be in P or NP-complete. We
describe an interactive protocol for its complement Graph Nonisomorphism.
Given G
0
and G
1
, how can Prover convince Verier that G
0

= G
1
? Note
that Prover cannot send a short certicate, since Graph Nonisomorphism is
not known to be in NP. Instead, Verier probabilistically tests Prover as
follows. Verier randomly selects G
0
or G
1
and randomly permutes its nodes
to obtain a graph H, which he sends to Prover with the question which of
G
0
, G
1
was the source of H. This is repeated for a polynomial, or constant,
number of rounds, say 100 times. If indeed G
0

= G
1
then Prover can always
reconstruct the answer, using his unbounded computing power. If on the
other hand G
0

= G
1
he has no better than a chance of
1
2
to answer correctly.
So after 100 rounds, if all of the answers of Prover are still correct, G
0

= G
1
with probability at least 1
1
2
100
. In summary, if P does his best to convince
V that G
0

= G
1
he can always succeed in doing so if indeed G
0

= G
1
, whereas
he will fail almost surely if G
0

= G
1
.
The complexity class IP consists of all languages that allow for a protocol
of the type just described. In the following we make this more precise. The
protocol will consist of several rounds of sending messages between Prover P
and Verier V , and at every round there will be a history string m
1
, . . . , m
i
of messages of previous rounds.

62
9.2. IP = PSPACE 63
Verier V is a polynomial time computable function that is given the
input of length n, as well as a random string of length p(n), where p
is a xed polynomial. Given message history m
1
, . . . , m
i
, V either
accepts, rejects, or sends a next message m
i+1
.
Prover P is an unrestricted function that is given the input, and that
outputs a message m
i+1
when given history m
1
, . . . , m
i
.
Dene the interaction V P by setting (V P)(w, r) = 1 if there is
a legal sequence of messages m
1
, . . . , m
k
between V and P for some k
such that m
k
is accept.
Without loss of generality all messages have length p(n), and the number of
rounds is bounded by p(n). Dene
Pr
[
V P accepts w
]
= Pr
|r|=p(n)
[
(V P)(w, r) = 1
]
.
Denition 9.1.1. We dene A IP if there exist a verier V such that for
all inputs w,
w A = P Pr
[
V P accepts w
]
2
3
,
w / A = P Pr
[
V P accepts w
]
1
3
.
As for BPP, we can boost the probability by polynomial iterations to make
the error exponentially small. Note that we have NP IP (P supplies V with
the certicate in just one round) and BPP IP. For the latter no interaction
is needed; given w and r, V simply performs a polytime computation and
accepts or rejects.
Theorem 9.1.2. IP PSPACE.
Proof. Though it is tedious to work out the details, this is a straightforward
derandomization of all possibilities in polyspace. We can get rid of the Prover
P by considering
Pr
[
V accepts w
]
= max
P
Pr
[
V P accepts w
]
.
Note that this value is
2
3
if w A and
1
3
if w / A. We can dene a
similar expression starting with an arbitrary message history.
9.2 IP = PSPACE
Towards the converse inclusion of Theorem 9.1.2, consider the following vari-
ant of the problem SA from page 42:
#SAT =
{
, k : has exactly k satisfying assignments
}
.
64 Chapter 9. Interactive proofs
Theorem 9.2.1. #SAT IP.
Proof. (Sketch.) Suppose = (x
1
, . . . , x
m
) is a Boolean formula and let
f
i
(a
1
, . . . , a
i
) be the number of satisfying assignments of where x
j
= a
j
for
j i. Hence f
0
() is the number of satisfying assignments of . Then we have
f
i
(a
1
, . . . , a
i
) = f
i+1
(a
1
, . . . , a
i
, 0) + f
i+1
(a
1
, . . . , a
i
, 1).
Consider the following protocol for #SAT. The input is a pair , k, where
k N is supposed to be the number of assignments of .
Round 0. P sends f
0
() to V , and V checks whether f
0
() = k, and rejects
if not.
Round 1. P sends f
1
(0) and f
1
(1) to V . V checks f
0
() = f
1
(0) + f
1
(1)
and rejects if not.
Round 2. P sends f
2
(0, 0), f
2
(0, 1), f
2
(1, 0), f
2
(1, 1) to V . V checks
whether
f
1
(0) = f
2
(0, 0) + f
2
(0, 1),
f
1
(1) = f
2
(1, 0) + f
2
(1, 1)
and rejects if not.
Round m. P sends f
m
(a
1
, . . . , a
m
) for all a
1
, . . . , a
m
{0, 1}. V checks
2
m
equations and rejects if any of them fail.
Round m + 1. If all f
m
(a
1
, . . . , a
m
) are correct V accepts, and rejects
otherwise.
This protocol for #SAT is obviously correct, but the problem is that it is
not polynomial: the rounds are exponential because at every next round the
number of equations and the size of the messages doubles. To remedy this,
write the f
i
s as polynomials over a nite eld F
q
, q 2
n
prime, where n
is the size of . Such polynomials are easily dened from by writing as
multiplication and x y as
x y = 1 (1 x)(1 y). (9.1)
(We could use + instead of , but has the advantage that it behaves the
same as on Boolean values.) Now, in Round 1 of the protocol, instead
of f
1
(0) and f
1
(1), choose a random r F
q
and consider f
1
(r). If

P lies
about f
0
() then it also has to lie about at least one of f
1
(0) and f
1
(1), hence
about the polynomial f
1
(z). Say it sends

f
1
(z) = f
1
(z). The key point of the
proof then is that for random r,

f
1
(r) = f
1
(r) with high probability, because
two dierent polynomials of degree n can agree on at most n points, for
a nonzero polynomial of degree n has at most n roots.
1
So working with
random inputs for the f
i
s prevents the blow-up in the above protocol, and
brings it down to polytime.
1
The precise formulation of this fact for multivariate polynomials is called the Schwartz-
Zippel theorem, cf. [29, p45], [24, p29].
9.3. Zero-knowledge proofs 65
Theorem 9.2.2. (Shamir) IP = PSPACE.
Proof. One inclusion was already proven in Theorem 9.1.2, so we only have
to prove PSPACE IP. We try to repeat the proof of #SAT IP with the
PSPACE-complete problem QBF instead of #SAT. Without loss of generality
we can work with closed q.b.f.s (no free variables) since this subset of QBF
is still PSPACE-complete. Given a closed q.b.f. = Q
1
x
1
. . . Q
m
x
m
, where
Q
i
{, }, dene
f
i
(a
1
, . . . , a
i
) =
{
1 if Q
i+1
x
i+1
. . . Q
m
x
m
is true,
0 otherwise.
f
0
() is the truth value of . Again we can nd polynomial expressions for the
f
i
s. We now have the identities
Q
i+1
= : f
i
(a
1
, . . . , a
i
) = f
i+1
(a
1
, . . . , a
i
, 0) f
i+1
(a
1
, . . . , a
i
, 1), (9.2)
Q
i+1
= : f
i
(a
1
, . . . , a
i
) = f
i+1
(a
1
, . . . , a
i
, 0) f
i+1
(a
1
, . . . , a
i
, 1), (9.3)
where was dened in (9.1). Now we run the same protocol as in the proof of
Theorem 9.2.1. A new problem is that in the identities (9.2) and (9.3) every
Q
i
potentially doubles the degree of the polynomial (so that they may grow
exponential), whereas it is essential in the proof of Theorem 9.2.1 that we
work with polynomials of a degree bounded by a xed n size(). To keep
the degree small we perform the following trick. Given a polynomial f(x) in
variable x, note that
f(x) = (1 x)f(0) + xf(1)

is linear in x and coincides with f on Boolean values of x. If every time (9.2)
or (9.3) is applied we rst perform this transformation for every x
j
, j i (i.e.
those variables whose degree might go up) then the degree is kept bounded
by 2, without changing the result on Boolean values. The obstacle thus being
removed, the rest of the proof is the same as before.
A full account of the proof of Theorem 9.2.2 can be found in Sipser [27].
9.3 Zero-knowledge proofs
Using public-key cryptography and interactive proofs, it is possible to con-
vince someone that you are in possession of a proof without actually giving
anything away about it. Such proofs (which are actually interactive proce-
dures) are called zero-knowledge proofs.
As an example consider the NP-complete problem 3COLORING. Suppose
Alice wants to convince Bob that a given graph G = (V, E) is 3-colorable.
66 Chapter 9. Interactive proofs
She could of course do this by giving Bob a 3-coloring after which Bob could
eciently check that it is correct, but Alice wants to keep her 3-coloring secret
and still convince Bob that she has it. This seemingly impossible task can be
achieved by the following interactive protocol.
In every round Alice chooses a random permutation of the 3 colors,
and for every node i chooses an RSA tuple (p
i
, q
i
, e
i
, d
i
). For every i she
announces to Bob (p
i
q
i
, e
i
) and the encoded version of ((i)), the color of i
after applying . At this point Bob cannot do anything with this information
since he does not have the decoding key d
i
, but Alice has made a commitment,
albeit encrypted, as to what the colors are. Next, it is Bobs turn to test Alice
by picking at random an edge (i, j) of the graph, after which Alice reveals to
Bob the secret keys d
i
and d
j
that allow Bob to decode ((i)) and ((j))
and see if they are dierent, as they should be. This ends one round of the
protocol.
Now if Alice uses indeed a 3-coloring of the graph, Bob will always be
satised. If not, in every round there is an edge (i, j) where ((i)) = ((j)).
There are |E| edges, so in every round Bob has a chance at least
1
|E|
of
discovering a mistake. By iterating a polynomial number of times this chance
becomes exponentially close to 1.
Let us check that Bob has indeed learned nothing about Alices 3-coloring
in the course of this procedure. In every round Bob receives a number of keys
and a pair of colors. But these are randomly permuted versions of the original
colors, hence nothing more than a random pair of dierent colors. Since in
the next round Alice chooses another random permutation, Bob might just as
well ip a coin himself to generate such random pairs and he learns absolutely
nothing from this.
The NP-completeness of 3COLORING can be used to show that all prob-
lems in NP have zero-knowledge proofs, cf. [13].
9.4 Exercises
Exercise 9.4.1. Show that the class IP does not change if we require that
the prover P is in EXP instead of unbounded.
Exercise 9.4.2. The original proof of Theorem 9.2.2 employed the notion
of a simple formula. A rst-order formula is simple if no occurrence of a
variable is separated by more than one universal quantier from its point of
quantication. Show that by introducing new variables every can be eec-
tively transformed to an equivalent simple formula. How many new variables
are needed at most?
Chapter 10
Approximations
If you cant do the real thing, you can try to settle for less. In this chapter
we discuss one particular way of what less could mean in the case of hard
computational problems.
10.1 Approximating the traveling salesman
Recall the problem TSP from section 2.5. In general this problem is NP-
complete (Corollary 3.4.2), so we have currently no feasible way to deal with
the problem in general. Does it become easier when we ask for approximate
solutions? In particular, say we only ask for a polynomial time algorithm
M that, given an instance x of TSP, always returns a solution M(x) that is
within a factor r of an optimal solution S(x), i.e. such that
c(M(x)) r c(S(x)), (10.1)
where c is the length of the tour and r [1, ) is xed. One can think
of many situations where having such an algorithm for some not too large
factor r would be quite helpful. The bad news is that no matter how large
we choose r, such an algorithm does not exist, unless we could already solve
the original problem in polynomial time!
Theorem 10.1.1. (Sahni and Gonzalez) Let r 1 be a constant. There is
no polynomial time algorithm M that approximates TSP to within a factor r,
as in (10.1), unless P = NP.
Proof. Suppose that such an M exists. We show how to solve the NP-
complete problem HAMILTON CIRCUIT in polynomial time. The idea is
very similar to that of Corollary 3.4.2. Given a graph G = (V, E) we use M
to decide whether G has a Hamilton circuit. Dene an instance I of TSP
as follows. The cities are the points in V = {1, . . . , n}, and we dene their
67
68 Chapter 10. Approximations
distances by
d
i,j
=
{
1 if (i, j) E,
rn otherwise.
Note that if G has a Hamilton circuit then I has a tour of length n, and
that if G does not have a Hamilton circuit then every tour in I has length at
least rn + (n 1), which is at least rn + 1 assuming that n 2, which we
may assume since the case n = 1 is trivial. Now apply M to the instance I.
Then G has a Hamilton circuit if and only if the length of the tour M(I) is
at most rn.
10.2 Approximation algorithms
In general, for every instance x of an optimization problem A we have a
possible set of solutions F(x) that we try to minimize or maximize. This
means that every possible solution s F(x) has a certain cost c(s). In case
of a minimization problem, the optimum cost is then dened as
OPT(x) = min
sF(x)
c(s).
(In case of a maximization problem of course we take the maximum.)
Denition 10.2.1. An -approximation algorithm M for a minimization
problem A is an algorithm that for each instance x always returns a solution
M(x) F(x) such that
c(M(x)) OPT(x)
c(M(x))
. (10.2)
The ratio on the left is called the relative error of M(x). The approximation
threshold of A is the greatest lower bound of all > 0 such that there exists a
polynomial time -approximation algorithm for A. Note that this threshold
is always a number in the interval [0, 1].
With the above denitions, we can reformulate Theorem 10.1.1 as follows:
Theorem 10.2.2. The approximation threshold of TSP is 1, unless P = NP.
Proof. Suppose that M is a polynomial time -approximation algorithm for
TSP. Then by (10.2) we have c(M(x)) OPT(x) c(M(x)), hence
c(M(x))
1
1
OPT(x).
By Theorem 10.1.1 this cannot hold for any < 1.
10.3. Probabilistically checkable proofs 69
Denition 10.2.3. A polynomial time approximation scheme (PTAS) for an
optimization problem is an algorithm that for each > 0 and each instance
x outputs a solution with relative error at most , as in (10.2), and that runs
in time polynomial in |x|. If the running time is also polynomial in
1
then
the scheme is called fully polynomial.
Denition 10.2.3 points to an interesting feature, namely that computational
problems may have several parameters, and that algorithms to solve them may
be polynomial in some of parameters but not in others. The eld in which
this phenomenon is systematically explored is called parameterized complexity
theory, and the standard textbook on this subject is Downey and Fellows [12].
The reader should note that approximability is not preserved by p-m-
reductions: Some NP-complete problems, such as TSP, do not have a PTAS
(unless P = NP, cf. Theorem 10.2.2), whereas others, such as KNAPSACK,
do have one. It is possible to devise a type of reduction that preserves approx-
imability, and to develop the theory of approximation problems analogous to
the theory of NP-completeness.
10.3 Probabilistically checkable proofs
In this section we discuss the notion of probabilistically checkable proof, and
the main theorem about this notion, the PCP Theorem. Although we will
not prove this theorem, we discuss some of its consequences for the theory of
approximations, where the theorem has had a profound inuence.
Suppose that L is a language in NP. By Proposition 4.2.4 there is a
polynomial time predicate R such that
L =
{
x :
p(|x|)
y (x, y) R
}
.
This implies the existence of a verier V for L, that, given any input x and
any certicate y of length at most p(|x|), checks whether y is indeed proof of
the fact that x L or not. The verication can be done in polynomial time,
and in principle uses all of the bits of y. In the following we will see that we
can greatly improve on the number of bits examined if we are willing to make
V probabilistic.
Denition 10.3.1. A probabilistic verier is a probabilistic algorithm (in
the sense of section 6.1) that given an input x and a proof as oracle,
examines the proof and accepts or rejects it with a certain probability. Given
functions r(n) and q(n), a language L is in the class PCP[r(n), q(n)] if there
is a polynomial time probabilistic verier V that on an input x of length n
uses r(n) random bits and at most q(n) queries to the oracle such that
x L Pr
[
V
(x) = 1
]
= 1,
x / L Pr
[
V
(x) = 1
]
1
2
.
Here the probabilities are taken over all choices of the r(n) random bits.
The following theorem is one of the landmarks of the subject. It was the
culmination of a series of papers by a large number of authors, with Sanjeev
Arora as one of the main contributors. For more information about the history
of the result we refer to [23].
Theorem 10.3.2. (PCP Theorem) There exists a constant q N such that
NP =
c1
PCP[c log n, q].
Hence in order to verify with high probability that a given instance x is in
some NP-language, we only need to examine a constant number of bits of the
proof, regardless the length of x. Furthermore, the constant is the same for all
languages in NP. The proof of Theorem 10.3.2 is too long and complicated
to discuss here. By now there is an somewhat easier proof by Irit Dinur
available. We recommend the exposition of Dinurs proof in [23]. Despite the
various improvements in presentation, the proof remains beyond the scope of
these course notes.
10.4 PCP theory and nonapproximability
Consider the following optimization problem MAX3SAT:
Input: a 3CNF formula (with three literals per clause),
Question: nd an assignment satisfying the largest number of clauses of .
Since 3SAT is NP-complete by Theorem 3.3.4, MAX3SAT is not solvable in
polynomial time, unless P = NP.
Proposition 10.4.1. The approximation threshold of MAX3SAT is at most
1/2.
Proof. Given a formula in CNF, consider the all-zero and all-one assignments.
Since every clause is satised by at least one of these, one of them satises at
least half of the clauses, so pick one that does this.
In fact, it is possible to improve Proposition 10.4.1 from
1
2
to
1
8
, cf. Exer-
cise 10.5.5.
By the following theorem, the approximation threshold of MAX3SAT is
bigger than 0. In particular there is no PTAS for MAX3SAT, unless P = NP.
Denition 10.4.2. We say that a CNF formula is -far from satisable if
for every assignment, at most a fraction 1 of the clauses of is satised.
Theorem 10.4.3. There exists > 0 such that for every L NP there is a
p-m-reduction x
x
of L to 3SAT such that
x L
x
satisable,
x / L
x
-far from satisable. (10.3)
10.4. PCP theory and nonapproximability 71
Proof. By the PCP theorem, let V be a probabilistic verier witnessing that
L PCP[O(log n), O(1)]. Let r {0, 1}
O(log |x|)
range over all the random
strings for V . For every r, V makes a constant number of queries to the
proof . Without loss of generality we may assume that the queries are
nonadaptive (since an exponential of a constant number of queries is still
constant). Suppose that the number of queries is . For each choice of r,
V
(x) depends on a Boolean function f

r
: {0, 1}
{0, 1} describing all

possible answers to the queries. For the translation into a 3CNF-formula,
introduce variables x
1
, . . . , x
k
that stand for all positions of that can be
queried, i.e. x
i
stands for (i). The number k equals the length of the proof
, which is polynomial in |x|. We can represent f
r
by a CNF-formula
r
with
2
clauses (cf. Proposition 7.1.2) using the variables x

q
1
, . . . , x
q
corresponding
to the queries V makes with r. To convert every clause of length to a clause
with three literals (as in Theorem 3.3.4) we need for every clause auxiliary
clauses, so in total
r
has at most 2
clauses. For every r we use a dierent

set of auxiliary variables. Finally, we let
x
=
r

r
. Now
x L Pr
[
V
(x) = 1
]
= 1
satises all clauses of
x
.
x / L Pr
[
V
(x) = 1
]
1
2
for at least half of the r, does not satisfy
r
, i.e.
at least one clause of
r
is false,

x
is -far from satisable, where =
1
2
1
2
.
Theorem 10.4.3 is a consequence of the PCP theorem. Interestingly, the
converse also holds:
Theorem 10.4.4. If L NP and L has a reduction as in (10.3) then L
PCP[O(log n), O(1)].
Proof. We dene a probabilistic verier V for L as follows. Given x, compute
x
as in (10.3), and interpret the proof as an assignment for
x
. V randomly
picks c =
1
ln 2 clauses of
x
and checks whether satises all of them. If
so, V accepts x, and V rejects x otherwise. To specify an item in a list of n
objects takes log n bits, and since the number of clauses in
x
is polynomial
in |x| we need c log |x|
O(1)
= O(log |x|) random bits. Checking these clauses
takes O(1) bits of . Now
x L
x
satisable
Pr
[
V
(x) = 1
]
= 1
x / L at least a fraction of the clauses of
x
is false
Pr
[
V
(x) = 1
]
(1 )
c
To nish the proof, we just have to see that (1 )
c
1
2
for c =
1
ln 2, which
easily follows from the inequality 1 x e
x
.
10.5 Exercises
Exercise 10.5.1. PCP[0, O(log n)] = P.
Exercise 10.5.2. PCP[0, poly n] = NP.
Exercise 10.5.3. Prove the easy direction of Theorem 10.3.2, namely that
PCP[O(log n), O(1)] NP.
Exercise 10.5.4. PCP[O(log n), 1] = P.
Exercise 10.5.5. Show that the approximation threshold of MAX3SAT is
at most
1
8
. (Hint: Use the probabilistic method: Assign the value 1 to every
variable x with probability
1
2
and compute the expected number of satised
clauses.)
Chapter 11
Proof complexity
In mathematical logic, the complexity of a theorem can sometimes be mea-
sured by considering the strength of the axioms that are needed to prove it.
There are interesting complexity theoretic analogues of this theme, that we
discuss in this chapter. Godels classic incompleteness theorem tells us that,
no matter how strong our axioms, not every true arithmetical formula is prov-
able. Here we consider the length of proofs, and show that it is unlikely that
every true propositional formula has a short proof. Short here of course
means of polynomial size in the length of the formula.
11.1 Propositional proof systems and Cooks program
Recall that SAT is the set of all satisable propositional formulas. Consider
its companion VALID consisting of all valid propositional formulas, that is,
those formulas that are valid under all assignments.
By Exercise 4.4.7, VALID is co-NP-complete. Hence, if VALID NP
then NP = co-NP, something that is believed to be unlikely. This means
that, if is indeed true that NP = co-NP, not every valid formula has a short
certicate that proves its validity. This can be seen as a bounded version of
Godels incompleteness theorem: Not every true propositional formula has a
short proof.
Denition 11.1.1. A propositional proof system is any polynomial time com-
putable function F : {0, 1}
VALID. (We can without loss of generality

require that F is also surjective, cf. Exercise 11.4.2.) If F(x) = , then x is a
proof that is valid. The restriction that F be polynomial time computable
is natural: given a proof we certainly want to be able to check eciently that
it indeed is a proof.
If NP = co-NP then for every propositional proof systemF and every constant
c there are innitely many valid formulas such that for || = n, for no
x {0, 1}
n
c
we have F(x) = . So if we ever want to prove that NP = co-NP,
73
74 Chapter 11. Proof complexity
we should at least be able to prove this for any given F. Cooks program
1
is
to come up with stronger and stronger proof systems, and with formulas that
have superpolynomial proof complexity in those systems. One nice aspect of
this program is that we can measure our progress on such a dicult question
as NP = co-NP.
Let us start simple. The most naive proof system for propositional logic is
the method of truth tables (Truth tables, apparently rst appeared, in their
modern form, in Wittgenstein, Tractatus Logico-Philosophicus 4.31 .) Since
the truth table of a formula with n propositional variables has size O(2
n
),
every proof of validity using truth tables has exponential size, regardless
which formula we are proving. So we have made our rst step in Cooks
program, namely we have proven an exponential lower bound for the proof
system of truth tables.
Of course the problem becomes more dicult for more sophisticated proof
systems F. The problem is open for several of the standard proof systems
for classical propositional logic. In section 11.3 we consider proofs based on
resolution. One possibility for making some progress is to consider proof sys-
tems that are weaker than those for classical logic. An obvious candidate is to
consider intuitionistic logic. A breakthrough result for this logic was obtained
by Hrubes [15]. He showed that in a proof system for intuitionistic proposi-
tional logic, exponential lower bounds for certain tautologies can indeed be
proven. (N.B. The set of intuitionistic validities is PSPACE-complete, and
it is a subset of VALID.) Hrubes uses a tautology similar to the pigeonhole
principle discussed below, namely k-colorability: a complete graph on k + 1
nodes is not k-colorable. This has a short classical proof, but it does not have
a short proof in an intuitionistic Frege system.
11.2 The pigeonhole principle
As discussed above, part of the task in Cooks program is, given a proof sys-
tem F, to come up with candidate formulas that are supposed to only have
superpolynomial proofs. For some time it was hoped that the combinato-
rial statement called the pigeonhole principle would provide such formulas.
This statement says that if we have m pigeons that we have to put into n
holes, and m > n, then there is a pigeonhole containing more than one pi-
geon. By Exercise 11.4.3, the pigeonhole principle can be formulated as a
Boolean formula PHP
m
n
. The hope that the formulas PHP
m
n
would only have
superpolynomial proofs was crushed when Buss [8] showed that in fact they
do have polynomial size proofs, in a certain Hilbert style proof system for
classical logic. Such systems, based on modus ponens, are called Frege sys-
1
The connection between proof systems and NP = co-NP was rst made in Cook and
Reckhow [11]. For a recent survey see Buss [9].
11.3. Resolution 75
tems in this context. The formulas PHP
m
n
can be used, however, to show an
exponential lower bound for resolution, see the proof of Theorem 11.3.4.
11.3 Resolution
In this section we discuss a simple propositional proof system, namely reso-
lution. In resolution there is only one rule, which speeds up the search for
proofs, but the method only works for formulas in CNF. (Recall that we have
seen in section 3.3 that converting formulas to CNF may be expensive. Recall
also that the CNF fragment of SAT is still NP-complete.) Resolution is also
the method underlying the declarative programming language Prolog.
In the following we use the terminology from section 1.4.
Denition 11.3.1. If C
1
= {l} C
1
and C
2
= {
l} C
2
are clauses, where
denotes that we are taking a union of disjoint sets, then C
1
C
2
is called a
resolvent of C
1
and C
2
.
Note that resolution is sound, i.e. preserves satisability. That is, if both
of the parent clauses are satisable then also their resolvent is satisable.
A resolution proof of a clause C from a formula S is a nite sequence
C
1
, C
2
, . . . , C
n
= C of clauses such that each C
i
is an element of S or a
resolvent of clauses earlier in the sequence. If such a proof exists we write
S
R
C. If S
R
we say that S is refutable.
We can in principle picture resolution proofs as binary trees. For example,
the following is a refutation proof from the set S =
{
{p, r}, {q, r}, {q},
{p, t}, {s}, {s, t}
}
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
{p} {p}
{p, q}
{q}
{p, s}
{s}
{p, r}
{q, r}
{p, t}
{s, t}
Note however that the representation of a resolution proof as a tree may be

costly in case a clause is repeated several times in the proof, which may result
in a repetition of parts of the tree.
Theorem 11.3.2. (Soundness of propositional resolution) S
R
= S
unsatisable.
Proof. Obviously, if V is an assignment satisfying C
1
and C
2
, then also V |= C
for any resolvent C of C
1
and C
2
. Hence if S is satisable then S
R
.
Resolution is not complete in the sense that for every set of clauses S and
every clause C, whenever S |= C then S
R
C. For example, |= p p but
R
{p, p}. However, resolution is complete in the sense that any inconsistent
S is refutable. This fact is called refutation completeness.
Theorem 11.3.3. (Refutation completeness of propositional resolution) S
unsatisable = S
R
.
Proof. Cf. [20].
The rst superpolynomial lower bounds for resolution were proven by Tseitin
[30], and the rst exponential ones by Haken [14]. The proof of the latter
result uses the following representation of PHP
m
n
as a CNF-formula: For
every i m and j n there is a variable p
ij
with the intended meaning
pigeon i is in hole j. For every i we have a clause p
i1
p
i2
. . . p
in
expressing that i is in some hole, and for every k and i = j we have a clause
p
ik
p
jk
expressing that no hole k gets two pigeons i and j. Taken together
these clauses express the formula PHP
m
n
, which is of course false if m > n.
Theorem 11.3.4. (Haken [14]) There are unsatisable formulas that only
have exponential size resolution refutations. More specic, for every n 2,
any resolution refutation of PHP
n
n1
has size at least 2
n
20
.
Proof. We follow the presentation in [2]. We view a resolution refutation as
an analysis showing that there is no assignment from the set of all possible
assignments that satises all the given clauses. The analysis remains valid if
we restrict attention to a subset of all possible assignments. We will restrict
attention to the following set of assignments: An assignment for PHP
n
n1
is i-critical if it assigns n 1 pigeons in an injective way to n 1 holes and
leaves pigeon i unassigned.
Restricting attention to the above set of critical assignments has the ad-
vantage that we can make all clauses in the refutation monotone, that is,
containing only positive literals. (Strictly speaking, changing the clauses
makes the refutation no longer a resolution proof, but it is equivalent to it
as far as the assignments under consideration are concerned.) We make the
proof monotone by replacing in every clause every negative literal p
ij
by
k=i
p
kj
.
Note that for critical assignments these are equivalent, since such an assign-
ment assigns a pigeon to every hole.
11.3. Resolution 77
Call a clause in the proof large if it has at least
n
2
10
variables, and let L
be the number of large clauses. All large clauses together have at least
n
2
10
L
occurrences of variables, and there are n
2
variables p
ij
, so there must be a
variable p
ij
that occurs at least
1
10
L times in the L large clauses. Setting
p
ij
= 1 and p
ij
= 0 for every j
= j and p
i
j
= 0 for every i
= i leaves at
most
9
10
L large clauses. Since this removes one pigeon and one hole, we now
have a monotone refutation of PHP
n1
n2
.
We repeat this t times, where t is so large that (
9
10
)
t
L < 1, i.e. (
10
9
)
t
> L,
so we let t = log10
9
L (rounded up, plus 1 if necessary). This removes all the
large clauses, and leaves us with a monotone refutation of PHP
nt
nt1
.
Now if L < 2
n
20
(which would be the case if the original refutation had
size less than 2
n
20
) then t = log10
9
L <
n
20
log10
9
2, from which it follows that
2
9
(n t)
2
>
1
10
n
2
.
So we have a monotone refutation of PHP
nt
nt1
with no clauses with
2
9
(nt)
2
variables, contradicting Lemma 11.3.5.
Lemma 11.3.5. Every monotone refutation of PHP
n
n1
contains a clause
with at least
2
9
n
2
variables.
Proof. For every clause C dene
witness(C) =
{
i : there is an i-critical assignment falsifying C
}
,
and dene the complexity comp(C) to be |witness(C)|. If C is a resolvent of
C
and C
then comp(C) comp(C
) + comp(C
) because any assignment

falsifying C must falsify at least one of C
and C
. Note that if comp(C)

2
3
n
then comp(C
) + comp(C
)
2
3
n, hence comp(C
)
1
3
n or comp(C
)
1
3
n.
So if C is the rst clause in the proof with comp(C)
1
3
n (which exists since
comp() = n) then
1
3
n comp(C)
2
3
n.
We show that C has at least
2
9
n
2
variables, namely, that if comp(C) = x then
it contains at least x(n x) variables. This suces since x(n x)
2
9
n
2
for
1
3
n x
2
3
n.
Suppose i witness(C) and is an i-critical assignment falsifying C. For
each j / witness(C), consider the assignment
obtained by replacing j by i:
if maps j to hole k,
maps i to hole k and leaves j unassigned. Then
is j-critical, and since j / witness(C) it satises C. Since
satises C
and does not, this can only be because
makes p
ik
true, and in particular
C must contain p
ik
. Repeating the argument for every j / witness(C), and
noting that for every j the assignment maps j to a dierent hole k, we see
that C contains (n x) distinct variables p
ik
. Since there are x dierent i in
witness(C), this gives x(n x) dierent variables p
ik
in total.
11.4 Exercises
Exercise 11.4.1. Show that if we drop the requirement in Denition 11.1.1
that F is polynomial time computable then there is a proof system in which
every valid formula has a short proof.
Exercise 11.4.2. Show that in Denition 11.1.1 we can require without loss
of generality that F is surjective.
Exercise 11.4.3. Formulate the pigeonhole principle for m pigeons and n
holes as a CNF-formula PHP
m
n
of propositional logic. Introduce variables x
i,j
with the meaning pigeon i is in hole j.
Exercise 11.4.4. Let PHP
m
n
the formula from Exercise 11.4.3. Write PHP
3
2
in clausal form, and prove it using resolution.
Further reading
By now there is a large number of textbooks available for the subject, and
several texts and notes are electronically available. The following is a very
incomplete list:
M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide
to the Theory of NP-Completeness, W. H. Freeman, 1979.
J. L. Balcazar, J. Daz, and J. Gabarro, Structural Complexity, two
volumes, Springer, 1988 (Vol. I) and 1990 (Vol. II).
C. H. Papadimitriou, Computational Complexity, Addison Wesley, 1994.
M. Sipser, Introduction to the Theory of Computation, Course Technol-
ogy, 2006.
S. Arora and B. Barak, Computational Complexity: A Modern Ap-
proach, Cambridge University Press, 2009.
L. Trevisan, Lecture Notes on Computational Complexity, available at
web pages of the author.
H. S. Wilf, Algorithms and complexity, available at web pages of the
author.
79
Bibliography
[1] M. Agrawal, N. Kayal, and N. Saxena, PRIMES is in P, Annals of Math-
ematics 160 (2004) 781793.
[2] S. Arora and B. Barak, Computational Complexity: A Modern Approach,
Cambridge University Press, 2009.
[3] T. Baker, J. Gill, and R. M. Solovay, Relativizations of the P=NP ques-
tion, SIAM Journal on Computing 4 (1975) 431442.
[4] J. L. Balcazar, J. Daz, and J. Gabarro, Structural Complexity I,
Springer, 1988.
[5] R. Beigel, N. Reingold and D. Spielman, PP is closed under intersection,
Journal of Computer and System Sciences 50(2) (1995) 191202.
[6] M. Blum, A machine-independent theory of the complexity of recursive
functions, Journal of the ACM 14 (1967) 322336.
[7] J. Buhler and S. Wagon, Basic algorithms in number theory, in: Algo-
rithmic number theory, MSRI Publications 44 (2008) 2568.
[8] S. R. Buss, Polynomial size proofs of the propositional pigeonhole prin-
ciple, Journal of Symboloc Logic 52 (1987) 916927.
[9] S. R. Buss, Towards NP-P via proof complexity and search, manuscript,
2011.
[10] S. A. Cook, A hierarchy for nondeterministic time complexity, Proc. 4th
ACM Symposium on Theory of Computing (1972) 187192.
[11] S. A. Cook and R. A. Reckhow, On the lengths of proofs in the propo-
sitional calculus, Proceedings of the Sixth Annual ACM Symposium on
the Theory of Computing, (1974) 135148.
[12] R. Downey and M. R. Fellows, Parameterized complexity, Springer, 1999.
80
Bibliography 81
[13] O. Goldreich, S. Micali, and A. Wigderson, Proofs that yield nothing
but their validity, and a methodology of cryptographic protocol design,
Proc. 27th IEEE Symp. on the Foundations of Computer Science (1986)
174187.
[14] A. Haken, The intractability of resolution, Theoretical Computer Science
39 (1985) 297308.
[15] P. Hrubes, A lower bound for intuitionistic logic, Annals of Pure and
Applied Logic 146 (2007) 7290.
[16] N. Immerman, Nondeterministic space is closed under complementation,
SIAM Journal on Computing 17 (1988) 935-938.
[17] R. Karp and R. Lipton, Turing machines that take advice, Lensignement
mathematique 28 (1982) 191210.
[18] R. Ladner, On the structure of polynomial time reducibility, Journal of
the ACM 22 (1975) 155-171.
[19] R. Ladner, N. Lynch, and A. Selman, Comparison of polynomial-time re-
ducibilities, Proc. 6th ACM Symposium on Theory of Computing (1974)
110121.
[20] A. Nerode and R. A. Shore, Logic for applications, 2nd edition, Springer
1997.
[21] P. G. Odifreddi, Classical recursion theory, Vol. 2, Studies in logic and
the foundations of mathematics Vol. 143, North-Holland, 1999.
[22] C. H. Papadimitriou, Computational complexity, Addison Wesley, 1994.
[23] J. Radhakrishnan and M. Sudan, On Dinurs proof of the PCP theorem,
Bulletin of the AMS 44 (2007) 1961.
[24] S. Rudich and A. Wigderson (eds.), Computational complexity theory,
American Mathematical Society, 2004.
[25] R. Szelepcsenyi, The method of forcing for nondeterministic automata,
Bulletin of the EATCS 33 (1987) 96100.
[26] R. Solovay and V. Strassen, A fast Monte-Carlo test for primality, SIAM
Journal on Computing 6(1) (1977) 8485.
[27] M. Sipser, Introduction to the theory of computation, 2nd edition, Brooks
Cole, 2005.
82 Bibliography
[28] S. A. Terwijn, Syllabus computability theory, Technical University of Vi-
enna, 2004. Available at web pages author.
[29] L. Trevisan, Lecture Notes on Computational Complexity, available at
web pages of the author.
[30] G. S. Tseitin, On the complexity of derivation in propositional calcu-
lus, In: A. O. Slisenko (ed.), Studies in constructive mathematics and
mathematical logic (1970) 115125.
Index
C/F, 52
C
A
, 30
A, complement, 2
A
x, restriction, 2
C, 28
p(n)
, 28
p(n)
, 28
, empty clause, 3
co-C, 2
xy, concatenation, 2
, disjoint union, 75
, 15
K, 17
|x|, length of x, 2
M(x), halting computation, 1
M(x), diverging computation, 1
x, negated literal, 3
O(f), 2
o(f), 2
(f), 3
(f), 3
M
B
, oracle machine, 26
p
T
, 26
p
m
, 15
m
, 15
, , pairing function, 2
PHP
m
n
, pigeonhole principle, 74
poly, 52
R
, resolution provable, 75
, 2
{0, 1}
n
, 2
BPP, 43, 44
E, 23
EXP, 9, 23
IP, 62, 63
LIN, 23
NLIN, 23
NP, 9
P, 9
PCP[r(n), q(n)], 69
PH, 27, 46
PP, 41, 46
PSPACE, 9, 63
P/poly, 52
RP, 46
SIZE(s(n)), 51
UP, 60
ZPP, 46
p
n
, 27
p
n
, 27
p
n
, 27
3COLORING, 16, 65, 66
CLIQUE, 19
GRAPH ISOMORPHISM, 34, 62
HAMILTON CIRCUIT, 24
HAMILTON PATH, 11, 20, 24
HORNSAT, 24
MAJ, 41
MAX3SAT, 70
PATH, 7
PRIMES, 11, 47
QBF, 18, 20, 34
SA, 42
SAT, 10, 1618, 23
SAT-CNF, 18
#SAT, 63
TSP, 11, 14, 22, 67
VALID, 30, 73
kDNF, 24
nSAT, 18
83
84 Index
adaptive query, 26
Adleman, 44
advice, 44, 52
advice function, 52
approximation threshold, 68
Arora, 70
Baker, 34
Berman-Hartmanis Conjecture, 36,
39
Blum, 38
Borodin, 37
bounded quantication, 28
Buss, 74
c.e., 2, 14
Cantor, 8, 32
certicate, 10
Chinese remainder theorem, 3
circuit, 44, 51
circuit size, 51
clause, 3
monotone, 76
CNF, 3, 18
collapse, 29
complete, 16
completeness
refutation, 76
computable, 2
conjunctive normal form, 3
context-sensitive, 13
Cook, 17, 74
delayed diagonalization, 32
density, 39
diagonalization, 8
Dinur, 70
disjunctive normal form, 3
DNF, 3
-far, 70
error probability, 41
Euler, 4
Eulers criterion, 47
Eulers function, 3
fan-in, 51
fan-out, 51
Fermat, 4
Fermats little theorem, 4
Frege system, 75
Friedberg-Muchnik Theorem, 39
gap theorem, 37
Gill, 34
Godel, 23, 73
guess, 10
Haken, 76
halting problem, 32
hard, 16
honest function, 14, 59
Horn formula, 24
Hrubes, 74
Immerman, 12
incomparable degrees, 39
index, 38
intermediate degree, 32
intuitionistic logic, 74
Jacobi symbol, 47
join operator, 15
Karp, 54
Kleene star operator, 2
Ladner, 32, 33, 36
language, 2
Las Vegas algorithm, 47
law of quadratic reciprocity, 47
Legendre symbol, 47
Levin, 17
linear speed-up, 7
Lipton, 54
literal, 3
low
2
, 34
Lynch, 36
m-reducible, 15
Index 85
Mahaney, 36
Monte Carlo algorithm, 46
Myhill-Nerode Theorem, 13
NP-complete, 17
one-way function, 59
optimization problem, 68
oracle machine, 26
p-isomorphic, 36
p-m-degree, 15
p-m-reducible, 15
p-T-degree, 27
padding, 22
parity function, 55
PCP Theorem, 69
pigeonhole principle, 74
polynomial hierarchy, 27
positive reductions, 23
Pratt, 11
primitive root, 4
probabilistic method, 45, 72
probabilistic Turing machine, 41
probabilistic verier, 69
problem, 2
propositional proof system, 73
prover, 62
PTAS, 69
quadratic residue, 47
quantied Boolean formula, q.b.f.,
18
query, 26
Reckhow, 74
refutable, 75
regular sets, 13
relative error, 68
relativize, 30
resolution, 75
proof, 75
resolvent, 75
RSA, 57
Savitch, 7, 20
scale, 38, 40
Sch oning, 34
Schwartz-Zippel theorem, 64
self-reducible, 23, 54
Selman, 36
set, 2
Sipser, 44, 54
Skolem function, 53
Skolemization, 53
Solovay, 34, 48
soundness
resolution, 75
space constructible, 6
space hierarchy theorem, 8
sparse, 36
speed-up theorem, 38
Sterne, 33
Strassen, 48
Szelepcsenyi, 12
tally, 36
tape compression, 7
time constructible, 6, 37
time hierarchy theorem, 8, 37
Toda, 46
tour, 11
Trakhtenbrot, 37
traveling salesman problem, 11, 67
truth table, 74
Tseitin, 76
Turing machine, 1
Turing reducible, 26
unambiguous machine, 60
Use Principle, 34
verier, 62, 69
von Neumann, 23
Wittgenstein, 74
word, 2
XOR, 55
zero-knowledge proof, 65

Complexity Theory

Uploaded by

Complexity Theory

Uploaded by

Complexity Theory

. (This set is also referred to

are routinely handled as such. Subsets A {0, 1}

x denotes the nite set A {0 . . . x}, or equivalently, the nite

x(x) to denote that (x) holds for almost

x(x) denotes that there are innitely many x such that

denotes the multiplicative group of integers x with gcd(x, n) = 1.

is denoted by (n). The function is

above holds. We prove that p is prime. If r

1 mod p then the order of r divides p 1. By (ii)

has order p 1, and p is

does not hold.

n(n < r t(n)). This result is called the linear speed-

(s), i.e. such that c > 0

(n) > c s(n)).

as above, consider an eective enumeration {M

is innitely often large enough to complete a computation of M

In this exercise we prove that the class of regular sets equals

Show that P is closed under Kleene star. This is much

= j + 1 and w[i, j] and w[i

is called honest if there is

is in CNF and of size at most c m

be the conjunction of all

is equivalent to F. Note that the -step

Figure 3.1: The truth module

Figure 3.3: The XOR module

Figure 4.1: The polynomial hierarchy

n and only queries smaller than n are made, the

is a p-isomorphism if f is bijective and has a polynomial time com-

Use delayed diagonalization to prove that the p-m-degrees

Same as Exercise 5.7.1, but now for

Assuming that NP = P, show that there are incomparable

There are no sparse p-m-complete sets for EXP.

Figure 6.2: Relations between complexity classes

X then the left

if for every length n and every

. We think of s as giving an advice

) then there is an implicit Skolem function f : N N

Finish the proof of Theorem 7.3.2.

of messages of previous rounds.

f(x) = (1 x)f(0) + xf(1)

(x) depends on a Boolean function f

{0, 1} describing all

clauses (cf. Proposition 7.1.2) using the variables x

clauses. For every r we use a dierent

VALID. (We can without loss of generality

Note however that the representation of a resolution proof as a tree may be

then comp(C) comp(C

) because any assignment

. Note that if comp(C)

maps i to hole k and leaves j unassigned. Then

is j-critical, and since j / witness(C) it satises C. Since

You might also like