Analysis of Algorithms

NP Completeness

1
 Analysis of Algorithms
• Module 1: Introduction to Algorithms
• Module 2 : Divide and Conquer Approach
• Module 3 : Greedy Approach
• Module 4: Dynamic Programming
• Module 5 : Backtracking & Branch and Bound
• Module 6 : String Matching

2
Learning Objectives
• P Problem
• NP Complete Problem
• NP Hard Problem

3
NP-Completeness
• So far we’ve seen a lot of good news!
• Such-and-such a problem can be solved quickly (i.e., in
close to linear time, or at least a time that is some small
polynomial function of the input size)
• NP-completeness is a form of bad news!
• Evidence that many important problems can not be
solved quickly.
• NP-complete problems really come up all the time!

4
Optimization & Decision
Problems
• Decision problems
• Given an input and a question regarding a problem,
determine if the answer is yes or no
• Optimization problems
• Find a solution with the “best” value
• Optimization problems can be cast as decision
problems that are easier to study
• E.g.: Shortest path: G = un-weighted directed graph
• Find a path between u and v that uses the fewest edges
• Does a path exist from u to v consisting of at most k edges?

5
Class of “P” Problems
• Class P consists of (decision) problems that are solvable
in polynomial time
• Polynomial-time algorithms
• Worst-case running time is O(nk), for some constant k

• Examples of polynomial time:

• O(n2), O(n3), O(1), O(n lg n)
• Examples of non-polynomial time:
• O(2n), O(nn), O(n!)

6
Tractable/Intractable Problems
• Problems in P are also called tractable
• Problems not in P are intractable or unsolvable
• Can be solved in reasonable time only for small inputs
• Or, can not be solved at all

• Are non-polynomial algorithms always worst than

polynomial algorithms?
- n1,000,000 is technically tractable, but really impossible
- nlog log log n is technically intractable, but easy

7
Example of Unsolvable Problem
• Turing discovered in the 1930’s that there are
problems unsolvable by any algorithm.
• The most famous of them is the halting problem
• Given an arbitrary algorithm and its input, will that
algorithm eventually halt, or will it continue forever in
an “infinite loop?”

8
Examples of Intractable Problems

9
Intractable Problems
• Can be classified in various categories based on
their degree of difficulty, e.g.,
• NP
• NP-complete
• NP-hard
• Let’s define NP algorithms and NP problems …

10
Nondeterministic and NP Algorithms
Nondeterministic algorithm = two stage procedure:
1) Nondeterministic (“guessing”) stage:
generate randomly an arbitrary string that can be thought
of as a candidate solution (“certificate”)

2) Deterministic (“verification”) stage:

take the certificate and the instance to the problem and
returns YES if the certificate represents a solution

NP algorithms (Nondeterministic polynomial)

verification stage is polynomial
11
Class of “NP” Problems
• Class NP consists of problems that could be solved by
NP algorithms
• i.e., verifiable in polynomial time

• If we were given a “certificate” of a solution, we could

verify that the certificate is correct in time polynomial
to the size of the input
• Warning: NP does not mean “non-polynomial”

12
E.g.: Hamiltonian Cycle

• Given: a undirected graph G = (V, E), determine a

simple cycle that contains each vertex in V
• Each vertex can only be visited once

• Certificate: hamiltonian

• Sequence: 〈v1, v2, v3, …, v|V|〉

not
hamiltonian

13
Is P = NP?
• Any problem in P is also in NP: P

NP
P ⊆ NP

• The big (and open question) is whether NP ⊆ P or

P = NP
• i.e., if it is always easy to check a solution, should it also
be easy to find a solution?

• Most computer scientists believe that this is false

but we do not have a proof …
14
NP-Completeness (informally)
P NP-complete
• NP-complete problems are
NP
defined as the hardest

problems in NP

• Most practical problems turn out to be either P or

NP-complete.

• Study NP-complete problems …

15
Reductions
• Reduction is a way of saying that one problem is “easier”
than another.
• We say that problem A is easier than problem B,
(i.e., we write “A ≤ B”)
if we can solve A using the algorithm that solves B.
• Idea: transform the inputs of A to inputs of B

yes
yes
α β
f Problem B no
no
Problem A
16
Polynomial Reductions
• Given two problems A, B, we say that A is

polynomially reducible to B (A ≤p B) if:

1. There exists a function f that converts the input of A

to inputs of B in polynomial time

2. A(i) = YES ⇔ B(f(i)) = YES

17
NP-Completeness (formally)
• A problem B is NP-complete if: P NP-complete

(1) B ∈ NP NP

(2) A ≤p B for all A ∈ NP

• If B satisfies only property (2) we say that B is NP-hard

• No polynomial time algorithm has been discovered for an

NP-Complete problem

• No one has ever proven that no polynomial time algorithm

can exist for any NP-Complete problem
18
Implications of Reduction
yes
yes
α β
f Problem B no
no
Problem A

- If A ≤p B and B ∈ P, then A ∈ P
- if A ≤p B and A ∉ P, then B ∉ P

19
Proving Polynomial Time
yes
yes
α β Polynomial time
f algorithm to decide B no
no
Polynomial time algorithm to decide A
1. Use a polynomial time reduction algorithm to
transform A into B
2. Run a known polynomial time algorithm for B
3. Use the answer for B as the answer for A

20
Proving NP-Completeness
Theorem: If A is NP-Complete and A ≤p B

⇒ B is NP-Hard

In addition, if B ∈ NP

⇒ B is NP-Complete

Proof: Assume that B ∈ P

Since A ≤p B ⇒ A ∈ P contradiction!

⇒ B is NP-Hard
21
Proving NP-Completeness In Practice

• Prove that the problem B is in NP

• A randomly generated string can be checked in
polynomial time to determine if it represents a solution

• Show that one known NP-Complete problem can

be transformed to B in polynomial time
• No need to check that all NP-Complete problems are
reducible to B

22
Revisit “Is P = NP?”
P NP-complete

NP

Theorem: If any NP-Complete problem can be solved

in polynomial time ⇒ then P = NP.

23
P & NP-Complete Problems
• Shortest simple path
• Given a graph G = (V, E) find a shortest path from a
source to all other vertices

• Polynomial solution: O(VE)

• Longest simple path

• Given a graph G = (V, E) find a longest path from a
source to all other vertices

• NP-complete
24
P & NP-Complete Problems
• Euler tour
• G = (V, E) a connected, directed graph find a cycle that
traverses each edge of G exactly once (may visit a vertex
multiple times)
• Polynomial solution O(E)

• Hamiltonian cycle
• G = (V, E) a connected, directed graph find a cycle that
visits each vertex of G exactly once
• NP-complete
25
NP-naming convention
• NP-complete - means problems that are 'complete'
in NP, i.e. the most difficult to solve in NP
• NP-hard - stands for 'at least' as hard as NP (but
not necessarily in NP);
• NP-easy - stands for 'at most' as hard as NP (but
not necessarily in NP);
• NP-equivalent - means equally difficult as NP, (but
not necessarily in NP);

26
Examples NP-complete and
NP-hard problems
NP-complete

NP-hard

27
Reference
• T.H. Coreman , C.E. Leiserson,R.L. Rivest, and C.
Stein, “Introduction to algorithms”, Fourth edition ,
PHI publication.

28