P, NP, NP-Hard and NP-Complete Problems

&
Approximation algorithms
P, NP, NP-Hard and NP-Complete Problems:
Based on Time Complexity:
1. Polynomial Time Algorithms: (Time complexity Less)
Example: Linear search-𝑂(𝑛)
Bubble sort-𝑂(𝑛 )
Merge sort-𝑂(𝑛𝑙𝑜𝑔𝑛)
2. Non-polynomial or Exponential Algorithms: (Time complexity is High)
Example: Travelling Salesman Problem-𝑂(𝑛² ∗ 2 )
Knapsack Problem -𝑂(2 / )
Based on Result:
1. Decision problems:
• Result is yes or no {0,1}
2. Optimization problems
• Result is a number representing an objective value
P, NP, NP-Hard and NP-Complete Problems:
Comparison:
P, NP, NP-Hard and NP-Complete Problems:
• Problems that yield solution (Algorithms that gives answer) in
Polynomial time are classified as “P Problems”
(Tractable/Deterministic Problems)
• Problems which can be solved in polynomial time, which take time
like O(n), O(n^2), O(n^3), O(log n), O(n*log n)….
• Eg: finding maximum element in an array or to check whether a string is
palindrome or not. so, there are many problems which can be solved in
polynomial time.
Konigsberg Bridge Problem :
• Pass over all 7 bridges exactly once!
• Solution: (Euler's analysis) Convert this
Problem into Graph theory.
• Land is denoted by vertices and bridges by
edges. For Graph to be traversable,
All vertices of Graph must be of EVEN degree
OR Exactly two vertices are of ODD, and rest
must be EVEN!
As all vertices of this graph are ODD degree,
solution does not exist (Proved) and this itself is
a solution for this problem!
Every Problem in the word have a Solution too.
In graph theory, an Eulerian trail (or Eulerian path) is a trail in a finite graph that visits every edge exactly once (allowing
for revisiting vertices). Similarly, an Eulerian circuit or Eulerian cycle is an Eulerian trail that starts and ends on the same
vertex.
P, NP, NP-Hard and NP-Complete Problems:
P Class Problems:
• P is set of problems that can be solved (deterministic) in Polynomial Time
Example: Linear Search, Binary search

Formally, an algorithm is polynomial time algorithm, if there exists a

polynomial p(n) such that the algorithm can solve any instance of size n in
time O(p(n))

NP Class Problem:
• NP is set of problems that can be solved in exponential (non-
deterministic Polynomial-NP) Time
• But these kind of problem can be verified in Polynomial Time
NP- Problems :
• Problems that take exponential time to yield solutions, these solutions can be
verified by Non-deterministic Polynomial time are classified as NP-Problems
(Non-deterministic/Non-tractable Problems).
• NP- Non-deterministic Polynomial time solving. Problem which can't be
solved in polynomial time but are checkable in polynomial time means that
given a solution of a problem , we can check that whether the solution is
correct or not in polynomial time.
• Example: TSP, Graph coloring, Hamiltonian cycle etc.…
• NP is set of decision problems that can be solved by a Non-deterministic
Turing Machine. P is subset of NP.
• Informally, NP is set of decision problems which can be solved by a
polynomial time via a “Lucky Algorithm or Guess Algorithm”, a magical
algorithm that always makes a right guess among the given set of choices
NP problems :
• Graph theory has these fascinating(annoying?) pairs of problems
– Shortest path algorithms?
– Longest path is NP complete (we’ll define NP complete later)
– Eulerian tours (visit every vertex but cover every edge only once,
even degree etc). Solvable in polynomial time!
– Hamiltonian tours (visit every vertex, no vertices can be
repeated). NP complete
Hamiltonian cycles :
• Determining whether a directed graph has a Hamiltonian cycle
does not have a polynomial time algorithm (yet!)
• However, if someone was to give you a sequence of vertices,
determining whether that sequence forms a Hamiltonian cycle
can be done in polynomial time
• Therefore, Hamiltonian cycles are in NP
SAT :
• A Boolean formula is satisfiable if there exists
some assignment of the values 0 and 1 to its variables that causes it
to evaluate
to 1.
• CNF – Conjunctive Normal Form. ANDing of clauses of ORs
2-CNF SAT :
• Each or operation has two arguments that are either variables or
negation of variables
• The problem in 2 CNF SAT is to find true/false(0 or 1) assignments
to the variables to make the entire formula true.
(x  y)(y  z)(xz)(z  y)

• Any of the OR clauses can be converted to implication clauses

2-SAT is in P :
• Create the implication graph

x

y
x

y

z

z
Satisfiability via path finding :
• If there is a path from
• And if there is a path from
• Then FAIL!
• How to find paths in graphs?
– DFS/BFS and modifications thereof
3 CNF SAT (3 SAT) :
• Not so easy anymore.
• Implication graph cannot be constructed
• No known polytime algorithm
• Is it NP?
– If someone gives you a solution how long does it take to verify it?
– Make one pass through the formula and check
• This is an NP problem
P, NP, NP-Hard and NP-Complete Problems:
NP-Class:
NP Class • Solved in Non-polynomial Time
• Verified in Polynomial Time
• Intractable Problem
P Class
P-Class:
• Solved in Polynomial Time
• Verified in Polynomial Time
• Tractable Problem

• P class problems are subset of NP class problem

• It's not known whether P=NP
• P!=NP
P, NP, NP-Hard and NP-Complete Problems:
Reduction :
• Let L1 and L2 be two decision problems. Suppose algorithm A2 solves L2. That is, if y is
an input for L2 then algorithm A2 will answer Yes or No depending upon whether y
belongs to L2 or not.
• The idea is to find a transformation from L1 to L2 so that the algorithm A2 can be part
of an algorithm A1 to solve L1.

• If we can convert (reduce/transform) instance of a problem A into problem B (NP

problem), the Algorithm of Problem B can be part of Algorithm of Problem A to
solve Problem A. Means that A is reducible to B.
Problem A α Problem B.
P, NP, NP-Hard and NP-Complete Problems:
An example reduction :

• CLIQUE problem
– A clique is a subgraph of a graph such that all the vertices in this
subgraph are connected with each other that is the subgraph is a
complete graph.
– The Maximal Clique Problem is to find the maximum sized clique of a
given graph G, that is a complete graph which is a subgraph of G and
contains the maximum number of vertices.
P, NP, NP-Hard and NP-Complete Problems:
An example reduction :

• CLIQUE problem • A clique in an undirected graph is

a subset of vertices such that
each pair is connected by an edge

• We want to take a problem

instance in 3-CNF SAT and convert
it to CLIQUE finding
P, NP, NP-Hard and NP-Complete Problems:
Reducing 3CNF SAT to CLIQUE :

• Given – A Boolean formula in 3 CNF SAT

• Goal – Produce a graph (in polynomial time) such that

• We will construct a graph where satisfying formula with k

clauses is equivalent to finding a k vertex clique.
P, NP, NP-Hard and NP-Complete Problems:
CLIQUE :

• An instance of a clique problem gives you 2 things as input

– Graph
– Some positive integer k
• Question being asked = do we have a clique of size k in this
graph
• Why can’t I just go through and pick all possible k-subsets?
P, NP, NP-Hard and NP-Complete Problems:
Decision problems versus optimization problems :

• Finding the maximum sized clique is an optimization problem

• But we can reduce it to a series of decision problems
– Can we find a clique of size 3 (why start at 3??)
– Can we find a clique of size 4
– Etc
• In general in our study of NP etc, we will focus on decision
problems
P, NP, NP-Hard and NP-Complete Problems:
REDUCE 3-CNF SAT to CLIQUE :

• For each clause, create a vertex for each literal

• For the edges

• Connect vertices if they come from different clauses

• Even if the vertices come from different clauses, do not connect if it results in incompatibility. No variable should be
connected to its not.

x1 ┐ x1 x1 There are more

edges in here.
Refer to the
CLRS book to get
┐x2 x2 x2 the complete
picture

┐x3 x3 x3
P, NP, NP-Hard and NP-Complete Problems:
Vertex cover problem :
• A vertex cover of an undirected graph 𝐺 = (𝑉, 𝐸) is a subset of vertices such that
every edge is incident to at least one of the vertices

• We’re typically interested in finding the minimum sized vertex cover

• To show vertex cover is NP-complete

• What problem should we try to reduce to it

• It sounds like the ‘reverse’ of CLIQUE

• Reduction is done from CLIQUE to vertex cover

Vertex cover problem : G’ =
G Complement
Of
G

Clique of size k in G exists iff a vertex cover of size

|V| - k exists in G’ where G’ is the complement
graph (vertices that had an edge between then in G
do not have one in G’ and vice versa)
P, NP, NP-Hard and NP-Complete Problems:
Vertex cover problem :

The original graph has a u,v,x,y CLIQUE. That is a clique of size 4

The complement graph has a vertex cover of size 6 (number of

vertices) – 4 (clique size). z,w is one such vertex cover.
P, NP, NP-Hard and NP-Complete Problems:
NP-Hard Problem:
• Every problem in NP class can be reduced into other set using polynomial
time, then its called as NP-Hard problem.

NP Class
Reduce NP-Hard
Class
P Class

• A problem (a language) is said to NP-hard if every problem in NP can be poly

time reduced to it.
P, NP, NP-Hard and NP-Complete Problems:
• NP-complete problems are the hardest
problems in NP set. A decision problem
L is NP-complete if:
1. L is in NP
(Any given solution for NP-complete
problems can be verified quickly, but
there is no efficient known solution).
2. Every problem in NP is reducible to L
in polynomial time

• The group of problems which are both in NP and NP-hard are

P ⊆ 𝑁𝑃 known as NP-complete problem

• All NP-Complete Problems are NP-Hard but not all NP-Hard

problems are not NP-Complete problem
P, NP, NP-Hard and NP-Complete Problems:
Prove Vertex cover Problem is a NP Complete:
• Requirement:
• Vertex Cover Problem is NP Problem
• Vertex cover problem can be reduced to a NP Complete problem (Which has already
solution). Eg: Vertex Cover problem can be reduced to Clique problem
• Steps:
• Already we proved Vertex Cover Problem is NP (1st Requirement is met).
• Vertex Cover Problem can be reduced to Clique Problem and hence we can say Vertex
Cover Problem (2nd Requirement) is NP Complete.
Approximation Algorithm:
Definition:
Approximation Algorithm:
Types of approximation:
Approximation Algorithm:
Examples:
1. Vertex Cover

2. Traveling Salesman Problem

Approximation Algorithm:
Vertex Cover:
• Instance An undirected graph G = (V, E).

• Feasible Solution A subset C ⊆ V such that at least one vertex of every edge of
G belongs C.

• Value The value of the solution is the size of the cover, |C|, and the goal is to
minimize it.
Approximation Algorithm:
Vertex Cover:
Example-
Approximation Algorithm:
Vertex Cover:
Greedy algorithm 1 :
1. C ← ∅

2. pick any edge (u, v) ∈ E.

3. C = C ∪ {u, v}

4. remove all edges incident on either u or v from E.

5. repeat the process till all edges are removed from E.

6. return C
Approximation Algorithm:
Vertex Cover:
Example- algorithm 1 :
Approximation Algorithm:
Vertex Cover:
Performance analysis of algorithm 1:

1. The edges picked by the algorithm is a maximal matching (say M), hence C is a vertex cover.

2. The algorithm runs in time polynomial of input size.

3. The optimum vertex cover (say C ∗ ) must cover every edge in M.

4. Hence C ∗ contains at least one of the end points of each edge in M, implies |C ∗ | ≥ |M|.

5. |C| = 2 ∗ |M| ≤ 2 ∗ |C ∗ |, where C ∗ is an optimal solution.

6. Thus, there is a 2-factor approximation algorithm for vertex cover problem.

Approximation Algorithm:
Traveling salesman problem (TSP):

• The TSP describes a salesman who must travel between n cities.

• The order in which he visits cities does not matter and he visits each city
during his trip and finishes where he was at first.

• Each city is connected to other cities by airplanes, or by road or railway.

• The salesman wants to keep the travel cost he travels as low as possible
Approximation Algorithm:
Reducing TSP to graph problem:

Instance A complete weighted undirected graph G = (V, E) with non-negative

edge weights.

Feasible Solution Find a cycle that visits each vertex exactly once.

Value The value of the solution is the sum of the weights associated with edges
in the cycle, and the goal is to find a cycle whose total weight is minimum.
Approximation Algorithm:
Approximation Algorithm for EUCLIDEAN TSP:
Approximation Algorithm:
Approximation Algorithm for EUCLIDEAN TSP (Example):
Approximation Algorithm:
Approximation ratio: