APPLIED ALGORITHMS

CONTENTS

• Introduction

• Problem classes P, NP, NPC

APPLIED ALGORITHMS • Decision problem and Optimization problem
NP-Complete • Reduction (Phép quy dẫn)

• Prove NP-complete, NP-hard

• Approaches to solve NP-hard problem

3 4
ĐỘ khó của bài toán History

• The difficulty evaluation of the problem is the estimation of the computation time • Since the 1960s, Steve Cook and Dick Karp decided that the
of the best algorithm among all the algorithms to solve the problem, both known minimum requirement for an efficient algorithm was that its
and unknown. computation time must be polynomial: O(nc) where c is a constant
• Two approaches: and n is the size of problem.
• Method 1: find a way to evaluate the lower bound of the problem’s complexity • It is also found that for many classes of problems, it is very difficult
(for the Advanced Algorithm Analysis course) to find such algorithms, and we do not even know if such an
• Method 2: focus on showing how hard it is comparable to any known problem algorithm exists.
(NP-complete theory) • Therefore, Cook and Karp and other researchers have proposed a
• The evaluation of the computational complexity of the problem plays a directional definition for the NP-complete problem class, which is believed until
role in the design of the algorithm to solve the given problem. now to be impossible to have an efficient algorithm to solve

5 6

CONTENTS Problem classes P and NP

• P is the class of problems that can be solved in polynomial time

• Introduction • The problem to check graph connectivity can be solved by an algorithm with a
computation time of O(n2) where n is the number of vertices in the graph, so it is a
• Problem classes P, NP, NPC problem of class P.
• The matrix multiplication problem solved by dynamic programming with time O(n3)
• Decision problem and Optimization problem where n is the size of matrix, so it is a problem of class P.
• NP is the class of problems verifiable in polynomial time
• Reduction (Phép quy dẫn) • “Verifiable” is to give an algorithm in polynomial time to verify the accuracy of an
output with the corresponding input data.
• Example: The problem of finding a Hamiltonian cycle, checking whether a sequence of
• Prove NP-complete vertices 𝑣 , 𝑣 , … , 𝑣 , 𝑣 is a Hamiltonian cycle of the given graph can be done after
polynomial time. so it is a problem of class NP.
• Approaches to solve NP-hard problem

7 8
Problem class NP-complete (NPC) Mối quan hệ giữa P, NP và NPC

• NP-complete is a class of problems in class NP and is no less difficult than any problem in • Sure
NP • 𝑷⊆ 𝑵𝑷
• This is the hardest problem class in the class NP • 𝑵𝑷𝑪⊆ 𝑵𝑷
• Why not say Solve in exponential time' or Can't solve in polynomial time’?
• Not solvable when size of the problem is large • Not sure:
• 𝑂 𝑛 và 𝑂 2 • 𝑷 = 𝑵𝑷 or 𝑷 ⊂ 𝑵𝑷 or 𝑷 ≠ 𝑵𝑷
• 𝑵𝑷𝑪 = 𝑵𝑷 or 𝑵𝑷𝑪 ⊂ 𝑵𝑷 or 𝑵𝑷𝑪 ≠ 𝑵𝑷
• The algorithm 𝑂 𝑛 is still a polynomial algorithm, but the computation time is not
practical
• Algorithms that run in polynomial time in real life have computation times much • 𝑷 = 𝑵𝑷?
smaller than 𝑂 𝑛 • One of the most difficult and central problems in the theory of computation is to prove or
disprove this equality!
• Usually, when a polynomial algorithm is found for a certain problem, many new • Until now this question is still an open question
effective algorithms will be found for the problem.

9 10

Mối quan hệ P, NP và NPC – Opinion (Not proven yet) Why care about NP-complete theory?

• If a problem is proved to be of the class NP-complete  we get a proof of the difficulty of

the problem ~ The most difficult in class of NP problems
• If a problem can be proven to be NP-complete -> Don't waste time trying to find an
efficient algorithm for it. Instead, focus on designing approximation or
heuristic/metaheuristic algorithms or finding efficient solutions to some special case of the
problem.
• Note: Some problems look very easy on the statement, but are in fact difficult as it
belongs to NP-complete class.
• The opinion of many computer science researchers, which have not yet been • For example: It is whether to divide a set of integers into 2 non-intersecting subsets so
proven, are as follows: that the sum of the elements in the 2 subsets is equal ?
𝑷 ∈ 𝑵𝑷, 𝑵𝑷𝑪 ∈ 𝑵𝑷, 𝑷 ∩ 𝑵𝑷𝑪 = ∅

11 12
CONTENTS Decision problem and optimization problem

• Optimization problem (PO) is a problem that requires requires finding a solution that
• Introduction satisfies a set of constraints and maximizes (max) or minimizes (min) a certain objective
function.
• Problem classes P, NP, NPC • Example: The problem finds shortest path on the graph: Given graph G, and 2 vertices
u, v, find the path from u to v that passing fewest edges on G.
• Decision problem and Optimization problem • Decision problem (PD) where the output can only be `YES' or `NO’ (true/false, 1/0,
accept/reject).
• For a decision problem, there are input data whose answer (output) is 'YES’; and there
• Reduction (Phép quy dẫn) are also input data whose answer is 'NO'
• Input data with the answer 'YES' ('NO') will be called input data 'YES' ('NO’ )
• Prove NP-complete, NP-hard • Example: Is there a path from u to v on graph G going through at most k edges?

• Approaches to solve NP-hard problem

13 14

Decision problem and optimization problem Decision problem and optimization problem

• The NP-complete class consists of decision problems only! • Consider optimization problem:
• For the optimization problem, if the optimization and accuracy of the output can be 𝑃𝑂 : max{𝑓 𝑥 : 𝑥 ∈ 𝐷}
verified in polynomial time, then the optimal solution can also be found in
polynomial time. • The decision problem (PD) corresponding to the optimization problem (PO) is:
• It is easier to reduce between decision problems than between optimization 𝑃𝐷 “𝐺𝑖𝑣𝑒𝑛 𝑡ℎ𝑒 𝑣𝑎𝑙𝑢𝑒 𝑘, 𝑐𝑎𝑛 𝑤𝑒 𝑓𝑖𝑛𝑑 𝑢 ∈ 𝐷 𝑠𝑢𝑐ℎ 𝑡ℎ𝑎𝑡 𝑓 𝑢 ≥ 𝑘 ? ”
problems

• The solution of the optimization problem directly leads to the solution of the corresponding
decision problem
• If the decision problem corresponds to an optimization problem that can be solved efficiently (for
example, by a polynomial algorithm), then that optimization problem can also be solved efficiently
(by a polynomial algorithm)

15 16
Relationship between (PO) and (PD) Example: The problem of finding maximum independent sets
• Assume that function f takes an integer value on D; 𝛼 and 𝛽 are the lower and upper
bounds of the value of f over D, respectively. • Optimization problem: Given undirected graph 𝐺 = 𝑉, 𝐸 . A subset of V whose any two
vertices are not adjacent on the graph is called the independent set of the graph. Find the
• Assume A is a problem solving algorithm (PD) with complexity 0 𝑛 independent set with largest cardinality (maximum independent set)
• Binary search algorithm solves the optimization problem. At each iteration step 𝑘 (𝑘 = • The corresponding decision problem: Given a positive integer k, whether the graph
1, 2, … ) contains an independent set with the cardinality of at least 𝑘.
• Calculate 𝜃 = (𝛼 + 𝛽 )/2 • Binary search algorithm, 𝛼 = 1, 𝛽 = 𝑛 where 𝑛 is the number of vertices in 𝐺. If the
• If the algorithm A finds 𝑥 ∈ 𝐷 that satisfy 𝑓 𝑥 > 𝜃 then 𝛼 = 𝜃 ,𝛽 =𝛽 decision problem is solvable by the polynomial algorithm then the correspondence
• Otherwise, update 𝛼 = 𝛼 ,𝛽 =𝜃 optimization problem can also be solved by the binary search shown above in no more
than 1 + log(𝑛) times applying the solving algorithm of the decision problem
• Continue to next step 𝑘 + 1 if 𝛽 −𝛼 ≥𝜖
• Apparently the above binary search algorithm gives the computation time 𝑶(𝒍𝒐𝒈(𝜷𝟎 −
𝜶𝟎 ) × 𝒏𝒄 )

17 18

CONTENTS Reduction

• Suppose A and B are two decision problems. We say that problem A can reduce in
• Introduction polynomial time to the problem B if there exists a polynomial time algorithm R that
transforms the input data into x of A into the input data R(x) of B such that x is the input
• Problem classes P, NP, NPC 'YES' (that is, the input to which the answer is 'YES') of A if and only R(x) is the input
'YES' of B

• Decision problem and Optimization problem • In the next section we will only consider the reduction after polynomial time, so for the
sake of brevity, we will call it the reduction instead of the referencing after the
polynomial time.
• Reduction (Phép quy dẫn)

• Prove NP-complete

• Approaches to solve NP-hard problem

19 20
Reduction Reduction

• Denote A reduces to B: A ≺ B
• If there exists a polynomial algorithm to solve B, then A will also be solved in polynomial
time
• We can say A is not harder than B (or B is not easier than A). If the problem A is difficult
(e.g. the class NP-complete), then the problem B is also difficult • C ≺ A and A ≺ B then C ≺ B
• This reduction is used to indicate that a problem is difficult.

21 22

CONTENTS NP-complete and NP-hard

• Definition: A decision problem A is called NP-complete when

• Introduction 1) A is a problem in NP
2) Ever problem in NP can reduce to A
• Problem classes P, NP, NPC • If A only satisfies (2) then A is called NP-hard.
• Comments:
• Decision problem and Optimization problem
• The concept of the hardest problem" in the class NP is built on the basis of reduction
• If all the problems in NP can be reduced to a problem A, then A is no less difficult than
• Reduction (Phép quy dẫn) any of them.
• Only with condition (2), if there exists a polynomial algorithm to solve A, it will entail
• Prove NP-complete, NP-hard the existence of a polynomial algorithm to solve all problems in NP

• Approaches to solve NP-hard problem

23 24
The relationship between NP-complete and NP-hard Diagram to prove problem L is NP-complete using reduction

• Opinion of many computer scientists, but unproven: 1. Prove L is NP (Usually quite easy to prove)
2. Choose the problem L’ is known NP-complete (Usually based on experience, and
not easy)
3. Develop a polynomial algorithm R to transform an input data set of L' into an
input data set of L (Usually not easy to show)
4. Prove that x is a YES input of L' if and only if the corresponding input R(x) is also
• To prove problem B is NP-complete by using reduction is to find the problem A a YES input of L.
belonging to NP-complete and the reduction A to B.
• Lemma: If B is NP-complete, A is NP, and 𝑨 ≺ 𝑩, then A is NP-complete.

25 26

Reduction tree structure of basic NP-complete problems CIRCUIT-SAT

• CIRCUIT-SAT (Circuit Satisfiability) is • CIRCUIT-SAT: Given a logical expression made up of n Bool variables, Is there an
the first problem proven to be NP- input data of Bool variables for the expression to take the value TRUE, or if the
complete. expression always outputs FALSE.?
• CIRCUIT-SAT: Given a logical • Example:
expression made up of n Bool
variables, Is there an input data of
Bool variables for the expression to
take the value TRUE, or if the
expression always outputs FALSE.?
• The expression outputs FALSE for all values of 𝑥 ∈ {0, 1}

• The expression outputs TRUE when (𝑥 , 𝑥 ) ∈ { 0,1 , 1,0 , 1,1 }

27 28
CIRCUIT-SAT CIRCUIT-SAT

• Theorem: CIRCUIT-SAT is NP-complete (Cook, 1971) • Theorem: CIRCUIT-SAT is NP-complete (Cook, 1971)
• Proof: • Proof:
• Step 1: CIRCUIT-SAT is NP. Given an input, it is easy to calculate the • Step 2: Every problem in NP can be reduced to CIRCUIT-SAT. The idea is as
corresponding output in polynomial time with a number of variables using a follows:
graph traversal algorithm. • Every problem in NP can be calculated using a logic circuit.
• This logic circuit has a number of components limited by a polynomial in the
number of variables and can therefore be computed in polynomial time.

29 30

Example: reduction HAM-CYCLE ≺ TSP Example: reduction HAM-CYCLE ≺ TSP

• Problem HAM-CYCLE: The undirected graph G = (V, E) contains Hamilton cycle? • Prove that the TSP decision problem is NP-complete
• Problem TSP: • TSP decision problem is NP: The procedure to check whether the solution is a
• Optimization problem: Find the cycle that goes through all vertices of the cycle containing all vertices of the graph and the length does not exceed k has
undirected graph G = (V, E, c) such that the cycle length is the shortest. polynomial computation time.
• Decision problem: Whether or not there exists a cycle that goes through all • Show the reduction: For each input data of the problem Ham-cycle 𝐺 = (𝑉, 𝐸),
vertices of the undirected graph G = (V, E, c) such that the cycle length is at we build an input data for TSP decision problem, 𝐺 = (𝑉, 𝐸, 𝑐) and 𝑘 = 0,
most k. where:
• 𝐸 = 𝑢, 𝑣 𝑢, 𝑣 ∈ 𝑉}
• Prove that the TSP decision problem is NP-complete
• Cost function 𝑐 is designed as follows:
• TSP decision problem of class NP: The procedure to check whether the • 𝑐 𝑢, 𝑣 = 1 if 𝑢, 𝑣 ∈ 𝐸
solution is a cycle containing all vertices of the graph and the length does not • 𝑐 𝑢, 𝑣 = 0 if ∉ 𝐸
exceed k has polynomial computation time. • We see, if 𝐻 is the solution of the Ham-cycle data set, then 𝐻 is also a solution
of the data set of the TSP decision problem. Conversely, if 𝐻′ is a solution of
the TSP decision problem, then 𝐻′ is also a solution of the Ham-cycle data set.

31 32
CONTENTS Approaches to solve NP-complete and NP-hard problem

• Introduction • Exact/complete/determistic approach

• Backtracking (Tìm kiếm quy lui)
• Problem classes P, NP, NPC • Branch-and-Bound (Nhánh cận)
• Divide-and-Conquer (Chia để trị)
• Decision problem and Optimization problem • Dynamic Programming (Quy hoạch động)
• Integer Programming (Quy hoạch nguyên tuyến tính)
• Reduction (Phép quy dẫn) • Constraint Programming (Quy hoạch ràng buộc)
• Column Generation/ Branch-and-Price (Sinh cột)
• Prove NP-complete, NP-hard • Combine different approaches

• Approaches to solve NP-hard problem

33 34

Approaches to solve NP-complete and NP-hard problem

• Incomplete (Gần đúng)

• Greedy Algorithm (Tham lam)
• Approximation Algorithm (Thuật toán xấp xỉ)
• Heuristics/Metaheuristics
• Local search, Multistage heuristic
• Tabu Search, Genetic Algorithm, Simulated Annealing, Practical Swarm, Ant Colony
Optimization
• Reinforcement Learning (Học tăng cường)
THANK YOU !
• Deep Learning (Học sâu)

35 36