1

DYNAMIC PROGRAMMING

24/05/2023
 24/05/2023

The shortest path

• To find a shortest path in a multi-stage graph
3 2 7

1 4
S A B 5
T

5 6

• Apply the greedy method :

the shortest path from S to T :
1+2+5=8
 24/05/2023

The shortest path in multistage graphs

4
A D
1 18
11 9
• e.g.
2 5 13
S B E T
16 2

5
C 2
F

• The greedy method can not be applied to this

case: (S, A, D, T) 1+4+18 = 23.
• The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
 24/05/2023

Dynamic Programming
• Dynamic Programming is a technique that can
be used to solve many optimization problems. In
most applications, dynamic programming
obtains solutions by working backward from the
end of problem toward the beginning, the
breaking up a large, unwieldy problem into a
series of smaller, more tractable problems.
 5

Compare DP & LP
24/05/2023

LP DP
Algorithm Can programming Cannot programming.
(Simplex)

Solution One stage Multi-stage

 6
24/05/2023

4 Steps in Dynamic Programming

• Divide the Problem into stages
• Solve the last stage first
• Work backwards solving stages
• Optimal solution obtained from all solved stages
 7
24/05/2023

TYPES OF DP PROBLEMS
2 types of DP:
- Network:
Ex: Shortest path problem
- Non – network:
Ex: Knapsack problem.
 8

Shortest path construction:

24/05/2023

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

 9

24/05/2023

Shortest path construction: 4th stage

• d(J, H) = 3

• d(J, I) = 4

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

 10

24/05/2023

Shortest path construction: 3rd stage

d(E,J) =d(E,H )+ d(J, H) = 1+3=4*
d(E,J) = d(E, I) +d(J, I) = 4+4=8

d(F,J )=d(F,H )+ d(J, H) = 6+3=9

d(F,J )= d(F, I) +d(J, I) = 3+4=7*

d(G,J )= d(G,H )+ d(J, H) = 3+3=6*

d(G,J )= d(G, I) +d(J, I) = 3+4=7
Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
 11

24/05/2023

Shortest path construction: 2nd stage

d(B,J)=d(B,E )+ d(E,J) = 7+4=11*
d(B,J)= d(B, F) + d(F,J)= 7+4=11*
d(B,J)=d(B, G) + d(G,J)=6+6=12

d(C,J)=d(C,F )+ d(F, J) = 6+3=9

d(C,J)=d(C,E)+ d(E, J) =3+4=7*
d(C,J)= d(C, G) +d(G, J) = 4+6=10

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

d(D,J)= d(D, E) +d(E, J) = 4+4=8*
d(D,J)= d(D, F) +d(F, J) = 1+7=8*
d(D,J)=d(D,G )+ d(G, J) = 5+6=11
 12

24/05/2023

Shortest path construction: 1st stage

Stage 1 Stage 2 Stage 3 Stage 4 Stage 5

d(A,J)=d(A,B )+ d(B,J) = 2+11=13

d(A,J)= d(A, C) + d(C,J)= 4+7=11*
d(A,J)= d(A, D) + d(D,J)= 3+8=11*
 13

The shortest path

24/05/2023

11 4

11 7 7 0

8 6

Route A-C-E-H-J: 4+3+1+3=11

 14
24/05/2023

KNAPSACK PROBLEM

Objective function of Knapsack problem can be:

- Minimization or
- Maximization
Ex:
- A knapsack can hold some limited items. Given a set
of items, each with a weight and a value, determine
the count of each item to include in a collection so
that the total weight is less than or equal to a given
limit and the total value is as large as possible
 15
24/05/2023

KNAPSACK PROBLEM CLASSIFY

• There are many types of Knapsack problem in practice.
Example:
- Choosing goods to carry on NASA space ships.
- Some scheduling problems, for example, there are
some tasks that need to be completed in the next 2
weeks, the 2-week duration can be considered as
knapsack, the tasks must be scheduled so that profit
can be maximized or the cost can be minimized; the
constraints are the working days, or hours in this 2
weeks.

15
 16

24/05/2023

Example : Knapsack problem

Given some items, pack the knapsack to get the
maximum total value. Each item has some weight and
some value. Total weight that we can carry is no more
than some fixed number W. So we must consider
weights of items as well as their values.

Item # Weight Value

1 1 8
2 3 6
3 5 5
 17

24/05/2023

Formula for subproblems

Recursive formula for subproblems:
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w],V [i  1, w  wi ]  bi } else

It means, that the best subset of Si that has total weight

w is:
1) the best subset of Si-1 that has total weight  w, or
2) the best subset of Si-1 that has total weight  w-wi
plus the item i
Recursive Formula
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else

• The best subset of Si that has the total weight  w, either

contains item i or not.
• First case: wi>w. Item k can’t be part of the solution,
since if it was, the total weight would be > w, which is
unacceptable.
• Second case: wi  w. Then the item k can be in the
solution, and we choose the case with greater value.
 19

24/05/2023

Example
Let’s run our algorithm on the following data:

n = 4 (# of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3),
(3,4),
(4,5),
(5,6)
How to pack the knapsack to achieve maximum total value of packed items?
 20

24/05/2023

Example (3)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0
 21

24/05/2023

Items:
Example (4) 1: (2,3)
i\W 2: (3,4)
0 1 2 3 4 5
0 0 0 0 0 0 0
3: (4,5)
i=1 4: (5,6)
1 0 0 bi=3
2 0
wi=2
3 0
w=1
4 0
w-wi =-1
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [1,1]  V [1  1,1]  0
 22
Items:
24/05/2023
1: (2,3)
2: (3,4)
Example (5) 3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3
wi=2
2 0
3 0 w=2
4 0 w-wi =0
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [1, 2]  max{V [1  1, 2], V [1  1, 2  2]  3}
3
 23

24/05/2023
Items:
1: (2,3)
Example (6) 2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
0 0 0 0 0 0 0 i=1 4: (5,6)
1 0 0 3 3 bi=3
2 0 wi=2
3 0 w=3
4 0 w-wi =1
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [1,3]  max{V [1  1,3], V [1  1,3  2]  3}
 max{0,3}  3
 24
Items:
24/05/2023
1: (2,3)
Example (7) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0
bi=3
1 0 0 3 3 3
wi=2
2 0
3 0 w=4
4 0 w-wi =2

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [1, 4]  max{V [1  1, 4], V [1  1, 4  2]  3}
 max{0,3}  3
 25

Items:
24/05/2023

Example (8) 1: (2,3)

2: (3,4)
i\W 0 1 2 3 4 5 i=1 3: (4,5)
0 0 0 0 0 0 0
bi=3 4: (5,6)
1 0 0 3 3 3 3
wi=2
2 0
3 0 w=5
4 0 w-wi =3

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [1,5]  max{V [1  1,5], V [1  1,5  2]  3}
 max{0,3}  3
 26

24/05/2023
Items:
Example (9) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
i=2
0 0 0 0 0 0 0 4: (5,6)
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0
3 0 w=1
4 0 w-wi =-2

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [2,1]  V [2  1,1] =0
 27

24/05/2023

Items:
Example (10) 1: (2,3)
i\W 0 1 2 3 4 5 i=2 2: (3,4)
0 0 0 0 0 0 0 3: (4,5)
bi=4
1 0 0 3 3 3 3 4: (5,6)
wi=3
2 0 0 3
3 0 w=2
4 0 w-wi =-1

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [1,5]  max{V [1  1,5], V [1  1,5  2]  3}
 max{0,3}  3
 28

24/05/2023
Items:
Example (11) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
i=2
0 0 0 0 0 0 0
bi=4 4: (5,6)
1 0 0 3 3 3 3
wi=3
2 0 0 3 4
3 0 w=3
4 0 w-wi =0

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [2,3]  max{V [2  1,3], V [2  1,3  3]  4}
 max{0, 4}  4
 29
Items:
24/05/2023
1: (2,3)
2: (3,4)
Example (12) 3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4
3 0 w=4
4 0 w-wi =1

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [2, 4]  max{V [2  1, 4], V [2  1, 4  3]  4}
 max{0, 4}  4
 30
Items:
24/05/2023
1: (2,3)
Example (13) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0
bi=4
1 0 0 3 3 3 3
wi=3
2 0 0 3 4 4 7
3 0 w=5
4 0 w-wi =2

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [2,5]  max{V [2  1,5], V [2  1,5  3]  4}
 max{0,3  4}  7
 31
Items:
24/05/2023
1: (2,3)
Example (14) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 w= 1..3
4 0
 32

24/05/2023
Items:
Example (15) 1: (2,3)
2: (3,4)
i\W 0 1 2 3 4 5 3: (4,5)
i=3
0 0 0 0 0 0 0 4: (5,6)
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 w= 4
4 0 w- wi=0
 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w], V [i  1, w  wi ]  bi } else
V [3, 4]  max{V [3  1, 4], V [3  1, 4  4]  5}
 max{3, 5}  5
 33
Items:
24/05/2023
1: (2,3)
Example (16) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0
bi=5
1 0 0 3 3 3 3
wi=4
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 w- wi=1

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w],V [i  1, w  wi ]  bi } else
V [3,5]  max{V [3  1,5], V [3  1,5  4]  5}
 max{7, 5}  7
 34
Items:
24/05/2023
1: (2,3)
Example (17) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 1..4
4 0 0 3 4 5
 35
Items:
24/05/2023
1: (2,3)
Example (18) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0
bi=6
1 0 0 3 3 3 3
wi=5
2 0 0 3 4 4 7
3 0 0 3 4 5 7 w= 5
4 0 0 3 4 5 7 w- wi=0

 V [i  1, w] if wi  w
V [i, w]  
max{V [i  1, w],V [i  1, w  wi ]  bi } else
V [4,5]  max{V [4  1,5], V [4  1,5  5]  6}
 max{7, 6}  7
How to find actual Knapsack Items

• All of the information we need is in the table.

• V[n,W] is the maximal value of items that can be
placed in the Knapsack.
• Let i=n and k=W
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1 // Assume the ith item is not in the knapsack
// Could it be in the optimally packed
knapsack?
 Items:
1: (2,3)
Finding the Items 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
1: (2,3)
Finding the Items (2) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=4 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
1: (2,3)
Finding the Items (3) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=3 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5
2 0 0 3 4 4 7 wi=4
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =7
i=n, k=W
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
1: (2,3)
Finding the Items (4) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=2 4: (5,6)
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7 wi=3
3 0 0 3 4 5 7 V[i,k] = 7
4 0 0 3 4 5 7 V[i1,k] =3
i=n, k=W
k  wi=2
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
1: (2,3)
Finding the Items (5) 2: (3,4)
3: (4,5)
i\W 0 1 2 3 4 5 i=1 4: (5,6)
0 0 0 0 0 0 0 k= 2
1 0 0 3 3 3 3 bi=3
2 0 0 3 4 4 7 wi=2
3 0 0 3 4 5 7 V[i,k] = 3
4 0 0 3 4 5 7 V[i1,k] =0
i=n, k=W
k  wi=0
while i,k > 0
if V[i,k]  V[i1,k] then
mark the ith item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
1: (2,3)
2: (3,4)
Finding the Items (6) 3: (4,5)
i\W 0 1 2 3 4 5 i=0 4: (5,6)
0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k]  V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1
 Items:
Finding the Items (7) 1: (2,3)
i\W 2: (3,4)
0 1 2 3 4 5
0 0 0 0 0 0 0
3: (4,5)
4: (5,6)
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7 The optimal
knapsack
4 0 0 3 4 5 7
should contain
i=n, k=W {1, 2}
while i,k > 0
if V[i,k]  V[i1,k] then
mark the nth item as in the knapsack
i = i1, k = k-wi
else
i = i1