Analysis of Algorithm

Lecture# 15

Dynamic Programming

1
Optimization Problems
• Some problems can have many possible/ feasible solutions
with each solution having a specific cost. We wish to find the
best solution with the optimal cost.
• Maximization problem finds a solution with maximum cost
• Minimization problem finds a solution with minimum cost

• A set of choices must be made in order to arrive at an optimal

(min/max) solution, subject to some constraints.

• Is “Sorting a sequence of numbers” optimization problem?

2
Optimization Problems
• Two common techniques:
• Greedy Algorithms (local)
• Make the greedy choice and THEN
• Solve sub-problem arising after the choice is made
• The choice we make may depend on previous choices, but
not on solutions to sub-problems
• Top down solution, problems decrease in size
• Dynamic Programming (global)
• We make a choice at each step
• The choice depends on solutions to sub-problems
3
• Bottom up solution, smaller to larger sub-problems
Dynamic Programming
• Dynamic Programming was introduced by Richard Bellman in 1955.

• A strategy for designing algorithms

• meta technique, not an algorithm

• Dynamic programming solves every sub problem just once.

• Saves its answer in a table (array)

• More efficient than “brute-force methods” as it avoids the work of

re-computing the answer every time the sub-problem is
encountered.

• Dynamic programming is typically applied to optimization problems.

4
Dynamic Programming
• How to know if an optimization problem can be solved by
applying dynamic programming?
• Two key ingredients
• Optimal substructure
• Overlapping sub-problems

• Optimal Substructure: A problem exhibits optimal substructure

if an optimal solution to the problem contains within its optimal
solutions to the sub-problems

• Overlapping sub problems: When a recursive algorithm revisits

the same problem over and over again, then the optimization
problem has overlapping sub-problems 5
 Dynamic Programming
Following steps are required in development of dynamic algorithms
1.Characterize the structure of an optimal solution
2.Recursively define the value of an optimal solution
3.Compute the value of an optimal solution in a bottom-up fashion
4.Construct an optimal solution from computed information

Note: Steps 1-3 form the basis of a dynamic programming

solution to a problem. Step 4 can be omitted only if the value of an
optimal solution is required.

6
Dynamic Programming
• Dynamic programming, like divide and conquer method, solves
problems by combining the solutions to sub-problems.

• Divide and conquer algorithms:

• partition the problem into independent sub-problem
• Solve the sub-problem recursively and
• Combine their solutions to solve the original problem

• In contrast, dynamic programming is applicable when the

sub-problems are not independent.

7
Divide-and-Conquer

Top-down
Prob

Subprob Subprob

…
…

Independent sub-problems (no overlapping work).

8
 Dynamic Programming

Prob

Subprob Subprob
1 2

Subproblem
Subprob sharing
Subprob 3
4 Subprob
7
Subprob
5 Subprob 9
Bottom-up 6
Knapsack Problem

10
Knapsack Problem
Given n items, each having a specific value vi and weight wi,
and a knapsack of fixed capacity W, pack the knapsack with
given items to maximize total value without exceeding the
total capacity W of the knapsack.

Item # Weight Value

1 1 8
2 3 6
3 5 5

11
Knapsack Problem
There are two versions of the problem:
1. “0-1 knapsack problem”
• Items are indivisible; you either take an item or not. Some
special instances can be solved with dynamic programming

2. “Fractional knapsack problem”

• Items are divisible: you can take any fraction of an item

12
0-1 Knapsack Problem
• Given a knapsack with maximum capacity W, and a set S consisting
of n items
• Each item i has some weight wi and benefit value bi (all wi and W
are integer values)

• Problem: How to pack the knapsack to achieve maximum total

value of packed items?

13
0-1 Knapsack Problem: Brute Force
Approach
Subset Total weight Total value
1. ∅ 0 0 # W V
2. {1} 2 20 1 2 20
3. {2} 5 30 2 5 30
4. {3} 10 50 3 10 50
5. {4} 5 10 4 5 10
6. {1,2} 7 50
7. {1,3} 12 70 knapsack capacity W = 16
8. {1,4} 7 30
9. {2,3} 15 80 Go through all combinations and
10. {2,4} 10 40 find the one with maximum value
11. {3,4} 15 60 and with total weight ≤ W
12. {1,2,3} 17 not feasible
13. {1,2,4} 12 60
14. {1,3,4} 17 not feasible 14
15. {2,3,4} 20 not feasible
16. {1,2,3,4} 22 not feasible Running time = O(2n)
0-1 Knapsack Problem: Brute Force
Approach
Knapsack-BF (n, V, W, C)
Compute all subsets, s, of S = {1, 2, 3, 4}
forall s ∈ S
weight = Compute sum of weights of these items
if weight > C, not feasible
new solution = Compute sum of values of these items
solution = solution ∪ {new solution}
Return maximum of solution

Approach: In brute force algorithm, we go through all combinations and find the
one with maximum value and with total weight less or equal to W = 16

Complexity
• Cost of computing subsets O(2n) for n elements
• Cost of computing weight = O(2n) 15
• Cost of computing values = O(2n)
• Total cost in worst case: O(2n)
0-1 Knapsack problem:
Dynamic Programming Approach
We can do better with an algorithm based on dynamic programming.
We need to carefully identify/define the sub-problems

Let’s try this:

•If items are labeled 1..n, then a sub-problem would be to find an
optimal solution for Sk = {items labeled 1, 2, .. k}

This is a reasonable sub-problem definition. The question is can we

describe the final solution (Sn ) in terms of sub-problems (Sk)?
Unfortunately, we can’t do that.

16
Defining a Sub-problem
As we have seen, the solution for S4 is not part of the solution for S5 so
our definition of a sub-problem is flawed and we need another one!

Let’s add another parameter: w, which will represent the maximum

weight for each subset of items.

The sub-problem then will be to compute V[k,w], i.e., to find an optimal

solution for Sk = {items labeled 1, 2, .. k} in a knapsack of size w

Assuming knowing V[i, j], where i = 0, 1, 2, … k-1, and j = 0, 1, 2, …w,

how to derive V[k,w]?

17
Recursive Formula for
sub-problems
Recursive formula for sub-problems:

The best subset of Sk that has the total weight ≤ w, either

contains item k or not.
•First case: wk>w. Item k can’t be part of the solution, since if it
was, the total weight would be > w, which is unacceptable.
•Second case: wk ≤ w. Then the item k can be in the solution, and
we choose the case with greater value.
18
0-1 Knapsack Algorithm
for w = 0 to W O(W)
V[0,w] = 0
for i = 1 to n
O(n)
V[i,0] = 0
for i = 1 to n
Repeat n times
for w = 0 to W
if wi <= w // item i can be part of the solution O(W)
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi] Remember that the
else brute-force algorithm
V[i,w] = V[i-1,w] takes O(2n)
else V[i,w] = V[i-1,w] // wi > w 19

What is the running time of this algorithm? O(n*W)

Example
Let’s run our algorithm on the following data:

n = 4 (# of elements)
W = 5 (max weight)

Elements (weight, benefit):

(2,3), (3,4), (4,5), (5,6)

20
Example (2)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1
2
3
4

for w = 0 to W
V[0,w] = 0
21
Example (3)
i\W 0 1 2 3 4 5
0 0 0 0 0 0 0
1 0
2 0
3 0
4 0

for i = 1 to n
V[i,0] = 0
22
 Items:
1: (2,3)
Example (4) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=1
0 0 0 0 0 0 0 bi=3
1 0 0 wi=2
2 0 w=1
3 0 w-wi =-1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
23
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (5) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=1
0 0 0 0 0 0 0 bi=3
1 0 0 3 wi=2
2 0 w=2
3 0 w-wi =0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
24
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (6) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=1
0 0 0 0 0 0 0 bi=3
1 0 0 3 3 wi=2
2 0 w=3
3 0 w-wi =1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
25
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (7) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=1
0 0 0 0 0 0 0 bi=3
1 0 0 3 3 3 wi=2
2 0 w=4
3 0 w-wi =2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
26
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (8) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=1
0 0 0 0 0 0 0 bi=3
1 0 0 3 3 3 3 wi=2
2 0 w=5
3 0 w-wi =3
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
27
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (9) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=2
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 wi=3
2 0 0 w=1
3 0 w-wi =-2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
28
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (10) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=2
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 wi=3
2 0 0 3 w=2
3 0 w-wi =-1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
29
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (11) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=2
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 wi=3
2 0 0 3 4 w=3
3 0 w-wi =0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
30
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (12) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=2
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 wi=3
2 0 0 3 4 4 w=4
3 0 w-wi =1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
31
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (13) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=2
0 0 0 0 0 0 0 bi=4
1 0 0 3 3 3 3 wi=3
2 0 0 3 4 4 7 w=5
3 0 w-wi =2
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
32
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (14) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=3
0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3 wi=4
2 0 0 3 4 4 7 w= 1..3
3 0 0 3 4
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
33
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (15) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=3
0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3 wi=4
2 0 0 3 4 4 7 w= 4
3 0 0 3 4 5 w- wi=0
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
34
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (16) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=3
0 0 0 0 0 0 0 bi=5
1 0 0 3 3 3 3 wi=4
2 0 0 3 4 4 7 w= 5
3 0 0 3 4 5 7 w- wi=1
4 0
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
35
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (17) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=4
0 0 0 0 0 0 0 bi=6
1 0 0 3 3 3 3 wi=5
2 0 0 3 4 4 7 w= 1..4
3 0 0 3 4 5 7
4 0 0 3 4 5
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
36
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
 Items:
1: (2,3)
Example (18) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=4
0 0 0 0 0 0 0 bi=6
1 0 0 3 3 3 3 wi=5
2 0 0 3 4 4 7 w= 5
3 0 0 3 4 5 7 w- wi=0
4 0 0 3 4 5 7
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
37
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
Comments
This algorithm only finds the max possible value that can be
carried in the knapsack
• i.e., the value in V[n,W]

To know the items that make this maximum value, an

addition to this algorithm is necessary

38
How to find actual Knapsack Items
All of the information we need is in the table.

V[n,W] is the maximal value of items that can be placed in the

Knapsack.

Let i=n and k=W

if V[i,k] ≠ V[i−1,k] then
mark the ith item as in the knapsack
i = i−1, k = k-wi
else
i = i−1 // Assume the ith item is not in the knapsack
// Could it be in the optimally packed knapsack?
39
 Items:
1: (2,3)
Finding the Items 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=4
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
V[i−1,k] =7
4 0 0 3 4 5 7
i=n, k=W
while i,k > 0
if V[i,k] ≠ V[i−1,k] then
mark the ith item as in the knapsack
40
i = i−1, k = k-wi
else
i = i−1
 Items:
1: (2,3)
Finding the Items (2) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=4
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=6
2 0 0 3 4 4 7 wi=5
3 0 0 3 4 5 7 V[i,k] = 7
V[i−1,k] =7
4 0 0 3 4 5 7
i=n, k=W
while i,k > 0
if V[i,k] ≠ V[i−1,k] then
mark the ith item as in the knapsack
41
i = i−1, k = k-wi
else
i = i−1
 Items:
1: (2,3)
Finding the Items (3) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=3
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=5
2 0 0 3 4 4 7 wi=4
3 0 0 3 4 5 7 V[i,k] = 7
V[i−1,k] =7
4 0 0 3 4 5 7
i=n, k=W
while i,k > 0
if V[i,k] ≠ V[i−1,k] then
mark the ith item as in the knapsack
42
i = i−1, k = k-wi
else
i = i−1
 Items:
1: (2,3)
Finding the Items (4) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=2
0 0 0 0 0 0 0 k= 5
1 0 0 3 3 3 3 bi=4
2 0 0 3 4 4 7 wi=3
3 0 0 3 4 5 7 V[i,k] = 7
V[i−1,k] =3
4 0 0 3 4 5 7
k − wi=2
i=n, k=W
while i,k > 0
if V[i,k] ≠ V[i−1,k] then
mark the ith item as in the knapsack
43
i = i−1, k = k-wi
else
i = i−1
 Items:
1: (2,3)
Finding the Items (5) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=1
0 0 0 0 0 0 0 k= 2
1 0 0 3 3 3 3 bi=3
2 0 0 3 4 4 7 wi=2
3 0 0 3 4 5 7 V[i,k] = 3
V[i−1,k] =0
4 0 0 3 4 5 7
k − wi=0
i=n, k=W
while i,k > 0
if V[i,k] ≠ V[i−1,k] then
mark the ith item as in the knapsack
44
i = i−1, k = k-wi
else
i = i−1
 Items:
1: (2,3)
Finding the Items (6) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5 i=0
0 0 0 0 0 0 0 k= 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
The optimal
i=n, k=W knapsack
while i,k > 0
should contain
if V[i,k] ≠ V[i−1,k] then
mark the nth item as in the knapsack {1, 2}
45
i = i−1, k = k-wi
else
i = i−1
 Items:
1: (2,3)
Finding the Items (7) 2: (3,4)
3: (4,5)
i\W 4: (5,6)
0 1 2 3 4 5
0 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
The optimal
i=n, k=W knapsack
while i,k > 0
should contain
if V[i,k] ≠ V[i−1,k] then
mark the nth item as in the knapsack {1, 2}
46
i = i−1, k = k-wi
else
i = i−1
0-1 Knapsack
Example
Input
• Given n items each
• weight wi
• value vi
• Knapsack of capacity W
Output: Find most valuable items that fit into the knapsack

Example:
item weight value knapsack capacity W = 16
1 2 20
2 5 30
3 10 50
4 5 10 47
Exercise

48