Introduction to Algorithms

Second Edition
by
Cormen, Leiserson, Rivest & Stein

Chapter 15
Assembly Line Scheduling
Problem
the fastest way through station S1,j-1 and then directly through
station S1,j, or
the fastest way through station S2,j-1, a transfer from line 2 to
line 1, and then through station S1,j.
the fastest way through station S2,j-1 and then directly through
station S2,j, or
the fastest way through station S1,j-1, a transfer from line 1 to
line 2, and then through station S2,j.
Step 1: The structure of the fastest way through the factory
the fastest way through station S1,j is either
the fastest way through station S2,j is either
Step 2: A recursive solution
Step 3: Computing the fastest times
PRINT-STATIONS (l, n)
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Matrix Chain Multiplication
Problem
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Step 1: The structure of an optimal parenthesization
let us adopt the notation A
ij
, where i j, for the matrix that results
from evaluating the product A
i
A
i+1
A
j
.

Ifthe problem is nontrivial, i.e., i < j, then any parenthesization of the
product A
i
A
i+1
A
j
must split the product between A
k
and A
k+1
for some
integer k in the range i k < j. That is, for some value of k, we first
compute the matrices A
ik
and A
k+1j
and then multiply them together to
produce the final product A
ij
.

The cost of this parenthesization is thus the cost of computing the
matrix A
ik
, plus the cost of computing A
k+1j
, plus the cost of
multiplying them together.
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Step 2: A recursive solution
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Step 3: Computing the optimal costs
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Longest Common Subsequence
Problem
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.


0 if i=0, or j=0
c[i,j] = c[i-1,j-1]+1 if i, j>0 and x
i
= y
j,

max{c[i-1,j], c[i,j-1]} if i, j>0 and x
i
= y
j,

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Optimal Binary Search Tree
Problem
Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

All Pair Shortest Path
The structure of a shortest path

For the all-pairs shortest-paths problem on a graph G = (V, E), all
subpaths of a shortest path should also be shortest paths.

Suppose that the graph is represented by an adjacency matrix W =
(w
ij
). Consider a shortest path p from vertex i to vertex j, and
suppose that p contains at most m edges.

Assuming that there are no negative-weight cycles, m is finite. If i =
j, then p has weight 0 and no edges.

If vertices i and j are distinct, then we decompose path p into ,
where path p now contains at most m - 1 edges.

p is a shortest path from i to k, and so (i, j) = (i, k) + w
kj
.
A recursive solution to the all-pairs shortest-paths problem
The latter equality follows since w
jj
= 0 for all j
Computing the shortest-path weights bottom up
0/1 Knapsack Problem
Given a knapsack with maximum capacity W, and a set S
consisting of n items.

Each item i has some weight w
i
and benefit value b
i
(all w
i
, b
i
and W are integer values)

Problem: How to pack the knapsack to achieve maximum
total value of packed items?
0-1 Knapsack problem
W = 20
w
i

b
i

10 9
8
5
5
4
4
3
3
2
Items
This is a knapsack
Max weight: W = 20
0-1 Knapsack problem:

e e
s
T i
i
T i
i
W w b subject to max
Problem, in other words, is to find
The problem is called a 0-1 problem, because each item
must be entirely accepted or rejected.
Just another version of this problem is the Fractional
Knapsack Problem, where we can take fractions of items.
Defining a Subproblem
If items are labeled 1..n, then a subproblem would be to find an
optimal solution for S
k
= {items labeled 1, 2, .. k}
Recursive Formula for subproblems

+
>
=
else } ] , 1 [ ], , 1 [ max{
if ] , 1 [
] , [
k k
k
b w w k B w k B
w w w k B
w k B
It means, that the best subset of S
k
that has total weight w is one
of the two:
1) the best subset of S
k-1
that has total weight w, or
2) the best subset of S
k-1
that has total weight w-w
k
plus the item k
The best subset of S
k
that has the total weight w,
either contains item k or not.
First case: w
k
>w. Item k cant be part of the
solution, since if it was, the total weight would be
> w, which is unacceptable
Second case: w
k
<=w. Then the item k can be in
the solution, and we choose the case with greater
value
0-1 Knapsack Algorithm
for w = 0 to W
B[0,w] = 0
for i = 0 to n
B[i,0] = 0
for w = 0 to W
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
Example:

n = 4 (Number of elements)
W = 5 (max weight)
Elements (weight, benefit):
(2,3), (3,4), (4,5), (5,6)
11/28/2013 45
Example (2)
for w = 0 to W
B[0,w] = 0
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
4
11/28/2013 46
Example (3)
for i = 0 to n
B[i,0] = 0
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
4
11/28/2013 47
Example (4)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=1
b
i
=3
w
i
=2
w=1
w-w
i
=-1
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
11/28/2013 48
Example (5)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=1
b
i
=3
w
i
=2
w=2
w-w
i
=0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
11/28/2013 49
Example (6)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=1
b
i
=3
w
i
=2
w=3
w-w
i
=1
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
11/28/2013 50
Example (7)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=1
b
i
=3
w
i
=2
w=4
w-w
i
=2
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
11/28/2013 51
Example (8)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=1
b
i
=3
w
i
=2
w=5
w-w
i
=2
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
11/28/2013 52
Example (9)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=2
b
i
=4
w
i
=3
w=1
w-w
i
=-2
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
0
11/28/2013 53
Example (10)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=2
b
i
=4
w
i
=3
w=2
w-w
i
=-1
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
0
3
11/28/2013 54
Example (11)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=2
b
i
=4
w
i
=3
w=3
w-w
i
=0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
0
3
4
11/28/2013 55
Example (12)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=2
b
i
=4
w
i
=3
w=4
w-w
i
=1
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
0
3
4
4
11/28/2013 56
Example (13)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=2
b
i
=4
w
i
=3
w=5
w-w
i
=2
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
0
3
4
4
7
11/28/2013 57
Example (14)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=3
b
i
=5
w
i
=4
w=1..3
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0
3
3
3
3
0 0
3
4
4
7
0
3
4
11/28/2013 58
Example (15)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=3
b
i
=5
w
i
=4
w=4
w- w
i
=0
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0 0 0
3
4
4
7
0
3
4
5
3
3
3
3
11/28/2013 59
Example (15)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=3
b
i
=5
w
i
=4
w=5
w- w
i
=1
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0 0 0
3
4
4
7
0
3
4
5
7
3
3
3
3
11/28/2013 60
Example (16)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=4
b
i
=5
w
i
=4
w=1..4
Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0 0 0
3
4
4
7
0
3
4
5
7
0
3
4
5
3
3
3
3
11/28/2013 61
Example (17)
if w
i
<= w // item i can be part of the solution
if b
i
+ B[i-1,w-w
i
] > B[i-1,w]
B[i,w] = b
i
+ B[i-1,w- w
i
]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // w
i
> w
0
0
0
0
0
0
W
0
1
2
3
4
5
i
0 1 2 3
0 0 0 0
i=4
b
i
=5
w
i
=4
w=5

Items:
1: (2,3)
2: (3,4)
3: (4,5)
4: (5,6)
4
0 0 0
3
4
4
7
0
3
4
5
7
0
3
4
5
7
3
3
3
3