COL 351: Analysis and Design

of Algorithms

Lecture 6

Keerti Choudhary
Department of Computer Science, IIT Delhi
Recall: Interval Selection Problem

1 Graph
2 Competitive
3
IOT
Theory programming

4 Machine
5 6
Cryptography Security
Learning

7 8 Quantum
NLP
Computing

8 am 10 am 12 pm 2 pm 4 pm 6 pm 8 pm

Goal: Find a set of maximum number of

non-overlapping seminars
Recall: Interval Selection Problem

1 Graph
2 Competitive
3
IOT
Theory programming

4 Machine
5 6
Cryptography Security
Learning

7 8 Quantum
NLP
Computing

8 am 10 am 12 pm 2 pm 4 pm 6 pm 8 pm

Optimal solution
S = {1,7,5,3,6}
Recall: Interval Selection Problem

1 Graph
2 Competitive
3
IOT
Theory programming

100 400 100

4 Machine
5 6
Cryptography Security
Learning

100 100 100

7 8 Quantum
NLP
Computing

100 400

8 am 10 am 12 pm 2 pm 4 pm 6 pm 8 pm

What if each interval has some

associated weight?
Recall: Interval Selection Problem

1 Graph
2 Competitive
3
IOT
Theory programming

100 400 100

4 Machine
5 6
Cryptography Security
Learning

100 100 100

7 8 Quantum
NLP
Computing

100 400

8 am 10 am 12 pm 2 pm 4 pm 6 pm 8 pm

What if each interval has some

associated weight?
Weighted Interval Selection

Formal Statement
Given: A collection C = {I1 = [s1, t1], …, In = [sn, tn]} with weight function wt : C → ℝ+.

Aim: Find a subset S ⊆ {I1, …, In} of non-overlapping intervals such that

∑
wt(I ) is maximised.
I∈S
Weighted Interval Selection

Formal Statement
Given: A collection C = {I1 = [s1, t1], …, In = [sn, tn]} with weight function wt : C → ℝ+.

Aim: Find a subset S ⊆ {I1, …, In} of non-overlapping intervals such that

∑
wt(I ) is maximised.
I∈S

Main Idea: Use Dynamic Programming !

Main Idea
Intervals in non-decreasing order of nish time
I1

I2

I3

I4

I5

I6

I7

Subproblem [1,j]: Find optimal of {I1, …, Ij}?

fi
Main Idea
Intervals in non-decreasing order of nish time
I1

I2

I3

I4

I5

I6

I7

Ques 1: What if I7 is in optimal for [1,n]? Remains to solve: ________________

fi
Main Idea
Intervals in non-decreasing order of nish time
I1

I2

I3

I4

I5

I6

I7

Ques 1: What if I7 is in optimal for [1,n]? Subproblem [1,4]

Remains to solve: ________________
fi
Main Idea
Intervals in non-decreasing order of nish time
I1

I2

I3

I4

I5

I6

I7

Ques 1: What if I7 is in optimal for [1,n]? Subproblem [1,4]

Remains to solve: ________________

Largest index k satisfying

finish-time(Ik) ⩽ start-time(I7)
fi
Main Idea
Intervals in non-decreasing order of nish time
I1

I2

I3

I4

I5

I6

I7

Ques 1: What if I7 is in optimal for [1,n]? Subproblem [1,4]

Remains to solve: ________________

Ques 2: What if I7 is not in opt for [1,n]? Remains to solve: ________________

fi
Main Idea
Intervals in non-decreasing order of nish time
I1

I2

I3

I4

I5

I6

I7

Ques 1: What if I7 is in optimal for [1,n]? Subproblem [1,4]

Remains to solve: ________________

Ques 2: What if I7 is not in opt for [1,n]? Subproblem [1,6]

Remains to solve: ________________
fi
Algorithm

1. Initialise an array OPT of size n.

2. Sort intervals in C = {I1, …, In} in non-decreasing order of nish-time

3. OPT[1] = wt(I1)
4. For j = 2 to n:
— k = Largest index satisfying finish-time(Ik) ⩽ start-time(Ij).

— OPT[ j] = max(wt(Ij) + OPT[k], OPT[ j − 1]).

5. Return OPT[n].

fi
Algorithm

1. Initialise an array OPT of size n.

2. Sort intervals in C = {I1, …, In} in non-decreasing order of nish-time

3. OPT[1] = wt(I1)
4. For j = 2 to n:
— k = Largest index satisfying finish-time(Ik) ⩽ start-time(Ij).

— OPT[ j] = max(wt(Ij) + OPT[k], OPT[ j − 1]).

5. Return OPT[n].

fi
Algorithm

1. Initialise an array OPT of size n.

2. Sort intervals in C = {I1, …, In} in non-decreasing order of nish-time

3. OPT[1] = wt(I1)
4. For j = 2 to n:
— k = Largest index satisfying finish-time(Ik) ⩽ start-time(Ij).

— OPT[ j] = max(wt(Ij) + OPT[k], OPT[ j − 1]).

5. Return OPT[n].

Homework: Provide an O(n log n) time implementation.

fi
Longest Common
Subsequence

2-Dimensional
Dynamic Programming
Longest Common Subsequence

Given: Two sequences X = (x1, …, xm) and Y = (y1, …, yn).

Find: Longest common subsequence (LCS) of X and Y.

Eg: X = EXPLAIN
Y = PLEASANT
Longest Common Subsequence

Given: Two sequences X = (x1, …, xm) and Y = (y1, …, yn).

Find: Longest common subsequence (LCS) of X and Y.

Eg: X = EXPLAIN
Y = PLEASANT

LCS(X,Y) = PLAN
Longest Common Subsequence

Given: Two sequences X = (x1, …, xm) and Y = (y1, …, yn).

Find: Longest common subsequence (LCS) of X and Y.

Eg: X = EXPLAIN Subproblem:

Y = PLEASANT
Xi = (x1, …, xi)
LCS(X,Y) = PLAN
Yj = (y1, …, yj)

1+LCS(Xi−1, Yj−1) if xi = yj
LCS(Xi, Yj) =
max(LCS(Xi−1, Yj), LCS(Xi−1, Yj)) otherwise
Longest Common Subsequence

Given: Two sequences X = (x1, …, xm) and Y = (y1, …, yn).

Find: Longest common subsequence (LCS) of X and Y.

Eg: X = EXPLAIN Subproblem:

Y = PLEASANT
Xi = (x1, …, xi)
LCS(X,Y) = PLAN
Yj = (y1, …, yj)

1+LCS(Xi−1, Yj−1) if xi = yj
LCS(Xi, Yj) =
max(LCS(Xi−1, Yj), LCS(Xi−1, Yj)) otherwise
Longest Common Subsequence

Given: Two sequences X = (x1, …, xm) and Y = (y1, …, yn).

Find: Longest common subsequence (LCS) of X and Y.

Eg: X = EXPLAIN Subproblem:

Y = PLEASANT
Xi = (x1, …, xi)
LCS(X,Y) = PLAN
Yj = (y1, …, yj)

1+LCS(Xi−1, Yj−1) if xi = yj
LCS(Xi, Yj) =
max(LCS(Xi−1, Yj), LCS(Xi−1, Yj)) otherwise
Algorithm

1. Initialise a 2d-array LCS of size (m + 1) × (n + 1).

2. Set LCS(i, j) = 0, for all pairs when at least one of i or j is 0

3. For i = 1 to m:
4. For j = 1 to n:
1+LCS(i − 1, j − 1) if xi = yj
LCS(i, j) =
max(LCS(i, j − 1), LCS(i − 1, j)) otherwise

5. Return LCS[m, n].

Algorithm

1. Initialise a 2d-array LCS of size (m + 1) × (n + 1).

2. Set LCS(i, j) = 0, for all pairs when at least one of i or j is 0

3. For i = 1 to m:
4. For j = 1 to n:
1+LCS(i − 1, j − 1) if xi = yj
LCS(i, j) =
max(LCS(i, j − 1), LCS(i − 1, j)) otherwise

5. Return LCS[m, n].

Ques 1: Provide an O(mn) time and O(min(m, n)) space algorithm to

compute the length of LCS.

Ques 2: Provide an O(mn) time algorithm to retrieve the LCS sequence.

Backtracking

P L E A S A N T

0 0 0 0 0 0 0 0 0

E 0 0 0 1 1 1 1 1 1

X 0 0 0 1 1 1 1 1 1

P 0 1 1 1 1 1 1 1 1

L 0 1 2 2 2 2 2 2 2

A 0 1 2 2 3 3 3 3 3

I 0 1 2 2 3 3 3 3 3

N 0 1 2 2 3 3 3 4 4
Backtracking

P L E A S A N T

0 0 0 0 0 0 0 0 0

E 0 0 0 1 1 1 1 1 1

X 0 0 0 1 1 1 1 1 1
P
P 0 1 1 1 1 1 1 1 1
L
L 0 1 2 2 2 2 2 2 2
A
A 0 1 2 2 3 3 3 3 3

I 0 1 2 2 3 3 3 3 3
N
N 0 1 2 2 3 3 3 4 4
Algorithm to find last character

1. Initialise 2d-arrays LCS, CHAR of size (m + 1) × (n + 1).

2. Set LCS(i, j) = 0, for all pairs when at least one of i or j is 0

3. For (i, j) ∈ [1,m] × [1,n] in dictionary order:
If (xi = yj):
LCS(i, j) = 1+LCS(i − 1, j − 1)
CHAR(i, j) = xi
Else If LCS(i − 1, j) ⩾ LCS(i, j − 1):
LCS(i, j) = LCS(i − 1, j)
CHAR(i, j) = CHAR(i − 1,j)
Else:
LCS(i, j) = LCS(i, j − 1)
CHAR(i, j) = CHAR(i, j − 1)
5. Return (LCS[m, n], CHAR[m, n]).

Ques: Modify the above function to provide an O(m 2n) time algorithm to nd
LCS sequence in O(n) space. fi
Partitioning Problem

Non-trivial
Dynamic Program
Partitioning Problem

Chapter-1 Chapter-2 Chapter-3 Chapter-4 Chapter-5

Problems Problems Problems Problems Problems

1..8 1..4 1..11 1..14 1..18

Goal: Divide chapters so that max of

sum total number of problems per person
is minimised.
Partitioning Problem

Chapter-1 Chapter-2 Chapter-3 Chapter-4 Chapter-5

Problems Problems Problems Problems Problems

1..8 1..4 1..11 1..14 1..18

Goal: Divide chapters so that max of

sum total number of problems per person
is minimised.
Partitioning Problem

Formal Statement
Given: An n-length sequence (x1, x2, …, xn).

Aim: Find a partitioning S1, S2, S3 of [1, n] such that following is minimized:

max (
∑ ∑ ∑ i)
xi , xi , x
i∈S1 i∈S2 i∈S3

Assumption: Each xi is at most O(n).

Partitioning Problem

A Simpler Problem
Given: An n-length sequence (x1, x2, …, xn), and an integer L.

Aim: Find a partitioning S1, S2, S3 of [1, n] such that following is bounded by L.

max (
∑ ∑ ∑ i)
xi , xi , x
i∈S1 i∈S2 i∈S3

Assumption: Each xi is at most O(n).

Partitioning Problem

Further Simplification
Given: An n-length sequence (x1, x2, …, xn), and integers L1, L 2, L 3.

Aim: Find a partitioning S1, S2, S3 of [1, n] such that following holds.

∑ ∑ ∑
xi ⩽ L1 , xi ⩽ L2 , and xi ⩽ L3
i∈S1 i∈S2 i∈S3

Assumption: Each xi is at most O(n).

Subproblem ( j, a, b, c)

Given: An j-length sequence (x1, x2, …, xj), and integers a, b, c.

Aim: Find a partitioning S1, S2, S3 of [1, j] such that following holds.

∑ ∑ ∑
xi ⩽ a , xi ⩽ b , and xi ⩽ c
i∈S1 i∈S2 i∈S3

Lemma: Subproblem ( j, a, b, c) has a valid partitioning i at least one of the following

has a valid solution:

∙ Subproblem ( j − 1, a−xj, b, c)
∙ Subproblem ( j − 1, a, b−xj, c)
∙ Subproblem ( j − 1, a, b, c−xj)
ff
Subproblem ( j, a, b, c)

Given: An j-length sequence (x1, x2, …, xj), and integers a, b, c.

Aim: Find a partitioning S1, S2, S3 of [1, j] such that following holds.

∑ ∑ ∑
xi ⩽ a , xi ⩽ b , and xi ⩽ c
i∈S1 i∈S2 i∈S3

Lemma: Subproblem ( j, a, b, c) has a valid partitioning i at least one of the following

has a valid solution:

∙ Subproblem ( j − 1, a−xj, b, c)
∙ Subproblem ( j − 1, a, b−xj, c)
∙ Subproblem ( j − 1, a, b, c−xj)
ff
Algorithm

1. Set m = x1 + ⋯ + xn.
2. Initialise a 4dim-array OPT of size (n + 1) × (m + 1)3.

3. OPT(0, a, b, c) = T, for each a, b, c ⩾ 0.

4. For ( j, a, b, c) ∈ (n × m 3) in dictionary order:
Set OPT( j,a, b, c) = T i at least one of following is also T.
∙ OPT( j − 1, a−xj, b, c)
∙ OPT( j − 1, a, b−xj, c)
∙ OPT( j − 1, a, b, c−xj)

(Here, OPT over negative indices is set to F).

5. Return min{L | OPT(n, L, L, L) = T}.

f
Algorithm

1. Set m = x1 + ⋯ + xn.
2. Initialise a 4dim-array OPT of size (n + 1) × (m + 1)3.

3. OPT(0, a, b, c) = T, for each a, b, c ⩾ 0.

4. For ( j, a, b, c) ∈ (n × m 3) in dictionary order:
Set OPT( j,a, b, c) = T i at least one of following is also T.
∙ OPT( j − 1, a−xj, b, c)
∙ OPT( j − 1, a, b−xj, c)
∙ OPT( j − 1, a, b, c−xj)

(Here, OPT over negative indices is set to F).

5. Return min{L | OPT(n, L, L, L) = T}.

f
Algorithm

1. Set m = x1 + ⋯ + xn.
2. Initialise a 4dim-array OPT of size (n + 1) × (m + 1)3.

3. OPT(0, a, b, c) = T, for each a, b, c ⩾ 0.

4. For ( j, a, b, c) ∈ (n × m 3) in dictionary order:
Set OPT( j,a, b, c) = T i at least one of following is also T.
∙ OPT( j − 1, a−xj, b, c)
∙ OPT( j − 1, a, b−xj, c)
∙ OPT( j − 1, a, b, c−xj)

(Here, OPT over negative indices is set to F).

5. Return min{L | OPT(n, L, L, L) = T}.

f
Algorithm

1. Set m = x1 + ⋯ + xn.
2. Initialise a 4dim-array OPT of size (n + 1) × (m + 1)3.

3. OPT(0, a, b, c) = T, for each a, b, c ⩾ 0.

4. For ( j, a, b, c) ∈ (n × m 3) in dictionary order:
Set OPT( j,a, b, c) = T i at least one of following is also T.
∙ OPT( j − 1, a−xj, b, c)
∙ OPT( j − 1, a, b−xj, c)
∙ OPT( j − 1, a, b, c−xj)

(Here, OPT over negative indices is set to F ).

5. Return min{L | OPT(n, L, L, L) = T}.

f
Algorithm

1. Set m = x1 + ⋯ + xn.
2. Initialise a 4dim-array OPT of size (n + 1) × (m + 1)3.

3. OPT(0, a, b, c) = T, for each a, b, c ⩾ 0.

4. For ( j, a, b, c) ∈ (n × m 3) in dictionary order:
Set OPT( j,a, b, c) = T i at least one of following is also T.
∙ OPT( j − 1, a−xj, b, c)
∙ OPT( j − 1, a, b−xj, c)
∙ OPT( j − 1, a, b, c−xj)

(Here, OPT over negative indices is set to F ).

5. Return min{L | OPT(n, L, L, L) = T}.

f
Algorithm

1. Set m = x1 + ⋯ + xn.
2. Initialise a 4dim-array OPT of size (n + 1) × (m + 1)3.

3. OPT(0, a, b, c) = T, for each a, b, c ⩾ 0.

4. For ( j, a, b, c) ∈ (n × m 3) in dictionary order:
Set OPT( j,
a, b, c) = T i at least one of following is also T.
∙ OPT( j − 1, a−xj, b, c)
∙ OPT( j − 1, a, b−xj, c)
∙ OPT( j − 1, a, b, c−xj)

(Here, OPT over negative indices is set to F ).

5. Return min{L | OPT(n, L, L, L) = T}.

Question: Design an e cient algorithm to retrieve the partitioning.

ffi
f