BITS, PILANI – K. K.

BIRLA GOA CAMPUS

Design & Analysis of Algorithms

Lecture No. 3

1
Course Objective 1:
Algorithm Design Techniques
We already know one popular strategy
Divide & Conquer
Consider the Coin Change Problem with coins of
denomination 1, 5, 10 & 25
Solution is Easy
What is the guarantee that solution works!
Introduce two more popular & widely applicable problem
solving strategies:
• Dynamic Programming
• Greedy Algorithms
 Key Questions
Course Objective 2:
One of the objectives of this course is to look at
Problems for which the known solution is non
polynomial and we can’t prove that no polynomial
time solution will ever exist.
Understand famous P vs NP problem
We strongly believe that certain important class of
problems will not have polynomial time solution.
 Course Objective - 3
• How to deal with the class of problems for which
we strongly believe that no polynomial time
algorithm will exist?
• Approximation Algorithms
• This class consists of important practical
problems
Example: Traveling Salesman Problem, 0-1
Knapsack Problem, Bin Packing Problem
And Many more
 Binary Search Tree
• Binary Search Tree
Well known, important data structure for efficient
search
If T is Binary tree with n nodes, then
min height of the tree is
logn
max height of the tree is
n-1
Disadvantage
Tree can be skewed making search inefficient
 Binary Search Tree
• How to fix the skewness
Balanced trees
- AVL Trees
- Red-Black Trees
- Multi-way Search Trees, (2, 4) Trees
- Few More
Technique used for balancing
- Rotations
Left Rotation or Right Rotation
 Binary Search Tree
• Disadvantage with Balanced Trees
Number of rotations needed to maintain balanced
structure is
O(logn)
Question
Are there type of binary search trees for which
number of rotations needed for balancing is
constant
(independent of n)
 Course Objective - 4
Course Objective – 4
Study type of binary search trees called Treap for
which expected number of rotations needed for
balancing is constant
We will be proving that expected number of
rotations needed for balancing in Treap is 2
(something really strong)
Treap is an Randomized Data Structure
Review - Algorithm Design Strategy
• Divide & Conquer
- binary search
- merge sort
- quick sort
- Matrix Multiplication (Strassen Algorithm)
Many More
Many standard iterative algorithms can be written
as Divide & Conquer Algorithms.
Example: Sum/Max of n numbers
Question: Any advantage in doing so?
 Divide & Conquer
Divide-and-conquer algorithms:
1. Dividing the problem into smaller sub-
problems
(independent sub-problems)
2. Solving those sub-problems
3. Combining the solutions for those smaller
sub-problems to solve the original problem
 Divide & Conquer
How to analyze Divide & Conquer Algorithms
Generally, using Recurrence Relations
• Let T(n) be the number of operations required to
solve the problem of size n.
Then T(n) = a T(n/b) + c(n) where
- each sub-problem is of size n/b
- There are a such sub-problems
- c(n) extra operations are required to combine the
solutions of sub-problems into a solution of the
original problem
 Divide & Conquer
• How to solve Recurrence Relations
- Substitution Method
- works in simple relations
- Masters Theorem
- See Text Book for details
 Coin Change Problem
Coin Change Problem
Given a value N, if we want to make change for
N paisa, and we have infinite supply of each of C
= { 1, 5, 10, 25} valued coins, what is the
minimum number of coins to make the change?
Solution – Easy (We all know this)
Greedy Solution
 Coin Change Problem
Key Observations
• Optimization Problem
• Making Change is possible
i.e. solution exists
• Multiple solution exists
For example: 60 - 25, 25, 10 OR 10, 10, six times
• Question has two parts
1. Minimum number of coins required
2. Which coins are part of the solution
• What is the guarantee that solution works?
 Coin Change Problem
Key Observations
Suppose we add coin of denomination 20 to the set
i.e. C = { 1, 5, 10, 20, 25}
If N = 40
Then the greedy solution fails.
Because the greedy solution will give answer as
3 which is {25, 10, 5}
Whereas correct answer is {20, 20}
 Course Objective
Main Question:
Given an problem (mostly optimization
problem)
1. How to decide whether to use
Greedy Strategy OR Dynamic Programming Strategy?
2. How to show that solution works?
We will explore these ideas in the initial few
lectures.
 Coin Change Problem
Suppose we want to compute the minimum
number of coins with values
d[1], d[2], …,d[n] where each d[i]>0
& where coin of denomination i has value d[i]
Let c[i][j] be minimum number of coins required
to pay an amount of j units 0<=j<=N using only
coins of denominations 1 to i, 1<=i<=n
C[n][N] is the solution to the problem
 Coin Change Problem
In calculating c[i][j], notice that:
• Suppose we do not use the coin with value
d[i] in the solution of the (i,j)-problem,
then c[i][j] = c[i-1][j]
• Suppose we use the coin with value d[i] in the
solution of the (i,j)-problem,
then c[i][j] = 1 + c[i][ j-d[i]]
Since we want to minimize the number of coins,
we choose whichever is the better alternative
Coin Change Problem – Recurrence
Therefore
c[i][j] = min{c[i-1][j], 1 + c[i][ j-d[i]]}
&
c[i][0] = 0 for every i
Alternative 1
Recursive algorithm
 Overlapping Subproblems
When a recursive algorithm revisits the same
problem over and over again, we say that the
optimization problem has overlapping
subproblems.
How to observe/prove that problem has
overlapping subproblems.
Answer – Draw Computation tree and observe
 Overlapping Subproblems
Computation Tree

Dynamic-programming algorithms typically take advantage of overlapping