Scribd
Upload
147 views

Recurrence Relation For Complexity Analysis of Algorithms

The document discusses three main methods for solving recurrence relations that arise when analyzing the time complexity of algorithms: the substitution method, recurrence tree method, and master method. The substitution method involves making a guess for the solution and proving it using mathematical induction. The recurrence tree method draws a tree representing the recurrence relation and sums the work done at each level. The master method directly solves recurrences of the form T(n) = aT(n/b) + f(n) by considering three cases based on how f(n) compares to nLogba.

Uploaded by

Awais Arshad
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
147 views

Recurrence Relation For Complexity Analysis of Algorithms

The document discusses three main methods for solving recurrence relations that arise when analyzing the time complexity of algorithms: the substitution method, recurrence tree method, and master method. The substitution method involves making a guess for the solution and proving it using mathematical induction. The recurrence tree method draws a tree representing the recurrence relation and sums the work done at each level. The master method directly solves recurrences of the form T(n) = aT(n/b) + f(n) by considering three cases based on how f(n) compares to nLogba.

Uploaded by

Awais Arshad
Copyright
© © All Rights Reserved
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 4

How to solve Recurrence Relation for Complexity

Analysis of Algorithms:
Many algorithms are recursive. When we analyze them, we get a recurrence relation for
time complexity. We get running time on an input of size n as a function of n and the
running time on inputs of smaller sizes. For example in Merge Sort, to sort a given array,
we divide it into two halves and recursively repeat the process for the two halves. Finally,
we merge the results. Time complexity of Merge Sort can be written as T(n) = 2T(n/2) +
cn. There are many other algorithms like Binary Search, Tower of Hanoi, etc. 
There are mainly three ways of solving recurrences:

Substitution Method: 
We make a guess for the solution and then we use mathematical induction to prove the
guess is correct or incorrect. 
For example consider the recurrence T(n) = 2T(n/2) + n
We guess the solution as T(n) = O(nLogn). Now we use induction to prove our guess.
We need to prove that T(n) <= cnLogn. We can assume that it is true for values smaller
than n.
T(n) = 2T(n/2) + n
     <= 2cn/2Log(n/2) + n
       =  cnLogn – cnLog2 + n
       =  cnLogn – cn + n
    <= cnLogn

Recurrence Tree Method:


In this method, we draw a recurrence tree and calculate the time taken by every level of
the tree. Finally, we sum the work done at all levels. To draw the recurrence tree, we start
from the given recurrence and keep drawing till we find a pattern among levels. The
pattern is typically arithmetic or geometric series. 
 
For example, consider the recurrence relation :

T(n) = T(n/4) + T(n/2) + cn2


            cn2
            
/      \
  T(n/4)     T(n/2)

If we further break down the expression T(n/4) and T(n/2), 


we get the following recursion tree.
                    cn2
              /            \     
    c(n2)/16          c(n2)/4
   /         \            /         \
T(n/16)  T(n/8)  T(n/8)    T(n/4) 
Breaking down further gives us following
                       cn2 
                /                \     
       c(n2)/16              c(n2)/4
    /          \                 /          \
c(n2)/256  c(n2)/64  c(n2)/64   c(n2)/16
 /    \            /    \      /    \        /    \  

To know the value of T(n), we need to calculate the sum of tree 


nodes level by level. If we sum the above tree level by level, 
we get the following series T(n)  = c(n^2 + 5(n^2)/16 + 25(n^2)/256) + ….
The above series is a geometrical progression with a ratio of 5/16.
To get an upper bound, we can sum the infinite series. We get the sum as (n2)/(1 – 5/16)
which is O(n2)

Master Method: 
Master Method is a direct way to get the solution. The master method works only for the
following type of recurrences or for recurrences that can be transformed into the
following type. 
 
T(n) = aT(n/b) + f(n) where a >= 1 and b > 1
There are the following three cases: 
 If f(n) = O(nc) where c < Logba then T(n) = Θ(nLog a) 
b

 If f(n) = Θ(nc) where c = Logba then T(n) = Θ(ncLog n) 


 If f(n) = Ω(nc) where c > Logba then T(n) = Θ(f(n)) 
How does this work? 
The master method is mainly derived from the recurrence tree method. If we draw the
recurrence tree of T(n) = aT(n/b) + f(n), we can see that the work done at the root is f(n),
and work done at all leaves is Θ(nc) where c is Logba. And the height of the recurrence
tree is Logbn 
 

In the recurrence tree method, we calculate the total work done. If the work done at
leaves is polynomially more, then leaves are the dominant part, and our result becomes
the work done at leaves (Case 1). If work done at leaves and root is asymptotically the
same, then our result becomes height multiplied by work done at any level (Case 2). If
work done at the root is asymptotically more, then our result becomes work done at the
root (Case 3). 
Examples of some standard algorithms whose time complexity can be evaluated
using the Master Method 
 Merge Sort: T(n) = 2T(n/2) + Θ(n). It falls in case 2 as c is 1 and Log ba] is also 1. So
the solution is Θ(n Logn) 
 Binary Search: T(n) = T(n/2) + Θ(1). It also falls in case 2 as c is 0 and Log ba is also
0. So the solution is Θ(Logn) 
Examples:
Example 1: Say you have derived the recurrence relation T(n) = 8T(n/2) + cn2, where c is
some positive constant. We see that this has the appropriate form for applying the master
method, and that a=8, b=2, and h(n) = cn2. cn2 is O(nlog28 − ε) = O(n3 − ε) for any ε ≤ 1, so this
falls into case 1. Therefore, T(n) is Θ(n3).
Example 2: Say you have derived the recurrence relation T(n) = T(n/2) + cn, where c is
some positive constant. We see that this has the appropriate form for applying the master
method, and that a=1, b=2, and h(n) = cn. Then h(n) is Ω(nlog21 + ε) = Ω(nε) for any ε ≤ 1, so
this falls into case 3. And ah(n/b) = cn/2 = ½h(n), therefore T(n) is Θ(n).
Example 3: Say you have derived the recurrence relation T(n) = 8T(n/4) + cn3/2, where c is
some positive constant. We see that this has the appropriate form for applying the master
method, and that a=8, b=4, and h(n) = cn3/2. cn3/2 is Θ(nlog48) = Θ(n3/2), so this falls into case
2. Therefore, T(n) is Θ(n3/2log n).
Notes: 
 It is not necessary that a recurrence of the form T(n) = aT(n/b) + f(n) can be solved
using Master Theorem. The given three cases have some gaps between them. For
example, the recurrence T(n) = 2T(n/2) + n/Logn cannot be solved using master
method. 
 Case 2 can be extended for f(n) = Θ(ncLogkn) 
If f(n) = Θ(ncLogkn) for some constant k >= 0 and c = Logba, then T(n) = Θ(ncLogk+1n) 

You might also like