Section IV.

5: Recurrence Relations from Algorithms

Given a recursive algorithm with input size n, we wish to find a (best big O)
estimate for its run time T(n) either by obtaining an explicit formula for T(n) or by
obtaining an upper or lower bound U(n) for T(n) such that T(n) = (U(n)). This can be
done with the following sequence of steps.
(a) Find the base cases and a recurrence relation for the algorithm.
(b) (1) Obtain an iterative formula for T(n) if possible. Then one can use methods in
Epp, chapter 8, or Section IV.1 of these notes or
(2) Obtain an upper or lower bound U(n) if the iterative formula is difficult or
impossible to obtain explicitly.
In Epp (section 8.2), there are examples of obtaining iterative formulas for T(n) for
simple cases like arithmetic or geometric sequences. Also illustrated are examples where
familiar summation identities can be used to get iterative formulas.
When iterative formulas for T(n) are difficult or impossible to obtain, one can use
either (1) a recursion tree method, (2) an iteration method, or (3) a substitution
method with induction to get T(n) or a bound U(n) of T(n) where T(n) = (U(n)).
Examples of these methods are found in Cormen (pp. 54-61). Examples of each of these
methods are given below.
Definition IV.5.1: Given a recursive algorithm (Definition in Section IV-1), a
recurrence relation for the algorithm is an equation that gives the run time on an input
size in terms of the run times of smaller input sizes.
Definition IV.5.2: A recursion tree is a tree generated by tracing the execution of a
recursive algorithm. (Cormen, p. 59)
Example IV.5.1: For Example IV.1.2. in Section IV.1 (Summing an Array), get a
recurrence relation for the algorithm and iterative formula T(n). Use (a) an iteration
method and (b) a recursion tree method. Get the estimate of T(n).
Solution: (a) T(n) = T(n-1) + 1, since addition of the n-th element can be done by adding
it to the sum of the n-1 preceding elements, and addition involves one operation. Also
T(0)=0. Therefore, T(k) = T(k-1) + 1 for k between 1 and n is the recurrence relation.
Iteration gives T(n) = T(n-1) + 1 = T(n-2)+1 +1 = T(n-3)+1+1+1 = ... = T(0)+1+1+...+1 =
T(0) + n = n. (At the last stage we have added one to itself n times). Therefore, T(n) =
(n).

Solution: (b) At each level of the tree, replace the parent of that subtree by the constant
term of the recurrence relation T(k) = T(k-1) + 1 and make T(k-1) the child. The tree is
really a list here. It is developed from left to right in the diagram below.
T(n)
1
1

1
1
|
|
|
|
T(n-1)

|
T(n-2)

1
|

1
|

|
1
|
T(0)

|
1
|
0

T(n) is computed by finding the sum of the elements at each level of the tree. Therefore
T(n) = n, and T(n) = (n).
Example IV.5.2: Binary Search (recursive version)
The pseudo code for recursive binary search is given below.
Algorithm Recursive Binary Search
Input: Value X of the same data type as array A, indices low and high, which are
positive integers, and A, the array A[low],..., A[high] in ascending order.
Output: The index in array if X is found, 0 (zero) if X not found.
Algorithm Body:
low + high
mid :=

if X = A[mid] then index := mid

else if X < A[mid] and low < mid then call Binary Search with inputs X, low,
mid-1, A
else if X > A[mid] and mid < high then call Binary Search with inputs X, mid+1,
high, A
else index := 0
end Algorithm Recursive Binary Search
Derive a recurrence relation on Binary Search and get a estimate of the worst case
running time T(n). Use a recursion tree method.
Solution to Example IV.5.2: One could count the number of comparisons of X to
n
A[mid]. Binary Search is called on a subarray of length approximately and there are 3
2
comparisons in the worst case before a recursive call is needed again. So the recurrence
relation is T(n) = T(n/2) + 3, or more specifically, T(k) = T(k/2) + 3 for input size k 2.
Also T(1) = 3.

The recursion tree is developed similar to that in Example 1. At each level, let the parent
be the constant term 3 and the child be the term T(k/2). The process is illustrated below.
T(n)

3
|
T(n/2)

3
|
3
|
T(n/4)

3
|
3
|
3

3
|
T(1)

3
|
3
|
3

3
|
3

The depth of the tree is approximately log 2 n. Adding the values in the levels of the tree
we get T(n) 3 log 2 n. Therefore T(n) = (log 2 n).
Example IV.5.3: Merge Sort
The pseudocode for Merge Sort is given in Epp, p. 529. Develop a recurrence relation
and get a estimate for this algorithm.
Algorithm Merge Sort
Input: positive integers bot and top, bot top, array items A[bot], A[bot+1], ... , A[top]
that can be ordered.
Output: array A[bot], A[bot+1], ... , A[top] of the same elements as in the input array but
in ascending order.
Algorithm Body:
if bot < top then do
bot + top
mid :=

call Merge Sort with input bot, mid, A[bot], ..., A[mid]
call Merge Sort with input mid + 1, top, A[mid+1], ..., A[top]
Merge A[bot], ..., A[mid] and A[mid+1], ..., A[top] [where Merge takes these
two arrays in ascending order and gives an array in ascending order]
end do
end Algorithm Merge Sort
Solution to Example IV.3.3: For simplicity let the input size be n = 2 k , k a positive
integer. Let T(n) denote the run time. In the worst case of Merge, there are about n
comparisons needed to determine the ordering in the merged array (actually n-1 since the
last element need not be compared). Since we desire a estimate and not an exact
formula, we can assume n. In the best case there are n/2 comparisons in Merge. This
algorithm is called recursively on two subarrays of length approximately n/2. Therefore,
the recurrence relation is T(n) = 2T(n/2) + n , so for particular k n,
k
T(k) = 2T( ) + k.
2

The recursion tree is developed below, using fact that, at each level of the tree, the parent
k
node is the term not involving T and there are two children due to the term 2T( ). The
2
tree is developed as shown, going from left to right and top to bottom.
T(n)

------

T(n/2)

n
2

T(

n
)
4

n
2

T(

n
)
4

T(n/2)

T(

n
2

------

n
)
4

T(

n
)
4

n
4

n
2
n
4

n
4

n
4

.
|
1

|
1

|
........................... 1

|
1

n
in each
m
node). There are k = log 2 n levels since we can cut the array in half k times. Thus, the
sum of the elements in all nodes of the tree is n log 2 n. Therefore, T(n) = ( n log2 n).

The sum of the values at each level of the tree is n (m nodes times the value

Example IV.5.4: Given the recurrence T(n) = T(

n
2n
) + T(
) + n, get a estimate for
3
3

T(n) using the recursion tree method.

Solution: The recursion tree is developed below.
n
T(

n
2n
) T(
)
3
3

n
3

T(

2n
3

n
2n
2n
) T(
) T(
)
9
9
9

T(

4n
)
9

The construction continues until we get to the leaves. We keep multiplying n by

2
3

until

( 23 ) k n = 1. The longest possible path of this tree is of length k where ( 23 ) k n = 1, i.e.,

where k = log 3 2 n . The sum of the values in each level of the tree is at most n. The left
part of the tree plays out before the right side; the length of the path from the root to the
leaves can vary from log 3 n to log 3 2 n , depending on whether argument k in T(k) goes to
k/3 or 2k/3. One obtains T(n) n log 3 2 n and T(n) n log3 n . By the fact that
growth rate of a log function is independent of its base (Theorem II.5.8), we can conclude
T(n) = O( n log 2 n ) and T(n) = ( n log 2 n ). Therefore T(n) = ( n log 2 n ).
The next two examples use the iteration method. Example IV.5.5 involves a convergent
geometric series, and Example IV.5.6 is more complicated since the geometric series
diverges.
Example IV.5.5: Given the recurrence T(n) = 3 T ( n 4 ) + n, show by an iteration
method that T(n) = (n).
Solution: For simplicity, we shall use m for m since the recurrence is in terms of the
floor function. The iteration proceeds as follows:
T(n) n + 3T(

n
n
n
n
n
n
) = n + 3(( ) + 3T(
))=n+3 +9
+ 27T( )
4
4
16
4
16
64

3
).
4
i=0
i=0
3
(The formula for T(n) involves a decreasing geometric series with ratio r = ).
4
Therefore, T(n) = O(n) (use C = 4 and X 0 = N 0 = 1 in the definition of big O). Also,
n
T(n) n since 3T(
) > 0. So T(n) = (n) (use C = 1 and X 0 = N 0 = 1 in the
4
definition of ). By the definition of (section II.5), we have T(n) = (n).
n

(3 / 4) = 4n from the formula (r ) = 1 r

i

Example IV.5.6: Given the recurrence T(n) = 3 T(

when |r| < 1 (here r =

n
) + n, use the iteration method to
2

get a big O estimate for T(n).

Solution: For simplicity, we shall use m for m since the recurrence is in terms of the
floor function. The iteration proceeds as follows:
n
n
n
n
n
n
T(n) = n + 3T( ) = n + 3(( ) + 3T( ) ) = n + 3 + 9 + 27T( ) = ... .
2
2
4
2
4
8

The coefficients form the terms of the geometric series

(r ) where r = 3/2.
i

i=0

Since

|r| > 1, the infinite series diverges. However, there are only finitely many terms in the
sum. The sum terminates for the smallest integer k where 2 k > n (the number of times
n
n
we divide n by 2 is log 2 n ), i.e., when k log 2 n . Since we have
2 2
n
n
T(n) n + 3 + 9 + + ( 3log2 n1 / 2 log2 n1 ) n
2
4
= n

log 2 n 1
3 i
2
i=0
3 log 2 n
2
log 2 n
log 2 n

) = n (( 32 ) log 2 n 1) /( 32 1) = 2 n ( ( 32 ) log 2 n -1)

= 2n ( )

= 2n 3

-2n

)- 2 n

= 2 3
- 2 n using 2 log2 n = n
= 2 n log 2 3 - 2 n .
log 2 n

In the last step, we used

(*) a logb n = n logb a ,
which is derived in Section IV.6.
Since log 2 3 > 1, we know that term n log2 3 dominates n. From section II.5, we can
conclude that T(n) = O (n log2 3 ).
Note: We use standard properties of logarithms in this argument. We also use the
n

identity

= (r n +1 1) (r 1) when r 1.

i=0

Note: The Master method given in Section IV.6 will give us a much easier way to do this
problem.
Substitution method with induction
The next two examples use the substitution method with induction. The method is
described as follows. We show T(n) = O(U(n)). Showing T(n) = (U(n)) is similar.
(1) Guess the form of a bound U(n) of T(n).
(2) Assume by strong induction that for some constants C and n0 , T(k) C U(k) for all
k n0 and k < n. Show that for these values of C and n0 , T(n) C U (n) for all
n > n0
(3) Find a pair C and n0 that also satisfy the initial conditions (values of T(1), T(2). etc.).
These must satisfy the conditions in (2) and (3); therefore, the values may change.
(4) If the given form does not work, try a different U(n) with same growth rate. If this
fails, try a U(n) with a different growth rate.
Example IV.5.7: Find a bound for T(n) where T(n) = T(n-1) + n and T(0) = 1. Use
the substitution method with induction.

Solution: From Epp, section 8.2, one can get the iterative formula
T(n ) = 1 + n(n + 1) 2 . So we know that T(n) = (n 2 ). However, we show how
substitution and induction works in this example
Assume T(n) C n 2 for all n > n0 , k < n, and find C and n0 that work.
Using the induction assumption, we try to find C and n0 where
T(n) = T(n-1) + n
C (n 1) 2 + n
= C (n 2 - 2n +1) + n
= C n2 - 2 C n + C + n
C n 2 for all n > n0 .
This inequality implies that 2 n C C n, i.e., n C (2n-1) or

n
C.
2n 1

n
is decreasing (verify by taking derivatives), we substitute 1
2n 1
for n and must have 1/(2-1) C, that is, C 1. However, no C works for T(0), and
T(1) = 2. For T(1) C (1) 2 to hold, we must have C 2. Putting together the
constraints C 1 and C 2, we know that C 2 works, and also that n0 1 must hold.
So take C = 2 and n0 = 1 in the definition of big O.

Since the expression

It is true that we did not need such a complicated procedure to solve Example II.5.7.
However, many times one cannot easily get a closed form for T(n). Yet the substitution
method with induction works. Example IV.5.8 illustrates this.
Example IV.5.8: Use the substitution and induction approach to show
n
T(n) = 2 T( ) + n, where T(1) = 1, is (n log 2 n ).
2
Solution: Show that T(n) C n log 2 n for all n > k n0 and find a C and n0 .
Using the induction assumption and recurrence formula, one obtains

T (n) 2(C 2n log 2 2n ) + n

Cn log 2 (n2 )+ n

= Cn log 2 n Cn log 2 2 + n
= Cn log 2 n Cn + n
Cn log 2 n if C 1.
n
n
(We use properties of logs and fact that in this derivation).
2 2

However, T(n) C n log 2 n is not true for n = 1, for 1 log 2 1 = 0. So we must choose
n0 2. Now, T(2) = 2T(1) + 2 = 4 and T(3) = 2T(1) + 3 = 5. We must pick C large
enough that T(2) C (2 log 2 2 ) and T(3) C (3 log 2 3 ). Any C 2 works. So we can
take C = 2 and n0 = 2 in the definition of big O.

Changing Variables
Sometimes one can do algebraic manipulation to write an unknown recurrence in
terms of a familiar one. Then one solves the familiar one and writes the solution in terms
of the original variable. Example IV.5.9 illustrates this.

Example IV.5.9: Solve T(n) = 2T( n ) + log 2 n .

Solution: Let m = log 2 n . Then this recurrence becomes T (2 m ) = 2T (2 m 2 ) + m . Let
S(m) = T( 2 m ) , i.e. S(m) = T(g(m)) where g(m) = 2 m . So S (m 2) = T (2 (m 2 ) and
the new recurrence S(m) = S(m/2) + m is produced. This is the same form as Example
IV.5.8; therefore it has the solution S(m) = (m log 2 m ). By the substitution n = 2 m , we
get T(n) = T( 2 m ) = S(m) = (m log 2 m ) = ( log 2 n log 2 (log 2 n)) .
Exercises:
(1) Use the recursion tree method to get a estimate, coefficient 1, for the recurrence
n
T(n) = 2T( ) + n 2 .
2
(2) Use the recursion tree method to get a estimate, coefficient 1, for the recurrence in
Example IV.5.5. You need to get the height of the recursion tree.
(3) Use the recursion tree method to get a estimate, coefficient 1, for the recurrence T
n
n
T(n) = T( ) + T( 3 ) + n 2 .
4
4
(4) Use an iteration method (similar to Example IV.5.5) to get a estimate, coefficient
n
1, for recurrence the T(n) = 4T + n .
5
(5) Use an iteration method (similar to Example IV.5.6) to get a estimate, coefficient 1,
n
for the recurrence T(n) = 5T + n.
4

n
n
(6) Given the recurrence T(n) = T ( ) + T ( ) + 1, show that T(n) C n for any C
2
2
cannot be shown using an induction argument. Show T(n) = O(n) is correct by finding a
C and b where T(n) C n b works. Assume that the initial condition is T(1) = 1.
(7) Solve the recurrence T (n) = T (n 1 / 3 ) + T (n 2 3 ) + log 3 n using a substitution m in terms
of n similar to that in Example IV.5.9. You will get a recurrence in m that is one of the
previous examples. Solve it in terms of m and solve the given recurrence in terms of n.