SOLVING RECURRENCES :-( Happen again (or) repeatedly)

The indispensable last step when analyzing an algorithm is often to solve a

recurrence equation.
With a little experience and intention, most recurrence can be solved by
intelligent guesswork.
However, there exists a powerful technique that can be used to solve certain
classes of recurrence almost automatically.
This is a main topic of this section the technique of the characteristic
equation.

1. Intelligent guess work:

This approach generally proceeds in 4 stages.

1. Calculate the first few values of the recurrence
2. Look for regularity.
3. Guess a suitable general form.
4. And finally prove by mathematical induction(perhaps constructive induction).

Then this form is correct.

Consider the following recurrence,

0 if n=0
T(n) = 3T(n 2)+n otherwise

First step is to replace n 2 by n/2

It is tempting to restrict n to being ever since in that case n2 = n/2, but
recursively dividing an even no. by 2, may produce an odd no. larger than 1.
Therefore, it is a better idea to restrict n to being an exact power of 2.
First, we tabulate the value of the recurrence on the first few powers of 2.

For instance, T(16) = 3 * T(8) +16

= 3 * 65 +16
= 211.
Instead of writing T(2) = 5, it is more useful to write T(2) = 3 * 1 +2.
Then,
T(A) = 3 * T(2) +4
= 3 * (3 * 1 +2) +4
= (32 * 1) + (3 * 2) +4
* We continue in this way, writing n as an explicit power of 2

The pattern is now obvious.

Proposition: (Geometric Series)

Let Sn be the sum of the first n terms of the geometric series a, ar, ar2.Then
Sn = a(1-rn)/(1-r), except in the special case when r = 1; when Sn = an.
* It is easy to check this formula against our earlier tabulation.

2. Homogenous Recurrences :

* We begin our study of the technique of the characteristic equation with the
resolution of homogenous linear recurrences with constant co-efficient, i.e the
recurrences of the form,

a0tn + a1tn-1 + .. + aktn-k = 0 where the ti are the values we are looking for.

* The values of ti on K values of i (Usually 0 i k-1 (or) 0 i k) are needed to

determine the sequence.
* The initial condition will be considered later.
* The equation typically has infinitely many solution.
* The recurrence is,

linear because it does not contain terms of the form t n-i, t n-j, t2 n-i, and
soon.

homogeneous because the linear combination of the t n-i is equal to zero.

With constant co-efficient because the ai are constants

* Consider for instance our non familiar recurrence for the Fibonacci sequence,

fn = f n-1 + f n-2

* This recurrence easily fits the mould of equation after obvious rewriting.
fn f n-1 f n-2 = 0

* Therefore, the fibonacci sequence corresponds to a homogenous linear recurrence

with constant co-efficient with k=2,a0=1&a1=a2 = -1.

* In other words, if fn & gn satisfy equation.

k
So ai f n-i = 0 & similarly for gn & fn
i=0
We set tn = C fn + d gn for arbitrary constants C & d, then tn is also a solution to
equation.

* This is true because,

a0tn + a1tn-1 + + aktn-k

= a0(C fn +d gn) + a1(C fn-1 + d gn-1) + + ak(C fn-k + d gn-k)

= C(a0 fn + a1 fn-1 + + ak fn-k)+
d(a0 gn + a1 gn-1 + + ak gn-k)
=C*0+d*0
= 0.

1) (Fibonacci) Consider the recurrence.

n if n=0 or n=1
fn =
f n-1 + f n-2 otherwise

We rewrite the recurrence as

,
fn f n-1 f n-2 = 0.
The characteristic polynomial is,
x2 x 1 = 0.

The roots are,

-(-1) ((-1)2 + 4)
x = ------------------------
2
1 (1 + 4)
= ----------------
2
1 5
= ----------
2
1+5 1 - 5
r1 = --------- and r2 = ---------
2 2
The general solution is,

fn = C1r1n + C2r2n

when n=0, f0 = C1 + C2 = 0
when n=1, f1 = C1r1 + C2r2 = 1

C1 + C2 = 0 (1)
C1r1 + C2r2 = 1 (2)

From equation (1)

C1 = -C2

Substitute C1 in equation(2)
-C2r1 + C2r2 = 1
C2[r2 r1] = 1

Substitute r1 and r2 values

1 - 5 1 - 5
C2 --------- - --------- = 1
2 2
1 5 1 5
C2 --------------------- = 1
2
-C2 * 25
-------------- = 1
2
5C2 = 1

C1 = 1/5 C2 = -1/5
Thus,

1 1 + 5 n -1 1 - 5 n
fn = ---- --------- + ---- --------
5 2 5 2
1 1 + 5n 1 5 n
= ---- --------- - ---------
5 2 2
3. Inhomogeneous recurrence :

* The solution of a linear recurrence with constant co-efficient becomes more

difficult
when the recurrence is not homogeneous, that is when the linear combination is
not

equal to zero.

* Consider the following recurrence

a0tn + a1 t n-1 + + ak t n-k = bn p(n)

* The left hand side is the same as before,(homogeneous) but on the right-hand
side
we have bnp(n), where,

b is a constant
p(n) is a polynomial in n of degree d.

4. Change of variables:
* It is sometimes possible to solve more complicated recurrences by making a
change of variable.
* In the following example, we write T(n) for the term of a general recurrences,
and ti for the term of a new recurrence obtained from the first by a change of
variable.

Example: (1)
Consider the recurrence,

1 , if n=1
T(n) =
3T(n/2) + n , if n is a power of 2, n>1

Reconsider the recurrence we solved by intelligent guesswork in the previous

section, but only for the case when n is a power of 2

1
T(n) =
3T(n/2) + n

* We replace n by 2i.
* This is achieved by introducing new recurrence ti, define by ti = T(2i)
* This transformation is useful because n/2 becomes (2i)/2 = 2 i-1
* In other words, our original recurrence in which T(n) is defined as a function of

T(n/2) given way to one in which ti is defined as a function of t i-1, precisely

the type of recurrence we have learned to solve.

ti = T(2i) = 3T(2 i-1) + 2i

ti = 3t i-1 + 2i
ti 3t i-1 = 2i (A)

In this case,

b = 2, p(n) = 1, degree = 0

So, the characteristic equation,

(x 3)(x 2) = 0

The roots are, r1 = 3, r2 = 2.

The general equation,

tn = C1 r1i + C2 r2i
sub. r1 & r2: tn = 3nC1 + C2 2n
tn = C1 3i + C2 2i

We use the fact that, T(2i) = ti & thus T(n) = tlogn when n= 2i to obtain,

T(n) = C1. 3 log2n + C2. 2log2n

T(n) = C1 . nlog23 + C2.n [i = logn]

When n is a power of 2, which is sufficient to conclude that,

T(n) = O(n log3) n is a power of 2
Example: (2)
Consider the recurrence,
T(n) = 4T(n/2) + n2 (A)

Where n is a power of 2, n 2.
ti = T(2i) = 4T(2 i-1) + (2i)2
ti = 4t i-1 + 4i

ti 4t i-1 = 4i

In this eqn,
b = 4, P(n) = 1, degree = 0

The characteristic polynomial,

(x 4)(x 4) = 0

The roots are, r1 = 4, r2 = 4.

So, the general equation,

ti = C14i + C24i.i [since i = logn]
= C1 4 log n + C2. 4 logn . logn [since 2i = n]
= C1 . n log 4 + C2. n log4.n log1

T(n) = O(n log4) n is the power of 2.

EXAMPLE : 3

T(n) = 2T(n/2) + n logn

When n is a power of 2, n 2

ti = T(2i) = 2T(2i/2) + 2i .i [since 2i = n; i =logn]

ti 2t i-1 = i. 2i

In this case,

b = 2, P(n) = i, degree = 1
(x 2)(x 2)2 = 0

The roots are, r1 = 2, r2 = 2, r3 = 2

The general solution is,

tn = C12i + C2. 2i . i + C3. i2. 2i

= nC1 + nC2 + nC3(log n22n)
tn = O(n.log22n)

Example: 4

T(n) = 2, n=1
5T(n/4) + Cn2 , n>1

ti = T(4i) = 5T(4i/4) + C(4i)2

= 5T 4 i-1 + C. 16i
= 5t i-1 + C.16i
ti 5t i-1 = C. 16i

In this case,
b = 16, P(n) = 1, degree = 0

The characteristic eqn,

(x 5)(x 16) = 0

The roots are, r1 =5, r2 = 16

The general solution,

ti = C1.5i + C2.16i
= C1.5i + C2.(42)i
tn = O(n2)

Example(1) :

Consider the recurrence,

tn 2t n-1 = 3n (A)

In this case, b=3, p(n) = 1, degree = 0.

The characteristic polynomial is,

(x 2)(x 3) = 0

The roots are, r1 = 2, r2 = 3

The general solution,

tn = C1r1n + C2r2n
tn = C12n + C23n (1)

when n=0, C1 + C2 = t0 (2)

when n=1, 2C1 + 3C2 = t1 (3)

sub n=1 in eqn (A)

t1 2t0 = 3
t1 = 3 + 2t0
substitute t1 in eqn(3),

(2) * 2 / 2C1 + 2C2 = 2t0

2C1 + 3C2 = (3 + 2t0)
-------------------------------
-C2 = -3
C2 = 3
Sub C2 = 3 in eqn (2)
C1 + C2 = t0
C1 + 3 = t0
C1 = t0 3
Therefore tn = (t0-3)2n + 3. 3n
= Max[O[(t0 3) 2n], O[3.3n]]

= Max[O(2n), O(3n)] constants

= O[3n]

Example :(2)

tn 2t n-1 = (n + 5)3n, n 1 (A)

This is Inhomogeneous
In this case, b=3, p(n) = n+5, degree = 1
So, the characteristic polynomial is, (x-2)(x-3)2 = 0
The roots are,
r1 = 2, r2 = 3, r3 = 3

The general equation,

tn = C1r1n + C2r2n + C3nr3n (1)
when n=0, t0 = C1 + C2 (2)
when n=1, t1 = 2C1 + 3C2 + 3C3 (3)

substituting n=1 in eqn(A),

t1 2t0 = 6 . 3
t1 2t0 = 18
t1 = 18 + 2t0

substituting t1 value in eqn(3)

2C1 + 3C2 + 3C3 = 18 + 2t0 (4)

C1 + C2 + = t0 (2)

Sub. n=2 in eqn(1)

4C1 + 9C2 + 18C3 = t2 (5)

sub n=2 in eqn (A)

t2 2t1 = 7. 9
t2 = 63 + 2t1
= 63 + 2[18 + 2t0]
t2 = 63 + 36 +4t0
t2 = 99 +4t0
sub. t2 value in eqn(3),
4C1 +9C2 + 18C3 = 99 + 4t0 (5)

solve eqn (2),(4) & (5)

n=0, C1 + C2 = t0 (2)
n=1, 2C1 + 3C2 + 3C3 = 18 + 2t0 (4)
n=2, 4C1 + 9C2 + 18C3 = 99 + 4t0 (5)

(4) * 6 12C1 + 18C2 +18C3 = 108 + 2t0 (4)

(5) 4C1 + 9C2 + 18C3 = 99 + 4t0 (5)
------------------------------------------------
8C1 + 9C2 = 9 + 8t0 (6)

(2) * 8 8C1 + 8C2 = 8t0 (2)

(6) 8C1 + 9C2 = 9 +8t0 (6)
--------------------------
-C2 = -9
C2 = 9

Sub, C2 = 9 in eqn(2)
C1 + C2 = t0
C1 + 9 = t0
C1 = t0-9
Sub C1 & C2 in eqn (4)
2C1 + 3C2 + 3C3 = 18 + 2t0
2(t0-9) + 3(9) + 3C3 = 18 + 2t0
2t0 18 + 27 + 3C3 = 18 + 2t0
2t0 + 9 + 3C3 = 18 + 2t0
3C3 = 18 9 + 2t0 2t0
3C3 = 9
C3 = 9/3
C3 = 3

Sub. C1, C2, C3, r1, r2, r3 values in eqn (1)

tn = C12n + C23n + C3.n.3n

= (t0 9)2n + 9.3n + 3.n.3n
= Max[O[(t0-9),2n], O[9.3n], O[3.n.3n]]
= Max[O(2n), O(3n), O(n3n)]

tn = O[n3n]