EE202 - EE MATH II Jitkomut Songsiri

5. Function of square matrices

• matrix polynomial

• rational function

• Cayley-Hamilton theotem

• infinite series

• matrix exponential

• applications to differential equations

5-1
 Matrix Power

the mth power of a matrix A for a nonnegative m is defined as

m
Y
Am = A
k=1

and define A0 = I

property: Ar As = AsAr = Ar+s

a negative power of A is defined as

A−n = (A−1 )n

n is a nonnegative integer and A is invertible

Function of square matrices 5-2

 Matrix polynomial

a matrix polynomial is a polynomial with matrices as variables

p(A) = a0I + a1A + · · · + anAn

 
2 1
for example A =
0 −1

     2
1 0 2 1 2 1
p(A) = 2I − 6A + 3A2 = 2 −6 +3
0 1 0 −1 0 −1
 
2 −3
=
0 11

Fact ✎ any two polynomials of A commute, i.e., p(A)q(A) = q(A)p(A)

Function of square matrices 5-3

similarity transform: suppose A is diagonalizable, i.e.,

Λ = T −1AT ⇐⇒ A = T ΛT −1
 
where T = v1 · · · vn , i.e., the columns of T are eigenvectors of A

then we have Ak = T Λk T −1

thus diagonalization simplifies the expression of a matrix polynomial

p(A) = a0I + a1A + · · · + anAn

= a0T T −1 + a1T ΛT −1 + · · · + anT ΛnT −1
= T p(Λ)T −1

where  
p(λ1) 0 ··· 0
 0 p(λ 2 ) 0 
p(Λ) = 
 .. .. ... .. 
0 0 · · · p(λn)

Function of square matrices 5-4

eigenvalues and eigenvectors ✌

if λ and v be an eigenvalue and corresponding eigenvector of A then

• p(λ) is an eigenvalue of p(A)

• v is a corresponding eigenvector of p(A)

Av = λv =⇒ A2v = λAv = λ2 v ··· =⇒ Ak v = λk v

thus

(a0I + a1A + · · · + anAn)v = (a0v + a1λ + · · · + anλn)v

which shows that

p(A)v = p(λ)v

Function of square matrices 5-5

 Rational functions

f (x) is called a rational function if and only if it can be written as

p(x)
f (x) =
q(x)

where p(x) and q(x) are polynomial functions in x and q(x) 6= 0

we define a rational function for square matrices as

p(A)
f (A) = , p(A)q(A)−1 = q −1(A)p(A)
q(A)

provided that q(A) is invertible

Function of square matrices 5-6

eigenvalues and eigenvectors ✌

if λ and v be an eigenvalue and corresponding eigenvector of A then

• p(λ)/q(λ) is an eigenvalue of f (A)

• v is a corresponding eigenvector of f (A)

both p(A) and q(A) are polynomials, so we have

p(A)v = p(λ)v, q(A)v = q(λ)v

and the eigenvalue of q(A)−1 is 1/q(λ), i.e., q(A)−1v = (1/q(λ))v

thus

f (A)v = p(A)q(A)−1v = q(λ)−1p(A)v = q(λ)−1 p(λ)v = f (λ)v

which says that f (λ) is an eigenvalue of f (A) with the same eigenvector

Function of square matrices 5-7

  
4 2
example: f (x) = (x + 1)/(x − 5) and A =
1 5

det(λI − A) = 0 = (λ − 4)(λ − 5) − 2 = λ2 − 9λ + 18 = 0

the eigenvalues of A are λ1 = 3 and λ2 = 6


 −1  
−1 5 2 −1 2 1 6
f (A) = (A + I)(A − 5I) = =
1 6 1 0 3 4

the characteristic function of f (A) is

det(λI − f (A)) = 0 = (λ − 1)(λ − 4) − 18 = λ2 − 5λ − 14 = 0

the eigenvalues of f (A) are 7 and −2

this agrees to the fact that the eigenvalues of f (A) are

f (λ1) = (λ1 − 1)/(λ1 − 5) = −2, f (λ2) = (λ2 − 1)/(λ2 − 5) = 7

Function of square matrices 5-8

 Cayley-Hamilton theorem

the characteristic polynomial of a matrix A of size n × n

X (λ) = det(λI − A)

can be written as a polynomial of degree n:

X (λ) = λn + αn−1 λn−1 + · · · + α1λ + α0

✌ Theorem: a square matrix satisfies its characteristic equation:

X (A) = An + αn−1An−1 + · · · + α1A + α0I = 0

result: for m ≥ n, Am is a linear combination of Ak , k = 0, 1, . . . , n − 1.

Function of square matrices 5-9

  
1 2
example 1: A = the characteristic equation of A is
0 3

X (λ) = (λ − 1)(λ − 3) = λ2 − 4λ + 3 = 0

the Cayley-Hamilton states that A satisfies its characteristic equation

X (A) = A2 − 4A + 3I = 0

use this equation to write matrix powers of A

A2 = 4A − 3I
A3 = 4A2 − 3A = 4(4A − 3I) − 3A = 13A − 12I
A4 = 13A2 − 12A = 13(4A − 3I) − 12A = 40A − 39I
.. ..

powers of A can be written as a linear combination of I and A

Function of square matrices 5-10

example 2: with A in page 5-10, find the closed-form expression of Ak
for k ≥ 2, Ak is a linear combination of I and A, i.e.,

Ak = α0I + α1A

where α1, α0 are to be determined

multiply eigenvectors of A on both sides

Ak v1 = (α0I + α1A)v1 ⇒ λk1 = α0 + α1λ1

Ak v2 = (α0I + α1A)v2 ⇒ λk2 = α0 + α1λ2

substitute λ1 = 1 and λ2 = 3 and solve for α0, α1

3 − 3k 3k − 1
    
1 1 1 α0
= ⇒ α0 = , α1 =
3k 1 3 α1 2 2
k k k
 
3−3 3 −1 1 3 −1
Ak = I+ A= k , k≥2
2 2 0 3

Function of square matrices 5-11

 Computing the inverse of a matrix

A is a square matrix with the characteristic equation

λn + an−1λn−1 + · · · + a1λ + a0 = 0

by the C-H theorem, A satisfies the characteristic equation

An + an−1An−1 + · · · + a1A + a0I = 0

if A is invertible, multiply A−1 on both sides

An−1 + an−1An−2 + · · · + a1I + a0A−1 = 0

thus the inverse of A can be alternatively computed by

−1 1 n−1 n−2


A =− A + an−1 A + · · · + a1 I
a0

Function of square matrices 5-12

  
2 −4 −4
example: given A = 1 −4 −5 find A−1
1 4 5

the characteristic equation of A is

det(λI − A) = λ3 − 3λ2 + 10λ − 8 = 0

0 is not an eigenvalue of A, so A is invertible and given by

−1 1 2

A = A − 3A + 10I
8
 
0 2 2
1
= −5 7 3
4
4 −6 2

compare the result with other methods

Function of square matrices 5-13

 Infinite series
P∞
Definition: a series k=0 ak converges to S if the partial sum

n
X
Sn , ak
k=0

converges to S as n → ∞

example of convergent series:

1 1 1
1+ + + + ··· = 2
2 4 8
1 1 1
1 − + − + · · · = log(2)
2 3 4

Function of square matrices 5-14

 Power series
a power series in scalar variable z is an infinite series of the form
∞
X
f (z) = ak z k
k=0

example: power series that converges for all values of z

z z2 z3
e = 1 + z + + + ···
2! 3!
z2 z4 z6
cos(z) = 1 − + − + · · ·
2! 4! 6!
z3 z5 z7
sin(z) = z − + − + · · ·
3! 5! 7!
z2 z4 z6
cosh(z) = 1 + + + + · · ·
2! 4! 6!
z3 z5 z7
sinh(z) = z + + + + · · ·
3! 5! 7!

Function of square matrices 5-15

 Power series of matrices
let A be matrix and Aij denotes the (i, j) entry of A
Definition: a matrix power series
∞
X
ak Ak
k=0

converges to S if all (i, j) entries of the partial sum

n
X
Sn , ak Ak
k=0

converges to the corresponding (i, j) entries of S as n → ∞

P∞
Fact ✌ if f (z) = k=0 ak z k is a convergent power series for all z then

f (A) is convergent for any square matrix A

Function of square matrices 5-16

 Matrix exponential

generalize the exponential function of a scalar

2 3
x x
ex = 1 + x + + + ···
2! 3!

to an exponential function of a matrix

define matrix exponential as

∞
A A2 A3 X Ak
e =I +A+ + + ··· =
2! 3! k!
k=0

for a square matrix A

the infinite series converges for all A

Function of square matrices 5-17

  
1 1
example: A =
0 0

find all powers of A

   
2 1 1 3 1 1
A = , A = ,..., Ak = A for k = 2, 3, . . .
0 0 0 0

so by definition,

∞ ∞ ∞
Ak Ak
  X    
X X 1 0 1 1 1 e e−1
eA = = I+ = + =
k! k! 0 1 k! 0 0 0 1
k=0 k=1 k=1

never compute eA by element-wise operation !

1 1
 
e e
eA 6=
e0 e0

Function of square matrices 5-18

 Eigenvalues of matrix exponential
✌ if λ and v be an eigenvalue and corresponding eigenvector of A then

• eλ is an eigenvalue of eA

• v is a corresponding eigenvector of eA

since eA can be expressed as power series of A:

A A2 A3
e =I +A+ + + ···
2! 3!
multiplying v on both sides and using Ak v = λk v give

A A2 v A3 v
e v = v + Av + 2!+ 3!+ ···
 
λ2 λ3
= 1 + λ + 2! + 3! + · · · v
= eλv

Function of square matrices 5-19

 Properties of matrix exponential

• e0 = I

• eA+B 6= eA · eB

• if AB = BA, i.e., A and B commute, then eA+B = eA · eB

• (eA)−1 = e−A

✌ these properties can be proved by the definition of eA

Function of square matrices 5-20

 Computing eA via diagonalization
if A is diagonalizable, i.e.,

T −1AT = Λ = diag(λ1, λ2, . . . , λn)

where λk ’s are eigenvalues of A then eA has the form

eA = T eΛT −1

• computing eΛ is simple since Λ is diagonal

• one needs to find eigenvectors of A to form the matrix T
• the expression of eA follows from
∞ ∞ ∞
X Ak X (T ΛT −1)k X T Λk T −1
eA = = = = T eΛT −1
k! k! k!
k=0 k=0 k=0

• if A is diagonalizable, so is eA

Function of square matrices 5-21

  
1 1 0
example: compute f (A) = eA given A = 0 2 1
0 0 0

the eigenvalues and eigenvectors of A are

     
1 1 1
λ1 = 1, v1 = 0 , λ2 = 2, v2 = 1 , λ3 = 0, v3 = −1
0 0 2

form T = v1 v2 v3 and compute eA = T eΛT −1

 

2 2
     
1 1 1 e 0 0 1 −1 −1 e e − e (e − 2e + 1)/2
eA = 0 1 −1 0 e2 0 0 1 1/2 = 0 e2 (e2 − 1)/2 
0 0 2 0 0 1 0 0 1/2 0 0 1

Function of square matrices 5-22

 Computing eA via C-H theorem

eA is an infinite series
2 3
A A
eA = I + A + + + ···
2! 3!

by C-H theorem, the power Ak can be written as

Ak = a0I + a1A + · · · + an−1An−1, k = n, n + 1, . . .

(a polynomial in A of order ≤ n − 1)

thus eA can be expressed as a linear combination of I, A, . . . , An−1

eA = α0I + α1A + · · · + αn−1An−1

where αk ’s are coefficients to be determined

Function of square matrices 5-23

 P∞ k
this also holds for any convergent power series f (A) = k=0 ak A

f (A) = α0I + α1A + · · · + αn−1An−1

(recursively write Ak as a linear combination of I, A, . . . , An−1 for k ≥ n)

multiplying an eigenvector v of A on both sides and using v 6= 0, we get

f (λ) = α0I + α1λ + · · · + αn−1λn−1

substitute with the n eigenvalues of A

λn−1
    
f (λ1) 1 λ1 · · · 1 α0
 f (λ2)  1 λ2 · · ·
 .  = . . λn−1
2 α1 

 .  . . .
. . .
.  . 


f (λn) 1 λn · · · λn−1
n αn−1

Fact ✎ if all λk ’s are distinct, the system is solvable and has a unique sol.

Function of square matrices 5-24

 Vandermonde matrix

a Vandermonde matrix has the form

λn−1
 
1 λ1 · · · 1
1 λ2 · · · n−1 
λ2 
V =
 .. .. .. .. 
1 λn · · · λnn−1

(with a geometric progression in each row)

✎ one can show that the determinant of V can be expressed as

Y
det(V ) = (λj − λi)
1≤i<j≤n

hence, V is invertible as long as λi’s are distinct

Function of square matrices 5-25

example: compute f (A) = eA given
 
1 1 0
A = 0 2 1 
0 0 0

the eigenvalues of A are λ = 1, 2, 0 (all are distinct)

form a system of equations: f (λi) = α0 + α1λi + α2λ2i for i = 1, 2, 3

 1   
e 1 1 1 α0
e2 = 1 2 22 α1
e0 1 0 0 α2

which has the solution

α0 = 1, α1 = 2e − e2/2 − 3/2, α2 = −e + e2/2 + 1/2

Function of square matrices 5-26

substituting α0, α1, α2 in

eA = α0I + α1A + α2A2

gives
     
1 0 0 1 1 0 1 3 1
eA = α0 0 1 0 + α1 0 2 1 + α2 0 4 2
0 0 1 0 0 0 0 0 0
 
α0 + α1 + α2 α1 + 3α2 α2
=  0 α0 + 2α1 + 4α2 α1 + 2α2
0 0 α0
2 2
 
e e − e (e − 2e + 1)/2
= 0 e2 (e2 − 1)/2 
0 0 1

(agree with the result in page 5-22)

Function of square matrices 5-27

 Repeated eigenvalues

A has repeated eigenvalues, i.e., λi = λj for some i, j

goal: compute f (A) using C-H theorem

however, we can no longer apply the result in page 5-24 because

• the number of independent equations on page 5-24 is less than n

• the Vandermonde matrix (page 5-25) is not invertible

cannot form a linear system to solve for the n coefficients, α0, . . . , αn−1

Function of square matrices 5-28

solution: for the repeated root with multiplicity r

get r − 1 independent equations by taking derivatives on f (λ) w.r.t λ

f (λ) = α0 + α1λ + · · · + αn−1λn−1

df (λ)
= α1 + 2α2λ + · · · + (n − 1)αn−1λn−2
dλ
.. = ..
dr−1f (λ) n−1−r
= (r − 1)!αr−1 + · · · + (n − r) · · · (n − 2)(n − 1)αn−1 λ
dr−1λ

Function of square matrices 5-29

example: compute f (A) = cos(A) given
 
1 1 0
A = 0 1 0 
0 0 2

the eigenvalues of A are λ1 = 1, 1 and λ2 = 2

by C-H theorem, write f (A) as a linear combination of Ak , k = 0, ..., n − 1

f (A) = cos(A) = α0 + α1A + α2A2

the eigenvalues of A must also satisfies this equation

f (λ) = cos(λ) = α0 + α1λ + α2λ2

the derivative of f w.r.t λ is given by

f ′(λ) = − sin(λ) = α1 + 2α2λ

Function of square matrices 5-30

thus we can obtain n linearly independent equations:

λ21
         
f (λ1) 1 λ1 α0 cos(1) 1 1 1 α0
f ′(λ1) = 0 1 2λ1 α1 =⇒ − sin(1) = 0 1 2 α1
f (λ2) 1 λ3 λ23 α2 cos(2) 1 2 4 α2

which have the solution

   
α0 2 sin(1) + cos(2)
α1 = 2 cos(1) − 3 sin(1) − 2 cos(2)
α2 − cos(1) + sin(1) + cos(2)

substitute α0, α1, α2 to obtain f (A)

f (A) = cos(A) = α0I + α1A + α2A2

 
cos(1) − sin(1) 0
=  0 cos(1) 0 
0 0 cos(2)

Function of square matrices 5-31

 Applications to ordinary differential equations

we solve the following first-order ODEs for t ≥ 0 where x(0) is given

scalar: x(t) ∈ R and a ∈ R is given

ẋ(t) = ax(t)

solution: x(t) = eatx(0), for t ≥ 0

vector: x(t) ∈ Rn and A ∈ Rn×n is given

ẋ(t) = Ax(t)

deAt
solution: x(t) = eAtx(0), for t ≥ 0 (use = AeAt = eAtA)
dt

Function of square matrices 5-32

 Applications to difference equations

we solve the difference equations for t = 0, 1, . . . where x(0) is given

scalar: x(t) ∈ R and a ∈ R is given

x(t + 1) = ax(t)

solution: x(t) = atx(0), for t = 0, 1, 2, . . .

vector: x(t) ∈ Rn and A ∈ Rn×n is given

x(t + 1) = Ax(t)

solution: x(t) = Atx(0), for t = 0, 1, 2, . . .

Function of square matrices 5-33

example: solve the ODE

ÿ(t) − ẏ(t) − 6y(t) = 0, y(0) = 1, ẏ(0) = 0

solution: define    
x1 (t) y(t)
x(t) , ,
x2 (t) ẏ(t)

write the equation into the vector form ẋ(t) = Ax(t)

   
ẏ(t) ẏ(t)
ẋ(t) = =
ÿ(t) ẏ(t) + 6y(t)
   
x2(t) 0 1
= = x(t)
6x1 (t) + x2(t) 6 1

the initial condition is   

y(0) 1
x(0) = =
ẏ(0) 0

Function of square matrices 5-34

thus it is left to compute eAt


0 1
A=
6 1

the eigenvalues and eigenvectors of A are

   
1 1
λ1 = −2, v1 = , λ2 = 3, v2 =
−2 3

all eigenvalues are distinct, so A is diagonalizable and

 
At Λt −1
  λ1 0
e = Te T , T = v1 v2 , Λ=
0 λ2
  −2t   
1 1 e 0 1 3 −1
eAt =
−2 3 0 e3t 5 2 1

Function of square matrices 5-35

the closed-form expression of eAt is

−2t 3t −2t 3t
 
At 1 3e + 2e −e +e
e =
5 −6e−2t + 6e3t 2e−2t + 3e3t

the solution to the vector equation is

−2t 3t −2t 3t
  
1 3e + 2e −e +e 1
x(t) = eAtx(0) =
5 −6e−2t + 6e3t 2e−2t + 3e3t 0
3t
 −2t 
1 3e + 2e
=
5 −6e−2t + 6e3t

hence the solution y(t) can be obtained by

  1 −2t 3t


y(t) = x1(t) = 1 0 x(t) = 3e + 2e , t≥0
5

Function of square matrices 5-36

example: solve the difference equation

y(t + 2) − y(t + 1) − 6y(t) = 0, y(0) = 1, y(1) = 0

solution: define    
x1(t) y(t)
x(t) , ,
x2(t) y(t + 1)

write the equation into the vector form x(t + 1) = Ax(t)

   
y(t + 1) y(t + 1)
x(t + 1) = =
y(t + 2) y(t + 1) + 6y(t)
   
x2(t) 0 1
= = x(t)
6x1(t) + x2 (t) 6 1

the initial condition is    

y(0) 1
x(0) = =
y(1) 0

Function of square matrices 5-37

thus it is left to compute At
 
0 1
A=
6 1

the eigenvalues and eigenvectors of A are

   
1 1
λ1 = −2, v1 = , λ2 = 3, v2 =
−2 3

all eigenvalues are distinct, so A is diagonalizable and

 
λ1 0
At = T ΛtT −1,
 
T = v1 v2 , Λ=
0 λ2

t
    
1 1 (−2) 0 1 3 −1
At =
−2 3 0 3t 5 2 1

Function of square matrices 5-38

the closed-form expression of At is

t t t t
 
t1 2(3 ) + 3(−2) 3 − (−2)
A =
5 2(3t+1) + 3(−2)t+1 3t+1 − (−2)t+1

for t = 0, 1, 2, . . .

the solution to the vector equation is

t t t t
  
1 2(3 ) + 3(−2) 3 − (−2) 1
x(t) = Atx(0) =
5 2(3t+1) + 3(−2)t+1 3t+1 − (−2)t+1 0
t t
 
1 2(3 ) + 3(−2)
=
5 2(3t+1) + 3(−2)t+1

hence the solution y(t) can be obtained by

1 t t


y(t) = x1(t) = 2(3 ) + 3(−2) , t = 0, 1, 2, . . .
5

Function of square matrices 5-39

 MATLAB commands

• expm(A) computes the matrix exponential eA

• exp(A) computes the exponential of the entries in A
   
1 1 e e−1
example from page 5-18, A = , eA =
0 0 0 1
>> A=[1 1;0 0];
>> expm(A)
ans =
2.7183 1.7183
0 1.0000
>> exp(A)
ans =
2.7183 2.7183
1.0000 1.0000

Function of square matrices 5-40

 References

Chapter 21 in
M. Dejnakarin, Mathematics for Electrical Engineers, 3rd edition,
Chulalongkorn University Press, 2006

Lecture note on
Linear algebra, EE263, S. Boyd, Stanford university

Function of square matrices 5-41