2 Matrix

Matrix Algebra
Matrix Algebra 1 / 101

Goals
Study matrix algebra with introduction to some applications

1 Algebra of matrix: addition, scalar multiplication, transposition,
product
2 Matrix multiplication
▶ ”Matrix transformation” such as rotation are important tool in

geometry and in return geometry provides a ”pictures” of the matrices
▶ Matrix multiplication is related to directed graphs
3 Matrix inverse
4 Introduction to matrix LU-factorization

Matrix Addition, Scalar Multiplication, and Transposition
Table of contents
1 Matrix Addition, Scalar Multiplication, and Transposition
Matrix addition
Scalar multiplication
Transpose of a Matrix
2 Matrix Multiplication
Matrix Multiplication
Application in Computer Graphics
3 Inverse Matrix
Inverse and Linear Systems
An Inversion Method
Application in Cryptography
4 Elementary Matrices
Elementary Matrices
Inverse and Elementary Matrices
5 Elimination equiv LU Factorization

Matrix
A rectangular array of numbers is called a matrix, denoted by uppercase
letters: A, B, C...
an m × n matrix or size m × n.
Simple notation A = [aij ]
(i, j) - entry : element lies in row i and column j, denoted by aij
Row matrix
A matrix of size 1 × n is called a row matrix
h i
1 2 4 9
Column matrix
A matrix of size m × 1 is called a column matrix
" #
1
2
Square matrix
A m × m matrix is a square matrix
" #
1 2
4 9

Example
" #
1 2 −1
B=
0 5 6
is a 2 × 3 matrix.
The (2, 1) - entry is
b21 = 0

Matrix and dataset

Graph and Incidence Matrix

Graph is a model consisting of nodes connected by edges. An incidence
matrix tell hows the nodes are connected by the edges. Every entry of an
incidence matrix is 0, -1 or 1.
In the first row -1, 1, 0, 0 show that the first edge goes from node 1 to
node 2 ( -1 for node 1 because the arrow goes out, +1 for node 2 with
arrow in)
This graph is complete - every pair of nodes is connected by an edge.
This graph is a tree - a graph has no closed loops

B&W or Greyscale Image

Image is distorted into small boxes called pixels. Dimension of the image
is simply the number of pixels across the image’s height(x) and width(y),
e.g, 24 × 16
Each of these pixels is denoted as a numerical value, called Pixel Values

which denote the intensity of the pixels. For a grayscale or b&w image,
pixel values range from 0 to 255.
RGB Color Images
The RGB color model is an additive color

model in which the red, green and blue
primary colors of light are added together in
various ways to reproduce a broad array of
colors.
RGB image can be viewed as three different

images (a red scale image, a green scale
image and a blue scale image) Full color image along
with its R, G, and B
Equal matrices
Two matrices A and B are equal, denoted by
A=B
if
1 They have the same size.
2 Corresponding entries are equal.

Example
Given
" # " # " #
a b 1 2 −1 1 0
A= ,B= and C =
c d 3 0 1 −2 2
Discuss the possibility that A = B, B = C, A = C.

Example
Given
" # " # " #
a b 1 2 −1 1 0
A= ,B= and C =
c d 3 0 1 −2 2
Discuss the possibility that A = B, B = C, A = C.

Solution
A = B is impossible because A and B are of different sizes: A is
2 × 2 whereas B is 2 × 3.
Similarly, B = C is impossible.
A = C is possible provided that corresponding entries are equal:
" # " #
a b 1 0
=
c d −2 2
means that a = 1, b = 0, c = −2 and d = 2

Matrix Addition, Scalar Multiplication, and Transposition Matrix addition
Matrix addition
If A and B are matrices of the same size, their sum A + B is the matrix
formed by adding corresponding entries.
If A = [aij ] and B = [bij ] then
A + B = [aij + bij ]
in other words
   
a11 a12 ··· a1n b11 b12 ··· b1n
 a21

a22 ···   b21
a2n  
b22 ··· b2n 

+ .
 . .. ..  .. .. 
 . .. 
..
 . . . .   .. . . . 

an1 bm2 · · · amn bn1 bm2 · · · bmn
 
a11 + b11 a12 + b12 ··· a1n + b1n
 a21 + b21

a22 + b22 ··· a2n + b2n 

= .. .. .. .. 

 . . . .


an1 + bn1 bm2 + bm2 ··· amn + bmn

Example
If " # " #
2 1 3 1 1 −1
A= and B =
−1 2 0 2 0 6
then " # " #
2 + 1 1 + 1 3 + (−1) 3 2 2
A+B = =
−1 + 2 2 + 0 0+6 1 2 6

Example
Find a, b, c such that

h i h i h i
a b c + c a b = 3 2 −1

Example
Find a, b, c such that

h i h i h i
a b c + c a b = 3 2 −1
Solution
Add the matrices on the left side to obtain
h i h i
a + c b + a c + b = 3 2 −1
This gives a + c = 3, b + a = 2 and c + b = −1

Solving these yields a = 3, b = −1, c = 0.

Zero matrix
The m × n matrix in which every entry is zero is called the m × n zero
matrix and is denoted as 0(or 0mn )
" #
0 0 0
023 =
0 0 0
Negative matrix
Negative of matrix A, denoted by −A obtained by multiplying each entry
of A by −1
−A = [−aij ]
Difference
If A and B are two m × n matrices, their difference A − B is defined by
A − B = A + (−B)
If A = [aij ] and B = [bij ] then A − B = [aij − bij ]

Example
Let
" # " # " #
3 −1 0 1 −1 1 1 0 −2
A= ,B = ,C =
1 2 −4 −2 0 6 3 1 1
Compute −A, A − B, A + B − C.

Example
Let
" # " # " #
3 −1 0 1 −1 1 1 0 −2
A= ,B = ,C =
1 2 −4 −2 0 6 3 1 1
Compute −A, A − B, A + B − C.
Solution
" #
−3 1 0
−A =
−1 −2 4
" # " #
3−1 −1 − (−1) 0 − 1 2 0 −1
A−B = =
1 − (−2) 2−0 −4 − 6 3 2 −10
" # " #
3 + 1 − 1 −1 − 1 − 0 0 + 1 − (−2) 3 −2 3
A+B−C = =
1−2−3 2+0−1 −4 + 6 − 1 −4 1 1

Example
Solve " # " #

3 2 1 0
+X =
−1 1 −1 2
where X is a matrix.

Example
Solve " # " #

3 2 1 0
+X =
−1 1 −1 2
where X is a matrix.
Solution
" #
3 2
Substract the matrix from both side
−1 1
" # " # " #
1 0 3 2 −2 −2
X= − =
−1 2 −1 1 0 1

Matrix Addition, Scalar Multiplication, and Transposition Scalar multiplication
If A is any matrix and k is any number, the scalar multiple kA is the

matrix obtained from A by multiplying each entry of A by k.
If A = [aij ] then
kA = [kaij ]

Example
Let " # " #

3 −1 4 1 2 −1
A= ,B = .
2 0 1 0 3 2
Compute 5A, 21 B, 3A − 2B.
Solution
" # " #
15 −5 20 1 0.5 1 −0.5
5A = , B=
10 0 5 2 0 1.5 1
" # " # " #
9 −3 12 2 4 −2 7 −7 14
3A − 2B = − =
6 0 3 0 6 4 6 −6 −1

Properties of matrix addition and scalar multiplication
Let A, B, and C denote arbitrary m × n matrices where m and n are

fixed. Let k and p denote arbitrary real numbers. Then
1 A + B = B + A (commutative law).
2 A + (B + C) = (A + B) + C (associative law)
3 0 + A = A for each A.
4 A + (−A) = 0
5 k(A + B) = kA + kB.
6 (k + p)A = kA + pA.
7 (kp)A = k(pA).
8 1A = A.

Example
Simplify 2(A + 3C) − 3(2C − B) − 3[2(2A + B − 4C) − 4(A − 2C)]

where A, B, and C are all matrices of the same size.

Example
Simplify 2(A + 3C) − 3(2C − B) − 3[2(2A + B − 4C) − 4(A − 2C)]

where A, B, and C are all matrices of the same size.
Solution
2(A + 3C) − 3(2C − B) − 3[2(2A + B − 4C) − 4(A − 2C)]

= 2A + 6C − 6C + 3B − 3[4A + 2B − 8C − 4A + 8C]
= 2A + 3B − 3(2B)
= 2A + 3B − 6B
= 2A − 3B

Matrix Addition, Scalar Multiplication, and Transposition Transpose of a Matrix
If A is an m × n matrix, the transpose of A, written AT , is the n × m

matrix whose rows are just the columns of A in the same order.
aTij = aji
The first row of AT is the first column of A (that is it consists of the
entries of column 1 in order). Similarly the second row of AT is the second
column of A, and so on.

Example
Write down the transpose of each of the following matrices.

     
1 h i 1 2 3 1 −1
A = 3 , B = 5 2 6 , C = 3 4 , D =  1 3 2 
     
2 4 6 −1 2 1

Example
Write down the transpose of each of the following matrices.

     
1 h i 1 2 3 1 −1
A = 3 , B = 5 2 6 , C = 3 4 , D =  1 3 2 
     
2 4 6 −1 2 1
Solution
   
5 " # 3 1 −1
T
h
T
i
T 1 3 4 T
A = 1 3 2 , B = 2 , C = ,D =  1 3 2  = D
   
2 4 6
6 −1 2 1

Another way to think of transposition
Main diagonal consists of the elements a11 , a22 , . . .
Forming the transpose of a matrix A can be viewed as “flipping” A about

its main diagonal, or as “rotating” A through 180o about the line
containing the main diagonal

Special types of matrix
Definition (Symmetric)
A matrix A is symmetric if AT = A
Definition (Skew symmetric)

A matrix A is skew symmetric if AT = −A

Example
 
1 2 3
A = 2 4 5 is a symmetric matrix
 
3 5 6
 
0 2 3
B = −2 0 −4 is a skew symmetric matrix
 
−3 4 0

Properties of Matrix Transpose
Let A and B denote matrices of the same size, and let k denote a scalar.
1 If A is an m × n matrix, then AT is an n × m matrix.
2 (AT )T = A.
3 (kA)T = kAT .
4 (A + B)T = AT + B T .

Example
Solve for A if " #!T " #
T 1 2 2 3
2A − 3 =
−1 1 −1 2

Example
Solve for A if " #!T " #
T 1 2 2 3
2A − 3 =
−1 1 −1 2
Solution
Applying the above theorem for the left hand side, we have
" #T " #
1 2
T T 1 −1
LHS = 2(A ) − 3 = 2A − 3
−1 1 2 1
The equation becomes

" # " # " # " # " #
1 −1 2 3 2 3 1 −1 5 0
2A − 3 = ⇒ 2A = +3 =
2 1 −1 2 −1 2 2 1 5 5
" # " #
1 5 0 2.5 0
So A = 2 =
5 5 2.5 2.5
Table of contents
Matrix addition
3 Inverse Matrix
An Inversion Method
Elementary Matrices

Matrix Multiplication Matrix Multiplication
Vector in Rn
Example
" #
x1
: a vector in R2
x2
 
x1
x2 : a vector in R3
 
x3
Definition
Vector in Rn is a n × 1 matrix
 
x1
x 
 2
x= 
. . .
xn
A vector in Rn is also called a point in Rn

 
0
0
Vector zero 0 =  
 
. . .
0
Two vectors are equal if and only if corresponding entries are the same
   
a1 b1
a   b 
 2  2 
  =   ⇐⇒ ai = bi ∀i
. . . . . .
an bn

Vector addition and scalar multiplication in Rn

Let    
x1 y1
x  y 
 2  2
x=  and y= 
. . . . . .
xn yn
be two vectors in Rn
Vector addition  
x 1 + y1
x + y 
x+y= 2 2

 ... 

x n + yn
Scalar multiplication  
ax1
 ax 
ax =  2 
 
 ... 
axn
Example
   
2 3
−1  1 
Let x =   , y =   then
   
 1   2 
2 −1
     
5 6 12
0 −3 −1
x+y=  3x =   3x + 2y =  
     
3  3   7 
1 6 4

Geometric meaning (1)
Vector addition v + w produces the diagonal of a parallelogram. The

reverse of w is -w

Each vector corresponds to a point


Definition (Dot product or standard inner product)

The dot product or standard inner product of two n - vectors in Rn
   
a1 b1
a  b 
a =  2  and b =  2 
   
. . . . . .
an bn
is defined as
a • b = a1 b1 + a2 b2 + · · · + an bn

Example
The dot product of

   
1 2
 2   5 
a=  and b = 
   
3 −1

   
4 0
is
a • b = (1)(2) + (2)(5) + (3)(−1) + (4)(0) = 9

Multiply a row by a column
Let  
b1
h i b 
a = a1 a2 . . . an and b =  2 
 
. . .
bn
then the multiplication of a and b is given by
ab = aT • b = a1 b1 + a2 b2 + · · · + an bn

Matrix-vector product
Let A be an m × n matrix. If
h i
A = a1 . . . an
where aj are the columns of A and
x1
 
 .. 
x= . 
xn
then
Ax = x1 a1 + · · · + xn an
which was motivated as a way of describing system of linear equation

Ax = b

The Matrix–Vector Product Written in Terms of Columns

 
a11 a12 ... a1n
 
x1
 a21 a22 ... a2n 
 x 
  2
Let A = 
 .. .. ..  and x = 
..   then
 . . . .  . . .
an1 an2 . . . ann xn
       
a11 a21 a1n a11 x1 + a12 x2 + · · · + a1n xn
a  a  a   a x + a x + · · · + a x 
 21   22   2n   21 1 22 2 2n n 
Ax = x1  +x2  +· · ·+xn  =
... ... ...  ...


an1 an2 ann an1 x1 + an2 x2 + · · · + ann xn
which can be written as
x1 col1 (A) + x2 col2 (A) + · · · + xn coln (A)

Example
 
" # 2
2 −1 −3
Let A = and x = −3 then
 
4 2 −2
4
" # " # " # " #
2 −1 −3 −5
Ax = 2 −3 +4 =
4 2 −2 −6

Revisit System of Linear Equations

The linear system

a11 x1 + a12 x2 + · · · + a1n xn = b1



a21 x1 + a22 x2 + · · · + a2n xn = b2

..



 .

am1 x1 + am2 x2 + · · · + amn xn = bm

can be written as
Ax ≡ x1 col1 (A) + x2 col2 (A) + · · · + xn coln (A) = b
The entries in the product Ax are merely the left sides of the equations in
the system. Hence the linear system can be written in matrix form as
Ax = b

Revisit System of Linear Equations

The linear system

a11 x1 + a12 x2 + · · · + a1n xn = b1



a21 x1 + a22 x2 + · · · + a2n xn = b2

..



 .

am1 x1 + am2 x2 + · · · + amn xn = bm

can be written as
Ax ≡ x1 col1 (A) + x2 col2 (A) + · · · + xn coln (A) = b
The entries in the product Ax are merely the left sides of the equations in
the system. Hence the linear system can be written in matrix form as
Ax = b
Theorem
Ax = b has solution (is consistent) if and only if b can be expressed as a
linear combination of the columns of matrix A.
The matrix A is called the coefficient matrix and the matrix

 
a11 a12 ... a1n b1
h  a21
i  a22 ... a2n b1 
A b =
 .. .. .. .. .. 
 . . . . .

an1 an2 . . . ann bn
is called the augmented matrix of the linear system

Example
The system of linear equation





−2x + z=5
2x + 3y − 4z = 7


 3x + 2y + 2z = 3
can be written is matrix form
Ax = b
     
−2 0 1 x 5
with A =  2 3 −4, x = y  and b = 7
     
3 2 2 z 3

Example
Consider the linear system Ax = b where
   
3 1 2   4
 4 −5 6  x1 1
A=  , x = x2  , b =  
     
 0 7 −3 0
x3
−1 2 0 2
then we can represent the system in terms of equation of columns

       
3 1 2 4
 4  −5  6  1
x1   + x2   + x3   =  
       
 0   7  −3 0
−1 2 0 2

Exercise
Determine the matrix A, x, b to write the following system of linear

equations in the form Ax = b.


 5x − x2 + 2x3 + x4 − 3x5 = 8
 1
1x + x + 3x − 5x4 + 2x5 = −2
2 3

−x + x − 2x

− 3x5 = 0
1 2 3

Matrix multiplication or matrix product
Definition
Let Ah be an m × ni matrix and B be an n × k matrix. Write
B = b1 . . . bk where bj is column j of B then the product matrix
AB is the m × k matrix defined by
h i h i
AB = A b1 . . . bk = Ab1 . . . Abk
Column j of AB this the matrix-vector product of A and column j of B

Example
   
2 3 5 8 9
Compute AB if A = 1 4 7 and B = 7 2
   
0 1 8 6 1

Example
   
2 3 5 8 9
Compute AB if A = 1 4 7 and B = 7 2
   
0 1 8 6 1
Solution
   
8 9
Columns of B are b1 = 7 and b2 = 2. So
   
6 1
         
2 3 5 8 67 2 3 5 9 29
Ab1 = 1 4 7 7 = 78 , Ab1 = 1 4 7 2 = 24
         
0 1 8 6 55 0 1 8 1 10
 
h i 67 29
Thus AB = Ab1 Ab2 = 78 24
 
55 10
Rule to multiply matrices

If A is an m × n matrix and B is an n × q matrix then
C = AB
is an m × q matrix and cij is found by multiplying the ith row of A by

the j th column of B.

Compatible rule

Example

Example (cont)

Example (cont)

Practice Problem
Write down the order of the matrices

 
1 2 " #
1 2
A= 0 1  B=
 
3 4
3 1
Find (if possible) AB and BA

Power of a square matrix

If A is a square matrix then
An = A.A
| {z. . . A}
n times
and A0 = In
where
Identity matrix
The identity matrix In is the n × n matrix with 1s on the main diagonal
(upper left to lower right), and zeros elsewhere.

Properties
A(B + C) = AB + AC (distributive law)

(A + B)C = AC + BC (distributive law)
A(BC) = (AB)C (associate law)
AB = BA not true
AI = A
IB = B
A(BC) = (AB)C
A(B + C) = AB + AC
a(AB) = (aA)B = A(aB)
(AB)T = B T AT

Matrix Multiplication Application in Computer Graphics
An Application in Computer Graphics
Computer graphics deals with images displayed on a computer screen, and

so arises in a variety of applications, ranging from word processors, to Star
Wars animations, to video games, to wire-frame images of an airplane.
These images consist of a number of points on the screen, together with
instructions on how to fill in areas bounded by lines and curves. Often
curves are approximated by a set of short straight-line segments, so that
the curve is specified by a series of points on the screen at the end of
these segments. Matrix transformations are important here because matrix
images of straight line segments are again line segments. Note that a
colour image requires that three images are sent, one to each of the
red,green, and blue phosphorus dots on the screen, in varying intensities.

Example - zoom in
" #
2 0
If A = then the result of AD is
0 2

Slant an image
Figure
1: slant
the letter to the right by multiplying by an x-shear matrix
1 0.2
A=
0 1

Shrink the x-coordinate

0.8 0
Figure 2: shrink x -coordinate by 0.8 with t an x-scale matrix B =
0 1

Rotate about origin
◦
Figure 3: rotate the letter about
the origin through 30 by matrix
cos(π/6) − sin(π/6)
R π6 =
sin(π/6) cos(π/6)

Exercise
What is the result of letter after applying a transform with matrix

" #
−1 0
B=
0 1

Inverse Matrix
Table of contents
Matrix addition
3 Inverse Matrix
An Inversion Method
Elementary Matrices

Inverse Matrix
Inverse matrix
If A is a square matrix, a matrix B is called an inverse of A if and

only if
AB = I and BA = I
 
1 0 ... 0
0 1 . . . 0
 
where I =   .. .. . . .  is an identity matrix of the same size as A
. . . .. 

0 0 ... 1
A matrix A that has an inverse is called an invertible matrix or
nonsingular. Otherwise, it is called noninvertible or singular
If A is an invertible matrix, the (unique) inverse of A is denoted A−1
A−1 (when it exists) is a square matrix of the same size as A with the
property that
AA−1 = A−1 A = I

Inverse Matrix
Example
" # " #
−1 1 0 1
Show that B = is an inverse of A = .
1 0 1 1

Inverse Matrix
Example
" # " #
−1 1 0 1
Show that B = is an inverse of A = .
1 0 1 1
Solution
We have " #
1 0
AB = BA =
0 1
Hence B is an inverse of A

Inverse Matrix
Example
Find the inverse of the matrix

" #
3 1
A=
5 2

Inverse Matrix
Solution
Since A is a 2 × 2 matrix, A−1 is also a 2 × 2 matrix and satisfies
AA−1 = A−1 A = I2
Suppose that " #

−1 x y
A =
z w
AA−1 = I2 yields
" #" # " #
3 1 x y 1 0
=
5 2 z w 0 1
or ( (
3x + z = 1 5x + 2z = 0
and
3y + w = 0 5y + 2w = 1

Inverse Matrix
" #
−1 2 −1
A =
−5 3

Inverse Matrix
" #
−1 2 −1
A =
−5 3
Need to verify that

" #" # " #
2 −1 3 1 1 0
=
−5 3 5 2 0 1

Inverse Matrix Inverse and Linear Systems
Use inverse matrix to solve system of linear equations
A system of linear equation can be written is a single matrix equation
Ax = b
where A, b are known and x is to be determine

If A is invertible then multiply each side of the equation on the left
by A−1 to get
A−1 Ax = A−1 b
Ix = A−1 b
x = A−1 b

Inverse Matrix Inverse and Linear Systems
Example
Solve the system equations
3x1 + x2 = 3
5x1 + 2x2 = 8
Solution
" # " #
3 1 x
Matrix form of the system is Ax = b where A = ,x= 1 ,
5 2 x2
" #
3
b= .
8
" #
2 −1
A has the inverse A−1 = . So the solution of the system is
−5 3
" #" # " #
−1 2 −1 3 −2
x=A b= = or x1 = −2, x2 = 9
−5 3 8 9
Inverse Matrix An Inversion Method
The Gauss − Jordan Method to find matrix inverse
If A is an invertible (square) matrix, there exists a sequence of elementary

row operations that carry A to the identity matrix I of the same size,
written A → I. This same series of row operations carries I to A−1 ; that
is, I → A−1 . The algorithm can be summarized as follows:
h i h i
A I → I A−1
This procedure will be justified in section Elementary Matrices

Example
Find inverse of

Solution
Step 1

Solution
Step 1
Step 2

Solution
Step 1
Step 2
Step 3
" #
6 −2
A−1 =
−2 1
Practice
Find the inverse of the matrix

 
2 7 1
A = 1 4 −1
 
1 3 0

Properties of Inverse Matrix
If A and B are invertible matrices of size n × n then

1 I is invertible and I −1 = I.
2 A−1 is invertible , and (A−1 )−1 = A
3 Inverse of product AB is invertible and (AB)−1 = B −1 A−1
(reverse order).
4 Ak is invertible for any k ≥ 1, and (Ak )−1 = (A−1 )k .
5 aA is invertible for a ̸= 0 and (aA)−1 = 1 A−1 .

a
6 Inverse of transpose (AT )−1 = (A−1 )T .

Conditions for Existence of Inverse Matrix
The following conditions are equivalent for an n × n matrix A

1 A is invertible.
2 The homogeneous system Ax = 0 has only the trivial solution x = 0.
3 A can be carried to the identity matrix In by elementary row
operations.
4 The system Ax = b has at least one solution x for every choice of
column b.
5 There exists an n × n matrix C such that AC = I .
n

Inverse Matrix Application in Cryptography
Crypography

Example
Number 26 + 2 also stands for B ...

Cryption
Encrypt a message of 4 letters such as STOP as following
Represent message |{z}
S |{z}
T |{z}
O |{z}
P by matrix
19 20 15 16
" #
19 20
D=
15 16
" #
1 −3
We use an encoding matrix M = to encrypt the message
−1 4
by transformation
" #" # " #
1 −3 19 20 −26 −28
MD = =
−1 4 15 16 41 44
The encoded message is−26 −28 41 44

|{z} |{z} |{z} |{z}
X O R

Encryption
You receive an encoded massege NCAA. Need to encrypt to find the origin
message.

Encryption
You receive an encoded massege NCAA. Need to encrypt to find the origin
message.
The principal is as following: If we denote D and E be the matrix
representation for origin message and encoded message then
E = MD
Given encoded matrix E and transformation matrix S, the decoded matrix

D is hence obtained with decoding matrix M −1 as following
D = M −1 E
Then you can find what origin message is. Maybe

The Hill Cipher
1 Codes can use finite fields as alphabets: letters in the message

become numbers 0, 1, . . . , p − 1
2 The numbers are added and multiplied mod p. Divide by p, keep
the remainder
3 A Hill Cipher multiplies blocks of the message by a secrete matrix E(
mod p)
4 To decode, multiply each block by the inverse matrix D( mod p).
Not a very secure cipher

Section 10.7 in the textbook of Strang, Section 8.8 in the textbook of
Nicholson

Elementary Matrices
Table of contents
Matrix addition
3 Inverse Matrix
An Inversion Method
Elementary Matrices

Elementary Matrices Elementary Matrices
Elementary Matrices
Elementary row operations takesan important role in linear algebra: solving

linear system (using gaussian algorithm), inverting a matrix (using
Gauss-Jordan method). It turns out that they can be performed by left
multiplying by certain invertible matrices
Definition
An n × n matrix E is called an elementary matrix if it can be obtained
from the identity matrix In by a single elementary row operation (called
the operation corresponding to E). Type of E is corresponding to type of
row operation:
Interchange two rows: E is of type I
Multiply one row by a nonzero number: E is of type II
Add a multiple of one row to a different row: E is of type III

Example
" #
0 1
E1 = : interchanging rows 1 and 2: elementary of type I
1 0
" #
1 0
E2 = : multiplying row 2 by 9: elementary of type II
0 9
" #
1 5
E2 = : adding 5 times row 2 to row 1: elementary of type III
0 1
" #
a b c
More concretely, consider A = then
p q r
" #" # " #
0 1 a b c p q r
E1 A = =
1 0 p q r a b c
Try to compute E2 A and E3 A

Elementary Matrices vs Elementary Row Operations
1 If an elementary row operation is performed on an m × n matrix A,

the result is EA where E is the elementary matrix obtained by
performing the same operation on the m × m identity matrix.
2 Every elementary matrix E is invertible, and E −1 is also a elementary
matrix (of the same type). Moreover, E −1 corresponds to the inverse

of the row operation that produces E.
The following table gives the inverse of each type of elementary row
operation:
Type Operation Inverse Operation
I Interchange rows p and q Interchange rows p and q
II Multiply row p by k ̸= 0 Multiply row p by 1/k
III Add k times row p Subtract k times row p
to row q ̸= p from row q

Example
 
0 1 0
E1 = 1 0 0
 
0 0 1
corresponds to exchanging rows 1 and 2 (type I). So the inverse
 
0 1 0
E1−1 = 1 0 0
 
0 0 1

Example
 
1 0 0
E2 = 0 1 0
 
0 0 9
corresponds to multiplyring row 3 by 9 (type II). So the inverse
 
1 0 0
E2−1 = 0 1 0 
 
0 0 19

Example
 
1 0 5
E3 = 0 1 0
 
0 0 1
corresponds to adding 5 times row 3 to row 1 (type III). So the inverse
 
1 0 −5
E3−1 = 0 1 0 
 
0 0 1
is to substract 5 times row 3 from row 1

Elementary Matrices Inverse and Elementary Matrices

Suppose that a matrix A is carried to matrix B, denoted byA → B by a
series of k elementary row operations corresponding to the elementary
matrices E1 , . . . , Ek then the reduction becomes
A → E1 A → E2 E1 A → · · · → Ek . . . E2 E1 A = B
In other word
A → U B where U = Ek . . . E2 E1 is invertible
U can be computed by
h i h i
A I → B U using the operations carrying A → B
If A is invertible then A can be carried to I, that is

h i h i
A I → I U where U A = I
Hence U = A−1 . So the Gauss - Jordan algorithm is valid

Furthermore
A−1 = U ⇒ A = U −1 = E1−1 E2−1 . . . Ek−1
Theorem
A square matrix is invertible if and only if it is a product of elementary
matrices.

Example
" #
−2 3
Express A = as a product of elementary matrices
1 0

Example
" #
−2 3
Express A = as a product of elementary matrices
1 0
Solution
The reduction of A → I is as follow
" # " # " # " #
−2 3 r1 ↔r2 1 0 r2 +2r1 1 0 13 r2 1 0
A= −−−−→ A = −−−−→ −−→ =I
1 0 E1 −2 3 E2 0 3 E3 0 1
" # " # " #
0 1 1 0 1 0
where E1 = , E2 = , E3 =
1 0 2 1 0 31
Hence E3 E2 E1 A = I, so
" #" #" #
0 1 1 0 1 0
A = E1−1 E2−1 E3−1 =
1 0 −2 1 0 3

LU Factorization
Table of contents
Matrix addition
3 Inverse Matrix
An Inversion Method
Elementary Matrices

LU Factorization
Triangular Matrices
h i
For a matrix A = aij , the elements a11 , a22 , . . . , form the main
diagonal of A.
A is called upper triangular if every entry below and to the left of
the main diagonal is 0
 
1 −1 0 3
0 0 1 1
 
0 0 −3 0
A is called lower triangular if every entry above and to the right of

the main diagonal is 0

LU Factorization
Example
Solve the system


 x + 2x2 − 3x3 − x4 + 5x5 = 3
 1
5x3 + x4 + x5 = 8


 2x5 = 6
where the coefficient matrix is upper triangular
It is easy to use backward substituition to find the solution of the system

equation. Similary, if the coefficient matrix is lower triangular then the
system can be solve by forward substitution where earlir variables are
substituted into later equation

LU Factorization
Solve Ax = b if A is LU-factorizable
If A = LU where L is lower triangular and U is upper triangular the the

system Ax = b can be solved in two stages as following
1 Solve Ay = b for y by forward substitution
2 Solve U x = y for x by backward substitution

LU Factorization
Matrix A can be carried to a row-echelon matrix U ( that is upper

triangular) with reduction
A → E1 A → E2 E1 A → · · · → Ek Ek−1 . . . E2 E1 A = U
where E1 , . . . , Ek are elementary matrices corresponding to the row

operation used. Hence
A = LU
where L = (Ek Ek−1 . . . E2 E1 )−1 = E1−1 E2−1 . . . Ek−1 .
Use the fact that Ei is lower triangluar and so is Ei−1 , we have L is also
lower triangular.
LU-factorization
If A can be lower reduced to a row - echelon matrix U then
A = LU
where L is lower triangular and U is upper triangular. The factorization

A = LU is called the LU-factorization of A
LU Factorization
LU-Algorithm
Suppose that A is a m × n matrix of rank r which can be lower reduced to
a row echelon matrix U then A = LU where the lower, invertible matrix L
is constructed as follows:
1 If A = 0 then take L = I
m and U = 0
2 If A ̸= 0 then write A = A and let c be the leading column of A ,
1 1 1
that is the first non zero column from the left in A1 . Use c1 to create
leading 1 and zeros below it using lower reduction. When this is
completed, let A2 be the matrix consisting of rows 2 to m of the
matrix just created
3 If A ̸= 0 then let c be leading columns of A and repeat Step 2 on
2 2 2
A2 to create A3
4 Continue until U is reached
5 Create L by replacing c , . . . , c at the bottom of the first r column

1 r
of Im
Proof for algorithm by induction on the number of columns n in A
LU Factorization
Example
 
5 −5 10 0 5
−3 3 2 2 1
Find LU-factorization of A = 
 
−2 2 0 −1 0

1 −1 10 2 5

LU Factorization
Solution
The reduction to row-echelon form is

LU Factorization
Solution (cont)
Let  
5 0 0 0
−3 8 0 0
L=
 
−2 4 −2 0

1 8 0 1
then
A = LU

LU Factorization
Practice
 
2 4 2
Find an LU-factorization for A =  1 1 2
 
−1 0 2

2 Matrix

Uploaded by

2 Matrix

Uploaded by

Matrix Algebra

Matrix Algebra 1 / 101

Study matrix algebra with introduction to some applications

▶ ”Matrix transformation” such as rotation are important tool in

Matrix Algebra 2 / 101

Matrix Algebra 3 / 101

Matrix Algebra 5 / 101

Matrix Algebra 6 / 101

Matrix and dataset

Matrix Algebra 6 / 101

Graph and Incidence Matrix

This graph is a tree - a graph has no closed loops

Matrix Algebra 8 / 101

B&W or Greyscale Image

Each of these pixels is denoted as a numerical value, called Pixel Values

RGB Color Images

The RGB color model is an additive color

RGB image can be viewed as three different

Two matrices A and B are equal, denoted by

Matrix Algebra 11 / 101

Discuss the possibility that A = B, B = C, A = C.

Matrix Algebra 12 / 101

Discuss the possibility that A = B, B = C, A = C.

means that a = 1, b = 0, c = −2 and d = 2

Matrix Algebra 13 / 101

Matrix Algebra 14 / 101

Find a, b, c such that

Matrix Algebra 15 / 101

Find a, b, c such that

This gives a + c = 3, b + a = 2 and c + b = −1

Matrix Algebra 15 / 101

If A = [aij ] and B = [bij ] then A − B = [aij − bij ]

Matrix Algebra 17 / 101

Matrix Algebra 17 / 101

Solve " # " #

Matrix Algebra 18 / 101

Solve " # " #

Matrix Algebra 18 / 101

If A is any matrix and k is any number, the scalar multiple kA is the

Matrix Algebra 19 / 101

Let " # " #

Compute 5A, 21 B, 3A − 2B.

Matrix Algebra 20 / 101

Properties of matrix addition and scalar multiplication

Let A, B, and C denote arbitrary m × n matrices where m and n are

Matrix Algebra 21 / 101

Simplify 2(A + 3C) − 3(2C − B) − 3[2(2A + B − 4C) − 4(A − 2C)]

Matrix Algebra 22 / 101

Simplify 2(A + 3C) − 3(2C − B) − 3[2(2A + B − 4C) − 4(A − 2C)]

2(A + 3C) − 3(2C − B) − 3[2(2A + B − 4C) − 4(A − 2C)]

Matrix Algebra 22 / 101

If A is an m × n matrix, the transpose of A, written AT , is the n × m

Matrix Algebra 23 / 101

Write down the transpose of each of the following matrices.

Matrix Algebra 24 / 101

Write down the transpose of each of the following matrices.

Matrix Algebra 24 / 101

Another way to think of transposition

Main diagonal consists of the elements a11 , a22 , . . .

Forming the transpose of a matrix A can be viewed as “flipping” A about

Matrix Algebra 25 / 101

Special types of matrix

Definition (Skew symmetric)

Matrix Algebra 26 / 101

Matrix Algebra 27 / 101

Properties of Matrix Transpose

Matrix Algebra 28 / 101

Matrix Algebra 29 / 101

The equation becomes

Matrix Algebra 30 / 101

A vector in Rn is also called a point in Rn

Matrix Algebra 32 / 101

Vector addition and scalar multiplication in Rn

Matrix Algebra 34 / 101

Geometric meaning (1)

Vector addition v + w produces the diagonal of a parallelogram. The