Topic:1

Matrix algebra and System of linear

equations

Gyalpozhing College of Information Technology

Prepared by Chiranjeevi Adhikari
Contents
1 Matrix 2
1.1 Matrix Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Matrix Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Transpose of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Trace of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.6 Types of Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.7 The inverse of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.8 Echelon Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.9 Gauss-Jordan Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Linear system of equations 11

2.1 Solving linear system using various methods . . . . . . . . . . . . . . . . 13
2.2 Application of linear system of equations . . . . . . . . . . . . . . . . . 14

3 Applying matrix algebra in Hill Cipher. 17

LINEAR ALGEBRA CSM 201

1 Matrix
Definition 1 (Matrix) If A is an m×n matrix- that is a matrix with m rows and n columns,then
the scalar entry in the ith row and jth column of A is denoted by ai j and is called the (i,j)
entry of A.

1.1 Matrix Addition

Definition 2 The sum of two matrices of the same order (i.e, the same number of rows and
the same number of columns) is the matrix obtained by adding together corresponding
elements of the original two matrices. (or)

If A and B are m × n metrices of same order, then the sum of A + B is m × n matrix whose
columns are the sums of the corresponding columns in A + B

Example
" # " # " #
4 0 5 1 1 1 2 −3
Let A= , B= and C=
−1 3 2 3 5 7 0 1

4+1 0+1 5+1

" # " #
5 1 6
Then A+B= =
−1 + 3 3 + 5 2 + 7 2 8 9
But A+C is not defined because A and C have different sizes/order.

2
LINEAR ALGEBRA CSM 201

Note:

If r is a real scalar and A is a matrix,then the scalar multiple r · A is the matrix whose
columns are r times the corresponding columns in A.

Example

Let A and
" B are the
# "matrices defined
# in the above examples ,then
1 1 1 2 2 2
2B =2 · =
3 5 7 6 10 14

" #" # " #

4 0 5 2 2 2 2 −2 3
A-2B = - =
−1 3 2 6 10 14 −7 −7 −12

Theorem 1 Let A,B and C be matrices of the same size, and let r and s be scalars then

• A+B=B+A

• (A+B)+C=A+(B+C)

• A+0=A where 0 is the zero matrix

• r(A+B)=rA+rB

• (r+s)A=rA+sA

• r(sA)=(rs)A

1.2 Matrix Multiplication

Definition 3 The multiplication of two matrices A and B requires that the matrix A has
the same number of the columns, as B has the rows.The result ,A × B, then has the same
number of rows as A and the same number of columns as B.

Am×p × B p×n = Cm×n

3
LINEAR ALGEBRA CSM 201

Example
" # " #
2 3 4 3 6
Let A= and B= .Then A × B is
1 −5 1 −2 3

(2 × 4) + (3 × 1) (2 × 3) + (3 × −2) (2 × 6) + (3 × 3)
" # " # " #
2 3 4 3 6
A×B = × =
1 −5 1 −2 3 (1 × 4) + (−5 × 1) (1 × 3) + (−5 × −2) (1 × 6) + (−5 × 3)
" #
11 0 21
=
−1 13 −9

Theorem 2 Let A be an m × n matrix ,and let B and C have sizes for which the indicated
sums.

• A(BC) = (AB)C

• A(B + C) = AB + AC

• (B + C)A = BA + CA

• r(AB) = (rA)B = A(rB) for any real scalar r.

• AB , BA

1.3 Transpose of a matrix

Definition 4 Let A be an m × n matrix,then transpose of A is the n × m matrix,denoted by
AT ,where columns are formed from the corresponding rows of A.

4
LINEAR ALGEBRA CSM 201

Example

 
−5 2  " #
−5 1 0
If A = 1 −3 , then A =
T
 
2 −3 4
0 4
 

Theorem 3 Let A and B denotes matrices whose size are appropriate for the following
sums

1. (AT )T = A

2. (A + B)T = AT + BT

3. For any real scalar r,(rA)T = rAT

4. (AB)T = BT AT

1.4 Trace of a matrix

Definition 5 If A is a square matrix, then the trace of A, denoted by tr(A), is defined to be
the sum of the entries on the main diagonal of A. The trace of A is undefined if A is not a
square matrix.
 
1 2 3
Example: Let A = 0 6 1 and the trace of A is
 
2 1 0
 

tr(A) = 1 + 6 + 0 = 7

Theorem 4 If A,B and C are n × n matrices ,then

• tr(A+B)=tr(B+A)

• tr(A)=tr(AT )

• tr(ABC)=tr(BCA)=tr(CAB)

5
LINEAR ALGEBRA CSM 201

1.5 Determinant
Definition 6 For n ≥ 2,the determinant of an n × n matrix A=[ai j ] is the sum of n terms of
the form ±a1 j detA1 j , with plus and minus signs alternating,where the entries a11 , a12 , ..., a1n
are from the first row of A.In symbols ,

detA = a11 detA11 − a12 detA12 + .... + (−1)1+n a1n detA1n

= nj=1 (−1)1+ j a1 j detA1 j
P

Example
 
1 5 0 
Compute the determinant of A = 2 4 −1
 
0 −2 0
 

Sol: |A| = (0 − 2) − 5(0 − 0) + 0(−4 − 0) = −2

1.6 Types of Matrices

1. A square matrix is a matrix with the same number of rows and columns. An n × n
matrix is known as asquare matrix of order n.
 2 5 10
eg: A=−3 −7 1 
 
0 2 1
 

2. A symmetric matrix is a matrix A such that AT = A.Such a matrix is necessarily

square.
" #
1 0
eg: A= is a symmetric matrix since A = AT .
0 −3
3. An Identity matrix is n × n matrix , denoted In , is a matrix with n rows and n
columns. The entries on the diagonal from the upper left to the bottom right are all
1’s, and all other entries are 0.
 
" # 1 0 0 0
1 0 0 1 0 0
eg:I2 = ,I = 
0 1 4 0 0 1 0

 
0 0 0 1

6
LINEAR ALGEBRA CSM 201

4. Matrix A of n × n is said to be Invertible Matrix(non-singular) if determinant of A

is not equal to zero .i.e,|A| , 0

1.7 The inverse of a matrix

Definition 7 An n × n matrix A is said to be in

Let A be n×n invertible matrix .Then an n×n matrix B is inverse of A iff AB=BA=In

Example
" # " #
2 5 −7 −5
Let A = and B = .Then B is the inverse of A since AB=I2
−3 −7 3 2
" # " #
a b d −d
Theorem 5 Let A= .If ad − bc , 0,then A is invertible and A−1 = 1
(ad−bc)
.
c d −c a

Note:

If A is a n × n invertible matrix of order n ,then A−1 = 1

|A|
(ad jA).

Examples:

Find the inverse of the given matrices and verify your answers.
 
1 −1 2 
1. A=0 2 −3
 
3 −2 4
 
" #
3 4
2. B=
5 6
 
 2 0 −1
Answers 1: A−1 = −9 −2 3  and A−1 A = I3
 
−1 −1 2
 
" #
−3 2
Answers 2: B−1 = 5 −3 and B−1 B = I2
2 2

7
LINEAR ALGEBRA CSM 201

1.8 Echelon Form

A matrix is in row echelon form if it has the following three properties :

• If there are any rows that consist entirely of zeros,they are placed together at the
bottom of matrix.

• Each leading entry(Pivot) of a row is in column to the right of the leading entry of
the row about it.

• All entries in a column below a leading entry are zero.

If a matrix in row echelon form satisfies the following additional conditions, then it is in
reduced row echelon form:

• The leading entry(Pivot) in each non-zeros row is 1.

• Each leading 1 is the only non-zero entry in its column.

Note:The following matrices are in echelon form. The leading entries(■) or Pivot have
any non-zero value.The starred entire(*) may have any values (including zero).
 
  0 ■ ∗ ∗ ∗ ∗ ∗ ∗ 
■∗ ∗ ∗   
0 0 0 ■ ∗ ∗ ∗ ∗ 
■ ∗ ∗  
 0
, 0 0 0 0 ■ ∗ ∗ ∗ 
 
 0
0 0 0  
 0 0 0 0 0 ■ ∗ ∗ 
 
0 0 0 0  
0 0 0 0 0 0 ■
The Row operations are :

• Interchange any two rows .

• Multiply /Divide a row by a nonzero number.

• Addition of any two rows.

8
LINEAR ALGEBRA CSM 201

Examples of row echelon form (ref):

 
1 5 0 2 −2 4 
   
1 2 3 −1 0 1 0 3 4 8 
0 −3 −6 7 , 0 0 0 1 7 −2
   
0 0 0 −2 0 0 0 0 0 0 
 
0 0 0 0 0 0
Examples of row echelon form (ref):
 
1 5 0 2 −2 4 
   
1 2 3 −1 0 1 0 3 4 8 
0 −3 −6 7 , 0 0 0 1 7 −2

  
0 0 0 −2 0 0 0 0 0 0 

 
0 0 0 0 0 0
Example:Find
 the row echelon and reduced echelon form of the following matrix
1 2 3 −1
A= 4 5 6 3 
 
7 8 9 5
 
   
1 2 3 −1 ]1 2 3 −1
 R2 −>R2 −4R1 
Ans: 4 5 6 3  −−−−−−−−−→  0 −3 −6 7  This matrix is in row echelon form
 
 R3 −>R3 −7R1 
7 8 9 5 −− −−−−−−−→ 0 0 0 −2
 

(ref).

Now for the reduced row echelon form.

     
]1 2 3 −1 R3 −> −1 R3 1 2 3 −1 1 0 −1 6 
2
 0 −3 −6 7  −−−−−− −→ 0 1 2 −7 −7 
0 1 2 2 
  
2  R1 −>R1 −2R2 
 −−
 
0 0 0 −2 R2 −> 3 R2 0 0 0 1 −1  − −−−− −−
→ 0 0 0 1
−−−−−−−→
 
1 0 −1 0
R2 −>R2 + 2 R3 
7

−−−−−−−−−→0 1 2 0 This matrix is in reduced row echelon form(rref).


R1 −>R1 −6R3 0 0 0 1
−−−−−−−−−→
Question:

9
LINEAR ALGEBRA CSM 201

 
 2 −2 4 −2
 2 1 10 7 
Find the row echelon and reduced row echelon form of the matrix A =  
−4 4 −8 4 
4 −1 14 6

1.9 Gauss-Jordan Method

We can used The Gauss Jordan Elimination, or Gaussian Elimination’s algorithm to find
the inverse of a matrix A.

Algorithm:

• Create an augmented matrix [A|In ], where A is the n × n matrix to be inverted and In

is the n × n identity matrix.

• Use elementary row operations on [A|In ] to transform the left partition A to row-reduced
form, applying each operation to the full augmented matrix.

• If the left partition of the row-reduced matrix has zero elements on its main diagonal,
stop: A does not have inverse. Otherwise,continue.

• Use elementary row operations on the row-reduced augmented matrix to transform

the left partition to the n × n identity matrix,applying each operation to the full
augmented matrix.

• The right partition of the final augmented matrix is the inverse of A.

Examples 1:
" #
1 2
Find the inverse of A=
3 4
" #
1 2 1 0
Ans:Augmented matrix for A =
3 4 0 1

" # " # " # " #

1 2 1 0 1 2 1 0 1 2 1 0 1 0 −2 1
= R2 −>R2 −3R1 R2 −> −1
2 R2 3 −1 R1 −>R1 −2R2 3 −1
3 4 0 1 −−−−−−−−−→ 0 −2 −3 1 −−−−−−−→ 0 1 2 2 −−−−−−−−−→ 0 1 2 2

Matrix A has been transformed into row-reduced form with a main diagonal of only ones;
A has an inverse.

10
LINEAR ALGEBRA CSM 201

" #
−2 1
A−1 = 3 −1
2 2

Examples 2:
" #
1 2
Find the inverse of A=
2 4
" #
1 2 1 0
Ans:Augmented matrix for A =
2 4 0 1

" # " #
1 2 1 0 1 2 1 0
= R2 −>R2 −2R1
2 4 0 1 −−−−−−−−−→ 0 0 −2 1
Matrix A has been transformed into row-reduced form. Because the main diagonal contains
a zero entry, Matrix A does not have an inverse; A is singular.

Question:
 
1 −1 2 
Find the inverse of the matrix A = 0 2 −3 using Gauss-Jordan method.
 
3 −2 4
 

2 Linear system of equations

Definition 8 A linear equation in the variables x1 , ..., xn is an equation that can be written
in the form

a11 x1 + a12 x2 + ... + a1n xn = b

where a11 , a12 ...a1n are real or complex numbers.

A system of linear equation(or linear system) is a collection of one or more linear equation
involving the same variables-say x1 , ...xn . The linear system has the general form :

a11 x1 + a12 x2 + ... + a1n xn = b1

a21 x1 + a22 x2 + ... + a2n xn = b1
. .
. .
. .

11
LINEAR ALGEBRA CSM 201

am1 x1 + am2 x2 + ... + amn xn = bm

where the coefficients ai j (i= 1, 2, . . . , m; j = 1, 2, . . . , n) and the quantities bi are all
known scalars.

Any linear system can be written in the matrix form AX = B where

... a1n 
     
 a11 a12 
 x1  b1 
... a2n 
 a   
 21 a22  x2  b 
 2 
A =  . . ... .  ,X =  .  and B =  . .
  
 . . ... .  .   . 
     

... amn
 
 
am1 am2 xn bn
A solution to linear system is a set of scalar values for the variables x1 , x2 , ..., xn that when
substituted into each equation of the system makes each equation true.

Examples :

The linear system

x1 + x2 − 2x3 = −3
2x1 + 5x2 + 3x3 = 11
−x1 + 3x2 + x3 = 5

can be written in matrix forms AX=B

     
 1 1 −2  x1  −3
A =  2 5 3 , X =  x2  and B =  11 .
     
−1 3 1 x3 5
     

which has a augmented matrix

 
 1 1 −2 −3
[A|b] =  2 5 3 11 
 
 
−1 3 1 5

12
LINEAR ALGEBRA CSM 201

2.1 Solving linear system using various methods

The linear system

x1 + x2 − 2x3 = −3
2x1 + 5x2 + 3x3 = 11
−x1 + 3x2 + x3 = 5

can be solved using various methods :

Method 1; Using reduced row echelon form(rref)

   
 1 1 −2 −3 1 0 0 0
[A|b] =  2 5 3 11  rre f 0 1 0 1
   
  −−−→  
−1 3 1 5 0 0 1 2
Hence,the solution set x1 = 0, x2 = 1, x3 = 2

Method 2; Using the matrix form

AX = B
X = A−1 B

The above linear system can be written in matrix forms AX=B

     
 1 1 −2  x1  −3
A =  2 5 3 , X =  x2  and B =  11 .
     
−1 3 1 x3 5
     
.  
 0.125 0.2258 −0.419
Then its A−1 =  0.1612 0.0322 0.2258 
 
−0.354 0.129 −0.096
 
     
 0.125 0.2258 −0.419 −3 0
Hence, X = A−1 B =  0.1612 0.0322 0.2258  ×  11 =1
     
−0.354 0.129 −0.096 5 2
     

13
LINEAR ALGEBRA CSM 201

Question:

Solve the system of equations:

2x + 3y − z = 12
4x − 5y + 6z = 35
2x + 6z = 1

2.2 Application of linear system of equations

Leontief Input Output Models

One way to analyze an economy is to divide it into sectors and study how the sectors
interact with one another. For example, a simple economy might be divided into three
sectors—manufacturing, agriculture, and utilities. Typically, a sector will produce certain
outputs but will require inputs from the other sectors and itself. For example, the agricultural
sector may produce wheat as an output but will require inputs of farm machinery from
the manufacturing sector, electrical power from the utilities sector, and food from its
own sector to feed its workers. Thus, we can imagine an economy to be a network
in which inputs and outputs flow in and out of the sectors; the study of such flows is
called input-output analysis. Inputs and outputs are commonly measured in monetary
units (dollars or millions of dollars, for example) but other units of measurement are also
possible.

The flows between sectors of a real economy are not always obvious. For example, in
World War II the United States had a demand for 50,000 new airplanes that required the
construction of many new aluminum manufacturing plants. This produced an unexpectedly
large demand for certain copper electrical components, which in turn produced a copper
shortage. The problem was eventually resolved by using silver borrowed from Fort Knox
as a copper substitute. In all likelihood modern input-output analysis would have anticipated
the copper shortage.

Most sectors of an economy will produce outputs, but there may exist sectors that consume
outputs without producing anything themselves (the consumer market, for example). Those
sectors that do not produce outputs are called open sectors. Economies with no open
sectors are called closed economies, and economies with one or more open sectors are
called open economies. In this section we will be concerned with economies with one
open sector, and our primary goal will be to determine the output levels that are required

14
LINEAR ALGEBRA CSM 201

for the productive sectors to sustain themselves and satisfy the demand of the open sector.

Leontief Model of an Open Economy

Let us consider a simple open economy with one open sector and three product-producing
sectors: manufacturing,agriculture, and utilities.Assume that inputs and outputs are measured
in dollars and that the inputs required by the productive sectors to produce one dollar’s
worth of output are in accordance with Table 1.

15
LINEAR ALGEBRA CSM 201

Usually, one would suppress the labeling and express this matrix as
 
0.5 0.1 0.1
C = 0.2 0.5 0.3
 
0.1 0.3 0.4
 

This is called the consumption matrix (or sometimes the technology matrix) for the
economy. The column vectors
     
0.5 0.1 0.1
c1 = 0.2 , c2 = 0.5 , c2 = 0.3
     
0.1 0.3 0.4
     

in C list the inputs required by the manufacturing, agricultural, and utilities sectors, respectively,
to produce $1.00 worth of output. These are called the consumption vectors of the sectors.
For example, c1 tells us that to produce .00 worth of output the manufacturing sector needs
$0.50 worth of manufacturing output, $0.20 worth of agricultural output, and $0.10 worth
of utilities output.

Continuing with the above example, suppose that the open sector wants the economy to
supply it manufactured goods, agricultural products, and utilities with dollar values:

d1 dollars of manufactured goods

d2 dollars of agriculture goods
d3 dollars of utilities goods

The column vector d that has these numbers as successive components is called the outside
demand vector. Since the product-producing sectors consume some of their own output,
the dollar value of their output must cover their own needs plus the outside demand.
Suppose that the dollar values required to do this are

x1 dollars of manufactured goods

x2 dollars of agriculture goods
x3 dollars of utilities goods

The column vector X that has these numbers as successive components is called the
production vector for the economy. For the economy with consumption matrix 1, that
portion of the production vector x that will be consumed by the three productive sectors is

16
LINEAR ALGEBRA CSM 201

The vector C x is called the intermediate demand vector for the economy. Once the intermediate
demand is met, the portion of the production that is left to satisfy the outside demand is
X − C x . Thus, if the outside demand vector is d, then X must satisfy the equation

which we will find convenient to rewrite as

(I − C)x = d.........(1)

The matrix (I − C) is called the Leontief matrix and equation(1) is called Leontief
equation

Question

Consider the economy described in Table 1. Suppose that the open sector has a demand for
$7900 worth of manufacturing products, $3950 worth of agricultural products, and $1975
worth of utilities.Find a production vector X that will meet it exactly.

3 Applying matrix algebra in Hill Cipher.

The coding and decoding of secret messages has been important in times of warfare, of
course, but it is also quite valuable in peacetime for keeping government and business

17
LINEAR ALGEBRA CSM 201

secrets under tight security.

Hill Cipher is a poly alphabetic cipher invented by Lester S. Hill in 1929, using linear
algebra and modular arithmetic via a numeric matrix that serves as an encryption and
decryption key.

Generally, the below-mentioned structure of numbers and letters are used in the Hill Cipher
Encryption, but this can be modified as per requirement.

Algorithm:

Encrypt ions

Encrypting with the Hill cipher is built on the following operation:

E(K, P) = (K × P) mod 26

Where K is our key matrix and P is the plaintext in vector form. Matrix multiplying these
two terms produces the encrypted ciphertext.

Decryption

Decrypting with the Hill cipher is built on the following operation:

D(K, C) = (K −1 × C) mod 26

Where K is our key matrix and C is the ciphertext in vector form. Matrix multiplying the
inverse of the key matrix with the ciphertext produces the decrypted plaintext.

Example 1:

18
LINEAR ALGEBRA CSM 201

 
17 17 5 
Encrypt ” pay more money ” using Hill cipher with the key matrix 21 18 21
 
2 2 19
 

p a y m o r e m o n e y
15 0 24 12 14 17 4 12 14 13 4 24

key= 3 × 3 matrix
Plain text= pay more money

Encrypting :pay
 
h i k11 k12 k13 
(c1 , c2 , c3 )= P1 P2 P3 k21 k22 k23  mod 26
k31 k32 k33
 
 
h i 17 17 5 
= 15 0 24 21 18 21 mod 26
2 2 19
 
h i
= 301 303 531 mod 26
h i
= 17 17 11 =(R,R,L) (ciphertext)

Encrypting :mor
 
h i 17 17 5 
= 12 14 17 21 18 21 mod 26
2 2 19
 
h i
= 532 490 677 mod 26
h i
= 12 22 1 =(M,W,B) (ciphertext)

Encrypting :emo
 
h i 17 17 5 
= 4 12 14 21 18 21 mod 26
2 2 19
 
h i
= 348 312 538 mod 26

19
LINEAR ALGEBRA CSM 201

h i
= 10 0 18 =(k,A,S) (ciphertext)

Encrypting :ney
 
h i 17 17 5 
= 13 4 24 21 18 21 mod 26
2 2 19
 
h i
= 353 341 605 mod 26
h i
= 15 3 7 =(P,D,H) (ciphertext)

Plaintext: p a y m o r e m o n e y
Ciphertext: R R L M W B K A S P D H

Example 2:
 
17 17 5 
Encrypt ” RRLMWBKASPDH ” using Hill cipher with the key matrix 21 18 21
 
2 2 19
 

R R L M W B K A S P D H
17 17 11 12 22 1 10 0 18 15 3 7

key= 3 × 3 matrix

Ciphertext= RRLMWBKASPDH

First we need to find the K −1 mod26

K −1 = 1
DetK
× Ad jK
 
14 25 7
Then K −1 mod26 = (23)−1 ×  7 1 8 mod 26
 
6 0 1
 
 
14 25 7
= 17 ×  7 1 8 mod 26
 
6 0 1
 

20
LINEAR ALGEBRA CSM 201

(Note: 17 is the multiplicative inverse of 23. Use Extented Euclidean Algorithm to find
the multiplicative inverse )
 
 4 9 15
A−1 = 15 17 6 
 
24 0 17
 

Decrypting :RRL
 −1
h i k11 k12 k13 
(p1 , p2 , p3 )= c1 c2 c3 k21 k22 k23  mod 26
k31 k32 k33
 
 
h i  4 9 15
= 17 17 11 15 17 6  mod 26
24 0 17
 
h i
= 587 442 544 mod 26
h i
= 15 0 24 =(p,a,y)

Decrypting :MWB
 
h i  4 9 15
= 12 22 1 15 17 6  mod 26
24 0 17
 
h i
= 402 482 329 mod 26
h i
= 12 14 17 =(m,o,r)

Decrypting :KAS
 
h i  4 9 15
= 10 0 18 15 17 6  mod 26
24 0 17
 
h i
= 472 90 456 mod 26
h i
= 4 12 14 =(e,m,o)

Decrypting :PDH

21
LINEAR ALGEBRA CSM 201

 
h i  4 9 15
= 15 3 7 15 17 6  mod 26
24 0 17
 
h i
= 273 186 362 mod 26
h i
= 13 4 24 =(n,e,y)

Plaintext: p a y m o r e m o n e y
Ciphertext: R R L M W B K A S P D H

22