ENGINEERING MATHEMATICS

KMM 511E

SOLVING LINEAR SYSTEMS OF

EQUATIONS
 Linear Systems of Equations
A linear system with m equations and n unknowns

a11 x1 + a12 x2 + a13 x3 + … = b1

a21 x1 + a22 x2 + a23 x3 + … = b2
…..
…..
am1 x1 + am2 x2 + am3 x3 + … = bm

a and b represent coefficients and x represents unknowns

When all b’s are equal to 0, then the system is homogeneous

We obtain systems of equations when relations between two or more variables are
provided for two or more situations
 Linear Systems of Equations
Ax = b
A relationship is written for the linear system of equations in matrix form

A is the coefficient matrix, consisting of the a coefficients of the linear system of equations
x and b are column matrices, representing the unknowns and b coefficients, respectively

When we multiply A and x matrices we obtain the related linear system of equations

Actually, matrix multiplication is related to the use of matrices in linear transformations

a11 x1 + a12 x2 = b1
a21 x1 + a22 x2 = b2

a11 a12 x1 = b1
a21 a22 x2 b2
 Linear Systems of Equations
Augmented matrix (Ã)

a11 a12 b1
a21 a22 b2

Solving Linear Systems of Equations

Elimination Methods

The first step is forming the augmented matrix of the related linear system
Then a triangular matrix is formed by eliminative operations

These operations may be:

Adding a row with another row
Multiplication of a row by a scalar value
Interchanging two rows (and sometimes columns)  pivoting

Finally, the unknowns may be calculated by back-substitution

 Linear Systems of Equations
Gauss Elimination Method

2x1 + 5 x2 = 2
4 x1 + 3 x2 = 18
A= 2 5 coefficient matrix
4 3

à = 2 5 2 augmented matrix
4 3 18

In order to form an upper triangular matrix, we may select the first row as the pivot row and
multiply it with -2, the result will be added to the second row forming a new second row. In this
manner, the first element of the second row will be eliminated.

2 5 2
0 -7 14

From the second row, we conclude that -7x2 = 14 and x2 = -2

We can back-substitute the result in the first row to obtain 2x1 + 5(-2) = 2 and x1 = 6
 Linear Systems of Equations
Pivoting

It should be applied when

- The element in the pivot row and column where elimination will take place is 0
- The values on the main diagonal are smaller than the values in the same row

0 2 4 6
5 3 2 8 pivoting is required
6 1 0 9

3 9 6 1
2 1 5 4 pivoting is recommended
8 5 1 6

We can interchange the rows

The third row may become the first row and the second row may become the third row while the first
row becomes the second row. This is for reducing rounding errors
 Linear Systems of Equations
Gauss-Jordan Elimination Method

2x1 + 5 x2 = 2
4 x1 + 3 x2 = 18
A= 2 5 coefficient matrix
4 3

à = 2 5 2 augmented matrix
4 3 18

Here, we should form a diagonal or unit matrix. If we select the first row as the pivot row and multiply it
with -2, the result will be added to the second row forming a new second row. In this manner, the first
element of the second row will be eliminated. We continue to eliminate the second element of the first
row. For this purpose, we multiply the second row by 5/7 and add it to the first row. If we divide the first
row by 2 and the second row by -7 we obtain a unit matrix.

2 5 2 2 0 12 1 0 6
0 -7 14  0 -7 14  0 1 -2

From the second row, we conclude that x2 = -2 and from the first row we see that x1 = 6
 Linear Systems of Equations
Linear Systems are

- Consistent if there is at least one solution

- Inconsistent if there is no solution
- Overdetermined if there are extra equations
- Underdetermined if the equations are less than the unknowns
- Determined if the numbers of equations and unknowns are equal

A linear system of equations may have

- No solution : parallel lines

- Single solution : intersecting lines
- Infinite number of solutions : coinciding lines

We may predict the existence of a solution for the linear systems by calculating the ranks of the
coefficient (nxn) and augmented matrices
When
- Rank of A  Rank of à then there is no solution
- Rank of A = Rank of à = n then there is a single solution
- Rank of A = Rank of à  n then there is an infinite number of solutions
 Linear Systems of Equations
Rank of a Matrix

We should find the determinant of the nxn square matrix. If the determinant is not equal to 0, then the
rank of the matrix is equal to n

If the determinant is equal to 0, then we find the ranks of all the minor (n-1)x(n-1) matrices. If the
determinant of even one of the minor matrices is not equal to 0, we say the rank of the matrix is equal to
n-1

We may continue wth the smaller matrices in the same manner, if necessary

If we have a matrix that is not square (n+1)xn, then we consider the largest possible square matrix (nxn)

-1 2 -3
0 4 -5 The rank of this 3x3 A matrix is
2 -4 6

det(A) = 0

det -1 2 = -4 so rank of A = 2
0 4
 Linear Systems of Equations
2 0 0
0 0 -1 The rank of this 2x3 B matrix is

B1 = 2 0
0 -1

det(B1) = -2 so rank of B = 2

Example of a single solution system

-x1 + x2 + 2x3 = 2
3x1 – x2 + x3 = 6
-x1 + 3x2 + 4x3 = 4

Example of a no solution system

3x1 + 2x2 + x3 = 3
2x1 + x2 + x3 = 0
6x1 + 2x2 + 4x3 = 6

When there is no solution, there will be an inconsistency for the result obtained at the end of
elimination. In one of the rows  0 0 0 12
 Linear Systems of Equations
Example of a system with infinite number of solutions

3x1 + 2x2 + 2x3 -5x4 = 8

0.6x1 + 1.5x2 + 1.5x3 – 5.4x4= 2.7
1.2x1 – 0.3x2 – 0.3x3 + 2.4x4= 2.1

When there are infinite number of solutions, one of the rows will have all 0s
In one of the rows  0 0 0 0
Actually, here we have 4 unknowns but 3 equations
 Linear Systems of Equations
Inverse of a Matrix

We may adapt Gauss-Jordan method to find the inverse of a square matrix

For square matrices

A A-1 = A-1 A = I

Ax=I
A-1 x-1 = I
I x = A-1-

-1 1 2
3 -1 1 = A
-1 3 4

-1 1 2 1 0 0 1 0 0 b11 b12 b13

3 -1 1 0 1 0 = A I  0 1 0 b21 b22 b23
-1 3 4 0 0 1 0 0 1 b31 b32 b33
 Linear Systems of Equations
-1 1 2 1 0 0 -1 1 2 1 0 0 1 0 0 -0.7 0.2 0.3
0 2 7 3 1 0  0 2 7 3 1 0 ….. 0 1 0 -1.3 -0.2 0.7
0 2 2 -1 0 1 0 0 -5 -4 -1 1 0 0 1 0.8 0.2 -0.2

-0.7 0.2 0.3

A-1 = -1.3 -0.2 0.7
0.8 0.2 -0.2

For singular matrices, there is no inverse matrix

 Linear Systems of Equations
Another method of solving linear systems of equations
Cramer’s Rule

a11 x1 + a12 x2 = b1
a21 x1 + a22 x2 = b2

Let us multiply the first row by a22 and the second row by –a12

a11 a22 x1 + a12 a22 x2 = b1 a22

-a21 a12 x1 - a22 a12 x2 = -b2 a12

(a11 a22 - a12 a21)x1 = b1 a22 - b2 a12

x1 = (b1 a22 - b2 a12) / (a11 a22 - a12 a21)

det(A) = a11 a12

a21 a22 = a11 a22 – a12 a21 = D

D1 = det b1 a12
b2 a22 = b1 a22 – b2 a12
 Linear Systems of Equations
x1 = D1 / D x2 = D2 / D

x2 is obtained by multiplying the first row by –a21 and the second row by a11
D2 is obtained by replacing a12 and a22 by b1 and b2

We may extend Cramer’s Rule to systems with n number of equations and unknowns
x1 = D 1 / D x2 = D2 / D x3 = D3 / D …….. xn = Dn / D

We will replace b coefficients in the consecutive columns

2x1 + 5 x2 = 2
4 x1 + 3 x2 = 18

D = det 2 5 = -14
4 3

D1 = det 2 5 = -84 x1 = D1/D = -84/-14 = 6

18 3

D2 =det 2 2 = 28 x2 = D2/D = 28/-14 = -2

4 18
 Linear Systems of Equations

LU Decomposition (Factorization)

This method is about factorizing a matrix as the product of a lower triangular matrix
and an upper triangular matrix

A=LU

where A is a square matrix

Ax=LUx=b

There are different versions of this method

 Linear Systems of Equations

Doolittle Method

For a matrix A of 2x2

A= 2 3 = 1 0 2 3 =LU
8 5 4 1 0 -7

We can form first the upper triangular matrix by elimination and then by considering matrix
multiplication obtain the elements of the lower triangular matrix

For a square matrix of nxn, we may construct the upper and lower matrices as

u11 u12 u13 1 0 0

0 u22 u23  m21 1 0
0 0 u33 m31 m32 1
 Linear Systems of Equations
We can find the unknown elements of the matrices as

a11 = u11 a12 = u12 a13 = u13

a21 = m21 u11 a22 = m21 u12 + u22 a23 = m21 u13 + u23

a31 = m31 u11 a32 = m31 u12 + m32 u22 a33 = m31 u13 + m32 u23 + u33

A= 3 5 2
0 8 2
6 2 8

u11 = 3 u12 = 5 u13 = 2

0 = m21 3  m21 = 0 8 = 0 u12 + u22  u22 = 8 2 = 0 u13 + u23  u23 = 2

6 = m31 3  m31 = 2 2 = 2x5 + m32 8  m32 = -1 8 = 2x2 - 1x2 + u33  u33 = 6

LU= 1 0 0 3 5 2
0 1 0 0 8 2
2 -1 1 0 0 6
 Linear Systems of Equations
By assuming that Ux = y  Ly = b

1 0 0 y1 8
0 1 0 y2 = -7
2 -1 1 y3 26

Solving the linear system starting from y1 gives

8
y = -7
3
Ux = y
3 5 2 x1 8
0 8 2 x2 = -7
0 0 6 x3 3
Solving the linear system starting from x3 gives
4
x = -1
1/2