Lecture Notes 3

CHAPTER 3
System of Linear Algebraic

Equations
CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

System of Linear Algebraic Equations
• This subject deals with simultaneous algebraic equations which

can be represented generally as:
a11 x1  a12 x2  ...  a1n xn  b1

a21 x1  a22 x2  ...  a2 n xn  b2
. . . .
. . . .
. . . .
an1 x1  an 2 x2  ...  ann xn  bn
• where the a’s are constant coefficients and the b’s are constants.

Solving Small Numbers Of Equations
• In this section, we will describe several methods that are

appropriate for solving small (n ≤ 3) sets of simultaneous
equations and that do not require a computer. These are:
1. The graphical method

2. Cramer’s rule and
3. The elimination of unknowns.

The Graphical Method
• A graphical solution is obtainable for two equations by

plotting them on Cartesian coordinates with one axis
corresponding to x1 and the other to x2. Because we are
dealing with linear systems, each equation is a straight line.
This can be easily illustrated for the general equations:
a11 x1  a12 x2  c1
a21 x1  a22 x2  c2
• If x2 is solved from each equations
 a11  c1  a21  c2
x2    x
 1  x2    x
 1 
a
 12  a12 a
 22  a22

The Graphical Method
• Thus, the equations are now in the form of straight lines;

that is:
x2 = (slope) x1 + intercept
• These lines can be graphed on Cartesian coordinates with

x2 as the ordinate and x1 as the abscissa. The values of x1
and x2 at the intersection of the lines represent the
solution.

Example
• Use the graphical method to solve
3x1 + 2x2 = 18
−x1 + 2x2 = 2
Solution
• Let x1 be the abscissa, then x2 is solved from each equation
x2 = (−3/2)x1 + 9
x2 = (1/2)x1 + 1

Example
• Then two line equations are plotted together as follow:
Then intersections of lines

represented solution
x1=4 and x2=3

Determinants and Cramer’s Rule
• Cramer’s rule is another solution technique that is best

suited to small numbers of equations.
• Before describing this method, we will briefly introduce

the concept of the determinant, which is used to
implement Cramer’s rule.
• In addition, the determinant has relevance to the

evaluation of the ill-conditioning of a matrix.

Determinants
• The value of the second-order determinant
D  a11a22  a21a12
• The value of the third-order determinant
*2 by 2 determinants are called as minors

Example
• Compute the determinant of matrices
 3 2 Non-Singular
A   D  3  2  (1)  2  8
 1 2 
 1/ 2 1 Singular
B   D  (1/ 2)  1  (1/ 2)  1  0
 1/ 2 1
 1/ 2 1 
C   D  (1/ 2)  2  (1)  1  0 Singular
 1 2 
 1/ 2 1 Almost
D   D  (1/ 2)  1  (23 / 50)  1  0.04 Singular
 23 / 50 1

Cramer Rule
• This rule states that each unknown in a system of linear

algebraic equations may be expressed as a fraction of two
determinants with:
• the denominator D (Determinant of Coefficient Matrix)
• the numerator obtained from D by replacing the column of

coefficients of the unknown in question by the constants b1,
b2, . . . , bn.

Cramer Rule
• Consider the following linear algebraic equations
a11 x1  a12 x2  a13 x3  b1

a21 x1  a22 x2  a23 x3  b2
a31 x1  a32 x2  a33 x3  b3
• These equations can be represented as matrices form:
 a11 a12 a13   x1   b1 

a a22 a23   x2   b2 
 21
 a31 a32 a33   x3   b3 

Cramer Rule
• Then the following determinants are calculated:
b1 a12 a13
Dx1  b2 a22 a23
b3 a32 a33
a11 a12 a13
D  a21 a22 a23 a11 b1 a13
a31 a32 a33 Dx2  a21 b2 a23
a31 b3 a33
a11 a12 b1
Determinant of
Coefficient matrix Dx3  a21 a22 b2
a31 a32 b3
Cramer Rule
• Finally, unknowns x1, x2 and x3 are then calculated as a

fraction of two determinants
Dx1 Dx2 Dx3

x1  x2  x3 
D D D
• Example:
Solve the following system of linear algebraic equations
by using Cramer's Rule
3 x1  2 x2  x3  9
x1  2 x2  2 x3  5
x1  x2  4 x3  2
Solution
3 2 1
2 2 1 2 1 2
D  1 2 2  3  (2) 1  3  (6)  (2)  (2)  1 (1)  23
1 4 1 4 1 1
1 1 4
9 2 1
2 2 5 2 5 2
Dx1  5 2 2  9  (2) 1  9  (6)  (2)  (16)  1 (1)  23
1 4 2 4 2 1
2 1 4
3 9 1
5 2 1 2 1 5
Dx2  1 5 2  3 9 1  3  (16)  9  (2)  1 (3)  69
2 4 1 4 1 2
1 2 4
3 2 9
2 5 1 5 1 2
Dx3  1 2 5  3  (2) 9  3 1  (2)  (3)  9  (1)  0
1 2 1 2 1 1
1 1 2
Solution
• Then
Dx1
23
x1   1
D 23
Dx2 69
x2    3
D 23
Dx30
x3   0
D 23

Elimination of Unknowns
• The elimination of unknowns by combining equations is an
algebraic approach that can be illustrated for a set of two
equations:
a11 x1  a12 x2  b1
a21 x1  a22 x2  b2
• The basic strategy is to multiply the equations by constants
so that one of the unknowns will be eliminated. For first
Equation might be multiplied by a21 and second Equation
by a11 to give
a21a11 x1  a21a12 x2  a21b1

a11a21 x1  a11a22 x2  a11b2
• Subtracting first equation from second one x1 term
will be eliminated from the equations to yield:
a22 a11 x2  a12 a21 x2  b2 a11  b1a21
• which can be solved for

a11b2  a21b1
x2 
a11a22  a12 a21
• Substituting x2 in one of the equations
a22b1  a12b2
x1 
a11a22  a12 a21
• Note that, the solutions obtained for x1 and x2 are
same with Cramer rule:
• Example
• Use the elimination of unknowns to solve following
equations.

Solution
• Applying the equations which has been already derived

Gauss Elimination
• In the previous section, the elimination of unknowns was
used to solve a pair of simultaneous equations. The
procedure consisted of two steps:
1. The equations were manipulated to eliminate one of the

unknowns from the equations. The result of this
elimination step was that we had one equation with one
unknown.
2. Consequently, this equation could be solved directly and

the result back-substituted into one of the original
equations to solve for the remaining unknown.

Gauss Elimination
• This basic approach can be extended to large sets of
equations by developing a systematic scheme or algorithm
to eliminate unknowns and to back-substitute. Gauss
elimination is the most basic of these schemes.
For a 3x3 Sets

of Equations
Upper triangle matrix

Example
• Use Gauss Elimination to solve following set of Equations
• Solution
• Solution starts with writing coefficient matrix together with
constants
 3 0.1 0.2 7.85   The numbers with blue circles

 0.1 7 0.3 19.3  must be eliminated to get a
  equivalent upper triangle
 0.3 0.2 10 71.4  matrix for coefficient matrix

Example
• Elimination procedure is performed with row operations:

• Row operation: The members (aij) in any rows of matrix is
multiplied with any number and added to or subtracted
from another row. The obtained matrix is equivalent with
previous matrix (This is same procedure what we did in
the case of elimination of unknown method!!!)
• Elimination procedure is started from 1st column. Then
continue 2nd, 3rd , …..columns.
• Diagonal members of each column (aii’s (a11, a22, …)) are
pivot members.
members
• The other members that must be eliminated, are target
members
Example
• For instance, In the 1st column
Pivot member  3 0.1 0.2 7.85 
Target member  0.1 7 0.3 19.3 
 
Target member  0.3 0.2 10 71.4 
• To eliminate 0.1 and 0.3 (Target members of 1st column)

 Multiply 1st row
 3 0.1 0.2 7.85  by (-0.1/3) =(-Target/Pivot)
 0.1 7 0.3 19.3  then add 2nd row
 
 0.3 0.2 10 71.4   Similarly, multiply 1st row
by (-0.3/3) =(-Target/Pivot)
then add 3rd row

Example
• Then following form is obtained
3 0.1 0.2 7.85 

0 7  (0.1)  (0.1/ 3) 0.3  (0.2)  ( 0.1/ 3) 19.3  7.85  ( 0.1/ 3) 
 
0 0.2  (0.1)  (0.3 / 3) 10  ( 0.2)  (0.3 / 3) 71.4  7.85  ( 0.3 / 3) 
 Target numbers of 1st

3 0.1 0.2 7.85 
column have been
Pivot member
0 7.0033 0.2933 19.5617  eliminated.
 
0 0.1900 10.020 70.6150   Now, to eliminate -0.19, 2nd
Target member row must be multiplied by
(0.19/7.0033) then added
to 3rd row
Example
• Then following form is obtained:
3 0.1 0.2 7.85 

0 7.0033  0.2933  19.5617 
 
0 0 10.020  (0.2933)  (0.19 / 7.0033) 70.6150  (19.5617)  (0.19 / 7.0033) 
3 0.1 0.2 7.85   Now, the elimination

0 7.0033 0.2933 19.5617  procedure is completed
 
0 0 10.012 70.0843 

Example
• Back substitution
3 0.1 0.2 7.85  3x1  0.1x2  0.2 x3  7.85

0 7.0033 0.2933 19.5617  0 x1  7.0033x2  0.2933x3  19.5617
 
0 0 10.012 70.0843  0 x1  0 x2  10.012 x3  70.0843
70.0843 19.5617  0.2933  7

x3  7 x2   2.5
10.012 7.0033
7.85  0.2(7)  0.1(2.5)

x1  3
3
Example
• Solve the following system of equations using Gauss

Elimination
3 x1  x2  2 x3  12
x1  2 x2  3x3  11
2 x1  2 x 2  x3  2
• Solution
Pivot member  3 1 2 12   Multiply 1st row
1 2 3 11 by (-1/3) then add 2nd row
Target member
   Similarly, multiply 1st row
Target member  2 2 1 2  by (-2/3) then add 3rd row

Example
• Target members of 1st column are eliminated
• Note that: 1st row is not changed
3 1 2 12   Multiply 2nd row

0 7 / 3 7 / 3 7  by (4/7) then add 3nd row
 
0 4 / 3 7 / 3 6 
 3 1 2 12   x3  2  x3  2
0 7 / 3 7 / 3 7 
  7 7
x2  x3  7  x2  1
0 0 1 2  3 3
3x1  x2  2 x3  12  x1  3
Pitfalls of Elimination Methods
• Division by Zero: A division by zero can occur during
both the elimination and the back-substitution phases. For
example, if we use naive Gauss elimination to solve:
• The normalization of the first row would involve division

by a11 = 0. Problems also can arise when a coefficient is
very close to zero. The technique of pivoting has been
developed to partially avoid these problems.

• Round-Off Errors: The problem of round-off error can
become particularly important when large numbers of
equations are to be solved. This is due to the fact that every
result is dependent on previous results. Consequently, an
error in the early steps will tend to propagate that is, it will
cause errors in subsequent steps.
• To avoid the round off errors
 More significant figures must be used

 Fractions must be used instead of decimals

• Ill Condition Systems:
 Well conditioned systems: are those where a small change in
one or more of the coefficients results in a similar small change
in the solution.
 Ill-conditioned systems: are those where small changes in

coefficients result in large changes in the solution.
• Because round-off errors can induce small changes in the

coefficients, these artificial changes can lead to large solution
errors for ill-conditioned systems, as illustrated in the
following example.

• Example: Solve the following system:
x1  2 x2  10
1.1x1  2 x2  10.4
• Solution:
2(10)  2(10.4) 1(10.4)  1.1(10)
x1  4 x2  3
1(2)  2(1.1) 1(2)  2(1.1)
• However with the slight change of the coefficient a21 from 1.1 to 1.05,
the result is changed dramatically to
2(10)  2(10.4) 1(10.4)  1.05(10)
x1  8 x2  1
1(2)  2(1.05) 1(2)  2(1.05)

• The primary reason for the discrepancy between the two
results is that the denominator represents the difference of
two almost-equal numbers.
numbers
• Notice that denominator of equations are the determinant
of the coefficient matrix. As a result, an ill-conditioned
system is one with a determinant close to zero.
• It is difficult to specify how close to zero the determinant
must be to indicate ill-conditioning. This is complicated by
the fact that the determinant can be changed by multiplying
one or more of the equations by a scale factor without
hanging the solution. Consequently, the determinant is a
relative value that is influenced by the magnitude of the
coefficients.
• Example:
Example Evaluate the determinant of the following
systems:

• a.
Well conditioned
• b.
ill conditioned
• c.
• The multiplication of an equation by a constant has no effect on its

solution. In addition, it is still ill-conditioned. Not only has it been raised
two orders of magnitude, but it is now over twice as large as the
determinant of the well-conditioned system in (a)
• Scaling: The magnitude of the coefficients interjects a
scale effect that complicates the relationship between
system condition and determinant size. One way to
partially solve this difficulty is to scale the equations so
that the maximum element in any row is equal to 1.
• Example: Scale the systems of equations in previous

Example to a maximum value of 1 and compute their
determinants.

• Solution:

Techniques for Improving Solutions
• The following techniques can be incorporated into the
naive Gauss elimination algorithm to circumvent some of
the pitfalls discussed in the previous section
• Use of More Significant Figures:

• The simplest remedy for ill-conditioning is to use more
significant figures in the computation. If your application
can be extended to handle larger word size, such a feature
will greatly reduce the problem. However, a price must be
paid in the form of the computational and memory
overhead connected with using extended precision.

• Pivoting
• Consider the following linear set of equations
10 7 0   x1   7  0
 3 2.099 6   x   3.901
  2   x   1
 5 1 5   x3   6   1 
• The diagonal elements of coefficient matrix are called

pivots. The kth pivot is the coefficient of the kth variable in
the kth equation at the kth step of the elimination.
• Both the computation of the multipliers and back
substitution require divisions by the pivots.

• Consequently, the algorithm can not be carried out if any
of the pivots are zero. It is also a bad idea to
complete the computation if any of the pivots are nearly
zero.
• Let us apply the Gauss Elimination steps using hypothetical
machine that does floating point arithmetic with five
significant digits chopping
10 7 0 7  Multiply 1st row by
 3 2.099 6 3.901 3/10 then add 2nd row
  Multiply 1st row by -5/10
 5 1 5 6  then add 3rd row

10 7 0 7  Multiply 2nd row by

 0 0.001 6 6.001 2.5/0.001 then add 3rd row
 
 0 2.5 5 2.5 
10 7 0 7 
 0 0.001 6 6.001 
 
 0 0 15005 15004  2500  6.001  15002.5
2.5
15004.5

• From back substitution, the unknowns are calculated as
follows:
15005 x3  15004  x3  0.99993
0.001x2  6  (0.99993)  6.001

5.99958
0.001x2  5.9995  6.001
0.001x2  0.0015  x2  1.5
10 x1  (7)(1.5)  7  x1  0.35

• Hence the solution becomes x=[-0.35, -0.15, 0.99993]T
instead of x=[0, -1, 1] T
• Where did things go wrong?
• There was no accumulation of “rounding error” caused by

doing thousand of arithmetic operations.
• The matrix is not close to singular det(A)=-150.05
• The difficulty comes from choosing a small pivot at second

step of elimination (-0.001)
• Partial Pivoting
• Use Gauss elimination to solve
0.0003x1  3.0000 x2  2.0001

1.0000 x1  1.0000 x2  1.0000
• Note that in this form the first pivot element,

element a11 = 0.0003,
is very close to zero. Then repeat the computation, but
partial pivot by reversing the order of the equations. The
exact solution is x1 = 1/3 and x2 = 2/3.

• Solution:
0.0003 3.0000 2.0001 Multiply 1st row by -1/0.0003
1.0000 1.0000 1.0000  then add 2nd row
 
0.0003 3.0000 2.0001

 0  9999 x2  6666  x2  2 / 3
  9999  6666 
• This result can be substituted back into the first equation
to evaluate x1:
0.0003 x1  3 x2  2.0001 2.0001  3(2 / 3)

x1 
0.0003

• However, due to subtractive cancellation, the result is very sensitive to
the number of significant figures carried in the computation:
x2  2 / 3
• Note how the solution for x1 is highly dependent on the number of

significant figures. This is because we are subtracting two almost-
equal numbers.

• On the other hand, if the equations are solved in reverse
order, the row with the larger pivot element is normalized.
The equations are:
Multiply 1st row by -0.0003

1.0000 1.0000 1.0000  then add 2nd row
0.0003 3.0000 2.0001
 
1.0000 1.0000 1.0000 

 0  2.9997 x2  1.9998  x2  2 / 3
 2.9997 1.9998 
x1  x2  1 x1  1  x2
3  ( 0.0003) *1  3  0.0003  2.9997
• This case is much less sensitive to the number of
significant figures in the computation
• Thus, a pivot strategy is much more satisfactory

• Effect of Scaling on Pivoting and Round-Off
(a) Solve the following set of equations using Gauss elimination and a
pivoting strategy:
2 x1  100, 000 x2  100, 000

x1  x2  2
(b) Repeat the solution after scaling the equations so that the maximum
coefficient in each row is 1.
(c) Finally, use the scaled coefficients to determine whether pivoting is
necessary
(a) However, actually solve the equations with the original coefficient values.
For all cases, retain only three significant figures.
(b) Note that the correct answers are x1 = 1.00002 and x2 = 0.99998 or, for
three significant figures, x1 = x2 = 1.00.

(a) Without scaling, forward elimination is applied to give:
 2 100000 100000  Multiply 1st row by (-1/2) then add 2nd

1 row
 1 2 
1  (1 / 2) *100000  1  50000  49999
 2 100000 100000 
 0 49999 49998 49999 x  49998  x  0.99997999
  2 2
• Using three significant figures x2=1, then
2 x1  100000 x2  100000  x1  0

(b) Scaling transforms the original equations to
0.00002 x1  x2  1
x1  x2  2
Therefore, the rows should be pivoted to put the greatest
value on the diagonal
x1  x2  2
0.00002 x1  x2  1
 1 1 2 Multiply 1st row by -0.00002

0.00002 1 1  then add 2nd row
 

• Forward elimination yields
1 1 2 
0 0.99998 0.99996 
 
• Using three significant figures
1 1 2 
0 1 1  x2  1 and x1  x2  2  x1  1
 
• Thus, scaling leads to the correct answer.

(c) The scaled coefficients indicate that pivoting is necessary.
We therefore pivot but retain the original coefficients to give
Multiply 1st row by -1/2 then
1 1 2 
add 2nd row
 2 100000 100000 
 
1 1 2 
 0 99998 99996  99998 x  99996  x  0.99997999
  2 2
•Using three significant figures

x2  1 and x1  x2  2  x1  1
•Thus, scaling was useful in determining whether pivoting
was necessary, but the equations themselves did not require
scaling to arrive at a correct result.
Determinant Evaluation Using Gauss
Elimination
• The determinant of a triangular matrix can be simply
computed as the product of its diagonal elements
• Proof:
 a11 a12 a13 

A   0 a22 a23  det( A)  a11
a22 a23
 a12
0 a23
 a12
0 a22
 0 0 a33  0 a33 0 a33 0 0
det( A)  a11  a22  a33

Elimination
• Example: Determine the determinant of following matrix
 3 1 2 
A   1 2 3 
 2 2 1
• Apply row operations to reduce the matrix a triangular

matrix
 3 1 2  Multiply 1st row by -1/3 then

add 2nd row
A  1 2 3 
 2 2 1 Multiply 1st row by -2/3 then
add 3nd row

Elimination
• Then the matrix becomes
 3 1 2  Multiply 2nd row by (-(-4/3)/(7/3)=4/7)

0 7 / 3 7 / 3  then add 3nd row
 
0 4 / 3 7 / 3
 3 1 2 
 0 7 / 3 7 / 3 Then determinant is calculated
 
0 0 1 
7
det( A)  3   (1)  7
3

Gauss Jordan Method
• The Gauss-Jordan method is a variation of Gauss

elimination.
• The difference is that when an unknown is eliminated in
the Gauss-Jordan method, it is eliminated from all other
equations rather than just the subsequent ones.
• In addition, all rows are normalized by dividing them by
their pivot elements.
• Thus, the elimination step results in an identity matrix
rather than a triangular matrix.
• Consequently, it is not necessary to employ back
substitution to obtain the solution.

Gauss Jordan Method
• The method is best illustrated by an example. Graphical
depiction of the Gauss-Jordan method.
Forward elimination
Unit (identity)
matrix
Note that: No need

back substitution

Gauss Jordan Method
• Example: Use the Gauss-Jordan technique to solve the following
system of equations
3 x1  x2  2 x3  12
x1  2 x2  3 x3  11
2 x1  2 x 2  x3  2
• First normalize first row with respect to pivot member (divide all
members in the first row by 3 (pivot member))
1 1/ 3 2 / 3 4  Multiply 1st row by -1 then

1 2  add 2nd row
 3 11
 2 2 1 2  Multiply 1st row by -2 then
add 3nd row

Gauss Jordan Method
• Note that first column of the system becomes to first
column of unit (identity matrix)
1 1/ 3 2 / 3 4  Multiply 1st row by -1 then

0 7 / 3 7 / 3 7  add 2nd row
 
0 4 / 3 7 / 3 6  Multiply 1st row by -2 then
add 3nd row
Identical with 1st
column of unit matrix
• Now 2nd column should be converted to 2nd column of unit

matrix. To do this first make the pivot member of 2nd row 1
dividing entire row member by 7/3

Gauss Jordan Method
• Then eliminate members other members of 2nd row (a12
and a32) except pivot member a22
Multiply 2nd row by 1/3 then
1 1/ 3 2 / 3 4  add 1st row
0 1 1 3 
  Multiply 2nd row by -4/3 then
0 4 / 3 7 / 3 6 add 3rd row
1 0 1 5 
0 1 1 3 
 
0 0 1 2
Identical with 1st
column of unit matrix

Gauss Jordan Method
• Now 3rd column should be converted to 3rd column of unit
matrix. To do this first make the pivot member of 3rd row 1
dividing entire row member by -1
Multiply 3rd row by -1 then
1 0 1 5  add 1st row
0 1 1 3
  Multiply 3rd row by -1 then
0 0 1 2 add 2nd row
• Finally, coefficient matrix is converted to unit matrix.

1 0 0 3  x1  3
0 1 0 1 
  Solution x2  1
0 0 1 2 x 2 3
Unit matrix
Gauss Jordan Method
• Home Exercise: Use the Gauss-Jordan technique to

solve the same system
• Answer:
x1  3
x2  2.5
x3  7

Inverse of A Matrix Using Gauss Jordan
• Gauss Jordan Method is also used to calculate inverse of a
matrix. Let us assume a 3x3 square matrix A.
 a11 a12 a13 
A   a21 a22 a23 
 a31 a32 a33 
• First, the matrix and identity matrix is written side by side
a11 a12 a13 1 0 0
a21 a22 a23 0 1 0
a31 a32 a33 0 0 1
[A] [I]

• Then row operations are applied to convert the matrix of A

to an identity matrix.
• Same operations are applied to identity matrix.
• Finally, The identity matrix is converted to inverse of A [A]-1
when the matrix A is converted to an identity matrix.
a11 a12 a13 1 0 0 Row 1 0 0 a111 a12 1 a131

a21 a22 a23 0 1 0 operations 0 1 0 a211 a22 1 a231
a31 a32 a33 0 0 1 0 0 1 a311 a32 1 a331
[A] [I] [I] [A]-1

• Example: Calculate the inverse of following matrix using
Gauss Jordan Method.
 3 1 2 
A  1 2 3 
 2 2 1
3 1 2 1 0 0 Divide 1st row by 3
1 2 3 0 1 0 Multiply 1st row by -1 then add 2nd row
Multiply 1st row by -2 then add 3nd row

2 2 1 0 0 1
1 1/ 3 2 / 3 1/ 3 0 0 Divide 2nd row by 7/3
0 7 / 3 7 / 3 1/ 3 1 0 Multiply 2nd row by 1/3 then add 1st row
Multiply 2nd row by 4/3 then add 3nd

0 4 / 3 7 / 3 2 / 3 0 1 row
1 0 1 0.2857 0.1429 0 Divide 3nd row by -1
0 1 1 0.1429 0.4286 0 Multiply 3nd row by -1 then add 1st row
Multiply 3nd row by -1 then add 2nd row

0 0 1 0.8566 0.5713 1
• Finally, the matrix A became the identity matrix.
1 0 0 0.5713 0.7141 1
0 1 0 1 1 1
0 0 1 0.8571 0.5713 1
[I] [A]-1
• The identity matrix (right hand side) became the inverse of A

System Condition
• The inverse of a matrix provides a means to identity
whether systems are ill-conditioned. Three methods are
available for this purpose
• 1. Scale the matrix of coefficients [A] so that the largest element in

each row is 1. Invert the scaled matrix and if there are elements of
[Ascaled]−1 that are several orders of magnitude greater than one, it is
likely that the system is ill-conditioned
• 2. Multiply the inverse by the original coefficient matrix and assess
whether the result is close to the identity matrix. If not, it indicates ill-
conditioning.
• 3. Invert the inverted matrix and assess whether the result is
sufficiently close to the original coefficient matrix. If not, it again
indicates that the system is ill-conditioned.
Condition Number
• Although these methods can indicate ill-conditioning, it would be
preferable to obtain a single number (condition number) that could
serve as an indicator. Attempts to formulate such a matrix condition
number are based on the mathematical concept of the norm.
• Vector and Matrix Norms:

• A norm is a real-valued function that provides a measure of the size or
“length” of multicomponent mathematical entities such as vectors and
matrices
• Euclidean norms can be employed to quantify the size of a vector

n
X   i 
x 2
i 1
x12  x2 2  ...  xn 2

Condition Number
• Or a matrix;
n n
A  a
i 1 j 1
ij
2
 a112  a12 2  ...a212  a22 2  ...  ann 2
• Condition Number:
We have introduced the concept of the norm, we can use it to define:
Cond  A  A  A1
• Here
A  norm of coefficent matrix ( A)
A1  norm of inverse of coefficent matrix ( A1 )
• Note that the larger the condition number is, the greater the system’s ill
conditioning

Condition Number
• Example: Determine the norm and condition number of

following matrix
 6 2 5 
A   8.5 1.1 3.5
 1.6 1 10.3 
• Solution:
• Norm of [A]:
3 3
A   a
i 1 j 1
ij
2
 62  (2) 2  52  (8.5)2  (1.1)2  (3.5)2  (1.6)2  (1)2  (10.3)2  16.1357

Condition Number
• Norm of [A-1]
 0.0442 0.088 0.0085 

A1   0.4625 0.3939 0.3584 
 0.038 0.0519 0.1332 
3 3
A 1
  (a 1
)  (0.0442)2  (0.088)2  ...  (0.1332)2  0.7275
ij
2
• Condition
i 1 j 1
number for [A]
• Cond  A number
Condition A A is notso
1
16.1357  0.7275accordingly
large number,  11.74 system is well-conditioned

Condition Number
• Example: Determine the norm and condition number of
following matrix.
 1 1/ 2 1/ 3 
 A  1/ 2 1/ 3 1/ 4
1/ 3 1/ 4 1/ 5 
n n
A   i j
a 2
i 1 j 1
 12
 (1 / 2) 2
 (1 / 3) 2
 ...  (1 / 5) 2
 1.41
 9 36 30 
 A1    36 192 180 
 
 30 180 180 
n n
A 1
  (a
i 1 j 1
1
ij )2  92  (36) 2  (30) 2  ...  (180) 2  372.2

Condition Number
• Condition number
Cond  A  A  A1  1.41 372.2  526.15
• Condition number is so large number, accordingly system is ill-

conditioned

Example
• Example: Given the equation
0.5 x1  x2  9.5
1.02 x1  2 x2  18.8
 a) Compute the determinant of coefficient matrix

 b) What is the system condition
 c) Solve system by elimination method
 d) Solve again, but with a11 modified slightly to 0.52.
Interpret your result.
• Solution:
 a) 0.5 1
det( A)   1  (1.02)  0.02
1.02 2
Example
 b) Determinant is too small, therefore system is ill-
condition
 c)
0.5 1 9.5 Multiply 1st row by -1.02/0.5 then add 2nd row
1.02 2 18.8
0.5 1 9.5 38.18

x2   954.5
0 0.04 38.18 0.04
9.5  954.5
x1   1890
0.5

Example
 d)
0.52 1 9.5 Multiply 1st row by -1.02/0.52 then add 2nd row
1.02 2 18.8
0.52 1 9.5 37.976

x2   949.4
0 0.04 37.976 0.04
9.5  949.4
x1   1844.04
0.52

Lecture Notes 3

Uploaded by

Lecture Notes 3

Uploaded by

CHAPTER 3

System of Linear Algebraic

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• This subject deals with simultaneous algebraic equations which

a11 x1  a12 x2  ...  a1n xn  b1

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• In this section, we will describe several methods that are

1. The graphical method

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• A graphical solution is obtainable for two equations by

• If x2 is solved from each equations

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Thus, the equations are now in the form of straight lines;

• These lines can be graphed on Cartesian coordinates with

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Use the graphical method to solve

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Then intersections of lines

x1=4 and x2=3

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Cramer’s rule is another solution technique that is best

• Before describing this method, we will briefly introduce

• In addition, the determinant has relevance to the

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• The value of the second-order determinant

• The value of the third-order determinant

*2 by 2 determinants are called as minors

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• This rule states that each unknown in a system of linear

• the denominator D (Determinant of Coefficient Matrix)

• the numerator obtained from D by replacing the column of

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

a11 x1  a12 x2  a13 x3  b1

 a11 a12 a13   x1   b1 

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Finally, unknowns x1, x2 and x3 are then calculated as a

Dx1 Dx2 Dx3

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

a21a11 x1  a21a12 x2  a21b1

• which can be solved for

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

1. The equations were manipulated to eliminate one of the

2. Consequently, this equation could be solved directly and

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

For a 3x3 Sets

Upper triangle matrix

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

 3 0.1 0.2 7.85   The numbers with blue circles

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• Elimination procedure is performed with row operations:

• To eliminate 0.1 and 0.3 (Target members of 1st column)

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

3 0.1 0.2 7.85 

 Target numbers of 1st

3 0.1 0.2 7.85 

3 0.1 0.2 7.85   Now, the elimination

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

3 0.1 0.2 7.85  3x1  0.1x2  0.2 x3  7.85

70.0843 19.5617  0.2933  7

7.85  0.2(7)  0.1(2.5)

• Solve the following system of equations using Gauss

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

3 1 2 12   Multiply 2nd row

• The normalization of the first row would involve division

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• To avoid the round off errors

 More significant figures must be used

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

 Ill-conditioned systems: are those where small changes in

• Because round-off errors can induce small changes in the

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

• The multiplication of an equation by a constant has no effect on its