Applied Mathematics and
Computation
Unit 3
Simultaneous Linear Equations
1
2
A set of Linear Algebraic Equations
• as are constant coefficient
• bs are constants
• Xs are unknowns
• n is number of unknown and number of equations
3
In Matrix Form
4
Linear Algebraic Equations
• Laws of conservation
– Mass, momentum and energy
– Multi-component system
– Discretized differential equation
• Equilibrium equation & Compatibility equation
• Kirchhoff's law
5
Solution methods
• Graphical
• Crammer rule
• Elimination of unknowns
• Gauss elimination
• Gauss Jordon
• LU decomposition
• Gauss Seidel
6
Graphical method
• Limited to three equations
7
Graphical method
8
Graphical Method
(a) no solution
(b) infinite solutions
(c) ill-conditioned system (difficult to detect intersection point)
9
Cramer’s Rule
• Determination of determinant is time
consuming, as number of equations increases 10
Elimination of unknown
• Same as Cramer’s Rule
11
Gauss Elimination
Multiply Eq (1) by a factor and subtract it from Eq (2).
12
Gauss Elimination
13
Gauss Elimination
(a) Forward elimination
(b) Backward substitution
14
Example
Multiply Eq (1) by (0.1/3) and subtract from Eq (2)
Multiply Eq (1) by (0.3/3) and subtract from Eq (3)
15
Multiply Eq (2) by (-0.19/7.00333) and subtract
from Eq (3)
16
• Backward substitution
x2 = -2.5
x1 = 3.0
17
Operation counting
• Forward elimination
• Backward substitution n n 1 2
18
Operation counting
21
Pitfall-I
• Pivot element is zero
• Example
• Factor = 4/0, 2/0
• Division by zero
22
Pitfall-II
• Pivot element is nearly zero
• Tolerance be specified in the program to detect this
condition
• Example-II
• Factor = 1/0.0003
• Division by a very small number
23
• X1 = (2.0001-3*0.7)/0.0003 = -333
• X1 = (2.0001-3*0.67)/0.0003 = -33
• X1 = (2.0001-3*0.667)/0.0003 = -3.33
24
Pitfall-III
• Round off error
– Important when dealing with 100 or more equations,
due to error propagation
• Remedy
– Increase significant figures
– This increases computation efforts
25
Remedy
• Pivoting
– Switching largest element to pivot position
• Complete pivoting
– If columns as well as rows are searched for the
largest element and both are switched.
• Partial pivoting
– If only column below pivot element is searched for
largest element and corresponding row is switched.
• Complete pivoting is rarely used because
switching columns change the order of Xs. Add
complexity in program
26
• In programming, rows are not switched
• But track of the pivot row number is kept
• Defines order of forward elimination and
backward substitution without changing place
• Moreover, Scaling is required to identify
whether pivoting is necessary
27
Example
• Without Scaling &Pivoting With scaling & pivoting
• Forward elimination Pivoting
49999 x2 49998
• x2 0.99998 1.0 Forward elimination
x1 1.00002 0.0
0.99998 x2 0.99996
x2 0.99998 1.0
• Round off error x1 1.00002 1.0
• Without scaling and with pivoting
99998 x2 99996
x2 0.99998 1.0
x1 1.00002 1.0
• Scaling should be done to identify the need of pivoting. Pivoting
and Gauss elimination be done with the original coefficient
values to avoid roundoff error introduced by the scaling.
Pitfall-IV
• Singular systems
– Slope are same
– Graphically, they will either be parallel or superimpose
– No solution or infinite solution
– Computer algorithm should recognize this case
– Determinant is zero
a11 a21
a11 a22 a21 a12 a11 a22 a21a12 0
a12 a22
Pitfalls
• Ill-conditioned equations
– Slope of equations are nearly same
– Determinant (denominator) is nearly zero
Determinant determination
• Example 1: well conditioned problem
• Example 2: Ill conditioned problem
• Example 3: Ill conditioned problem
• The value of determinant is not right indicator
of ill conditioned equation. 36
Determinant determination with scaling
• Example 1: well conditioned problem
• Example 2: Ill conditioned problem
• Example 3: Ill conditioned problem
• The value of determinant after scaling is right
indicator of ill conditioned equation. However,
difficult to determine for more than 3 equations 37
Another Indicator
• Ill-conditioned equations: matrix
– Scale the matrix of coefficient [A] so that largest
element in each row is 1.
– Invert the matrix.
– If there are elements that are several order of
magnitude greater than one => Ill conditioned
38
Other indicators
• If multiplication of the inverse with the
original matrix is not close to identity matrix
=> ill condition
• If the invert of the inverted matrix is not close
to original matrix => ill condition
• Computationally inefficient methods
39
Another indicator
• Ill-conditioned equations:
– A small changes in coefficient, results in large
changes in solution
• Example
• A change of <5% in coefficient, brings a change in x1 by 100 %40
Pitfalls
• Ill-conditioned equations: When matrix condition
number is greater than unity
x A
Cond A
x A
≥1
A matrix that is not invertible has condition
number equal to infinity.
Matrix norm
• The norm of a matrix is a non-negative real number
which measures how large its elements are. It is a
way of determining the “size” of a matrix that is not
necessarily related to how many rows or columns the
matrix has but the magnitude of the matrix
• Frobenius (Euclidean) norm
(Root Sum Square)
• Infinity norm
(Max Absolute Row Sum)
• 1-norm
(Max Absolute Column Sum)
Example
45
46
Gauss Jordon
• Divide to make pivot element equal to 1
• Full elimination in all the (n-1) equations, not only
forward elimination
• Results in identity matrix instead of upper triangular
matrix.
• No need of backward substitution
49
50
• Multiplication: (n)n+(n-1)n+(n-2)n…=(n(n+1)/2).n=n3/2+n2/2
• i.e., S number of terms in each equation (n-i+1) * number of equations (n)
• Addition: n(n-1)+(n-1)(n-1)+(n-2)(n-1)…=(n.(n+1)/2)(n-1)=n3/2-n/2
• i.e., S number of terms in each equation (n-i+1) * number of equations (n-1)
• Number of flops
• Computation 50% more than Gauss elimination
Special matrices
• Banded Matrix
– Tridiagonal Matrix
(ODE discretization)
• Sparse Matrix
(PDE discretization)
Tri-diagonal Matrix
57
Iterative procedure
• Initial guess may be taken as zero or 1
• Stopping criteria
58
Jacobi Method
Parallelization possible
x1 = 0 2.616667 3.000762 3.000806 3.000022 2.999999 3
x2 = 0 -2.75714 -2.48852 -2.49974 -2.5 -2.5 -2.5
x3= 0 7.14 7.006357 7.000207 6.999981 6.999999 7
Gauss Seidel Method
Parallelization difficult
x1 = 0 2.616667 2.990557 3.000032 3
x2 = 0 -2.79452 -2.49962 -2.49999 -2.5
x3= 0 7.00561 7.000291 6.999999 7
Convergence Condition
• Diagonally dominant
63
64
Reducing flops
• Pre-divide the coefficient of the row with the
diagonal element
65
Relaxation
• l = 0 to 1 under relaxation (damping)
• l = 1 to 2 over relaxation (moving in right direction)
As obtained by Jacobi or
As obtained after Gauss-Seidel Methods
applying relaxation factor
