CHAPTER 3

System of Linear Algebraic

Equations

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method
• Consider the following system of linear equations:
a11 x1  a12 x2  ...  a1n xn  b1
a21 x1  a22 x2  ...  a2 n xn  b2
. . . .
. . . .
. . . .
an1 x1  an 2 x2  ...  ann xn  bn
• This equation can be written by solving the ith iteration for
the unknown xi (i=1,2,3…,n) as follows:

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method
1
x1  [b1  a12 x2  a13 x3  ...  a1n xn ]
a11
1
x2  [b2  a21 x1  a23 x3  ...  a2 n xn ]
a22
.
.
1
xn  [bn  an1 x1  an 2 x2  ...  ann 1 xn 1 ]
ann
•These equations can be expressed in a compact form

1 n
xi  [bi 
aii

j 1, j  i
aij x j ] ; i  1, 2,3,..., n

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method

• In the Jacobi iteration method, we start with a set of initial

(approximate) values, x1(1), x2(1),…,xn(1) for x1,x2,…,xn respectively. If no
better initial estimate is available, we can assume each component to
be zero [i.e. xi(1)=0, (i=1,2,…,n)]. The assumed solution xi(1), is
substituted into the right hand side of eqn. to generate a new solutions

1 n

i   aij x j ] , i  1, 2,..., n
(2) (1)
• This iterative procedure
xi  is[bcontinued
aii j 1, j  i

1 n
xi ( k 1)
 [bi   aij x j ( k ) ] , i  1, 2,..., n k  1, 2,...
aii j 1, j  i

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method

• Until
xi ( k 1)  xi ( k )
( k 1)
  s i  1, 2,3...
xi

• Where ε is a small member. It has been found that a

sufficient condition for the convergence of Jacobi method
is
n
aii  
j 1, j  i
aij (Diagonally Dominant)

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method
• Example: Determine whether the following system of
equations is diagonally dominant if it is, find its solution
using Jacobi iteration method.
5 x1  x2  2 x3  1
2 x1  x2  7 x3  32
2 x1  6 x2  3 x3  2

• As it can be seen second and third equations are not large

with the other coefficients of the same equations. However,
interchanging 2nd and 3rd equations yields he following
system

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method
5 x1  x2  2 x3  1
2 x1  6 x2  3 x3  2
2 x1  x2  7 x3  32
• Then, this system is diagonally dominant:
5  1  2 5  3
6  2  3 6  5
7  2  2 73

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method
• Solving equations for x1, x2 and x3 respectively:
1
x1   (1  x2  x3 )
5
1
x2  (2  2 x1  3 x3 )
6
1
x3  (32  2 x1  x2 )
7

• Choosing initial approximation for variables as:

xi (1)  0 , (i  1, 2, 3)
1
x1(2)   (1  0  2  0)  0.2
5
1
x2 (2)  (2  2  0  3  0)  0.333
6
1
x3(2)  (32  2  0  0)  4.5714
7 CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN
Jacobi Iteration Method
• These values are again substituted into the right hand sides

1
x1(3)   (1  0.333  2  4.5714)  1.5619
5
1
x2 (3)  (2  2  (0.2)  3  4.5714)  2.6857
6
1
x3(3)  (32  2  ( 0.2)  0.333)  4.5810
7

• If the iterations are continued, following results are

obtained:

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Jacobi Iteration Method

No of iteration x1 x2 x3
1 0 0 0
2 -0.2000 0.3333 4.5714
3 1.5619 2.6857 4.5810
4 1.0952 2.1032 3.7415
5 0.8760 1.8390 3.9580
. . . .
. . . .
. . . .
11 0.9998 1.9997 3.9991

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Gauss-Seidel Method
• It can be seen that in Jacobi iteration method, all the
variables xi(k+1) are computed using the values of the
previous xi(k). This method can be improved by using Gauss-
Seidel iteration method in which the values of x1(k+1), x2(k+1),
…, xi(k+1) computed in the current iteration, along with the
values of xi+2(k), xi+3(k),…, xn(k) , are used in finding the xi+1(k+1) .

1 i 1 n
xi ( k 1)
 [bi   aij x j ( k 1)
  aij x j ( k ) ] , i  1, 2,..., n k  1, 2,...
aii j 1 j  i 1

• The method requires an initial guess for the values of the

unknowns x2, x3, x4,…, xn
CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN
Gauss-Seidel Method
• Example: Find the solution of the following equations
using the Gauss-seidel iteration method.
1
x1   (1  x2  x3 )
5
1
x2  (2  2 x1  3 x3 )
6
1
x3  (32  2 x1  x2 )
7

• By choosing the initial guess as x2(1)=0 and x3(1)=0 and

substituting them into the right-hand side
1
x1(2)   (1  0  2  0)  0.2
5

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Gauss-Seidel Method
• By substituting x1(2)  0.2 and x3(1)  0
(2) 1
x2  (2  2  (0.2)  2  0)  0.4
6
• Next, x1(2)  0.2 and x2 (2)  0.4

1
x3(2)  (32  2  (0.2)  0.4)  4.5714
7

• These values are again used to generate xi(3) i=1,2,3…

using general iterations, the following values in Table is
obtained.

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Gauss-Seidel Method

No of iteration x1 x2 x3
1 0 0 0
2 -0.2000 0.4 4.5714
3 1.5586 2.1029 3.8286
4 0.9109 1.9440 4.0335
5 1.0246 2.0085 3.9918
6 0.9950 1.9975 4.0018
7 1.0012 2.0005 3.9996
8 0.9997 1.9999 4.0001

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Newton-Raphson Method for System
of Nonlinear Equations
• Many engineering analyses require the solution of a set of
simultaneous nonlinear algebraic equation. The Newton-
Raphson method can be extended to solve system of
nonlinear equations:
f1 ( x1 , x2 , ..., xn )  0
f 2 ( x1 , x2 ,..., xn )  0
. . . . . .
. . . . . .
. . . . . .
f n ( x1 , x2 ,..., xn )  0

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Newton-Raphson Method for System
of Nonlinear Equations
• If the functions f1, f2, f3,…,fn are expanded in Taylor’s series
about an arbitrary (guess or approximate) solution (x1, x2,
… ,xn) and if only the linear terms are retained, we can
obtain the values of the functions at the exact solution
( x1 , x2 ,..., xn ) as:

f1 f f
f1 ( x1 , x2 ,..., xn )  f1 ( x1 , x2 ,..., xn )  ( x1  x1 )  1 ( x2  x2 )  ...  1 ( xn  xn )
x1 x2 xn
.
.
f n f n f n
f n ( x1 , x2 ,..., xn )  f1 ( x1 , x2 ,..., xn )  ( x1  x1 )  ( x2  x2 )  ...  ( xn  xn )
x1 x2 xn

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Newton-Raphson Method for System
of Nonlinear Equations
• where partial derivatives are evaluated at (x1,x2,…,xn). By
setting above equations to zero and denoting
xi  xi  xi (i  1, 2,..., n)
 f1 f1 f1   x1    f1 
...
 x x2 xn    
 


 1 
 f 2 f 2 f n   x2    f 2 
 x ...    
x2 x2   
  
 1 
 : : ::: :  :   : 
    
 f n f n
...
f n   xn    f n 
 x1 x2 xn    
 


fi
• where the partial derivatives x and the function fi are
j

evaluated at the current approximation (x1, x2,…, xn).

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN
Newton-Raphson Method for System
of Nonlinear Equations
• The matrix of first partial derivatives of fi is called as the
Jacobian of fi. Solution of above linear equations yields the
unknowns Δx1 ,Δx2 ,…, Δxn. Then the solution at the next
iteration; ( i 1) (i ) (i )
x1  x1  x1
x2 ( i 1)  x2 ( i )  x2 ( i )
. . .
. . .
. . .
xn ( i 1)  xn ( i )  xn ( i )
CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN
Newton-Raphson Method for System
of Nonlinear Equations
• Example: Use the multiple-equations of Newton
Raphson method to determine roots of the following
equations. Initial values are x1(1) =1.5 and x2(1) =3.5.
f1 ( x1 , x2 )  x12  x1 x2  10  0
f 2 ( x1 , x2 )  x2  3 x1 x2 2  57  0
• Solution: First compute the partial derivatives
(1) (1)
f1  f  f 2  f 
 2 x1  x2   1   2  1.5  3.5  6.5  3 x2 2   2   3(3.5) 2  36.75
x1  x1  x1  x1 

(1) (1)
f1  f1  f 2  f 
 x1    1.5  1  6 x1 x2  2   1  6  (1.5)  (3.5)
x2  x2  x2  x2 

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Newton-Raphson Method for System
of Nonlinear Equations
f1 ( x1 , x2 ) (1)  (1.5) 2  (1.5)  (3.5)  10  2.5
f 2 ( x1 , x2 ) (1)  (3.5) 2  (3)  (1.5)  (3.5) 2  57  1.625

• First Iteration
 f1 f1 
 x   x1    f1 
x2     6.5 1.5   x1(1)   2.5 
 1  36.75 32.5  (1)    1.625
 f 2 f 2         x2   
 x  x    f 
 1 x2   2   2 
x1(1)  0.536
x2 (1)  0.6561
x1(2)  x1(1)  x1(1)  1.5  0.536  2.036
x2 (2)  x2 (1)  x2 (1)  3.5  0.6561  2.8439
CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN
Newton-Raphson Method for System
of Nonlinear Equations
• Second Iteration
 f1 f1 
 x  x    f 
x2   1   1   6.9159 2.0360   x1(2)   0.0645 
 1   24.2633 35.7411  (2)    4.7560 
 f 2 f 2        x2   
 x  x    f 
 1 x2   2   2 
x1(2)  0.0373
x2 (2)  0.1584

x1(3)  x1(2)  x1(2)  2.036  0.0373  1.9987

x2 (3)  x2(2)  x2(2)  2.8439  0.1584  3.0023

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN

Newton-Raphson Method for System
of Nonlinear Equations
• Third Iteration
 f1 f1 
 x  x    f 
x2   1   1   6.9997 1.9987   x1(3)   0.0045 
 1   27.0414 37.0042   (3)    0.050 
 f 2 f 2        x2   
 x  x    f 
 1 x2   2   2 
x1(3)  0.0013
x2 (3)  0.0023

x1(4)  x1(3)  x1(3)  1.9987  0.0013  2.000

x2 (4)  x2 (3)  x2(3)  3.0023  0.0023  3.000

CENG 202: NUMERICAL ANALYSIS, DR. SEMIH ERHAN