Chapter 1| Introduction to Numerical Methods

INTENDED LEARNING OUTCOMES:

At the end of this chapter, it is expected that the students will be able to
1. Define and recognize the difference between numerical method and analytical
method. 2. Familiarize themselves with error and error analysis.
3. Understand the concept of significant figures.
4. Differentiate “accuracy” and “precision”.

01| Introduction

Generally, we can divide mathematics into two broad categories the pure mathematics and the
applied mathematics.  Physics
 Engineering Economy
The simplest way to distinct these two broad categories of math is that pure mathematics is the study  Mechanics of Rigid Bodies
of math independent of any application outside mathematics. In other words, it is the math solely for  Mechanics of Deformable Bodies
its own sake.  Circuits 1, 2 and 3 (Complex Numbers, System of Linear Equations, Laplace Transform,
Subject under the category of pure mathematics commonly taken by college students Fourier Series, Gamma Function, etc.)
from:  College Algebra  Electromagnetics
 Advance Algebra  Vector analysis
 Plane and Spherical Trigonometry  Electrical Machines
 Solid Mensuration  Power Plant Engineering
 Analytical Geometry  Power System Analysis (Fault Analysis, Load Flow Analysis, Optimization, Surge and
 Differential Equations Transient Analysis, Arc Flash Analysis)
 Integral Calculus
Numerical analysis involves the study of methods of computing numerical data. In many problems
On the other hand, applied math is the application of mathematical methods in different fields (such this implies producing a sequence of approximations by repeating the procedure again and again.
as engineering). In short, it is the use of math for practical applications here is also the list of People who employ numerical methods for solving problems have to worry about the following
subjects under applied math category from: issues: the rate of convergence (how long does it take for the method to find the answer), the
 Differential Equations (AC and DC Transient) accuracy (or even validity) of the answer, and the completeness of the response (do other solutions,
 Advanced Engineering Mathematics in addition to the one found, exist).
 Probability and Statistics
 Numerical Methods Numerical methods provide approximations to the problems in question. No matter how accurate
 Chemistry they are they do not, in most cases, provide the exact answer. In some instances, working out the
exact answer by a different approach may not be possible or may be too time consuming and it is in
these cases where numerical methods are most often used.
Generally speaking, Numerical Metod is the consolidation of pure mathematics as one subject it is iterative solution in short, a trial-and-error approach until we arrive in a value of x where it will
the study of algorithm or process that utilize the numerical approximation to solve mathematical satisfy the equation or the so-called convergence.
problems.
x୬ାଵ = x୬ −f(x୬)
We can say that most of the solution under the pure math are exact solutions while numerical
fᇱ(x୬)
method is one of the approximate solutions to solve alternatively a mathematical problem.
Using the Newton-Raphson Method,
So, to make it clear say for example that we are finding the roots of the polynomial

x୬ାଵ = x୬ −x୬ଶ + 8x୬ + 15

equation: xଶ + 8x + 15 = 0
2x୬ + 8

If we apply factoring to the equation, we get From this equation we can now refine our initial guess by substituting x୬ = 2, and the refined value of
x or in this case xଵ = −0.91667.
(x + 3)(x = 5) = 0

xଶ = 2 −(2)ଶ + 8(2) + 15
Therefore, we can say that the roots of the equation are,
2(2) + 8 = −0.91667
x = −3
and Obviously, this value is still far away from our expected value. But if you will compare it to our initial it
x = −5. is quite nearer than expected one. So, by continuously updating our x୬ after few iterations we
managed to arrive at the expected value of the root.
This is the Exact Solution.

xଶ = −0.9167 −(−0.91667)ଶ + 8(−0.91667) + 155

Exact or Analytical Solution gives us the exact answer just like in the previous problem.
For example, the algorithm of Newton-Raphson Method we can refine our initial guess by means of
Whereas in numerical methods, we don’t need to simplify 2(−0.91667) + 8
the given equation, but instead we will start the = −2.29617

solution by giving an initial guess of what might be the roots.

xଷ = −2.29617 −(−2.29171)ଶ + 8(−2.29617) + 155

Let us say that x = 2 or any number you can think of. 2(−2.29617) + 8
= −2.85463

If we substitute 2 to the given equation,

xସ = −2.85463 −(−2.85463)ଶ + 8(−2.85463) + 155

xଶ + 8x + 15 = 0
 2(−2.85463) + 8
= −2.99077

(2)ଶ + 8(2) + 15 ≠ 0

xହ = −2.99077 −(−2.99077)ଶ + 8(−2.99077) + 155

Obviously, the equation will not be satisfied but remember processed under numerical methods. 2(−2.99077) + 8
that is only our initial guess using algorithms are
= −2.99996
x଺ = −2.9996 −(−2.9996)ଶ + 8(−2.9996) + 155 Non-Computer Methods
Computers are undeniably one of the greatest inventions of computational capabilities, the extensive accessibility of
= −3.00000
man. Many things which cannot be done manually are made computers (especially laptops and personal computers) and
2(−2.9996) + 8
possible by these machines. Aside from providing improved
Most of the solution and their numerical methods will be handled the same way as this example  Partial Differential Equation which includes the solution of finite elements
since numerical methods are iterative solutions are simply a solution that requires continuous
repetition until a condition of a particular method is satisfied.
Now this solution or approach is very helpful especially when we are dealing with transcendental 02| Modeling, Computers and Error Analysis
equations or equations that involve trigonometric function or logarithmic functions. Since it is too
difficult to simplify the equation to extract the value of the roots, for example, Numerical methods are techniques used in the formulation of mathematical problems so that they
can be solved with arithmetic operations. Although there are many types of numerical methods, they
3 + sin x − e୶ = 0
have one mutual characteristic: they always contain large numbers of tedious arithmetic
computations. With the increased development of fast and efficient computers, the role of numerical
Lastly numerical method is important because this is the way how computers solve our mathematical
methods in solving engineering problem has increased dramatically in recent years.
problems and we are now living in a computerized world whereas doing a daily job routine that will
their partnership with numerical methods has had a substantial impact on the actual engineering
take a week to get done can now be shortened by incorporating computer algorithms like
problem-solving process. In the pre-computer era there were generally three different ways in which
engineering designs.
engineers approached problem solving. In general, we can say that the Three Different ways in
Which Engineers solved problems can be categorized as follows:
They are so tedious you will surely spend a lot of time in front of your computer but with the advent of
1. Solutions were derived from some problems by using analytical or exact
technology you can do it smarter.
methods. 2. Graphical Solution were utilized to identify behavior of systems
3. Calculators and slide rules were used to implement numerical methods
Learning numerical methods is essential as well as exciting since we understand the background
calculation of our computer and also, we can write our own algorithms to solve different problems.
Figure 1.1 shows the comparison between the phases of engineering problem solving in the pre
computer and the computer era.
So, throughout this course we will going to tackle these topics from:
 System of Equations - determination of roots of nonlinear and linear
equations  Function Approximation and Interpolation, Curve Fitting
 Numerical Differentiation and Integration Ordinary Differential Equation
 03| Mathematical Modeling and Engineering Problem Solving

To effectively and successfully implement any tool, knowledge and understanding are needed. For
example, it will be hard to repair a car if you do not understand how the tool chest works.

This is particularly true when using computer to solve engineering problems. Though computers
have great potential utility, it will be useless without the basic knowledge and understanding of how
engineering system works.

So basically, what is Mathematical Modelling?

Mathematical modeling is any physical problem which you have in your daily life. If you convert that
physical problem into by applying the basic principles of science basically known as the
mathematical modelling.

Example of a mathematical model used in electrical engineering are:

1. Power Flow Equations

Figure 1.1 The three phases of engineering problem solving (a) the pre-computer era and (b) the computer era

Numerical Methods and Engineering Practice

The extensive accessibility of digital computers since the late 1940s has led to an absolute explosion 2. Ohm’s Law
in the use and development of numerical methods. During earlier times, growth was limited by the 3. Optimization Models
cost of large mainframe computers, therefore engineers are left to use simple analytical approaches 4. Load Forecasting
in important parts of their work. But the recent evolution and availability of inexpensive and portable
laptop computers has given us the powerful computational capabilities.
04| A Simple Mathematical Model
Additional reasons why we should study numerical methods are:
1. Numerical methods are undoubtedly one of the most powerful tools A mathematical model can be broadly defined as a formulation or equation that expresses the
available 2. Commercially computer programs involve numerical methods. essential features of a physical system or process in mathematical terms. In a very general sense, it
3. Many problems cannot be solved using the readily available computer can be represented as a functional relationship of the form
programs 4. Numerical methods are an efficient vehicle for learning to use
computers. Dependent Variable = f{independent variables, parameters, forcing functions}
5. Numerical methods deliver a way for you to strengthen your understanding of mathematics.
We can actually convert the physical models into a mathematical equation or the physical
phenomenon into a mathematical equation. So, we have some dependent variables and some
dependent independent variables

Let us discuss each component one by one.

 Dependent variable is a characteristic that shows the behavior or state of the system 
Independent variable indicates the dimensions, such as time and space, along with the
system’s behavior is being determined,
 Parameters are reflective of the system’s properties or composition, and
 Forcing functions are external influences acting upon the system.

Figure 1.2 shows the engineering problem solving problems.

Mathematical expression can range from a simple algebraic relationship to a very large complicated
set of differential equations.

Analytical or Exact Solution satisfies the original differential equation, however, there are many
mathematical models that cannot be solved exactly. This is where we use approximation which is
what numerical solutions are all about.

Figure 1.2 The engineering problem solving process

05| Significant Figures

Whenever we use numerical values in our computations, we must have assurance that this can be
used with confidence. This is where the concept of significant figures come in.

The concept of a significant figure, or digit, has been developed to formally designate the reliability of
a numerical value. The significant digits of a number are those that can be used with confidence.
They correspond to the number of certain digits plus one estimated digit.
 nearest 1/100th milliliter; meanwhile his colleague who measured 92 milliliters only knows his
The concept of significant figures has two important implications for our study of numerical methods: value to the nearest 1 milliliter. It's important to understand that "zero" does not mean "nothing."
1. Numerical methods yield approximate results. We must, therefore, develop criteria to Zero denotes
specify how confident we are in our approximate result. One way to do this is in terms of actual information, just like any other number. You cannot tag on zeros that aren't certain to
significant figures. For example, we might decide that our approximation is acceptable if it is belong there.
correct to four significant figures. 5. Trailing zeros in a whole number with the decimal shown ARE significant. Placing a decimal at the
2. Although quantities such as π, e or √7, represent specific quantities, they cannot be end of a number is usually not done. By convention, however, this decimal indicates a significant
expressed exactly by a limited number of digits. zero. For example, "540." indicates that the trailing zero IS significant; there are THREE
significant figures in this value.
Take for example Figure 1.3.
6. Trailing zeros in a whole number with no decimal shown are NOT significant. Writing just "540"
indicates that the zero is NOT significant, and there are only TWO significant figures in this value. 7.
Exact numbers have an INFINITE number of significant figures. This rule applies to numbers that are
definitions. For example, 1 meter = 1.00 meters = 1.0000 meters = 1.0000000000000000000
meters, etc.

8. For a number in scientific notation: N x 10x, all digits comprising N ARE significant by the first 6
rules; "10" and "x" are NOT significant. 5.02 x 104 has THREE significant figures: "5.02." "10 and
"4" are not significant.

Figure 1.3 Automobile Speedometer and Odometer Rule 8 provides the opportunity to change the number of significant figures in a value by
manipulating its form. For example, let's try writing 1100 with THREE significant figures. By rule
6, 1100 has TWO significant figures; its two trailing zeros are not significant. If we add a decimal
RULES FOR SIGNIFICANT FIGURES to the end, we have 1100., with FOUR significant figures (by rule 5.) But by writing it in scientific
notation: 1.10 x 103, we create a THREE-significant-figure value.
1. All non-zero numbers ARE significant. The number 33.2 has THREE significant figures because
all of the digits present are non-zero. Example:
1. Calculate the answer to 1 significant figure.
2. Zeros between two non-zero digits ARE significant. 2051 has FOUR significant figures. The zero 0.0075 ÷ 0.0000715
is between a 2 and a 5.
2. Calculate the answer to 3 significant figures.
3. Leading zeros are NOT significant. They're nothing more than "place holders." The number 0.54 0.0075 ÷ 0.0000715
has only TWO significant figures. 0.0032 also has TWO significant figures. All of the zeros are
leading. 3. State the number of significant figures.
350.701
4. Trailing zeros to the right of the decimal ARE significant. There are FOUR significant figures in
92.00. 92.00 is different from 92: a scientist who measures 92.00 milliliters knows his value to the 4. State the number of significant figures.
 67.030 kg

MATHEMATICAL RULES FOR SIGNIFICANT FIGURES

RULES FOR ROUNDING OFF NUMBERS
1. Addition and Subtraction 1. If the digit to be dropped is greater than 5, the last retained digit is increased by one. For example,
When adding or subtracting numbers, the number of significant figures in the result is 12.6 is rounded to 13.
determined from the position relative to the decimal point of the least significant figure of the 2. If the digit to be dropped is less than 5, the last remaining digit is left as it is. For example, 12.4 is
numbers being added or subtracted. rounded to 12.
3. If the digit to be dropped is 5, and if any digit following it is not zero, the last remaining digit is
Example: increased by one. For example, 12.51 is rounded to 13.
Add 472.1, 3.192 and 5000.86 4. If the digit to be dropped is 5 and is followed only by zeroes, the last remaining digit is increased
Because 472.1 has only 1 sig. fig. after the decimal point, the final answer can by one if it is odd, but left as it is if even. For example, 11.5 is rounded to 12 and 12.5 is
have only 1 sig. fig. after the decimal point—the correct answer is 5476.2 rounded to 12.
This rule means that if the digit to be dropped is 5 followed only by zeroes, the result is always
rounded to the even digit. The rationale is to avoid bias in rounding: half of the time we round
up, half the time we round down
Subtract 126.5419 from 8000

Because 8000 has only 1 sig. fig. four places to the left of the decimal point, the 06| Accuracy and Precision
least significant figure in the final answer must also be four places to the left of
the decimal point—correct answer is 8000
Mathematical Models area as good as the modeler itself. Because as we create these mathematical
models there would be assumptions and approximations that we will be incorporating with our
solution. And when we talk about numerical assumptions and approximations, we cannot prevent
2. Multiplication and Division
ourselves in committing some numerical error. Therefore, our models must have precise and
The number of significant figures in the result is determined from the number of significant
accurate or must be close to the true value.
figures in the least significant value used in the calculation.

Example: The errors associated with both calculations and

Multiply 88.037 by 0.00721 measurements can be characterized with regard to their
accuracy and precision.
88.037 has 5 sig. figs. and .00721 has 3 sig. figs. This limits the result to a
total of three sig. figs. —the correct answer is 0.635 Accuracy refers to how closely a computed or measured
value agrees with the true value.

Precision refers to how closely individual computed or

measured values agree with each other. being evaluated is to normalize the error to the true value, as in

An example from marksmanship illustrating concepts of �������� ��������������������

accuracy and precision. a) Inaccurate and imprecise, b)
accurate and imprecise, c) inaccurate and precise; d) ���������������� ���������� =��������
����������
accurate and precise. Figure 1.5 Accuracy vs Precision �������� ����������
07| Error Definitions
Where, as specified by Eq. (3.2), error = true value − approximation. The relative error can also be
Numerical errors arise from the use of approximations to represent exact mathematical operations multiplied by 100 percent to express it as
and quantities
Two Major Forms of Numerical Errors: ���� =�������� ����������
1. Round-off Error
�������� ���������� ��������% (eq. 3)
Due to the fact that computers can represent only quantities with a finite number of digits.
Where εt designates the true percent relative error.

2. Truncation Error
Example:
The discrepancy introduced by the fact that numerical methods may employ approximations to Suppose that you have the task of measuring the lengths of a bridge and a rivet and come up with
represent exact mathematical operations and quantities. 9999 and 9 cm, respectively. If the true values are 10,000 and 10 cm, respectively, compute (a) the
true error and (b) the true percent relative error for each case.
For both types, the relationship between the exact, or true, result and the approximation can be
formulated as: Notice that for Eqs. (2) and (3), E and ε are subscripted with a t to signify that the error is normalized
to the true value. In the example above, we were provided with this value. However, in actual
�������� ���������� = situations such information is rarely available. For numerical methods, the true value will be known
�������������������������� +
only when we deal with functions that can be solved analytically.
���������� (eq. 1)

Such will typically be the case when we investigate the theoretical behavior of a particular technique
By rearranging Eq. 1, we find that the numerical error is equal to the discrepancy between the truth
for simple systems. However, in real-world applications, we will obviously not know the true answer
and the approximation, as in
a priori. For these situations, an alternative is to normalize the error using the best available
estimate of the true value, that is, to the approximation itself, as in
���� = �������� ���������� −
�������������������������� (eq. 2)

Where Et is used to designate the exact value of the error. The subscript t is included to designate
���� = ���������������������� ����������
that this is the “true” error. ����������������������������������%
(eq. 4)
A shortcoming of this definition is that it takes no account of the order of magnitude of the value
Where the subscript a signifies that the error is normalized to an approximate value. Note also that
under examination. For example, an error of a centimeter is much more significant if we are
for real-world applications, Eq. (3.2) cannot be used to calculate the error term for Eq. (4). One of
measuring a rivet rather than a bridge. One way to account for the magnitudes of the quantities
the can be shown (Scarborough, 1966) that if the following criterion is met, we can be assured that the
computations, we may not be concerned with the sign of the error, but we are interested in whether result is correct to at least n significant figures.
the percent absolute value is lower than a pre-specified percent tolerance εs. Therefore, it is often
useful to employ the absolute value of Eqs. (2) through (5). For such cases, the computation is ���� = (��. �� × ������ି��)% (eq. 7)
repeated until

|����| < ���� (eq. 6)

If this relationship holds, our result is assumed to be within the pre-specified acceptable level εs.
Note that for the remainder of this text, we will almost exclusively employ absolute values when we
use relative errors. YOUTUBE LINKS:
It is also convenient to relate these errors to the number of significant figures in the approximation. It
negative; if the approximation is less than the true value, the error is
challenges of numerical methods is to determine error estimates in the
positive. Also, for Eqs. (3) to (5), the denominator may be less than
absence of knowledge regarding the true value. For example, certain zero, which can also lead to a negative error. Often, when performing
numerical methods use an iterative approach to compute answers. In
such an approach, a present approximation is made on the basis of a
previous approximation. This process is performed repeatedly, or
iteratively, to successively compute (we hope) better and better
approximations.

For such cases, the error is often estimated as the difference between

previous and current approximations. Thus, percent relative error is
determined according to
https://youtu.be/KjBCjVAobVs 
https://youtu.be/p1nXahmcQwg
���� = �������������� https://youtu.be/uZkYpvCK2pw
�������������������������� ି
����������������
��������������������������

��������������

����������������������������������
% (eq. 5)

The signs of Eqs. (2) through (5) may be either positive or negative. If
the approximation is greater than the true value (or the previous
approximation is greater than the current approximation), the error is