Lecture 6:

Truncation Error, Consistency, and

Convergence

J.K. Ryan@tudelft.nl

WI3097TU
Delft Institute of Applied Mathematics
Delft University of Technology

8 October 2012

() Truncation Error, Consistency, and Convergence 8 October 2012 1 / 54

Outline

1 Review
Single Step Methods
Stability
IVP Stability
Amplification factor

2 Local Truncation Error & Consistency

3 Global truncation error & Convergence

() Truncation Error, Consistency, and Convergence 8 October 2012 2 / 54

Single-step methods
Summary
We have 4 schemes to solve

y 0 (t) = f (t, y (t))

y (t0 ) = y0 .

Forward Euler: wn+1 = wn + hf (tn , wn )

Backward Euler: wn+1 = wn + hf (tn+1 , wn+1 )
Trapezoidal method:
h
wn+1 = wn + (f (tn , wn ) + f (tn+1 , wn+1 ))
2
Modified Euler:

w
n+1 = wn + hf (tn , wn )
h
wn+1 = wn + (f (tn , wn ) + f (tn+1 , w
n+1 ))
2
() Truncation Error, Consistency, and Convergence 8 October 2012 3 / 54
Stability

Lax Equivalence Theorem: For a linear scheme,

Stability + Consistency = Convergence

Stability is how much small perturbations affect the solution.

For this we have to look at
Stability of the initial value problem (IVP)
Stability of the numerical method

() Truncation Error, Consistency, and Convergence 8 October 2012 4 / 54

Stability
IVP Stability

For the stability of the initial value problem (IVP), we consider

(t) = y (t) y (t)

|{z}
perturbed problem

There are three possibilities:

1 It is unstable if limt |(t)|
2 It is stable if limt |(t)| < (finite)
3 It is unconditionally stable if limt |(t)| 0

() Truncation Error, Consistency, and Convergence 8 October 2012 5 / 54

Stability
IVP Stability

For the test equation

y 0 = y
y (0) = y0 , (1)

limt |(t)| 0 equates to < 0.

() Truncation Error, Consistency, and Convergence 8 October 2012 6 / 54

Stability
Numerical Stability

For numerical stability, we must consider the amplification

factor, Q(h) in

wn+1 = Q(h)wn = (Q(h))n+1 y0 .

The possiblities are:

1 The method is unstable if limh0 |Q(h)| > 1.
2 The method is stable if limh0 |Q(h)| 1.

We use these conditions to choose an appropriate bound on h.

() Truncation Error, Consistency, and Convergence 8 October 2012 7 / 54

Stability
Amplification factor

wn+1 = Q(h)wn
The amplification factor for our methods is given by:
2
Forward Euler: Q(h) = 1 + h, stable for 0 < h < || .
1
Backward Euler: Q(h) = 1h , stable for 0 < h.
1+ h2
Trapezoidal method: Q(h) = 1 h2
, stable for 0 < h.

modified Euler: Q(h) = 1 + h + 12 (h)2 , stable for

2
0 < h < || .

() Truncation Error, Consistency, and Convergence 8 October 2012 8 / 54

Local Truncation Error

Last time we asked: How good are these methods?

For this, we need convergence.

But, convergence is difficult to prove directly.

The Lax equivalence theorem says that we can instead prove

stability and consistency.

() Truncation Error, Consistency, and Convergence 8 October 2012 9 / 54

Stability

Last time, we covered stability.

Stability is given by the amplification factor,

wn+1 = Q(h)wn .

For this, we choose h so that |Q(h)| 1.

() Truncation Error, Consistency, and Convergence 8 October 2012 10 / 54

Local Truncation Error

For consistency, we will need the local truncation error. But

first we need a few definitions:

yn+1 = exact solution at time tn+1 .

zn+1 = Q(h)yn = approximation applied to the exact
solution at time tn .
wn+1 = (Q(h))n+1 y0 = approximation applied n + 1
times.

() Truncation Error, Consistency, and Convergence 8 October 2012 11 / 54

Local Truncation Error

Definition ( Local truncation error)

The local truncation error at time step n + 1 is given by
yn+1 zn+1
n+1 (h) = .
h

Notice, it is the difference between the exact solution and the

approximation applied to the exact solution at time tn .

() Truncation Error, Consistency, and Convergence 8 October 2012 12 / 54

Consistency

Definition (Consistent)
A numerical method is consistent if

lim n+1 (h) = 0.

h0

Consistent
We can replicate the results.
Solution of the numerical method converges to the exact
solution of the IVP.
yn+1 zn+1 0 faster than h 0.

By definition, this means that we need to examine the

truncation error.

() Truncation Error, Consistency, and Convergence 8 October 2012 13 / 54

Local truncation error

Consider again the test equation:

y 0 = y ,
y (0) = y0

which has solution y (t) = y0 e t .

The solution at tn+1 is

yn+1 = y (tn+1 ) = y0 e tn+1 = e (n+1)h y0 = e h yn = yn+1 .

() Truncation Error, Consistency, and Convergence 8 October 2012 14 / 54

Local truncation error

The numerical method is given by

wn+1 = Q(h)wn .

Applied to the exact solution it is

zn+1 = Q(h)yn .

The local truncation error is then

yn+1 zn+1 e h yn Q(h)yn (e h Q(h))yn

n+1 (h) = = = .
h h h

() Truncation Error, Consistency, and Convergence 8 October 2012 15 / 54

Local truncation error

This means that the truncation error is determined by

e h Q(h).

For this, we need two things:

1 The Taylor series for e h
rX1
1 1 1
e h = 1+h+ (h)2 + (h)3 + = (h)i +O(hr )
2! 3! i=0
i!

2 Q(h), which is given by the numerical method.

() Truncation Error, Consistency, and Convergence 8 October 2012 16 / 54

Local truncation error
Forward Euler
The amplification factor for Forward Euler is given by

Q(h) = 1 + h.

Therefore, the local truncation error is

(e h Q(h))yn
n+1 (h) =
h

1 1

1 + h + (h)2 + O(h3 )

= (1 + h) yn
h
2! | {z }

| {z } Amplification factor
Taylor series for e h
1 1
 
= (h)2 + O(h3 ) yn
h 2!
1 2
 
= h + O(h2 ) yn n+1 (h) = O(h)
2!
() Truncation Error, Consistency, and Convergence 8 October 2012 17 / 54
Forward Euler

Euler forward,
wn+1 = wn + hF (tn , wn ),

Is an explicit method.
Has amplification factor Q(h) = 1 + h.
2
Is stable for 0 < h < || .
Has local truncation error of order one.
Stability + Consistency = Convergence. It is a first order
method.

() Truncation Error, Consistency, and Convergence 8 October 2012 18 / 54

Consistency

Let us do the same for the other methods.

Convergence is difficult to prove.
Lax theorem says that we can instead prove stability and
consistency.
Stability is given by ensuring the amplification factor is less
than one.
|Q(h)| 1.
For consistency, we need the local truncation error to
vanish as the step size becomes smaller.
1
lim n+1 (h) = lim (yn+1 zn+1 ).
h0 h0 h

() Truncation Error, Consistency, and Convergence 8 October 2012 19 / 54

Local truncation error
Modified Euler
For the modified Euler, the amplification factor is given by
(h)2
Q(h) = 1 + h + .
2
2
It is stable for 0 < h < || .
For consistency, we examine the local truncation error:
(e h Q(h))yn
n+1 (h) =
 h
1 1 1

2 3 4
= 1 + h + (h) + (h) + O(h ) ..
h 2! 3!
!#
(h)2
.. 1 + h + yn
2
1 1
 
= (h)3 + O(h4 ) yn
h 3!
1
=( h2 3 + O(h3 ))yn n+1 (h) = O(h2 )
3!
() Truncation Error, Consistency, and Convergence 8 October 2012 20 / 54
Modified Euler

Modified Euler is given by

w
n+1 = wn + hF (tn , wn ),
h
wn+1 = wn + (F (tn , wn ) + F (tn+1 , w
n+1 )) (2)
2

Is an explicit method.
(h)2
Has amplification factor Q(h) = 1 + h + 2 .
2
Is stable for 0 < h < || .
Has local truncation error of order two.
Stability + Consistency = Convergence. It is a second
order method.

() Truncation Error, Consistency, and Convergence 8 October 2012 21 / 54

Local truncation error
Implicit methods

For implicit methods we generally have to work a bit harder.

Amplification factor is not in polynomial form.
Have to put in polynomial form.
Requires Taylor expansion.

() Truncation Error, Consistency, and Convergence 8 October 2012 22 / 54

Local truncation error
Backward Euler

For Backward Euler, the amplification factor is given by

1
Q(h) = .
1 h

Before we plug this into the formula for the local truncation
error, we must perform a Taylor expansion:
1
= 1 + h + (h)2 + (h)3 + O(h4 ),
1 h
when |h| < 1.

() Truncation Error, Consistency, and Convergence 8 October 2012 23 / 54

Local truncation error
Backward Euler

1
Plugging in 1h = 1 + h + (h)2 + (h)3 + O(h4 ) into the
formula for local truncation error gives

(e h Q(h))yn
n+1 (h) =
 h
1 1

= 1 + h + (h)2 + O(h3 ) ..
h 2!
i
.. (1 + h + (h)2 + O(h3 )) yn
1 1
 
= (h)2 + O(h3 ) yn
h 2
1 2
 
2
= h + O(h ) yn n+1 (h) = O(h)
2

() Truncation Error, Consistency, and Convergence 8 October 2012 24 / 54

Backward Euler

Euler backward,

wn+1 = wn + hF (tn+1 , wn+1 ),

Is an implicit method.
1
Has amplification factor Q(h) = 1h .
Is stable for h > 0.
Has local truncation error of order one.
Stability + Consistency = Convergence. It is a first order
method.

() Truncation Error, Consistency, and Convergence 8 October 2012 25 / 54

Local truncation error
Trapezoidal method

For the Trapezoidal method, the amplification factor,

h
1+ 2
Q(h) = h
1 2

also has to be expanded into a Taylor series:

h
1+ h h h h
  
2
h
= 1+ + ( )2 + ( )3 + O(h4 )
1+
1 2
2 2 2 2
1 1
= 1 + h + (h)2 + (h)3 + O(h4 )
2 4
 
when  h
2  < 1.
 

() Truncation Error, Consistency, and Convergence 8 October 2012 26 / 54

Local truncation error
Trapezoidal method
1+ h
Plugging in 2
1 h
= 1 + h + 12 (h)2 + 14 (h)3 + O(h4 ) into the
2
formula for local truncation error gives

(e h Q(h))yn
n+1 (h) =
 h
1 1 1

= 1 + h + (h)2 + + (h)3 + O(h4 ) ..
h 2! 3!
1 1

.. (1 + h + (h)2 + (h)3 + O(h4 )) yn
2 4
1 1
 
= (h)3 + O(h4 ) yn
h 6
1
 
= h2 3 + O(h3 ) yn n+1 (h) = O(h2 )
6

() Truncation Error, Consistency, and Convergence 8 October 2012 27 / 54

Trapezoidal method

Trapezoidal method,
h
wn+1 = wn + (F (tn , wn ) + F (tn+1 , wn+1 )),
2

Uses the average of Euler forward and Euler backward.

Is an implicit method.
1+ h
Has amplification factor Q(h) = 2
1 h
.
2
Is stable for h > 0.
Has local truncation error of order two.
Stability + Consistency = Convergence. It is a second
order method.

() Truncation Error, Consistency, and Convergence 8 October 2012 28 / 54

Local truncation error
To find the local truncation error
1
n+1 (h) = (yn+1 zn+1 )
h
where zn+1 = Q(h)yn .
1 Write the exact solution in terms of yn : yn+1 = yn e h .
yn h
n+1 (h) = (e Q(h))
h
2 Expand e h in a Taylor series.
3 Expand the amplification factor, Q(h) in a Taylor series
(if necessary).
4 Simplify the quantity e h Q(h)
yn
5 Multiply by h.

() Truncation Error, Consistency, and Convergence 8 October 2012 29 / 54

Local truncation error
Observe: All methods can be written as

wn+1 = Q(h)wn .

Explicit methods can be written as

zn+1 = yn + h(zn , tn , h).

This gives a truncation error of:

1 1
n+1 (h) = [yn+1 zn+1 ] = [yn+1 (yn + h(zn , tn , h))]
h h
1
= (yn+1 yn ) (zn , tn , h)
|h {z } | {z }
Depends upon method
All explicit methods

() Truncation Error, Consistency, and Convergence 8 October 2012 30 / 54

Local truncation error

For explicit methods, the local truncation error can be written

as
1
n+1 (h) = (yn+1 yn ) (zn , tn , h).
h

We can simplify this further. Expand yn+1 around yn :

h2
yn+1 = yn + h (yn )0 + (yn )00 +O(h3 )
| {z } 2 | {z }
=f (tn ,yn ) =(f (tn ,yn ))0 =(ft +ffy )n

h2
= yn + hf (tn , yn ) + (ft + ffy )n + O(h3 )
2

() Truncation Error, Consistency, and Convergence 8 October 2012 31 / 54

Local truncation error

Then
1
n+1 (h) = (yn+1 yn ) (zn , tn , h)
h !
1 h2 3
= (yn + hf (tn , yn ) + (ft + ffy )n + O(h )) yn
h 2
(zn , tn , h)
h
= f (tn , yn ) + (ft + ffy )n + O(h2 ) (zn , tn , h)
2

Euler Forward: (zn , tn , h) = f (tn , yn )

Modified Euler: (zn , tn , h) = f (tn , yn ) + h(ft + ffy )

() Truncation Error, Consistency, and Convergence 8 October 2012 32 / 54

Local truncation error
Finding (zn , tn , h) for modified Euler:

zn+1 = yn + hf (tn , yn )
h
zn+1 = yn + (f (tn , yn ) + f (tn+1 , zn+1 ))
2
We need to expand f (tn+1 , zn+1 ) in a Taylor series around
(tn , yn ).

zn+1 yn )
f (tn+1 , zn+1 ) =f (tn , yn ) + ft (tn , yn ) (tn+1 tn ) +fy (tn , yn ) (
| {z } | {z }
=h =hf (tn ,yn )

+ ftt (tn , yn )(tn+1 tn )2 ..

.. + 2fty (tn+1 tn )( zn+1 yn )2
zn+1 yn ) + fyy (
+ O(h3 )
=f (tn , yn ) + h(ft (tn , yn ) + f (tn , yn )fy (tn , yn )) + O(h2 )

() Truncation Error, Consistency, and Convergence 8 October 2012 33 / 54

Global truncation error

The local truncation error,

1
n+1 (h) = (yn+1 zn+1 ),
h
says how well the method performs over one time-step. It tells
us whether the method is consistent.

What is the cumulative effect?

() Truncation Error, Consistency, and Convergence 8 October 2012 34 / 54

Global truncation error

For that we need:

Definition (Global truncation error)
The global truncation error of a scheme is defined as

en+1 = yn+1 wn+1 ,

where yn+1 = y (tn+1 ) is the exact solution at tn+1 and

wn+1 = (Q(h))n+1 y0 is the approximation applied to the initial
condition n + 1 times.

This is the cumulative effect of the errors from the

approximation over the entire interval.

() Truncation Error, Consistency, and Convergence 8 October 2012 35 / 54

Global truncation error

Definition (Convergence)
A numerical method is convergent if

lim en+1 = 0.
h0

The global truncation error goes to zero as we increase the

number of subintervals.
The local & global truncation errors are of the same order.

() Truncation Error, Consistency, and Convergence 8 October 2012 36 / 54

Convergence

We now have the three elements in the Lax Equivalence

Theorem: For a linear scheme,

Stability + Consistency = Convergence

Proof is in the book...

() Truncation Error, Consistency, and Convergence 8 October 2012 37 / 54

Higher Order Methods

The methods we have discussed are all 1st or 2nd order

methods. Sometimes, we would like a higher order method.

When we require high accuracy.

When we must solve the equation over long periods in
time.

() Truncation Error, Consistency, and Convergence 8 October 2012 38 / 54

Higher Order Methods
4th order Runge-Kutta (RK4)
One higher-order method is the family of Runge-Kutta methods.
The 4th order Runge-Kutta scheme is:

k1 = hf (tn , wn )
h k1
k2 = hf (tn + , wn + )
2 2
h k2
k3 = hf (tn + , wn + )
2 2
k4 = hf (tn + h, wn + k3 )

RK4 is the average of these slopes

1
wn+1 = wn + (k1 + 2(k2 + k3 ) + k4 )
6

() Truncation Error, Consistency, and Convergence 8 October 2012 39 / 54

Higher Order Methods
4th order Runge-Kutta (RK4)

RK4:
Uses an average of different slopes.
This allow us to avoid taking higher derivatives
This is a 4th order method.
The amplification factor is

1 1 1
Q(h) = 1 + h + (h)2 + (h)3 + (h)4
2 3! 4!

2.8
The stability region is 0 < h < .
||

() Truncation Error, Consistency, and Convergence 8 October 2012 40 / 54

Example

Example
Consider the numerical scheme given by

u ? = un + hf (tn , un )
un+1 = u ? + (1 )hf (tn + h, u ? )

1 Show the truncation error is O(h) for each value of .

Show that there does not exist a for which the scheme is
O(h2 ).
2 What is the amplification factor?
3 Consider the nonlinear equation y 0 = 2y 4y 2 and take
= 12 . Determine the maximal step size such that the
method is stable in the neighbourhood of y = 12 .

() Truncation Error, Consistency, and Convergence 8 October 2012 41 / 54

Example
Truncation error

The numerical method applied to the exact solution, yn , is

given by

z ? = yn + hf (tn , yn )
zn+1 = z ? + (1 )hf (tn + h, z ? )

We first need to expand f (tn + h, z ? ) in a Taylor series around

(tn , yn ) :

f (tn + h, z ? ) =f (tn , yn ) + (ft )n (tn + h tn ) + (fy )n (z ? yn )

1 1
+ 2 h2 (ftt )n (tn + h tn )2 + (fyy )n (z ? yn )2
2 2
+ h(fty )n (tn + h tn )(z ? yn ) + O(h3 )

() Truncation Error, Consistency, and Convergence 8 October 2012 42 / 54

Example
Truncation error

Substituting z ? = yn + hf (tn , yn ) into the expansion, we have

f (tn + h, z ? ) = f (tn , yn ) + h((ft )n + (f fy )n ) + O(h2 ).

Now, we can consider zn+1 :

zn+1 =z ? + (1 )hf (tn + h, z ? )

=yn + hf (tn , yn )
+ (1 )h(f (tn , yn ) + h((ft )n + (f fy )n ) + O(h2 ))
=yn + hf (tn , yn )
+ h(f (tn , yn ) + h(ft + f fy )n + O(h2 ))
h(f (tn , yn ) + h(ft + f fy )n + O(h2 ))

() Truncation Error, Consistency, and Convergence 8 October 2012 43 / 54

Example
Truncation error

Therefore we have that

zn+1 = yn +hf (tn , yn )+h2 (ft +f fy )n +(h)2 (ft +f fy )n +O(h3 )

We also need
1
yn+1 = yn + h (yn )0 + h2 (yn )0 0
+ O(h3 )
| {z } 2 | {z }
=f (tn ,yn ) =(ft +ffy )n
1
= yn + hf (tn , yn ) + h2 (ft + ffy )n + O(h3 ) = yn+1
2

() Truncation Error, Consistency, and Convergence 8 October 2012 44 / 54

Example
Truncation error

Which gives the local truncation error as

1
n+1 (h) = (yn+1 zn+1 )
h
1 1
= (yn + hf (tn , yn ) + h2 (ft + ffy )n + O(h3 )) ..
h 2

.. (yn + hf (tn , yn ) + h2 (ft + f fy )n + . . .)
1
= h(ft + ffy )n h(ft + f fy )n + . . .
2
=O(h)

The local truncation error is O(h) for any value of .

() Truncation Error, Consistency, and Convergence 8 October 2012 45 / 54

Example
Amplification factor

To determine the amplification factor, we apply the numerical

method to the test equation,

y 0 = y , y (0) = y0 , < 0.

This gives

u ? = un + hun
un+1 = u ? + (1 )hu ?

() Truncation Error, Consistency, and Convergence 8 October 2012 46 / 54

Example
Amplification factor

Simplifying,

un+1 = u ? + (1 )hu ?
= (un + hun ) + (1 )h(un + hun )
*+ h + (h)2 h 2 2
= (1 + 
h  (h) )un

 *


= (1 + h + (1 )(h)2 ) un
| {z }
Q(h)

The Amplification factor is given by

Q(h) = (1 + h + (1 )(h)2 )

() Truncation Error, Consistency, and Convergence 8 October 2012 47 / 54

Example
Nonlinear equation

The nonlinear equation is

y 0 = f (y ) = 2y 4y 2 .

Notice, this is zero at y = 0 and y = 12 . If = 12 , we want to

determine the maximum step-size, h, for which the scheme is
stable. We proceed as follows

First, we linearize f (y ) around y = 12 .

() Truncation Error, Consistency, and Convergence 8 October 2012 48 / 54

Example
Nonlinear equation
The linearization is given by
1 1 1
 
L(y ) = f ( ) + f 0 ( ) y .
2 2 2
This gives
L(y ) = 1 2y .
Therefore, to consider the stablity of the nonlinear equation

y 0 = 2y 4y 2 .

it is enough to consider the stability of the linear equation

y 0 = 2y

This is just our test equation with = 2.

() Truncation Error, Consistency, and Convergence 8 October 2012 49 / 54

Example
Nonlinear equation

1
For stability, we consider the amplification factor with = 2
and = 2 :

Q(h) =1 + h + (1 )(h)2
1 1
 
=12h + 1 (2h)2
2 2
=1 2h + h2

The method is stable if

|Q(h)| = |1 2h + h2 | 1.

() Truncation Error, Consistency, and Convergence 8 October 2012 50 / 54

Example
Nonlinear equation

|Q(h)| = |1 2h + h2 | 1
This means

1 1 2h + h2 1
1 (h 1)2 1
2 h(h 2) 0

Recall: h > 0. Thus:

0<h2 for stability.

() Truncation Error, Consistency, and Convergence 8 October 2012 51 / 54

Summary table

FE BE ME TM
(h)2 1+ h
Amplification factor 1 + h 1 h 1 + h + 2
2
1+ h
2
Type method Explicit Implicit Explicit Implicit
2 2
Stability range 0 < h < || h>0 0 < h < || h>0
Method order One One Two Two
(zn , tn , h) fn fn + h(ft + ffy )n

FE = Forward Euler; BE = Backward Euler; ME Modified

Euler; TM = Trapezoidal method

() Truncation Error, Consistency, and Convergence 8 October 2012 52 / 54

Summary
Lax Equivalence Theorem: For a linear scheme,

Stability + Consistency = Convergence

Stability: given by amplification factor, choose h such that

|Q(h)| < 1.

Consistency: given by the local truncation error:

1
n+1 = (yn+1 zn+1 ).
h

Convergence: given by the global truncation error:

en+1 = yn+1 wn+1 .

() Truncation Error, Consistency, and Convergence 8 October 2012 53 / 54

Material addressed

3 Global truncation error &

1 Review Convergence
Single Step Methods
Stability Material in book:
IVP Stability
Amplification factor
Chapter 6, Sections 1-7
2 Local Truncation Error &
Consistency Useful exercises: 1-5

() Truncation Error, Consistency, and Convergence 8 October 2012 54 / 54