A Semi-Implicit Mid-Point Rule For Stiff Systems of Ordinary Differential Equations
A Semi-Implicit Mid-Point Rule For Stiff Systems of Ordinary Differential Equations
O. Introduction
For many years, the numerical solution of stiff ordinary differential equations
(ODEs) has been an active field of research motivated by challenging real life
applications. The presently most effective algorithm for this type of problems
seems to be the BDF method of Gear [10] in the advanced version due to
Hindmarsh [13]. As this method is based on an implicit discretization, it re-
quires the iterative solution of a nonlinear algebraic system per each integration
step. In view of this feature, recent attention has focussed on so-called semi-
implicit methods, which require the solution of just one linear system per each
step (or stage). Examples of this latter kind are the Rosenbrock-Wanner meth-
ods (see e.g. Kaps and Rentrop [15]) or the new extrapolation method to be
introduced in the present paper.
The latter method is based on a new type of discretization, which is called
the semi-implicit mid-point rule, as it has been derived from the well-known
explicit mid-point rule (see Gragg [11, 12], and Bulirsch and Stoer [2]). In
Sect. 1, this semi-implicit mid-point discretization is shown to permit a qua-
dratic asymptotic expansion in terms of the stepsize h. This expansion and cer-
tain additional stability considerations (see Sect. 2) back the expectation that
the new method should be useful for solving stiff problems. Details of the al-
gorithmic realization are described in Sect. 3. Finally, in Sect. 4, numerical
374 G. Bader and P, Deuflhard
comparisons of the new extrapolation code with the codes due to Gear and
Hindmarsh [10, 13] and Kaps and Rentrop [15] are given.
Y'=f(Y) (1.1)
y(0)=y0.
Upon applying the explicit mid-point rule as realized in the algorithm due to
Gragg [12], Bulirsch and Stoer [2 3 one obtains:
a) rto: = Yo
r/p =Yo +hf(yo)
(1.2)
b) k = 1..... [: qk+ P=t/k- i +2hf(r/k),
c) S e: = ~1[~le- t +2q~+r/e+ t].
The discretization error for both tl(t; h) and S(t; h) is known to have an asymp-
totic h2-expansion of the form (with t'. =Eh fixed):
N
t/(t; h ) - y ( t ) = ~. [ u j ( t ) + ( - 1)e"vj(t)], hEJ+EN+ ~(t; h) h =N+2
~=1 for he[0, HI (1.3)
where N depends on the order of differentiability o f f The functions uj, vj are
known to satisfy the linear systems (dropping inhomogeneous terms):
u)= fy(y(t)) uj + . . .
(1.4)
v] = -fy(y(t)) Vj + ....
Because of this behaviour, the functions vj are called the "weakly unstable"
components. The final replacement of re by Se just eliminates v 1 in the asymp-
totic expansion.
A Semi-Implicit Mid-Point Rule for Stiff Systems 375
Now, discretize this equation by means of the explicit mid-point rule and use
(1.6) for back-substitution. Then, with the notation
f (y):= f (y) - A y,
one obtains the following discretization:
a) F/o:= Yo
Fll: = eAh(y o + hf(yo)),
b) k = l , . . . , • : FIk+l:=eah(eAhTlk_l+2hf(Flk)), (1.2')
C)
Se:=z[e
- 1
rle_l+2Fle+e-AhT]e+l].
A h --
Now, following the lines of the construction of (1.2'), one may easily verify the
existence of the asymptotic expansion
N
F/(t; h ) - y ( t ) = ~ [ff~(t)+(-1)e~j(t)].hZJ+ff~u+~(t; h).h :u+z, (1.3')
j=l
For actual realization, however, one will replace the matrix exponential by
some Pad~ approximation, say
enh~E(Ah). (1.10)
E(Ah)'.=I+hA (1.13)
will be made. This choice will turn out to produce the outstanding candidate
among the discretization schemes of the type (1.11) - see the stability con-
siderations of Sect. 2.1. In fact, even the exact matrix exponential will be ruled
out for stability reasons! Thus, in contrast to any approaches as the one of
Lawson [16], the discretization proposed here should not be regarded as an
"exponential fitting" method.
a) r/o" =Yo
r/,: = (I - h A)- 1 [Yo + hf(yo)],
(1.14)
b) k = l ..... d: rlk+l:=(I-hA)-l[(I+hA)rlk_l+2hf(qk)],
c) - 1
S2m:=~(~/2m+ 1 +q2m_ 0 for ( = 2 m .
From the derivation, (1.14.b) is called the semi-implicit mid-point rule. This
two-step scheme is started by the semi-implicit Euler step (1.14.a), which may
be derived in a similar way as the mid-point step (1.14.b). The final step
(1.14.c), which is due to I-1], performs some kind of smoothing that will be
analyzed in Sect. 2.2 below. Note that this final step is not just obtained by
transforming Gragg's final smoothing step (1.2.c) via (1.2'.c) with (1.10).
Theorem 1. Let y denote the unique solution of the IVP (1.1) and r/(t; h)-~le the
solution of the discretization scheme (1.14.a,b) for t = f h fixed. Let for some
inner product ( . , . ) the following relation hold:
(x, A x ) <__~( x , x ) - ~ llxll 2. (1.15)
A Semi-lmplicit Mid-Point Rule for Stiff Systems 377
H#<l-e (1.17)
for some e in the range 0 < e < 1 such that, for all hG[0,H], the following asymp-
totic expansion holds:
N
with ai, bj, e~, dj defined in the proof (see (1.25) and (1.30)), and EN+ t is uni-
formly bounded for all h~EO, HI.
In particular, with the additional notation
for s=O,
the following estimates hold:
a) for ~<0:
lIEN+ a(t; h)ll <[le2Lt+ ctt q~(2Lt), (1.20)
Now, recall that extrapolation methods are just a special case of one-step
methods (cf. [26]). As a consequence, without further restrictions on y and f
(and their derivatives), the uniform bound on EN+ 1 must contain at least poly-
nomials (of degree (2N+2)) in [IA[I. Recall that with explicit discretizations
e llallt appears.
With this notation, the scheme (1.14.a,b) is equivalent to the following recur-
sion
a) Xk+l:--=Xk+h[g(Xk+2"~Xk)+f(Zk)],
(1.21)
with
xo:=yo, zo:=xo+~[g(zo)+f(xo)].
This semi-implicit recursion is well-defined, if
I - hA nonsingular,
For kT~0, his difference system is seen to be consistent with the differential
system
a) x'-Ax=f(w), x(0)=y o,
(1.23)
b) w'-Aw=f(x), w(0)= y o .
Thus x ( t ) - w ( t ) = y ( t ) follows. Moreover (1.21') is invariant under the common
interchanges
Xk+--')'Xk+ 1~ Wk ~4'Wk+ 17 h~--)" - ~ .
c) G(o)=o,
G(O)= _
Z~
•(2k--
d--k
1)tA~I
~vl|
380 G. Bader and P. Deuflhard
c+(t),= - W -~ '" )
where
N
A(:= ~ (k(t) h2k+z(t; h) h 2N+2,
k=l
N
A~:= ~, ~k(t) h2k-Fx(t; h)h 2N+2,
k=l
bl(t)=-dl(t)=-O.
The results (1.25) have essentially already been found by Mirbeth [-17]. In or-
der to verify the boundedness of the remainder terms X, Z, one may con-
veniently introduce the further notation:
1 1
S f<N+I)(Y +6"AOA~N+I d6
( N + 1)! ~=o
----:B~+ 1((o .... , (n ; Z(t; h) h 2N+ 2; h) h 2N+2,
1 1
fm+a)(y+f'A~)A~N+l d6
( N + 1)! ~=o
=..O~+l(~O .... ,~,; X(t; h)h2N+2; h)h 2N+2,
N 1 (~2N+ 1 - 2 i
R(t;h)'.= ~ ~ (2N+l-2i)!r
i=0 6=0
N 1 ~2N+1-21
P(t; h):=,~=o'= ~=_o
~ (2N + 1 - 2 i ) I (12N+2-2i)(tq-r~h) dr~'
,_ 0 DN+I,=O
Note that, from the definitions of at(t ), ce(t), one has
~i,~iEC 2N+ 3-2i,
so that both R and P may be uniformly bounded. With these preparations, the
following system of nonlinear difference equations may be shown to hold:
A Semi-ImplicitMid-Point Rule for StiffSystems 381
c) x(0; h)=0
N+ 1 r(2k) [O~ (1.26)
Z(h; h ) = - e ( h ; - h ) + ( I - h A ) - i ~ ~N+l-k~ v,
k= 0 (2k) !
For the convenience of the subsequent estimates, one may introduce the func-
tions
q~(t; h)'.=(I + h A) R ( t - h ; h ) - ( I - h A) R(t + h; -h) + 2h BN+ l(. ; h),
~(t; h):=(I+hA)P(~-h; h ) - ( I - h A ) P ( t + h ; - h ) + 2 h DN+I('; h).
Moreover, they are Cl-functions with respect to h. In order to derive the es-
timates (1.20), one may adopt a technique similar to the one used by Dahlquist
and Lindberg [-4-]: let I1"LIdenote the norm induced by the inner product ( . , . )
defined in (1.15). For the time being, let /~<0, which readily implies that (for
the subordinate matrix norm) for all h > 0
II(I-hA)- 1 (I + h a ) l I < 1
1 (1.27)
<1.
I](I-hA)-lll <1 - h # =
Note that, as a consequence of (1.25), the above bounds e,/3 depend on bounds
for polynomials in IIAI[ via the dependence upon ~i, ~i - compare Remark 1
above. The additional implicit dependence upon the terms X, Z will be dis-
cussed at the end of the proof.
As a next step, introduce the upper bounds
Eo:=0, El:= fl
(1.28)
Ek+ 1: = E k - 1 -t- 2hL E k -t- hot, k = l ..... :.
From this, by means of repeated induction, one may come to the upper bound
This verifies (1.20.a). Next, assume that # > 0 and #H < 1 - ~ for some ee]0, 1[.
In this situation, (1.27) must be replaced by
[ 2h(L+#)~ k
/1-+2h(L+#)\k ~1-~ i ~ ] 1
Ek <
=l-h# ~ i---~-~ ) + e 2(/,+#) (1.29')
with fire+1 uniformly bounded. In comparison with (1.18) and with the notation
~j(t):=Uj(t)--Vj(t),
the following results hold."
(o(t)=--y(t)
~(t)
a) (~(t)= k=o (2k)! { = 1 .... , N + I , (1.32)
/~s+l(t; h),=89 +h; h)+ P(t +h;-h)+ Z(t-h; h)+ P(t-h; h)]. (1.34)
Since Z and P are uniformly bounded, the same is true for /~. Moreover, the
representation (1.32.a) is verified. Next, in order to verify (1.32.b), apply (1.26.b)
for t = 0 (recalling that t/(0; h)=yo) , which reads
2. Stability Considerations
y(t)-g(t).
which has been derived in (1.11). Note that (2.2) is just the non-autonomous
version of (1.11). One may identify the quantities
A:=2, f(t):=g'(t)-2g(t)
g(t):=g o + a t (2.3)
will lead to a strict determination of only one Pad6 approximation. For the
above specification, (2.2) is equivalent to
Hence, any steady state solution (2.3) is a solution of the above difference
equation, if the following two conditions hold:
E(z) - E( - z) = 2z
(2.4)
E(z) + E ( - z) = 2.
the specification that has been used throughout Sect. 1.2 above - see (1.13).
One may also observe that the exact exponential ez is ruled out by the equa-
tions (2.4)! In view of the above outstanding property for the simple Pad6-
approximation (2.5), the concentration of the present study to this special case
seems to be justified.
z:=fyh=2h, Re(z)<0,
1 ( l + z ] "`-I
q2m-1-- 1--Z \ l - - z ! YO
=(I+z~ m
~]2m \~ZI Yo (2.8)
1 (1 + z~ "-1
S2m = ( 1 - z ) 2 \ l - - z f Yo"
q 2 , , ~ ( - 1)"yo
(-1)"
q2,,- 1~ Yo~ 0 (2.10)
Z
g2 m__a.(-- 1)m-1
Z2 Y0~0"
This means that, for f odd, the r/e are strongly absolutely stable. In particular,
the final step due to [1] performs some additional smoothing.
The above stability considerations have been made under the assumption
that
A =fr(Y) = constant.
In the more general nonlinear case one has
fr(Y) ~ constant.
Zo:=Ah, Re(z0)<0.
A Semi-Implicit Mid-Point Rule for Stiff Systems 387
(1 -Zo)~2-2(Z-Zo)Z-(l+zo)=O. (2.11)
Let zj(Z, Zo), j= 1, 2 denote the characteristic roots of (2.11). Then the associated
stability region is defined by
a(zo) := {z~ffg[lzi(z, Zo)[ < 1, j = 1, 2}.
For the b o u n d a r y 0G one may introduce the p a r a m e t e r representation
"c(~)=e ie.
This representation leads to
which means that the stability regions are elliptic. In Fig. 1, such regions are
shown for various z o. F o r z o approaching the imaginary axis, the elliptic do-
mains shrink d o w n to the degenerate case of just the interval [0, 2Zo], if z o lies
on the imaginary axis. This behaviour nicely reflects the fact that the concept
of absolute stability actually ends on the imaginary axis.
[m (z~
3. Algorithmic Realization
3.1. Discretization
The realization of the (n, n)-matrix
a) qo' =Yo
A o' = (I - h A ) - 1 hf(yo) '
b) k = l ..... # - 1 : qk:=tlk_lq-Ak_l
Ak' = AR- 1 + 2 ( 1 - h A ) - 1 [ h f (qk)_ Ak - 1], (3.2)
C) ~]g,:~-/~_ 1 -t- AE- 1
zle,= (I - h A ) - a [-hf 07e)-Ar 1]
A k = O(h),
where D is the above weighting matrix. Note that this latter test is formally
invariant under componentwise re-gauging of the variable y. If one of the two
tests is activated, then the integration step is repeated with a reduced stepsize.
Remark. The authors do not favour any monitoring devices that involve any
condition number of the matrix ( I - h A ) . In fact, typical linear systems arising
from extremely stiff ODE systems usually exhibit extremely ill-conditioned
matrices occurring in nevertheless well-conditioned linear systems. As an exam-
ple, a linear system of the type
may occur after the decomposition. Here cond(A) ,,~-1, while the solution x 1
= x 2 = l is well-defined for all e~0. The peculiarity of such systems is that in
the singular limiting case (here e ~ 0 ) the right-hand sides remain in the column
space of the matrix. For a treatment of the rounding error propagation in such
cases see Wilkinson [25].
3.2. Extrapolation
The existence of the quadratic asymptotic expansion (1.31) permits one to ap-
ply Richardson extrapolation. For this purpose, the problem is repeatedly dis-
cretized over the same basic step, say [0, H], with successively decreasing step-
sizes H
hi , = - - , i = 1, 2..... (3.6)
nl
~={nl,nz . . . . },
for R e ( z ) ~ - oo. In this asymptotic case, one may reasonably require that the
sign of the above approximation agrees with the sign of e z along the negative
real axis, i.e. that
S2mi>0,
which readily leads to the representation
The special value of a has been selected on an empirical basis. These con-
siderations lead to the sequence
The analysis for A4:0 is less simple, since then (1.32.b) replaces (1.33.b). As
(I + hA) may well be singular, a more detailed analysis is necessary. On the basis
and with the notation of (1.34) together with (1.26), one may derive the follow-
ing difference equation (dropping the indices (k+ 1)):
where (1.32.b) and (1.26.c) have been used. Obviously, the second right-hand
term is (9(h), whereas the first right-hand term is just (9(1). If the second term
dominates, then
1l/~(2h; h)ll = (9(h) (3.13)
for "sufficiently large" h. This starting point together with the recursion (3.11)
then implies (3.10.a) for sufficiently large h. On the other hand, if the first term
dominates, then
[b/~(2h; h)ll =(9(1). (3.13')
This starting point together with (3.11) then implies
f, the second right-hand term in (3.12) may be large compared with the first
term, which represents the "smoothing" performed by the final step. This
smoothing will be the more significant, the stiffer the problem is. Hence, for
nonlinear and extremely stiff ODE systems, the representation (3.10) based on
(3.13) will be appropriate. This interpretation nicely agrees with numerical ex-
periments that have been made both under the assumption (3.10) and under
the assumption (3.10'). Roughly speaking, the even orders as in (3.10') have
appeared to be slightly preferable in linear problems, whereas the odd orders
as in (3.10) have appeared to be significantly preferable in more difficult non-
linear problems. A further analysis shows that assuming (3.10) for the stepsize
control in a situation where (3.10') would be more appropriate, just leads to a
more cautious stepsize strategy - thus improving the robustness of the algo-
rithm. Hence, the odd order assumption (3.10) has been implemented in the
stepsize control of METAN 1.
4. Numerical Comparisons
4.2. Kidney Model Problem (due to Babugka, see Scott and Watts [19])
Originally, this ODE system arises in the context of a two-point boundary
value problem, where
Y'I=a~(Y3--Yl)
a) Ys(0) =0.990268835,
b) ys(0) =0.99 , (4.2)
c) ys(0)=0.9
A peculiarity of the problem is that it is non-stiff for the first value, but rather
stiff for the second and the third value. Moreover, the solution of the initial
value problem is extremely sensitive to small changes of the initial values ys(0)
- see Fig. 2. Comparative results for both analytic Jacobian and numerical
difference approximation of the Jacobian are listed in Table 3. Due to the
sensitivity of the problem, the final accuracies obtained by the three codes were
not comparable for TOL > 1.D-6.
A Semi-Implicit Mid-Point Rule for Stiff Systems 395
y, ltl_-- /
/
2- /"
104. Ys(O)=0.9/.
3- /
2- /
103" /"
, /
3-
2- f"
102.=_ /" Ys(O)=O99 / j
-: / I
~- / /
/
2- / I
lO1-~ / I
,7 / //
3-1 / ' / y~(0)=0990268635
2-] .J lJ
100. ~
0 0.2 0 Z, 0.6 0.8 1.0 t
Fig. 2. Graph yl(t) of the IVP (4.1) for 3 different initial values ys(0)
| OLU
~1 H~S
9 O*~ " .
4 L?F
5 H|P
MKU .":::.
?
I
AOP
GL~ e , ~ 1 4 9 1 4 9 ,~149 9
9 .. : 9
lit!
10 IS~ o 9
It ISG
12 FRF
1) PPIt
14 pFr 9 I 9149149
15 rPP 9 1 4 9 1 4 9
1~ PFI
IT kl, O 9 ... 9 9
11 AZ,F
19 ALA
;10 CA[* "::,.,',,.,
31 DIIA
~l ?In
l] TID
24 ? J ~ i i t l i ii
315 tllt~
2~ /rOD
2? ItOll D
28 DP|
2'I Fie 9 e~ 9 a o 9
30 ROll
31 OG~
l~ Dim D 9 i
l) I~i Q 6 9 9149
34
35 I
36
17 I~P
151 t~n
40 |gt(* 9 9 19
41 ILIIP
4) I'SP
41 Ill!
44 l~rlc O I
4s I'I~0
441 Pyp
4T PT?
48
49
SO
P~it
LDO
LOL
"!!!..."
1t N
54 Oll
SS DII
S6 %.|
S? OZY
SO XS! e.
S9 asp I ,.
60 ~rp
It Ol11 9 o9
il COle
45 ImtJl m
14 PPP
45 A4P
Conclusion
Acknowledgements. The authors wish to thank G. Dahlquist and H.J. Stetter for extremely
helpful comments on a former draft of this paper. Moreover, they are indebted to their colleague
E. Hairer for his careful reading of the manuscript.
References
1. Bader, G.: Numerische Behandlung steifer Differentialgleichungen mit einer modifizierten Mit-
telpunktsregel. Technische Universit~it Mtinchen, Institut ftir Mathematik: Diploma thesis, 1977
2. Bulirsch, R., Stoer, J.: Numerical Treatment of Ordinary Differential Equations by Extrapola-
tion Methods. Numer. Math. 8, 1-t3 (t966)
3. Dahlquist, G.: A Special Stability Problem for Linear Multistep Methods. BIT 3, 27-43 (1963)
4, Dahlquist, G., Lindberg, B.: On Some Implicit One-Step Methods for Stiff Differential Equa-
tions. Royal Institute of Technology, Stockholm: Tech. Rep. TRITA-NA-7302, 1973
5. Deuflhard, P.: A Study of Discretization Schemes due to Hersch with Application to Stiff
Differential Systems. Lecture given at the Oberwolfach Meeting on "Numerical Treatment of
Differential Equations", July 4-July 10, 1976 (Unpublished)
6. Deuflhard, P.: Order and Stepsize Control in Extrapolation Methods. Numer. Math. 41, 399-
422 (1983)
7. Deuflhard, P., Bader, G., Nowak, U.: LARKIN - A Software Package for the Numerical
Simulation of LARge Systems arising in Chemical Reaction KINetics. Springer Series
chem. Phys. 18, 38-55 (1981)
8. Enright, W.H., Hull, T.E., Lindberg, B.: Comparing Numerical Methods for Stiff Systems of
ODEs. BIT 15, 10-48 (1975)
9. Garfinkel, D., Hess, B.: Metabolic Control Mechanisms VII. A Detailed Computer Model of
the Glycolytic Pathway in Ascites Cells. J. Bio. Chem. 239, 971-983 (1964)
398 G. Bader and P. Deuflhard
10. Gear, C.W.: Numerical Initial Value Problems in Ordinary Differential Equations. New York:
Prentice Hall, 1971
11. Gragg, W.B.: Repeated Extrapolation to the Limit in the Numerical Solution of Ordinary
Differential Equations. University of California, Los Angeles: Thesis, 1963
12. Gragg, W.B.: On Extrapolation Algorithms for Ordinary Initial Value Problems. SIAM J. Nu-
mer. Anal. 2, 384-404 (1965)
13. Hindmarsh, A.C.: GEAR-Ordinary Differential Equation System Solver. Lawrence Livermore
Laboratory: Tech. Rep. UCID-30001, Rev. 3 (Dec. 1974)
14. Hofer, E.: A Partially Implicit Method for Large Stiff Systems of ODEs with only Few Equa-
tions Introducing Small Time-Constants. SIAM J. Numer. Anal. 13, 645-663 (1976)
15. Kaps, P., Rentrop, P.: Generalized Runge-Kutta Methods of Order Four with Step Size Con-
trol for Stiff Ordinary Differential Equations. Numer. Math. 33, 55-68 (1979)
16. Lawson, J.D.: Generalized Runge-Kutta Processes for Stable Systems with Large Lipschitz
Constants. SIAM J. Numer. Anal. 4, 372-380 (1967)
17. Mirbeth, M.: Eine modifizierte Mittelpunktsregel zur numerischen Integration steifer Systeme
von Differentialgleichungen. Technische Universit~it Miinchen, Institut f'tir Mathematik: Di-
ploma thesis, 1975
18. Prothero, A., Robinson, A.: On the Stability and Accuracy of One-Step Methods for Solving
Stiff Systems of Ordinary Differential Equations. Math. Comput. 28, 145-162 (1974)
19. Scott, M.R., Watts, H.A.: A Systematized Collection of Codes for Solwng Two-Point Bound-
ary-Value Problems. SANDIA Lab., Albuquerque: Tech. Rep. SAND 75-0539, 1975
20. Shampine, L.F.: Evaluation of a Test Set for Stiff ODE Solvers. ACM TOMS 7, 409-420 (1981)
21. Sherman, A.H.: NSPIV-A FORTRAN Subroutine for Gaussian Elimination with Partial
Pivoting. University of Texas at Austin: Tech. Rep. TR-65, CNA-118, 1977
22. Stetter, H.J.: Symmetric Two-Step Algorithms for Ordinary Differential Equations. Comput. 5,
267-280 (1970)
23. Stetter, H.J.: Analysis of Discretization Methods for Ordinary Differential Equations. Springer
Tracts in Natural Philosophy. Berlin, Heidelberg, New York: Springer, 1973
24. Stoer, J., Bulirsch, R.: Einfiihrung in die Numerische Mathematik II. Berlin, Heidelberg, New
York: Springer, 1973
25. Wilkinson, J.H.: The Algebraic Eigenvalue Problem. Clarendon, Oxford, 1965
26. Hairer, E., Bader, G., Lubich, C.: On the Stability of Semi-Implicit Methods for Ordinary
Differential Equations. BIT 22, 211-232 (1982)