Linear Multistep Methods page 1

Linear Multistep Methods1

Note: The authoritative reference for the material on convergence is the book by
Peter Henrici, Discrete Variable Methods in Ordinary Differential Equations, Wi-
ley, 1962. The best reference on absolute stability is the book by Jack Lambert,
Numerical Methods for Ordinary Differential Systems, Wiley, 1991.
We consider the Initial Value Problem (IVP)

y ′ = f (x, y), y(a) = y0 , x ∈ [a, b], y(x), f (x, y) ∈ IRm (1)

where we assume that there exists some Lipschitz Constant L such that

kf (x, y) − f (x, z)k ≤ Lky − zk, for all (x, y), (x, z) ∈ [a, b] × IRm (2)

Assumption (2) implies that the initial value problem (1) possesses a unique solu-
tion.
We use the notation

xn = a + nh, yn ≈ y(xn ), fn = f (xn , yn ), n = 0, 1, 2, · · · (3)

Our fundamental problem for the first few weeks of this semester is how to compute
the yn . They may be defined by Linear Multistep Methods (LMMs):

k
X k
X
αj yn+j = h βj fn+j (4)
j=0 j=0

Here, k is the step-number , h is the step-size (assumed constant for the moment),
αk = 1, |α0 | + |β0 | =
6 0, the method is explicit if βk = 0, otherwise it is implicit .
We are concerned with the Global Truncation Error

en = y(xn ) − yn . (5)

The LMM (4) is said to be convergent if, for all IVPs (1) satisfying (2), all x ∈ [a, b],
and all starting strategies yµ = ηµ (h), µ = 0, 1, · · · , k − 1 satisfying

lim ηµ (h) = y0 (6)

h−→0

the following holds:

lim yn = y(x) (7)
h−→0,nh=x−a

(Note that the requirement on the starting values is rather weak, it is satisfied e.g.
by the “strategy” yµ = y0 .)
An obvious minimum requirement for a LMM is that it be convergent.

Two simple Examples

To get an idea of what’s involved in convergence, let’s look at two extremely simple
examples. This will illustrate a

1
by Peter Alfeld. TEX processed January 29, 2010

1
 Linear Multistep Methods page 2
Major Technique. Consider a test problem, compute the general solutions of the
numerical method and the analytical problem, and compare them.

What’s the simplest IVP imaginable? How about

y ′ = 0, y(0) = y0 . (8)

Clearly, the solution is

y(x) = y0 . (9)
Now let‘s compute the general solution of the difference method

k
X
αj yn+j = 0. (10)
j=0

This is a homogeneous linear difference equation whose solutions define a k dimen-

sional linear space of sequences yn , n = 0, 1, 2, 3, · · ·.
To understand the space of solutions better we associate with the LMM (4) its first
and second characteristic polynomial ρ and σ, respectively by

k
X k
X
j
ρ(ξ) = αj ξ , and σ(ξ) = βj ξ j . (11)
j=0 j=0

It is obvious that if
ρ(r) = 0 (12)
then
yn = r n (13)
is a solution of the difference equation (10). Any linear combination of solutions of
the form (13) also solves (10). If there are k distinct roots of ρ then all solutions
are linear combinations of solutions of the form (13). If there is a multiple root r
satisfying
ρ(r) = ρ′ (r) = · · · = ρq (r) = 0 (14)
then q + 1 corresponding linearly independent solutions of (10) are given by2

y n = nj r n where j = 0, 1, · · · , q. (15)

In what follows let’s suppose for simplicity that all roots r1 , r2 , · · ·, rk of ρ are
distinct. The general solution of the difference equation is then given by

k
X
yn = γµ rµn . (16)
µ=1

The coefficients γj are defined by the starting values.

How can this solution converge to the true solution of the IVP (8)? Clearly we must
have:
— 1 must be a root of ρ.
— No root of ρ can exceed 1 in absolute value.
— In view of what was said above about multiple roots, any root of ρ of absolute value
equal to 1 must be simple.
2
Verifying this is a good exercise.

2
 Linear Multistep Methods page 3
Let’s note these properties formally

a. ρ(1) = 0
b. ρ(r) = 0 =⇒ |r| ≤ 1 (17)
′
c. ρ(r) = 0 and |r| = 1 =⇒ ρ (r) 6= 0

Definition. A method that satisfies properties b. and c. is said to be zero-stable.

To gain more insight, let’s look at a slightly more complicated IVP, say

y ′ = 1, y(0) = 0 (18)

which obviously has the solution

y(x) = x. (19)

The difference equation now becomes:

k
X k
X
αj yn+j = h βj (20)
j=0 j=0

This is an inhomogeneous linear difference equation and its general solution can
be written as any particular solution plus the general solution of the homogeneous
equation (8). To find a particular solution we have to do some inspired guessing.
Considering the analytical solution suggests to try a solution of the form

yn = γxn = γnh (21)

where γ is as yet undetermined. Plugging this into the difference equation and using
property a. above yields
 
Xk X k Xk
γ αj (n + j)h = γ nh αj + h αj j 
j=0 j=0 j=0
k
X
= γh αj j (22)
j=0
k
X
=h βj
j=0

Clearly, the last equation requires

k
X
βj
j=0 σ(1)
γ= = . (23)
k
X ρ′ (1)
αj j
j=0

It is now almost obvious3 that for convergence we have to have that

γ = 1. (24)
3
Work out the details! Also, what happens if ρ′ (1) = 0?

3
 Linear Multistep Methods page 4
Let’s summarize: For the LMM to converge just for the simple DEs y ′ = 0 and
y ′ = 1 we have to have properties a.–c. above, and, in addition:
d. σ(1) = ρ′ (1). (25)

Definition. A method that satisfies a. and d. is said to be consistent.

Why did we do all this? Most amazingly, it turns out that these properties are also
sufficient for convergence in general! Thus
the LMM will converge for all IVPs (1) if
it converges just for y ′ = 0 and y ′ = 1!
Marvel at that! A proof of this fact is given in Henrici’s book but is beyond the
scope of this course.
Thus it turns out that

LMM convergent ⇐⇒ it is consistent and zero-stable

Local Truncation Error

Associated with the LMM is its Local Truncation Error LTE4 defined by
k
X k
X
LTE = αj y(xn+j ) − h βj y ′ (xn+j ) (28)
j=0 j=0

where y(x) is the true solution of the IVP (1). Assuming that the solution can be
expanded into a power series about xn 5 it turns out that
∞
X
LTE = Ci hi y (i) (xn ) (29)
i=0

where
k
X  
C0 = αj = ρ(1)
j=0
k
X k
X  
C1 = jαj − βj = ρ′ (1) − σ(1) (30)
j=1 j=0
k k
1 X q 1 X
Cq = j αj − j q−1 βj , q = 2, 3, · · ·
q! j=1 (q − 1)! j=1
The LMM is of order p if
C0 = C1 = · · · = Cp = 0, Cp+1 6= 0. (31)
The number Cp+1 is the error constant of the LMM .
4
There is also an “error under the localizing assumption”
yn+j = y(xn+j ), j = 0, 1, · · · , k − 1. (26)
It follows by a simple application of the Mean Value Theorem that under that
assumption
y(xn+k ) − yn+k = LT E + O(hp+1 ). (27)
Thus, at least asymptotically, the local truncation error, and the error under the
localizing assumption, are equivalent.
5
as seen in class, the order and error constant are independent of the point about
which the LTE is expanded.

4
 Linear Multistep Methods page 5
Note. According to our earlier definition, a LMM is consistent if it is of order at
least 1.

Note. We can now reinterpret the two requirements for convergence: Stability
means that errors do not get unduly amplified, and consistency means that the
error introduced at each step is not too large.

The following table gives the maximum order of convergent LMMs

explicit implicit
k even k k+2
k odd k k+1

Examples
Of particular importance among the following examples are the Adams Methods
and the Backward Differentiation formulas. These are the ones used in the many
versions of Gear’s package.
Here are some examples for explicit convergent LMMs:
1. Euler’s Method. p = k = 1, C2 = 12 :

yn+1 − yn = hfn (32)

2. The Midpoint Rule. p = k = 2, C3 = 13 :

yn+2 − yn = 2hfn+1 (33)

3. Adams-Bashforth Methods. p = k, k = 1, 2, 3, · · ·:
k−1
X
yn+k − yn+k−1 = h βj∗ fn+j (34)
j=0

(Note that the requirement p = k uniquely defines the βj∗ .)

Here are some examples for implicit convergent LMMs:
4. Backward (or Implicit) Euler Method. p = k = 1, C2 = − 21 :

yn+1 − yn = hfn+1 (35)

1
5. The Trapezoidal Rule. p = 2, k = 1, C3 = − 12 :

h
yn+1 − yn = (fn+1 + fn ) (36)
2
1
6. Simpson’s Rule. p = 4, k = 2, C5 = − 90 :

h
yn+2 − yn = (fn+2 + 4fn+1 + fn ) (37)
3
In spite of being convergent, the performance of Simpson’s Rule is mediocre (due
to the spurious root of the first characteristic polynomial being −1).
7. Adams-Moulton Methods. p = k + 1, k = 1, 2, 3, · · ·:
k
X
yn+k − yn+k−1 = h βj fn+j (38)
j=0

5
 Linear Multistep Methods page 6
(Note that the requirement p = k + 1 uniquely defines the βj .
8. Backward Differentiation Methods. p = k, k = 1, 2, 3, 4, 5, 6:
k
X
αj yn+j = hβk fn+k (39)
j=0

These methods are not zero-stable if k > 6. Note that the requirement p = k
uniquely defines the αj .

Non-Convergent Maximal Linear Multistep Methods

The coefficients of an LMM can be chosen so as to maximize the order. Matching
the number of parameters with the number of conditions suggests that an explicit
method can have order 2k − 1, and implicit one order 2k. It turns out that the
linear equations are always consistent, and that you don’t get a higher order for
free. Thus these values are indeed attained. LMMs of maximum possible order
are called maximal. Note that they may not be convergent. Examples of maximal
methods listed above include: Euler’s Method, the Trapezoidal Rule, and Simpson’s
Rule. Following are some additional divergent methods:

Explicit LMMs
9. k = 2, p = 3, C4 = 16 :

yn+2 + 4yn+1 − 5yn = h (4fn+1 + 2fn ) (40)

10. k = 3, p = 5, C6 = 12 .

yn+3 + 18yn+2 − 9yn+1 − 10yn = h (9fn+2 + 18fn+1 + 3fn ) (41)

Implicit LMMs
−3
11. k = 3, p = 6, C7 = 140 .

11yn+3 + 27yn+2 − 27yn+1 − 11yn = h (3fn+3 + 27fn+2 + 27fn+1 + 3fn ) (42)

Absolute Stability
We now know how to find convergent methods, but these are not necessarily good
methods (because they may be inefficient). To get better insight we need another
test equation. However, that equation is not as obvious as the previous ones (y ′ = 0
and y ′ = 1). It can be motivated as follows:
y ′ = f (x, y)
Linearize



≈ y ′ (x) + fy x, y(x) y − y(x)
= A(x)y + g(x)
Freeze
(43)
≈ Ay + g
Homogenize
−→ y ′ = Ay
Diagonalize
−→ y ′ = λy where λ ∈ C

6
 Linear Multistep Methods page 7
The individual steps can be justified as follows:
Linearize This is simply a Taylor expansion of first order about the exact solution of the
differential equation. It is reasonable as long as we are close to the exact solution.
Freeze We move along the x-axis in small steps, so this assumption is locally valid. Globally
it is of course totally unrealistic.
Homogenize The inhomogeneous term g only adds a constant term to the solution, and can
be ignored (since we already dealt with constant components of the solution by
considering the test equation y ′ = 0.
diagonalize Suppose there is a similarity transform

D = C −1 AC (44)

where C and D are complex, and D is diagonal This is possible whenever A is

non-defective, i.e., it possesses a full set of linearly independent eigenvectors. In
that case, the columns of C are the eigenvectors of A, and the diagonal entries of
D are the corresponding eigenvalues.
We are thus led to the test equation

y ′ = λy (45)

where λ is a complex number that plays the role of one of the eigenvalues of the
Jacobian of f . Note that this closely models the situation in solving the problem
y ′ = Ay analytically6 .
Plugging the differential equation (45) into the LMM (4) yields the difference equa-
tion
X k
(αj − κβj ) yn+j = 0 (46)
j=0

where
κ = hλ ∈ C. (47)
This is once more a homogeneous linear difference equation whose solutions are
governed by the roots of the stability polynomial

k
X
π(κ, ξ) = (αj − κβj ) ξ j = ρ(ξ) − κσ(ξ). (48)
j=0

The roots of a polynomial depend continuously on the coefficients of the polynomial.

The roots of the stability polynomial therefore depend continuously on κ.

Definition. The root r = r(κ) of π(κ, ξ) satisfying r(0) = 1 is said to be the

essential root of π(κ, ξ). The other roots are called the spurious roots of π(κ, ξ).

Definition. We say that the LMM (4) is absolutely stable 7 for a given κ = hλ if
all solutions of the test equation (45) with stepsize h converge to zero as n tends to
infinity.

Equivalently, the LMM is absolutely stable for a given κ if all roots of the stability
polynomial π are less than 1 in absolute value. The set of all κ for which the LMM
is absolutely stable is the region of absolute stability of the LMM. This is a subset
of the complex plane.
6
As an exercise you may wish to work out the connection.
7
There is also a much less important concept of relative stability which requires that
the spurious roots grow no faster than the essential root

7
 Linear Multistep Methods page 8
Note. It is important to realize that the LMM being absolutely stable does not
imply a value judgment. The analytic solutions of the test equation (45) will tend
to zero as x tends to infinity iff the real part of λ is negative, and it will grow
unboundedly if the real part of λ is positive. Thus the ideal region of absolute
stability of an LMM is the open left half plane, no larger or smaller.

The boundary of the region of absolute stability is a subset of the set

ρ(eiθ ) √
 
BL(ρ, σ) = z ∈ C| z = , i= −1, θ ∈ [0, 2π] . (49)
σ(eiθ )

(The notation BL means “boundary locus”).

Example. For Euler’s Method we obtain

eiθ − 1
z= = cos θ + i sin θ − 1 (50)
1
which is the unit circle shifted to the left by 1. For κ = 0 the only root of the
stability polynomial is ξ = 1. Euler’s method is thus absolutely stable for κ inside
the circle. Clearly, it is not absolutely stable outside of the circle.

Example. For the Trapezoidal Rule we obtain

1
ρ(ξ) = ξ − 1, and σ(ξ) = (ξ + 1). (51)
2
Thus





2 eiθ − 1 2 eiθ − 1 e−iθ + 1 2 1 + eiθ − e−iθ − 1 4i sin θ
z= = = 2 = 2.
eiθ + 1 (eiθ + 1) (e−iθ + 1) |eiθ + 1| |eiθ + 1|
(52)
Thus the boundary locus is the imaginary axis. A simple check shows that the
region of absolute stability is the open half plane, and that the method is absolutely
unstable for positive real parts of κ. Thus the Trapezoidal Rule has precisely the
“right” region of absolute stability.

We need a more powerful tool to verify that all roots of a given polynomial are
in the unit circle. A polynomial Φ with complex coefficients is said to be a Schur
Polynomial if all of its roots have absolute value less than 1. The following criterion8
may be used to identify Schur polynomials:
(Schur Criterion.) Let

Φ(r) = ck rk + ck−1 rk−1 + · · · + c1 r + c0 , (53)

where c0 ck 6= 0, and the ci denote complex coefficients. Define

Φ̂(r) = c̄0 rk + c̄1 rk−1 + · · · + c̄k−1 r + c̄k (54)

(where c̄ denotes the conjugate complex of c) and

1

Φ1 (r) = Φ̂(0)Φ(r) − Φ(0)Φ̂(r) . (55)
r
8
An excellent reference on this and related topics is: J.J.H. Miller, On the location
of zeros of certain classes of polynomials with applications to numerical analysis,
J. Inst. Math. Appl., 8, 397–406 (1971). The journal (JIMA) is available in the
Marriott Library.

8
 Linear Multistep Methods page 9
Clearly, Φ1 has degree at most k − 1. Then, Φ is a Schur Polynomial if and only if

|Φ̂(0)| > |Φ(0)| (56)

and Φ1 is a Schur Polynomial.
The set BL(ρ, σ) divides the complex plane into a finite number of subsets. The
Schur Criterion can be applied to one point in each of these subsets, and the region
of absolute stability can be identified in this manner.
Next is a (casual) argument that shows that for a convergent LMM the essential
root of ρ behaves as it should for small real values of hλ. (How should it behave?
It should be greater than 1 for positive κ, and smaller than 1 for negative κ.)
The roots of the stability polynomial are defined by




π κ, r(κ) = ρ r(κ) − κσ r(κ) = 0. (57)
Differentiating implicitly with respect to κ yields




ρ′ r(κ) r′ (κ) − σ r(κ) − κσ ′ r(κ) r′ (κ) = 0. (58)
Solving for r′ (κ) yields the differential equation


′ σ r(κ)
r (κ) = ′

. (59)
ρ r(κ) − κσ ′ r(κ)
The essential root r1 satisfies r1 (0) = 1. Thus we obtain
σ(1)
r1 (0) = 1 and r1′ (0) = =1 (60)
ρ′ (1)
Increasing κ somewhat will r1 (to beyond 1) and decreasing κ will decrease r1 (below
1).
This kind of argument can be extended to the complex plane.
The following consequences are immediate:
1. A convergent LMM cannot be absolutely stable for small positive values of ℜ(κ).
2. A convergent LMM for which all spurious roots of the first characteristic polynomial
are strictly within the unit circle will have a non-empty region of absolute stability
to the left of the origin.
Note. Item 2. is the reason for the success of the Adams methods. Here the
spurious roots are all equal to zero. Hence it will take a while for them to move out
of the unit circle as κ changes. As a consequence the Adams methods have fairly
large regions of absolute stability.

Following is another application of the above kind of reasoning:

Example. Consider the spurious root of Simpson’s Rule. It satisfies

2


σ(−1) 1/3 ξ + 4ξ + 1 1
r2 (0) = −1 and r2′ (0) = ′ = =+ . (61)

ρ (−1) 2ξ 3


ξ=−1

Thus the spurious root will become more negative and thus larger than 1 in absolute
value as κ becomes negative. Simpson’s Rule is therefore absolutely unstable for
small negative (as well as for small positive) values of κ. In fact, the region of
absolute stability of Simpson’s Rule is empty (exercise!).
We next ask for methods with infinite regions of absolute stability, including the
entire left half plane if possible.
However, we are stymied immediately by the following

9
 Linear Multistep Methods page 10
Observation. The region of absolute stability of an explicit method is bounded.

To see this note that as κ tends to infinity (in some direction in the complex plane)
the leading coefficient of the stability polynomial will converge to zero relative to
some of the others. So in a sense the stability polynomial approaches a polynomial
of degree less than k. Thus at least one root somehow has to “vanish”. The only way
this can happen is that one of the roots of the stability polynomial tends to infinity
as κ tends to infinity. (Working out the details will be a home-work problem.)
We are now at a crossroads. We could either
— Insist on using explicit methods because of their simplicity, or
— Explore the use of implicit methods.
Both branches have their merits, and we will explore them both, beginning with
the first.

10