Numerical Simulation of Difference Equations
Numerical Simulation of Difference Equations
Numerical Simulation of Difference Equations
Numerical Simulation
of Difference Equations
An important ingredient in school mathematics is solution of algebraic equations like x + 3 = 4. The challenge is to determine a numerical value for x such
that the equation holds. In this chapter we are going to give a brief review of
difference equations or recurrence relations. In contrast to traditional equations,
the unknown in a difference equation is not a single number, but a sequence of
numbers.
For some simple difference equations, an explicit formula for the solution
can be found with pencil-and-paper methods, and we will review some of these
methods in section 6.4. For most difference equations, there are no explicit solutions. However, a large group of equations can be solved numerically, or simulated, on a computer, and in section 6.3 we will see how this can be done.
In chapter 5 we saw how real numbers can be approximated by floatingpoint numbers, and how the limitations inherent in floating-point numbers sometimes may cause dramatic errors. In section 6.5 we will see how round-off errors
affect the simulation of even the simplest difference equations.
A common type of puzzle goes like this: Suppose a man has a son that is half
his age, and the son will be 16 years younger than his father in 5 years time. How
old are they?
With equations we do not worry about the ages, but rather write down what
we know. If the age of the father is x and the age of the son is y, the first piece of
information can be expressed as y = x/2, and the second as y = x 16. This has
given us two equations in the two unknowns x and y,
y = x/2,
y = x 16.
Once we have the equations we use standard techniques to solve them. In this
case, we find that x = 32 and y = 16. This means that the father is 32 years old,
and the son 16.
Exercises
1 One of the oldest known age puzzles is known as Diophantus riddle. It comes from the
Greek Anthology, a collection of puzzles compiled by Metrodorus of Chios in about 500
AD. The puzzle claims to tell how long Diophantus lived in the form of a riddle engraved
on his tombstone:
God vouchsafed that he should be a boy for the sixth part of his life; when a
twelfth was added, his cheeks acquired a beard; He kindled for him the light
of marriage after a seventh, and in the fifth year after his marriage He granted
him a son. Alas! late-begotten and miserable child, when he had reached the
measure of half his fathers life, the chill grave took him. After consoling his
grief by this science of numbers for four years, he reached the end of his life.
How old were Diophantus and his son at the end of their lives?
Example 6.1. A simple difference equation arises if we try to model the growth
of money in a bank account. Suppose that the amount of money in the account
after n years is x n , and the interest rate is 5 % per year. If interest is added once
a year, the amount of money after n + 1 years is given by the difference equation
x n+1 = x n + 0.05x n = 1.05x n .
(6.1)
This equation characterises the growth of all bank accounts with a 5 % interest
rate in order to characterise a specific account we need to know how much
money there was in the account to start with. If the initial deposit was 100 000 (in
your favourite currency) at time n = 0, we have an initial condition x 0 = 100 000.
This gives the complete model
x n+1 = 1.05x n ,
x 0 = 100 000.
(6.2)
Example 6.2. Suppose that we withdraw 1 000 from the account every year. If
we include this in our model we obtain the equation
x n+1 = 1.05x n 1 000.
(6.3)
If we start with the same amount x 0 = 100 000 as above, we now find x 1 = 104 000,
x 2 = 108 200, and x 10 150 312.
Example 6.3. As the capital accumulates, it is reasonable that the owner increases the withdrawals. If for example the amount withdrawn increases by 300
each year, we get the model
x n+1 = 1.05x n (1 000 + 300n).
In this case we find x 1 = 104 000, x 2 = 107 900, and x 10 = 134 844.
113
(6.4)
250 000
200 000
150 000
10
15
20
Figure 6.1. The growth of capital according to the models (6.1) (largest growth), (6.3) (middle growth), and
(6.4) (smallest growth).
Plots of the development of the capital in the three different cases are shown
in figure 6.1. Note that in the case of (6.4) it appears that the growth of the capital
levels out. In fact, it can be shown that after about 45 years, all the capital will be
gone, and x 46 will in fact be negative, i.e., money must be borrowed in order to
keep up the withdrawals.
After these simple examples, let us define difference equations in general.
Definition 6.4 (Difference equation). A difference equation or recurrence relation is an equation that involves the terms of an unknown sequence {x n }.
The equation is said to be of order k if a term in the sequence depends on k
previous terms, as in
x n+k = f (n, x n , x n+1 , . . . , x n+k1 ),
(6.5)
day a person with the illness infects one new person, and the infected person
becomes ill after three days. This means that on day n, the total number of ill
persons are the people who were ill yesterday, plus the number of people who
were infected three days ago. But this latter number equals the number of people that were ill three days ago. If x n denotes the number of ill people on day n,
we therefore have
x n = x n1 + x n3 , n = 3, 4, . . . ,
or
x n+3 = x n+2 + x n ,
n = 0, 1, . . .
n = 0, 1, . . .
(6.6)
x1 = a1 ,
...
xk = ak ,
(6.7)
This corresponds to starting with a population of one new-born pair which remains the only one until this pair gives birth to another pair after k months.
6.2.2 Linear difference equations
It is convenient to classify difference equations according to their structure, and
for many purposes the simplest ones are the linear difference equations.
Definition 6.7. A kth-order difference equation is said to be linear and inhomogenous if it has the form
x n+k = g (n) + f 0 (n)x n + f 1 (n)x n+1 + + f k1 (n)x n+k1 ,
where g and f 0 , . . . , f k1 are functions of n. The equation is said to have constant coefficients if the functions f 0 . . . , f k1 do not depend on n. It is said to
be homogenous if g (n) = 0 for all n.
From this definition we see that all the difference equations we have encountered so far have been linear, with constant coefficients. The equations (6.3) and
(6.4) are inhomogenous, the others are homogenous.
Linear difference equations are important because it is relatively simple to
predict and analyse their solutions, as we will see in section 6.4.
6.2.3 Solving difference equations
In examples 6.16.3 we saw how easy it is to compute the terms of the sequence
determined by a difference equation since the equation itself is quite simply a
formula which tells us how one term can be computed from the previous ones.
Provided the functions involved are computable and the calculations are done
correctly (without round-off errors), we can therefore determine the exact value
of any term in the solution sequence in this way. We refer to this as simulating
the difference equation.
There is another way to solve a difference equation, namely by determining
an explicit formula for the solution. For instance, the difference equations in
examples 6.16.3 turn out to have the solutions that are given by the formulas
x n = 100 000 1.05n ,
n
(6.8)
(6.9)
(6.10)
The advantage of these formulas is that we can compute the value of a term immediately, without first computing all the preceding terms. With such formulas
116
we can also deduce the asymptotic behaviour of the solution. For example we
see straightaway from (6.10) that in the situation in example 6.3, all the capital will eventually be used up, since x n becomes negative for sufficiently large n.
Another use of solution formulas like the ones in (6.8)(6.10) is for predicting the
effect of round-off errors on the numerical solutions computed by simulating a
difference equation, see section 6.5.
Observation 6.8 (Solving difference equations). There are two different ways
to solve a difference equation:
1. By simulating the equation, i.e., by starting with the initial values, and
then successively computing the numerical values of the terms of the
solution sequence, as in examples 6.16.3.
2. By finding an explicit formula for the solution sequence, as in (6.8)
(6.10).
Exercises
1 Compare with (6.5) and determine the function f for the difference equations below. Also
compute the values of x 2 , . . . , x 5 in each case.
a) x n+2 = 3x n+1 x n ,
b) x n+2 = x n+1 + 3x n ,
x 0 = 2, x 1 = 1.
x 0 = 4, x 1 = 5.
c) x n+2 = 2x n+1 x n , x 0 = 1, x 1 = 2.
d) x n+1 = 4 x n , x 0 = 0.
e) 5x n+2 3x n+1 + x n = n,
2
f ) x n+1
+ 5x n = 1,
x 0 = 3.
x 0 = 0, x 1 = 1.
117
b) x n+3 x n+1 + x n2 = 0.
c) x n+2 + x n+1 x n = 0.
d) nx n+2 x n+1 e n + x n = n 2 .
x0 = a0 , x1 = a1 ,
(6.11)
where g , f 0 and f 1 are given functions of n, and a 0 and a 1 are given real numbers.
Let us consider an example to remind ourselves how the terms are computed.
Example 6.9. We consider the difference equation
x n+2 = n + 2x n 3nx n+1 ,
x 0 = 1, x 1 = 2,
This algorithm computes all the N + 1 terms and saves them in the array
x = [x 0 , . . . , x N ]. Sometimes we are only interested in the last term x N , or we just
want to print out the terms as they are computed then there is no need to
store all the terms.
Algorithm 6.11. The following algorithm computes the solution of (6.11), just
like algorithm 6.10, but prints each term instead of storing them:
x pp = a 0 ;
x p = a1 ;
for i = 2, 3, . . . , N
x = g (i 2) + f 0 (i 2)x pp + f 1 (i 2)x p ;
print x;
x pp = x p ;
x p = x;
The algorithm is based on the simple fact that in order to compute the next
term, we only need to know the two previous terms, since the equation is of
second order. At time i , the previous term x i 1 is stored in x p and the term x i 2
is stored in x pp . Once x i has been computed, we must prepare for the next step
and make sure that x p is shifted down to x pp , which is not needed anymore, and
x is stored in x p . Note that it is important that these assignments are performed
in the right order. At the beginning, the values of x p and x pp are given by the
initial values.
In both of these algorithms it is assumed that the coefficients given by the
functions g , f 0 and f 1 , as well as the initial values a 0 and a 1 , are known. In practice, the coefficient functions would usually be entered as functions (or methods) in the programming language you are using, while the initial values could
be read from the terminal or via a graphical user interface.
Algorithms (6.10) and (6.11) can easily be generalised to 3rd or 4th order, or
equations of any fixed order, and not only linear equations. The most convenient is to have an algorithm which takes the order of the equation as input.
Algorithm 6.12. The following algorithm computes and prints the first N + 1
terms of the solution of the kth-order difference equation
x n+k = f (n, x n , x n+1 , . . . , x n+k1 ),
n = 0, 1, . . . , N k
(6.12)
for i = 0, 1, . . . , k 1
zi = ai ;
print z i ;
for i = k, k + 1, . . . , N
x = f (i k, z 0 , . . . , z k1 );
print x;
for j = 0, . . . , k 2
z i = z i +1 ;
z k1 = x;
Algorithm 6.12 is similar to algorithm 6.11 in that it does not store all the
terms of the solution sequence. To compensate it keeps track of the k previous
terms in the array z = [z 0 , . . . , z k1 ]. The values x k , x k+1 , . . . , x N are computed in
the second for-loop. By comparison with (6.12) we observe that i = n + k; this
explains i k = n as the first argument to f . The initial values are clearly correct
the first time through the loop, and at the end of the loop they are shifted along
so that the value in z 0 is lost and the new value x is stored in z k1 .
Difference equations have the nice feature that a term in the unknown sequence is defined explicitly as a function of previous values. This is what makes
it so simple to generate the values by algorithms like the ones sketched here.
Provided the algorithms are correct and all operations are performed without
errors, the exact solution will be computed. When the algorithms are implemented on a computer, this is the case if all the initial values are integers and all
computations can be performed without introducing floating-point numbers.
One example is the Fibonacci equation
x n+2 = x n + x n+1 ,
x 0 = 1, x 1 = 1.
Exercises
1 Program algorithm 6.11 and test it on the Fibonacci equation
x n+2 = x n+1 + x n ,
120
x 0 = 0, x 1 = 1.
2 Generalise algorithm 6.11 to third order equations and test it on the Fibonacci like equation
x n+3 = x n+2 + x n+1 + x n , x 0 = 0, x 1 = 1, x 2 = 1.
3 A close relative of the Fibonacci numbers is called the Lucas numbers, and these are defined by the difference equation
L n+2 = L n+1 + L n ,
L 0 = 2, L 1 = 1.
(6.13)
where b is some real number. Often we are interested in x n for all n 0, but any
value of n Z makes sense in the following. From (6.13) we find
x n+1 = bx n = b 2 x n1 = b 3 x n2 = = b n+1 x 0 .
This is the content of the first lemma.
121
(6.14)
n Z,
n Z.
if |b| < 1;
0,
lim |x n | =
,
if |b| > 1;
|x |, if |b| = 1.
0
Phrased differently, the solution of the difference equation will either tend
to 0 or , except in the case where |b| = 1.
6.4.2 Second-order homogenous equations
The general second-order homogenous equation is
x n+2 + b 1 x n+1 + b 0 x n = 0.
(6.15)
The basic idea behind solving this equation is to try with a solution x n = r n in
the same form as the solution of first-order equations, and see if there are any
values of r for which this works. If we insert x n = r n in (6.15) we obtain
0 = x n+2 + b 1 x n+1 + b 0 x n = r n+2 + b 1 r n+1 + b 0 r n = r n (r 2 + b 1 r + b 0 ).
In other words, we must either have r = 0, which is uninteresting, or r must be a
solution of the quadratic equation
r 2 + b1 r + b0 = 0
122
which is called the characteristic equation of the difference equation. If the characteristic equation has the two solutions r 1 and r 2 , we know that both y n = r 1n
and z n = r 2n will be solutions of (6.15). And since the equation is linear, it can be
shown that any combination
x n = C r 1n + Dr 2n
is also a solution of (6.15) for any choice of the numbers C and D. However,
in the case that r 1 = r 2 this does not give the complete solution, and if the two
solutions are complex conjugates of each other, the solution may be expressed
in a more adequate form that does not involve complex numbers. In either case,
the two free coefficients can be adapted to two initial conditions like x 0 = a 0 and
x1 = a1 .
Theorem 6.15. The solution of the homogenous, second-order difference
equation
x n+2 + b 1 x n+1 + b 0 x n = 0
(6.16)
is governed by the solutions r 1 and r 2 of the characteristic equation
r 2 + b1 r + b0 = 0
as follows:
1. If the two roots r 1 and r 2 are real and distinct, the general solution of
(6.16) is given by
x n = C r 1n + Dr 2n .
2. If the two roots are equal, r 1 = r 2 , the general solution of (6.16) is given
by
x n = (C + Dn)r 1n .
3. If the two roots are complex conjugates of each other so that r 1 = r and
r 2 = r, and r has the polar form as r = e i , then the general solution of
(6.16) is given by
x n = n (C cos n + D sin n).
In all three cases the solution can be determined uniquely by two initial conditions x 0 = a 0 and x 1 = a 1 , where a 0 and a 1 are given real numbers, since this
determines the two free coefficients C and D uniquely.
The proof of the theorem is not so complicated and can be found in a text
123
x 0 = 2, x 1 = 9.
9 = x 1 = 2C + 7D,
whose solution is C = 1 and D = 1. The solution that satisfies the initial conditions is therfore
x n = 2 n + 7n .
Example 6.17. The difference equation
x n+2 2x n+1 + 2x n = 0,
x 0 = 1, x 1 = 1,
x n = 2 C cos(n/4) + D sin(n/4) .
We determine C and D by enforcing the initial conditions
0
2 (C cos 0 + D sin 0) = C ,
124
Corollary 6.18. Suppose that one root, say r 1 , of the characteristic equation
satisfies |r 1 | > 1, that C = 0, and that |r 2 | < |r 1 |. Then
lim |x n | = .
Note that in cases 2 and 3 in theorem 6.15, the two roots have the same absolute value (in case 2 the roots are equal and in case 3 they both have absolute
value ). This means that it is only in the first case that we need to distinguish
between the two roots in the conditions in corollary 6.18.
Proof of corollary 6.18. In cases 2 and 3 in theorem 6.15 |r 1 | = |r 2 |, so if |r 1 | > 1
and |r 2 | < |r 1 | we must have two real roots. Then we can write the solution as
r n
2
x n = r 1n C + D
r1
and therefore
r n
2
= |C | lim |r 1 |n = .
lim |x n | = lim |r 1 |n C + D
n
n
n
r1
If both |r 1 | < 1 and |r 2 | < 1 and both roots are real, the triangle inequality
leads to
(6.17)
will also be a solution for any choice of the coefficients {C i }. As we have already
seen, an equation of order k can be adapted to k initial values.
To determine k solutions, we follow the same procedure as for second-order
equations and try the solution x n = r n . We then find that r must solve the characteristic equation
r k + b k1 r k1 + + b 1 r + b 0 = 0.
From the fundamental theorem of algebra we know that this equation has k distinct roots, and complex roots occur in conjugate pairs since the coefficients are
real. A theorem similar to theorem 6.15 can therefore be proved.
Observation 6.19. The general solution of the difference equation
x n+k + b k1 x n+k1 + + b 1 x n+1 + b 0 x n = 0
is a combination of k terms
x n = C 1 x n1 +C 2 x n2 + +C k x nk
where each term x ni is a solution of the difference equation. The solution
i
x n is essentially of the form x ni = r in where r i is the i th root of the characteristic equation
r k + b k1 r k1 + + b 1 r + b 0 = 0.
Note the word essentially in the last sentence: just like for quadratic equations we have to take special care when there are double roots (or roots of even
higher multiplicity) or complex roots.
Closed formulas for the roots can be found for quadratic, cubic and quartic
equations, but the expressions even for cubic equations can be rather complicated. For higher degree than 2, one therefore has to resort to numerical techniques, like the ones in chapter 9, for finding the roots.
126
There is also an analog of corollary 6.18 which shows that a solution will
tend to zero if all roots have absolute value less than 1. And if there is a root
with absolute value greater than 1, whose corresponding coefficient in (6.17) is
nonzero, then the solution will grow beyond all bounds when n becomes large.
6.4.4 Inhomogenous equations
So far we have only discussed homogenous difference equations. For inhomogenous equations there is an important, but simple lemma, which can be found in
standard text books on difference equations.
p
(6.18)
Then all other solutions of the inhomogenous equation will have the form
p
x n = x n + x nh
where x nh is some solution of the homogenous equation
More informally, lemma 6.20 means that we can find the general solution of
(6.18) by just finding one solution, and then adding the general solution of the
homogenous equation. The question is how to find one solution. The following
observation is useful.
Observation 6.21. One of the solutions of the inhomogenous equation
x n+k + b k1 x n+k1 + + b 1 x n+1 + b 0 x n = g (n)
has the same form as g (n).
Some examples will illustrate how this works.
Example 6.22. Consider the equation
x n+1 2x n = 3.
127
(6.19)
Here the right-hand side is constant, so we try with the a particular solution
x nh = A, where A is an unknown constant to be determined. If we insert this in
the equation we find
A 2A = 3,
p
In general, when g (n) is a polynomial in n of degree d , we try with a particular solution which is a general polynomial of degree d . When this is inserted
in the equation, we obtain a relation between two polynomials that should hold
for all values of n, and this requires corresponding coefficients to be equal. In
this way we obtain a set of equations for the coefficients.
Example 6.23. In the third-order equation
x n+3 2x n+2 + 4x n+1 + x n = n
(6.20)
From these equations we find B = 1/4 and A = 3/16, so one solution of (6.20) is
p
xn =
n
3
.
4 16
There are situations where the technique above does not work because the
trial polynomial solution is also a homogenous solution. In this case the degree
of the polynomial must be increased. For more details we refer to a text book on
difference equations.
Other types of right-hand sides can be treated similarly. One other type is
given by functions like
g (n) = p(n)a n ,
where p(n) is a polynomial in n and a is a real number. In this case, one tries
p
with a solution x n = q(n)a n where q(n) is a general polynomial in n of the same
degree as p(n).
128
(6.21)
x n = (A + B n)3n .
When this is inserted in the difference equation we obtain
= 3n 3 A + B (n + 1) + 4A + 4B n
= 3n (7A + 3B + 7B n).
Here we can cancel 3n , which reduces the equation to an equality between two
polynomials. If these are to agree for all values of n, the constant terms and the
linear terms must agree,
7A + 3B = 0,
7B = 1.
This system has the solution B = 1/7 and A = 1/49, so a particular solution of
(6.21) is
1
1 n
p
xn = n
3 .
7
49
x n = x nh + x n = C 4n +
1 n
3 .
49
Exercises
1 Find a unique solution for x n for the following difference equations:
a) x n+1 = 3x n ,
x 0 = 5/3
b) x n+2 = 3x n+1 + 2x n ,
c) x n+2 = 2x n+1 x n ,
d) x n+2 = 2x n+1 + 3x n ,
x 0 = 3, x 1 = 4.
x 0 = 1, x 1 = 1
x 0 = 2, x 1 = 1
129
x 0 = 2, x 1 = 4.
b) x n+2 3x n+1 + 2x n = 2n + 1,
c) 2x n+2 3x n
= 15 2n ,
d) x n+1 x n = 5n2n ,
x 0 = 1, x 1 = 3.
x 0 = 3, x 1 = 6.
x 0 = 1, x 1 = 5
3 Find the unique solution of the difference equation described in equation 6.4 with initial
condition x 0 = 100000, and show that all the capital is indeed lost after 45 years.
4 Remember that the Fibonacci numbers are defined as:
F n+2 = F n+1 + F n , F 0 = 0, F 1 = 1.
Remember also from exercise 6.3.3 that the Lucas numbers are defined as:
L n+2 = L n+1 + L n , L 0 = 2, L 1 = 1.
Use this to prove the following identities:
a) L n = F n+1 + F n1
b) F 2n = F n L n
therefore focus on the case where floating-point numbers must be used. Note
that we use 64-bit floating-point numbers in all the examples in this chapter.
The effect of round-off errors becomes quite visible from a couple of examples.
Example 6.26. Consider the equation
2
1
x n+2 x n+1 x n = 0,
3
3
x 0 = 1, x 1 = 0.
(6.22)
C + D = 1,
C D/3 = 0,
which has the solution C = 1/4 and D = 3/4. The solution of (6.22) is therefore
xn =
1
1 + (1)n 31n .
4
19
x n+1 + 2x n = 10,
3
131
x 0 = 2, x 1 = 8/3.
(6.23)
The two roots of the characteristic equation are r 1 = 1/3 and r 2 = 6, so the general solution of the homogenous equation is
x nh = C 3n + D6n .
p
To find a particular solution we try a solution x n = A which has the same form
as the right-hand side. We insert this in the difference equation and find A = 3,
so the general solution is
p
x n = x nh + x n = 3 +C 3n + D6n .
(6.24)
8/3 = x 1 = 3 +C /3 + 6D.
(6.25)
(6.26)
C + 18D = 1.
which has the solution C = 1 and D = 0. The final solution is therefore
x n = 3 3n ,
(6.27)
x10 = 2.99998306646,
x15 = 3.00001192858.
These values appear to approach 3 as they should. However, some later values
are
x20 = 3.09329859009,
x30 = 5641411.98633,
(6.28)
14
x40 = 3.41114428655 10 ,
and at least the last two of these are obviously completely wrong!
132
19
x n+1 2x n .
3
The number 10, and the coefficient 2 can be represented exactly. The middle
coefficient 19/3, however, cannot be represented exactly by floating-point numbers, and is replaced by the nearest floating-point number c = 19/3 + , where
is a small number of magnitude roughly 1017 .
Observation 6.28. When the difference equation (6.23) is simulated numerically, round-off errors cause the difference equation and initial conditions to
become
19
x n+2
+ x n+1 + 2x n = 10, x 0 = 2, x 1 = 8/3 + ,
(6.29)
3
where and are both small numbers of magnitude roughly 1017 .
The effect of round-off errors in the coefficients
So the actual computations are based on the difference equation (6.29), and not
(6.23), but we can still determine a formula for the exact solution that is being
computed. The characteristic equation now becomes
19
r2
+ r +2 = 0
3
which has the two roots
1
r 1 = 19 + 3 289 + 114 + 92 ,
6
r2 =
133
1
19 + 3 + 289 + 114 + 92 .
6
The dependence on in these formulas is quite complicated, but can be simplified by the help of Taylor-polynomials which we will learn about in chapter 8.
Using this technique, it is possible to show that
r1
1
,
3 17
r2 6 +
18
.
17
In addition, since the right-hand side of (6.29) is constant, we try with a particular solution that is constant. If we do this we find the particular solution
p
xn =
30
.
10 + 3
This means that the general formula for the solution of the difference equation
(6.29) is
1 n
30
18 n
xn =
+C
+D 6+
.
10 + 3
3 17
17
When is of magnitude 1017 , this expression will be very close to
x n = 3 +C
1 n
+ D6n
(6.30)
for all values of n that we typically encounter in practice. This simplifies the
analysis of round-off errors for linear difference equations considerably: We can
simply ignore round-off in the coefficients.
Observation 6.29. The round-off errors that occur in the coefficients of the
difference equation (6.29) do not lead to significant errors in the solution of
the equation. This is true for general, linear difference equations with constant coefficients: Round-off errors in the coefficients (and the right-hand
side) are not significant and may be ignored.
The effect of round-off errors in the initial values
We next consider the effect of round-off errors in the initial values. From what
we have just seen, we may assume that the result of the simulation is described
by the general formula (6.30). The initial values are
x 0 = 2,
x 1 = 8/3 + ,
8/3 + = x 1 = 3 +C /3 + 6D.
134
3
,
17
D=
3
.
17
(6.31)
This is summarised in the next observation where for simplicity we have introduced the notation = 3/17.
Observation 6.30. Because of round-off errors in the second initial value, the
result of numerical simulation of (6.24) corresponds to using a solution in the
form (6.24), where C and D are given by
C = 1 + ,
D =
(6.32)
(6.33)
From observation 6.30 it is easy to explain where the values in (6.28) come
from. Because of round-off errors, the computed solution is given by (6.33),
where is a small nonzero number. Even if is small, the product 6n will eventually become large, since 6n grows beyond all bounds when n becomes large.
We can in fact use the result of the numerical simulation to estimate . From
(6.28) we have x40 3.4 1014 , and for n = 40 we also have 3n 8.2 1020 and
6n 1.3 1031 . Since we have used 64-bit floating-point numbers, this means
that only the last term in (6.33) is relevant (the other two terms affect the result
in about the 30th digit and beyond). This means that we can find from the
relation
3.4 1014 x40 640 1.3 1031 .
From this we see that 2.6 1017 . This is a reasonable value since we know
that is roughly as large as the round-off error in the initial values. With 64bit floating-point numbers we have about 1518 decimal digits, so a round-off
error of about 1017 is to be expected when the numbers are close to 1 as in this
example.
Observation 6.31. When is nonzero in (6.33), the last term 6n will eventually dominate the computed solution of the difference equation completely,
and the computations will end in overflow.
135
It is important to realise that the reason for the values generated by the numerical simulation in (6.28) becoming large is not particularly bad round-off
errors; any round-off error at all would eventually lead to the same kind of behaviour. The general problem is that the difference equation corresponds to a
family of solutions given by
x n = 3 +C 3n + D6n ,
(6.34)
and different initial conditions pick out different solutions (different values of C
and D) within this family. The exact solution has D = 0. However, for numerical simulation with floating-point numbers it is basically impossible to get D to
be exactly 0, so the last term in (6.34) will always dominate the computed solution for large values of n and completely overwhelm the other two terms in the
solution.
6.5.2 Round-off errors for linear equations of general order
The difference equation in example 6.27 is not particularly demanding we
will get the same effect whenever we have a difference equation where the exact solution remains significantly smaller than the part of the general solution
corresponding to the largest root of the characteristic equation.
Observation 6.32. Suppose the difference equation
x n+k + b nk x nk + + b 1 x n+1 + b 0 x n = g (n)
is simulated numerically with floating-point numbers, and let r be the root of
the characteristic equation,
r k + b k1 r k1 + + b 1 r + b 0 = 0,
with largest absolute value. If the particular solution of the inhomogenous
equation does not grow as fast as |r |n (in case |r | > 1), or decays faster than
|r |n (in the case |r | < 1), then the computed solution will eventually be dominated by the solution corresponding to the root r , regardless of what the initial
values are.
In example 6.27, the solution family has three components: the two solutions 6n and 3n from the homogenous equation, and the constant solution 3
from the inhomogenous equation. When the solution we are interested in just
involves 3n and 3 we get into trouble since we invariably also bring along 6n
because of round-off errors. On the other hand, if the exact initial values lead to
136
a solution that includes 6n , then we will not get problems with round-off: The
coefficient multiplying 6n will be accurate enough, and the other terms are too
small to pollute the 6n solution.
Example 6.33. We consider the third-order difference equation
16
17
4
17
107
x n+2 + x n+1 x n = 10 2n , x 0 = 2, x 1 = , x 2 =
.
3
3
3
3
9
The coefficients have been chosen so that the roots of the characteristic equation are r 1 = 1/3, r 2 = 1 and r 3 = 4. To find a particular solution we try with
p
x n = A2n . If this is inserted in the equation we find A = 3, so the general solution is
x n = 3 2n + B 3n +C + D4n .
(6.35)
x n+3
(6.36)
The discussion above shows that this is bound to lead to problems. Because of
round-off errors, the coefficients B and D will not be exactly 0 when the equation
is simulated. Instead we will have
xn = 3 2n + 1 3n + (1 + 2 ) + 3 4n
Even if 3 is small, the term 3 4n will dominate when n becomes large. This is
confirmed if we do the simulations. The computed value x100 is approximately
4.5 1043 , while the exact value is 3.8 1030 , rounded to two digits.
Exercises
1 In each of the cases, find the analytical solution of the difference equation, and describe
the behavior of the simulated solution for large values of n:
a) x n+1 13 x n = 2,
x0 = 2
x 0 = 1/7, x 1 = 1/7
x 0 = 1, x 1 = 2/3
x 0 = 5/14.
5 n
5
14
137
5 n
5 .
14
(6.37)
x 0 = 0, x 1 = (1 5)/2.
(6.38)
n
n
1+ 5
1 5
xn = C
+D
,
2
2
1 5
.
2
x 0 = 1, x 1 = 1/15.
(6.39)
a) Determine the general solution of (6.39) as well as the solution selected by the initial
condition.
b) Why must you expect problems when you do a numerical simulation of the equation?
c) Determine approximately the value of n when the numerical solution has lost all
significant digits.
d) Perform the numerical simulation and check that your predictions are correct.
5 In this exercise we consider the difference equation
5
x n+2 x n+1 + x n = 0,
2
x 0 = 1, x 1 = 1/2.
a) Determine the general solution, and the solution corresponding to the initial conditions.
b) What kind of behaviour do you expect if you simulate the equation numerically?
c) Do the simulation and explain your results.
138
6.6 Summary
In this chapter we met the effect of round-off errors on realistic computations for
the first time. We saw that innocent-looking computations like the simulation of
the difference equation in example 6.27 led to serious problems with round-off
errors. By making use of the theory behind linear difference equations with constant coefficients, we were able to understand why the simulations behave the
way they do. From this insight we also realise that for this particular equation
and initial values, the blow-up is unavoidable, just like cancellation is unavoidable when we subtract two almost equal numbers. Such problems are usually
referred to as being badly conditioned. On the other hand, a different choice of
initial conditions may lead to calculations with no round-off problems; then the
problem is said to be well conditioned.
139