Main

Numerical Methods for Engineers
Leif Rune Hellevik
Department of Structural Engineering, NTNU
Jan 26, 2018

Contents
1 Introduction 7
1.1 Acknowledgements and dedications . . . . . . . . . . . . . . . . . 7
1.2 Check Python and LiClipse plugin . . . . . . . . . . . . . . . . . 8
1.3 Scientific computing with Python . . . . . . . . . . . . . . . . . . 10
2 Initial value problems for ODEs 11

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.1 Example: A mathematical pendulum . . . . . . . . . . . . 12
2.1.2 n-th order linear ordinary differential equations . . . . . . 13
2.2 Existence and uniqueness of solutions for initial value problems . 13
2.3 Taylor’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.3.1 Example: Taylor’s method for the non-linear mathematical
pendulum . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3.2 Example: Newton’s first differential equation . . . . . . . 17
2.4 Reduction of Higher order Equations . . . . . . . . . . . . . . . . 17
2.4.1 Example: Reduction of higher order systems . . . . . . . 19
2.5 Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.5.1 Example: Discretization of a diffusion term . . . . . . . . 26
2.6 Euler’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.6.1 Example: Eulers method on a simple ODE . . . . . . . . 28
2.6.2 Example: Eulers method on the mathematical pendulum 29
2.6.3 Example: Generic euler implementation on the mathemat-
ical pendulum . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6.4 Example: Sphere in free fall . . . . . . . . . . . . . . . . . 32
2.6.5 Euler’s method for a system . . . . . . . . . . . . . . . . . 34
2.6.6 Example: Falling sphere with constant and varying drag . 35
2.7 Python functions with vector arguments and modules . . . . . . 42
2.8 How to make a Python-module and some useful programming
features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.8.1 Example: Numerical error as a function of ∆t . . . . . . . 50
2.9 Heun’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.9.1 Example: Newton’s equation . . . . . . . . . . . . . . . . 54
2.9.2 Example: Falling sphere with Heun’s method . . . . . . . 56
2.10 Generic second order Runge-Kutta method . . . . . . . . . . . . 59
2
CONTENTS 3
2.11 Runge-Kutta of 4th order . . . . . . . . . . . . . . . . . . . . . . 60

2.11.1 Example: Falling sphere using RK4 . . . . . . . . . . . . 62
2.11.2 Example: Particle motion in two dimensions . . . . . . . 64
2.12 Basic notions on numerical methods for IVPs . . . . . . . . . . . 70
2.13 Variable time stepping methods . . . . . . . . . . . . . . . . . . . 72
2.14 Numerical error as a function of ∆t for ODE-schemes . . . . . . 75
2.15 Absolute stability of numerical methods for ODE IVPs . . . . . . 92
2.15.1 Example: Stability of Euler’s method . . . . . . . . . . . 92
2.15.2 Example: Stability of Heun’s method . . . . . . . . . . . 95
2.15.3 Stability of higher order RK-methods . . . . . . . . . . . 96
2.15.4 Stiff equations . . . . . . . . . . . . . . . . . . . . . . . . 97
2.15.5 Example: Stability of implicit Euler’s method . . . . . . . 99
2.15.6 Example: Stability of trapezoidal rule method . . . . . . 100
2.16 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3 Shooting Methods 108

3.1 Shooting methods for boundary value problems with linear ODEs 108
3.1.1 Example: Couette-Poiseuille flow . . . . . . . . . . . . . . 112
3.1.2 Example: Simply supported beam with constant cross-
sectional area . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.1.3 Example: Simply supported beam with varying cross-
sectional area . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.2 Shooting methods for boundary value problems with nonlinear
ODEs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
3.2.1 Example: Large deflection of a cantilever . . . . . . . . . 129
3.3 Notes on similarity solutions . . . . . . . . . . . . . . . . . . . . . 135
3.3.1 Example: Freezing of a waterpipe . . . . . . . . . . . . . . 139
3.3.2 Example: Stokes’ first problem: flow over a suddenly
started plate . . . . . . . . . . . . . . . . . . . . . . . . . 140
3.3.3 Example: The Blasius equation . . . . . . . . . . . . . . . 141
3.4 Shooting method for linear ODEs with two unknown initial con-
ditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
3.4.1 Example: Liquid in a cylindrical container . . . . . . . . . 150
3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
4 Finite differences for ODEs 163

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
4.2 Errors and stability . . . . . . . . . . . . . . . . . . . . . . . . . . 164
4.2.1 Local truncation error . . . . . . . . . . . . . . . . . . . . 165
4.2.2 Global error . . . . . . . . . . . . . . . . . . . . . . . . . . 165
4.2.3 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.2.4 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . 166
4.2.5 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 167
4.2.6 Stability - Example in 2-norm . . . . . . . . . . . . . . . . 167
4.3 Tridiagonal systems of algebraic equations . . . . . . . . . . . . . 168
4.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
CONTENTS 4
4.4.1 Example: Heat exchanger with constant cross-section . . 172

4.4.2 Cooling rib with variable cross-section . . . . . . . . . . . 181
4.4.3 Example: Numerical solution for specific cooling rib . . . 184
4.5 Linearization of nonlinear algebraic equations . . . . . . . . . . . 189
4.5.1 Picard linearization . . . . . . . . . . . . . . . . . . . . . 190
4.5.2 Newton linearization . . . . . . . . . . . . . . . . . . . . . 194
4.5.3 Example: Newton linearization of various nonlinear ODEs 197
4.5.4 Various stop criteria . . . . . . . . . . . . . . . . . . . . . 200
4.5.5 Example: Usage of the various stop criteria . . . . . . . . 201
4.5.6 Linerarizations of nonlinear equations on incremental form 204
4.5.7 Quasi-linearization . . . . . . . . . . . . . . . . . . . . . . 206
4.5.8 Example: . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
5 Mathematical properties of PDEs 217

5.1 Model equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
5.1.1 List of some model equations . . . . . . . . . . . . . . . . 217
5.2 First order partial differential equations . . . . . . . . . . . . . . 218
5.3 Second order partial differenatial equations . . . . . . . . . . . . 222
5.4 Boundary conditions for 2nd order PDEs . . . . . . . . . . . . . 226
5.4.1 Hyberbolic equations . . . . . . . . . . . . . . . . . . . . . 226
5.4.2 Elliptic equations . . . . . . . . . . . . . . . . . . . . . . . 227
5.4.3 Parabolic equations . . . . . . . . . . . . . . . . . . . . . 228
6 Elliptic PDEs 229

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
6.2 Finite differences. Notation . . . . . . . . . . . . . . . . . . . . . 231
6.2.1 Example: Discretization of the Laplace equation . . . . . 233
6.3 Direct numerical solution . . . . . . . . . . . . . . . . . . . . . . 233
6.3.1 Neumann boundary conditions . . . . . . . . . . . . . . . 241
6.4 Iterative methods for linear algebraic equation systems . . . . . . 247
6.4.1 Stop criteria . . . . . . . . . . . . . . . . . . . . . . . . . 252
6.4.2 Optimal relaxation parameter . . . . . . . . . . . . . . . . 254
6.4.3 Example using SOR . . . . . . . . . . . . . . . . . . . . . 255
6.4.4 Initial guess and boundary conditions . . . . . . . . . . . 257
6.4.5 Example: A non-linear elliptic PDE . . . . . . . . . . . . 258
7 Diffusjonsproblemer 264
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
7.2 Confined, unsteady Couette flow . . . . . . . . . . . . . . . . . . 265
7.3 Stability: Criterion for positive coefficients. PC-criterion . . . . . 270
7.4 Stability analysis with von Neumann’s method . . . . . . . . . . 272
7.4.1 Example: Practical usage of the von Neumann condition . 276
7.5 Some numerical schemes for parabolic equations . . . . . . . . . 279
7.5.1 Richardson scheme (1910) . . . . . . . . . . . . . . . . . . 279
7.5.2 Dufort-Frankel scheme (1953) . . . . . . . . . . . . . . . . 280
CONTENTS 5
7.5.3 Crank-Nicolson scheme. θ-scheme . . . . . . . . . . . . . 282

7.5.4 Generalized von Neumann stability analysis . . . . . . . . 288
7.6 Truncation error, consistency and convergence . . . . . . . . . . . 290
7.6.1 Truncation error . . . . . . . . . . . . . . . . . . . . . . . 290
7.6.2 Consistency . . . . . . . . . . . . . . . . . . . . . . . . . . 292
7.6.3 Example: Consistency of the FTCS-scheme . . . . . . . . 292
7.6.4 Example: Consistency of the DuFort-Frankel scheme . . . 292
7.6.5 Convergence . . . . . . . . . . . . . . . . . . . . . . . . . 293
7.7 Example with radial symmetry . . . . . . . . . . . . . . . . . . . 293
7.7.1 Example: Start-up flow in a tube . . . . . . . . . . . . . . 296
7.7.2 Example: Cooling of a sphere . . . . . . . . . . . . . . . . 303
8 hyperbolic PDEs 307

8.1 The advection equation . . . . . . . . . . . . . . . . . . . . . . . 307
8.2 Forward in time central in space discretization . . . . . . . . . . 308
8.3 Upwind schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
8.4 The modified differential equation . . . . . . . . . . . . . . . . . 314
8.5 Errors due to diffusion and dispersion . . . . . . . . . . . . . . . 316
8.5.1 Example: Advection equation . . . . . . . . . . . . . . . . 317
8.5.2 Example: Diffusion and dispersion errors for the upwind
schemes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318
8.5.3 Example: Diffusion and disperision errors for the Lax-
Wendroff scheme . . . . . . . . . . . . . . . . . . . . . . . 319
8.6 The Lax-Friedrich Scheme . . . . . . . . . . . . . . . . . . . . . . 322
8.7 Lax-Wendroff Schemes . . . . . . . . . . . . . . . . . . . . . . . . 322
8.7.1 Lax-Wendroff for non-linear systems of hyperbolic PDEs . 323
8.7.2 Code example for various schemes for the advection equation325
8.8 Order analysis on various schemes for the advection equation . . 328
8.8.1 Separating spatial and temporal discretization error . . . 331
8.9 Flux limiters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
8.10 Example: Burger’s equation . . . . . . . . . . . . . . . . . . . . . 342
8.10.1 Upwind Scheme . . . . . . . . . . . . . . . . . . . . . . . . 342
8.10.2 Lax-Friedrich . . . . . . . . . . . . . . . . . . . . . . . . . 342
8.10.3 Lax-Wendroff . . . . . . . . . . . . . . . . . . . . . . . . . 343
8.10.4 MacCormack . . . . . . . . . . . . . . . . . . . . . . . . . 344
8.10.5 Method of Manufactured solution . . . . . . . . . . . . . . 344
9 Python Style Guide 347

9.1 The conventions we use . . . . . . . . . . . . . . . . . . . . . . . 347
9.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
9.3 The code is the documentation... i.e. the docs always lie! . . . . . 348
9.4 Docstring Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
10 Sympolic computation with SymPy 349

10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
10.2 Basic features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
CONTENTS 6
Bibliography 352
Chapter 1
Introduction
1.1 Acknowledgements and dedications

This digital compendium is based on the compendium Numeriske Beregninger
by my good friend and colleague Jan B. Aarseth who shared all of his work
so generously and willingly. His compendium served as an excellent basis for
further development and modernisation in terms of accessibility on the web,
programming language, and naturally numerous theoretical extensions and
removals. Primarily, the digital compendium is intended to be used in the course
TKT-4140 Numerical Methods at NTNU.
The development of the technical solutions for this digital compendium results
from collaborations with professor Hans Petter Langtangen, who has developed
Doconce for flexible typesetting, and associate professor Hallvard Trætteberg
at IDI, NTNU, who has developed the webpage-parser which identifies and
downloads Python-code from webpages for integration in Eclipse IDEs. The the
development of the digital compendium has been funded by the project IKTiSU.
and NTNU Teaching Excellense 2015-2016.
Johan Kolstø Sønstabø was instrumental in the initial phase and the develop-
ment of the first version of the Digital Compendium consisting of only Chapter
1 and for programming examples and development of exercises.
Fredrik Eikeland Fossan has contributed enormously by implementing python
codes for most chapters. In particular, his contributions to the test procedures
for MMS and to algorithms for testing of the order of accuracy of numerical
schemes are highly appreciated.
Lucas Omar Müller taught the course TKT 4140 in the spring semester
2017. Apart from doing an excellent job in teaching by bringing in a wide range
of new elements, he has also made numerous contributions in translating the
compendium from Norwegian to English and by adding theoretical sections.
Marie Kjeldsen Bergvoll made important contributions during her summer-
job with the translation of chapters 2-7 from LATEX to Doconce. Tragically,
Marie passed away due to a sudden illness while being in China working on her
7
CHAPTER 1. INTRODUCTION 8
master thesis in March 2017. Her choice of courses were motivated by interest,
curiousity and a hunger for knowledge. The very same mentality brought her to
countries like Germany and China, where English is not the official language.
She was a energetic, independent, warm, and dedicated young student, with
good analytical skills and a talent for mathematics.
Hans Petter Langtangen, passed away on October 10th in 2016. Hans Petter
was an extraordinarily productive author, lecturer, supervisor and researcher.
His books on software and methods for solving differential equations are widely
used and have influenced these fields considerably. Ever since I meet Hans
Petter for the first time, when he acted as my opponent at my PhD-defense in
1999 and later on as a collaborator at CBC, I have learned to know him as an
enthusiastic and playful scholar with a great passion for science. He was truly
a unique character in our field. Who else would write his own flexible markup
language producing output to a range of web-formats and PDF, before setting
off to produce piles of textbooks? But maybe even more important was his
inspirational, energetic and natural ways of interaction with friends, colleagues,
and collaborators in a great variety of disciplines. I miss him dearly both as a
friend and collaborator.
During the work with this digital compendium, we have suffered the loss of
Hans Petter and Marie, two fantastic but quite different persons. Hans Petter
at the full bloom of his career, whereas Marie was a talented, young master
student, just about to start her career.
To honor the memories of both Hans Petter and Marie, I dedicate this digital
compendium to them.
Leif Rune Hellevik
Trondheim, October 19, 2017.
1.2 Check Python and LiClipse plugin

This section has the objective to verify that you have Python and the necessary
Python Packages installed, and also that the Eclipse/LiClipse IDE plugin is
working. Download and run the code systemCheck.py in your Eclipse/LiClipse

IDE. For an illustration on how to download the Eclipse/LiClipse plugin and
how to use it see this video LiClipsePlugin.mp4 LiClipsePlugin.ogg
# src-ch0/systemCheck.py
def systemCheck():
’’’
Check for necessary moduls needed in the course tkt4140:
matplotlib
numpy
scipy
sympy
’’’
installed = " ... installed!"
print ""
print "Check for necessary modules needed for the course tkt4140"
print ""
print "check for matplotlib ",

try:
import matplotlib.pyplot
print installed
except:
print " IMPORT ERROR; no version of matplotlib.pyplot found"
print "check for numpy ",

try:
import numpy
print installed
except:
print " IMPORT ERROR; no version of numpy found"
print "check for scipy ",

try:
import scipy
print installed
except:
print " IMPORT ERROR; no version of scipy found"
print "check for sympy ",

try:
import sympy
print installed
except:
print " IMPORT ERROR; no version of sympy found"
if __name__ == ’__main__’:
systemCheck()
1.3 Scientific computing with Python

In this course we will use the programming language Python to solve numerical
problems. Students not familiar with Python are strongly recommended to work
through the example Intro to scientific computing with Python before proceeding.
If you are familiar with Matlab the transfer to Python should not be a problem.
Chapter 2
Initial value problems for

Ordinary Differential
Equations
2.1 Introduction
With an initial value problem for an ordinary differential equation (ODE) we
mean a problem where all boundary conditions are given for one and the same
value of the independent variable. For a first order ODE we get e.g.
y 0 (x) = f (x, y) (2.1)
y(x0 ) = a (2.2)
while for a second order ODE we get
y 00 (x) = f (x, y, y 0 ) (2.3)

y(x0 ) = a, y 0 (x0 ) = b (2.4)
A first order ODE, as shown in Equation (2.2), will always be an initial value
problem. For Equation (2.4), on the other hand, we can for instance specify the
boundary conditions as follows,
y(x0 ) = a, y(x1 ) = b
With these boundary conditions Equation (2.4) presents a boundary value problem.
In many applications boundary value problems are more common than initial
value problems. But the solution technique for initial value problems may often
be applied to solve boundary value problems.
Both from an analytical and numerical viewpoint initial value problems are
easier to solve than boundary value problems, and methods for solution of initial
value problems are more developed than for boundary value problems.
11
CHAPTER 2. INITIAL VALUE PROBLEMS FOR ODES 12
2.1.1 Example: A mathematical pendulum

Consider the problem of the mathematical pendulum (see Figure 2.1). By using
Newton’s second law in the θ-direction one can show that the following equation
must be satisfied:
∂2θ g
2
+ sin(θ) = 0 (2.5)
∂τ l
dθ
θ(0) = θ0 , (0) = 0 (2.6)
dτ
g
θ
θ0
Figure 2.1: An illustration of the mathematical pendulum.
To present the governing equation (2.6) on

p a more convenient form, we
introduce a dimensionless time t given by t = gl · τ such that (2.5) and (2.6)
may be written as:
θ̈(t) + sin(θ(t)) = 0 (2.7)

θ(0) = θ0 , θ̇(0) = 0 (2.8)
The dot denotes derivation with respect to the dimensionless time t. For small
displacements we can set sin(θ) ≈ θ, such that (2.7) and (2.8) becomes
θ̈(t) + θ(t) = 0 (2.9)

θ(0) = θ0 , θ̇(0) = 0 (2.10)
The difference between (2.7) and (2.9) is that the latter is linear, while the
first is non-linear. The analytical solution of Equations (2.7) and (2.8) is given
in Appendix G.2. in the Numeriske Beregninger.
2.1.2 n-th order linear ordinary differential equations

An n-th order linear ODE may be written on the generic form:
an (x)y (n) (x) + an−1 (x)y (n−1) (x) + · · · + a1 (x)y 0 (x) + a0 (x)y(x) = b(x) (2.11)
where y (k) , k = 0, 1, . . . n is referring to the k’th derivative and y (0) (x) = y(x).
If one or more of the coefficients ak also are functions of at least one y (k) , k =
0, 1, . . . n, the ODE is non-linear. From (2.11) it follows that (2.7) is non-linear
and (2.9) is linear.
Analytical solutions of non-linear ODEs are rare, and except from some
special types, there are no general ways of finding such solutions. Therefore
non-linear equations must usually be solved numerically. In many cases this is
also the case for linear equations. For instance it doesn’t exist a method to solve
the general second order linear ODE given by
a2 (x) · y 00 (x) + a1 (x) · y 0 (x) + a0 (x) · y(x) = b(x)
From a numerical point of view the main difference between linear and
non-linear equations is the multitude of solutions that may arise when solving
non-linear equations. In a linear ODE it will be evident from the equation if
there are special critical points where the solution changes character, while this
is often not the case for non-linear equations.
For instance the equation y 0 (x) = y 2 (x), y(0) = 1 has the solution y(x) = 1−x
1
such that y(x) → ∞ for x → 1, which isn’t evident from the equation itself.
2.2 Existence and uniqueness of solutions for ini-

tial value problems
If we are to solve an initial value problem of the type in Equation (2.2), we
must first be sure that it has a solution and that the solution is unique. Such
conditions are guaranteed by the following criteria:
The criteria for existence and uniqueness

Consider the domain D = {(y, x) : |y − a| ≤ c, x0 ≤ x ≤ x1 }, with c an
arbitrary constant. If f (y, x) is Lipschitz continuous in y and continuous
in x over D, then there is a unique solution to the initial value problem
(2.2)-(2.2) at least up to time X ∗ = min(x1 , x0 + c/S), where
S = max |f (y, x)|.

(y,x)∈D
We say that f (y, x) is Lipschitz continuous in y over some domain D if there

exists some constant L ≥ 0 so that
|f (y, x) − f (y ∗ , x)| ≤ L|y − y ∗ |
for all (y, x) and (y ∗ , x) in D. Note that this requirement is more restrictive than
plain continuity, which only requires that |f (y, x) − f (y ∗ , x)| → 0 as y → y ∗ .
In fact, Lipschitz continuity requires that |f (y, t) − f (y ∗ , t)| → O(|y − y ∗ |) as
y → y ∗ . To give a definition for L, consider that f (y, x) is differentiable with
respect to y in D and that its derivative ∂f ∂y (y,x)
is bounded, then we can take
∂f (y, x)
L = max | |, (2.12)
(y,x)∈D ∂y
since
∂f (v, x)
f (y, x) = f (y ∗ , x) + (y − y ∗ )
∂y
for some value v between y and y ∗ .
For (2.4), or higher order ODEs in general, one has to consider that such
equations can be written as systems of first order ODEs. Similar criteria apply
in that case to define the existence and uniqueness of solutions. However, special
attention must be paid to the definition of Lipschitz continuity, in particular to
the definition of appropriate bounds in terms of a given norm. This aspect will
not be covered in this notes.
Fulfillment of the criteria for existence and uniqueness
Consider the initial value problem
y 0 (x) = (y(x))2 , y(0) = a > 0.
It is clear that f (y) = y 2 is independent of x and Lipschitz continuous in y over
any finite interval |y − a| ≤ c with L = 2(a + c) and S = (a + c)2 . In this case, we
can say that there is unique solution at least up to time c/(a + c)2 and then take
the value of c that maximizes that time, i.e. c = a, yielding a time of 1/(4a).
In this simple problem we can verify our considerations by looking at the
exact solution
1
y(x) = .
a−1 − x
It can be easily seen that y(x) → ∞ as x → 1/a, so that there is no solution
beyond time 1/a.
Violation of the criteria for existence and uniqueness
Consider the initial value problem
p
y 0 (x) = y(x) , y(0) = 0.
√
In this case we have that f (y) = y is not Lipschitz continuous near y = 0 since
√
f 0 (y) = 1/(2 y) → ∞ as y → 0. Therefore, we can not find a constant L so
that the bound in (2.12) holds for y and y ∗ near 0. This consideration implies
that this initial value problem does not have an unique solution. In fact, it has
two solutions:
y(x) = 0
and
1 2
y(x) = x .
4
2.3 Taylor’s method

Taylor’s formula for series expansion of a function f (x) around x0 is given by
(x − x0 )2 00 (x − x0 )n (n)
f (x) = f (x0 )+(x−x0 )·f 0 (x0 )+ f (x0 )+· · ·+ f (x0 )+higher order terms
2 n!
(2.13)
Let’s use this formula to find the first terms in the series expansion for θ(t)
around t = 0 from the differential equation given in (2.9):
θ̈(t) + θ(t) = 0 (2.14)

θ(0) = θ0 , θ̇(0) = 0 (2.15)
First we observe that the solution to the ODE in (2.15) may be expressed as
an Taylor expansion around the initial point:
t2 t3 ... t4
θ(t) ≈ θ(0) + t · θ̇(0) + θ̈(0) + θ (0) + θ(4) (0) (2.16)
2 6 24
By use of the initial conditions of the ODE in (2.15) θ(0) = θ0 , θ̇(0) = 0 we
get
t2 t3 ... t4
θ(t) ≈ θ0 + θ̈ + θ (0) + θ(4) (0) (2.17)
2 6 24
From the ODE in (2.15) we obtain expressions for the differentials at the initial
point:
θ̈(t) = −θ(t) → θ̈(0) = −θ(0) = −θ0 (2.18)
Expressions for higher order differentials evaluated at the initial point may
be obtained by further differentiation of the ODE (2.15)
...
θ (t) = −θ̇(t) → θ̈(0) = −θ(0) = −θ0 (2.19)
and
θ(4) (t) = −θ̈(t) → θ(4) (0) = −θ̈(0) = θ0

Substitution of these differentials into (2.17) yields
t2 t4 t2 t4

θ(t) ≈ θ0 1 − + = θ0 1 − + (2.20)
2 24 2! 4!
If we include n terms, we get
t2 t4 t6 t2n

θ(t) ≈ θ0 · 1 − + − + · · · + (−1)n
2! 4! 6! (2n)!
If we let n → ∞ we see that the parentheses give the series for cos(t). In this
case we have found the exact solution θ(t) = θ0 cos(t) of the differential equation.
Since this equation is linear we manage in this case to find a connection between
the coefficients such that we recognize the series expansion of cos(t).
Taylor’s Method for solution of initial value ODE..

1. Use the ODE (e.g. (2.2) or (2.4)) to evaluate the differentials at the
initial value.
2. Obtain an approximate solution of the ODE by substitution of the
differentials in the Taylor expansion (2.13).
3. Differentiate the ODE as many times needed to obtain the wanted

accuracy.
2.3.1 Example: Taylor’s method for the non-linear math-

ematical pendulum
Let’s try the same procedure on the non-linear version (2.7)
θ̈(t) + sin (θ(t)) = 0

θ(0) = θ0 , θ̇(0) = 0
2 3 ...
t4 (4)
We start in the same manner: θ(t) ≈ θ(0) + t2 θ̈(0) + t6 θ (0) + 24 θ (0). From
the differential equation we have θ̈ = − sin(θ) → θ̈(0) = − sin(θ0 ), which by
consecutive differentiation gives
... ...
θ = − cos(θ) · θ̇ → θ (0) = 0
θ(4) = sin(θ) · θ̇2 − cos(θ) · θ̈ → θ(4) (0) = −θ̈(0) cos(θ(0)) = sin(θ0 ) cos(θ0 )
2 4
Inserted above: θ(t) ≈ θ0 − t2 sin(θ0 ) + 24t
sin(θ0 ) cos(θ0 ).
We may include more terms, but this complicates the differentiation and it
is hard to find any connection between the coefficients. When we have found an
approximation for θ(t) we can get an approximation for θ̇(t) by differentiation:
3
θ̇(t) ≈ −t sin(θ0 ) + t8 sin(θ0 ) cos(θ0 ).
Series expansions are often useful around the starting point when we solve
initial value problems. The technique may also be used on non-linear equations.
Symbolic mathematical programs like Maple and Mathematica do this
easily.
2.3.2 Example: Newton’s first differential equation

We will end with one of the earliest known differential equations, which Newton
solved with series expansion in 1671.
y 0 (x) = 1 − 3x + y + x2 + xy, y(0) = 0
Series expansion around x = 0 gives
x2 00 x3 x4
y(x) ≈ x · y 0 (0) + y (0) + y 000 (0) + y (4) (0)
2 6 24
From the differential equation we get y 0 (0) = 1. By consecutive differentiation
we get
y 00 (x) = −3 + y 0 + 2x + xy 0 + y → y 00 (0) = −2
y (x) = y 00 + 2 + xy 00 + 2y 0
000
→ y 000 (0) = 2
y (4) (x) = y 000 + xy 000 + 3y 00 → y (4) (0) = −4
3 4
Inserting above gives y(x) ≈ x − x2 + x3 − x6 .
3 4 5 6
Newton gave the following solution: y(x) ≈ x − x2 + x3 − x6 + x30 − x45 .
Now you can check if Newton calculated correctly. Today it is possible to
give the solution on closed form with known functions as follows,
√ √
√

h x i 2 2
y(x) =3 2πe · exp x 1 + · erf (1 + x) − erf
2 2 2
h x i
+4 · 1 − exp[x 1 + −x
2
√
Note the combination 2πe. See Hairer et al. [6] section 1.2 for more details
on classical differential equations.
2.4 Reduction of Higher order Equations

When we are solving initial value problems, we usually need to write these as
sets of first order equations, because most of the program packages require this.
Example 1:
y 00 (x) + y(x) = 0, y(0) = a0 , y 0 (0) = b0

We may for instance write this equation in a system as follows,
y 0 (x) =g(x)
g 0 (x) = − y(x)
y(0) =a0 , g(0) = b0
Example 2:
Another of a third order ODE is:
y 000 (x) + 2y 00 (x) − (y 0 (x))2 + 2y(x) = x2 (2.21)

0 00
y(0) = a0 , y (0) = b0 , y (0) = c0
We set y 0 (x) = g(x) and y 00 (x) = g 0 (x) = f (x), and the system may be
written as
y 0 (x) =g(x)
g 0 (x) =f (x)
f 0 (x) = − 2f (x) + (g(x))2 − 2y(x) + x2
with initial values y(0) = a0 , g(0) = b0 , f (0) = c0 .
This is fair enough for hand calculations, but when we use program packages
a more systematic procedure is needed. Let’s use the equation above as an
example.
We start by renaming y to y0 . We then get the following procedure:
y 0 = y00 = y1
y 00 = y000 = y10 = y2
Finally, the third order ODE in (2.21) may be represented as a system of
first order ODEs:
y00 (x) =y1 (x)

y10 (x) =y2 (x)
y20 (x) = − 2y2 (x) + (y1 (x))2 − 2y0 (x) + x2
with initial conditions y0 (0) = a0 , y1 (0) = b0 , y2 (0) = c0 .
General procedure to reduce a higher order ODE to a system of

first order ODEs.
The general procedure to reduce a higher order ODE to a system of
first order ODEs becomes the following:
Given the equation
y (m) = f (x, y, y 0 , y 00 , . . . , y (m−1) ) (2.22)

y(x0 ) = a0 , y 0 (x0 ) = a1 , . . . , y (m−1) (x0 ) = am−1
where
dm y
y (m) ≡
dxm
with y = y0 , we get the following system of ODEs:
y00 = y1
y10 = y2
. (2.23)
.
0
ym−2 = ym−1
0
ym−1 = f (x, y0 , y1 , y2 , . . . , ym−1 )
with the following boundary conditions:
y0 (x0 ) = a0 , y1 (x0 ) = a1 , . . . , ym−1 (x0 ) = am−1
2.4.1 Example: Reduction of higher order systems

Write the following ODE as a system of first order ODEs:
y 000 − y 0 y 00 − (y 0 )2 + 2y = x3
y(0) = a, y 0 (0) = b, y 00 (0) = c
First we write y 000 = y 0 y 00 + (y 0 )2 − 2y + x3 .
By use of (2.23) we get
y00 = y1
y10 = y2
y20 = y1 y2 + (y1 )2 − 2y0 + x3
y0 (0) = a, y1 (0) = b, y2 = c
2.5 Differences
We will study some simple methods to solve initial value problems. Later we
shall see that these methods also may be used to solve boundary value problems
for ODEs.
For this purpose we need to introduce a suitable notation to enable us to
formulate the methods in a convenient manner. In Figure 2.2) an arbitrary
function y is illustrated as a continuous function of x, ie y = y(x). Later y will
be used to represent the solution of an ODE, which only may be represented
and obtained for discrete values of x. We represent these discrete, equidistant
values of x by:
xj = x0 + jh
where h = ∆x is assumed constant unless otherwise stated and j is an integer
counter referring to the discrete values xj for which the corresponding discrete
value
y
y(xj)
y(xj-1)
y(xj-2)
y(x1)
y(x0)
h h h
x0 x1 xj-2 xj-1 xj x
Figure 2.2: A generic function y(x) sampled at equidistant values of x. .
yj = y(xj )
may be found (see Figure 2.3).
y yj+1½ yj+2
yj+1
yj+½
yj
yj-½
yj-1
½h ½h ½h ½h
xj-1 xj-½ xj xj+½ xj+1 xj+1½ xj+2 x
Figure 2.3: Illustration of how to obtain difference equations.
Having introduced this notation we may the develop useful expressions and
notations for forward differences, backward differences, and central differences,
which will be used frequently later:
Forward differences:
∆yj = yj+1 − yj
Backward differences:
∇yj = yj − yj−1 (2.24)

Central differences:
δyj+ 21 = yj+1 − yj
The linear difference operators ∆, ∇ and δ are useful when we are deriving
more complicated expressions. An example of usage is as follows,
δ 2 yj = δ(δyj ) = δ(y1+ 12 − y1− 12 ) = yj+1 − yj − (yj − yj−1 ) = yj+1 − 2yj + yj−1
However, for clarity we will mainly write out the formulas entirely rather
than using operators.
We shall find difference formulas and need again:
Taylor’s theorem:
1
y(x) =y(x0 ) + y 0 (x0 ) · (x − x0 ) + y 00 (x0 ) · (x − x0 )2 + (2.25)
2
1 (n) n
· · · + y (x0 ) · (x − x0 ) + Rn
n!
The remainder Rn is given by
1
Rn = y (n+1) (ξ) · (x − x0 )n+1 (2.26)
(n + 1)!
where ξ ∈ (x0 , x)
Now, Taylor’s theorem (2.25) may be used to approximate the value of y(xj+1 ),
i.e. the forward value of y(xj ), by assuming that y(xj ) and it’s derivatives are
known:
h2 00
y(xj+1 ) ≡y(xj + h) = y(xj ) + hy 0 (xj ) + y (xj )+ (2.27)
2
hn y (n) (xj )
··· + + Rn
n!
where the remainder Rn = O(hn+1 ), h → 0.
From (2.27) we also get
h2 00 hk (−1)k y (k) (xj )

y(xj−1 ) ≡ y(xj − h) = y(xj ) − hy 0 (xj ) + y (xj ) + · · · + +...
2 k!
(2.28)
In the following we will assume that h is positive. By solving (2.27) with
respect to y 0 we may obtain a discrete forward difference approximation of y 0 at
xj :
y(xj+1 ) − y(xj )
y 0 (xj ) = + O(h) (2.29)
h
By solving (2.28) with respect to y 0 we obtain a discrete approximation at xi :
y(xj ) − y(xj−1 )
y 0 (xj ) = + O(h) (2.30)
h
By adding (2.28) and (2.27) together we get an approximation of the second
derivative at the location xj :
y(xj+1 ) − 2y(xj ) + y(xj−1 )

y 00 (xj ) = + O(h2 ) (2.31)
h2
By subtraction of (2.28) from (2.27) we get a backward difference approxi-
mation of the first order derivative at the location xj :
y(xj+1 ) − y(xj−1 )
y 0 (xj ) = + O(h2 ) (2.32)
2h
Notation:
We let y(xj ) always denote the function y(x) with x = xj . We use yj both
for the numerical and analytical value, the intended meaning is hopfullye clear
from the context.
Equations (2.29), (2.30), (2.31) and (2.32) then may be used to deduce the
following difference expressions:
yj+1 − yj
yj0 = ; truncation error O(h) (2.33)
h
yj − yj−1
yj0 = ; truncation error O(h) (2.34)
h
yj+1 − 2yj + yj−1
yj00 = ; truncation error O(h2 ) (2.35)
h2
yj+1 − yj−1
yj0 = ; truncation error O(h2 ) (2.36)
2h
In summary, equation (2.33) is a forward difference, (2.34) is a backward
difference while (2.35) and (2.36) are central differences.
The expressions in (2.33), (2.34), (2.35) and (2.36) may also conveniently be
established from Figure 2.4.
Whereas, (2.33) follows directly from the definition of the derivative, whereas
the second order derivative (2.35) may be obtained as a derivative of the derivative
by:

00 yj+1 − yj yj − yj−1 1 yj+1 − 2yj + yj−1
yj (xj ) = − · =
h h h h2
and an improved expression for the derivative (2.36) may be obtained by
averaging the forward and the backward derivatives:

0 yj+1 − yj yj − yj−1 1 yj+1 − yj−1
yj = + · =
h h 2 2h
y yj+1
yj
yj-1
½h ½h ½h ½h
xj-1 xj xj+1 x
Figure 2.4: Illustration of how the discrete values may be used to estimate
various orders of derivatives at location xi .
To find the truncation error we proceed in a systematical manner by:

y 0 (xj ) = a · y(xj−1 ) + b · y(xj ) + c · y(xj+1 ) + O(hm ) (2.37)
where we shall determine the constants a, b and c together with the error
term. For simplicity we use the notation yj ≡ y(xj ), yj0 ≡ y 0 (xj ) and so on.
From the Taylor series expansion in (2.27) and (2.28) we get
a · yj−1 + b · yj + c · yj+1 =
h2 h3

a · yj − hyj0 + yj00 − yj000 (ξ) + b · yj + (2.38)
2 6
2
h 00 h3 000

0
c · yj + hyj + yj + yj (ξ)
2 6
By collecting terms in Eq. (2.38) we get:
a · yj−1 + b · yj + c · yj+1 =
(a + b + c) yj + (c − a) h yj0 + (2.39)
2 3
h 00 h 000
(a + c) y + (c − a) y (ξ)
2 j 6
From Eq.(2.39) we may then find a, b and c such that yj0 gets as high accuracy
as possible by :
a+b+c=0
(c − a) · h = 1 (2.40)
a+c=0
The solution to (2.40) is

1 1
a=− , b = 0 and c =
2h 2h
which when inserted in (2.37) gives
yj+1 − yj−1 h2
yj0 = − y 000 (ξ) (2.41)
2h 6
By comparison of (2.41) with (2.37) we see that the error term is O(hm ) =
2
− h6 y 000 (ξ),
which means that m = 2. As expected, (2.41) is identical to (2.32).
Let’s use this method to find a forward difference expression for y 0 (xj ) with
accuracy of O(h2 ). Second order accuracy requires at least three unknown
coefficients. Thus,
y 0 (xj ) = a · yj + b · yj+1 + c · yj+2 + O(hm ) (2.42)

The procedure goes as in the previous example as follows,
a · yj + b · yj+1 + c · yj+2 =
h2 h3

a · yj + b · yj + hyj0 + yj00 + y 000 (ξ) +
2 6
3

8h 000
c · yj + 2hyj0 + 2h2 yj00 + yj (ξ)
6
=(a + b + c) · yj + (b + 2c) · hyj0
h3

b
+h2 + 2c · yj00 + (b + 8c) · y 000 (ξ)
2 6
We determine a, b and c such that yj0 becomes as accurate as possible. Then we
get,
a+b+c=0
(b + 2c) · h = 1 (2.43)
b
+ 2c = 0
2
The solution of (2.43) is
3 2 1
a=− , b= , c=−
2h h 2h
which inserted in (2.42) gives
−3yj + 4yj+1 − yj+2 h2
yj0 = + y 000 (ξ) (2.44)
2h 3
2
The error term O(hm ) = h3 y 000 (ξ) shows that m = 2.
Here follows some difference formulas derived with the procedure above:
Forward differences:
dyi yi+1 − yi 1
= + y 00 (ξ)∆x
dx ∆x 2
dyi −3yi + 4yi+1 − yi+2 1
= + y 000 (ξ) · (∆x)2
dx 2∆x 3
dyi −11yi + 18yi+1 − 9yi+2 + yi+3 1
= + y (4) (ξ) · (∆x)3
dx 6∆x 4
d2 yi yi − 2yi+1 + yi+2
= + y 000 (ξ) · ∆x
dx2 (∆x)2
d2 yi 2yi − 5yi+1 + 4yi+2 − yi+3 11
= + y (4) (ξ) · (∆x)2
dx2 (∆x)2 12
Backward differences:
dyi yi − yi−1 1
= + y 00 (ξ)∆x
dx ∆x 2
dyi 3yi − 4yi−1 + yi−2 1
= + y 000 (ξ) · (∆x)2
dx 2∆x 3
dyi 11yi − 18yi−1 + 9yi−2 − yi−3 1
= + y (4) (ξ) · (∆x)3
dx 6∆x 4
d2 yi yi − 2yi−1 + yi−2 000
= + y (ξ) · ∆x
dx2 (∆x)2
d2 yi 2yi − 5yi−1 + 4yi−2 − yi−3 11
= + y (4) (ξ) · (∆x)2
dx2 (∆x)2 12
dyi yi+1 − yi−1 1
= − y 000 (ξ)(∆x)2
dx 2∆x 6
dyi −yi+2 + 8yi+1 − 8yi−1 + yi−2 1
= + y (5) (ξ) · (∆x)4
dx 12∆x 30
d2 yi yi+1 − 2yi + yi−1 1
= − y (4) (ξ) · (∆x)2
dx2 (∆x) 2 12
d2 yi −yi+2 + 16yi+1 − 30yi + 16yi−1 − yi−2 1
= + y (6) (ξ) · (∆x)4
dx2 12(∆x)2 90
d3 yi yi+2 − 2yi+1 + 2yi−1 − yi−2 1
= + y (5) (ξ) · (∆x)2
dx3 2(∆x)3 4
2.5.1 Example: Discretization of a diffusion term

d d

This diffusion term dx p(x) dx u(x) often appears in difference equations, and
it may be beneficial to treat the term as it is instead of first execute the
differentiation. Below we will illustrate how the diffusion term may be discretized
with the three various difference approaches: forward, backward and central
differences.
Central differences: We use central differences (recall Figure 2.3) as follows,
[p(x) · u0 (x)]|i+ 21 − [p(x) · u0 (x)]|i− 21

d d
p(x) · u(x) ≈
dx dx i h
p(xi+ 12 ) · u (xi+ 21 ) − p(xi− 21 ) · u0 (xi− 21 )
0
=
h
Using central differences again, we get
ui+1 − ui ui − ui−1
u0 (xi+ 12 ) ≈ , u0 (xi− 21 ) ≈ ,
h h
which inserted in the previous equation gives the final expression
pi− 12 · ui−1 − (pi+ 12 + pi− 21 ) · ui + pi+ 12 · ui+1

d d
p(x) · u(x) ≈ +error term
dx dx i h2
(2.45)
where
h2 d

000 0 00
error term = − · p(x) · u (x) + [p(x) · u (x)] + O(h3 )
24 dx
If p(x1+ 21 ) and p(x1− 21 ) cannot be found directly, we use
1 1
p(x1+ 12 ) ≈ (pi+1 + pi ), p(x1− 12 ) ≈ (pi + pi−1 ) (2.46)
2 2
Note that for p(x) = 1 = constant we get the usual expression
d2 u

ui+1 − 2ui + ui−1
= + O(h2 )
dx2 i h2
Forward differences: We start with

p(xi+ 21 ) · u0 (xi+ 12 ) − p(xi ) · u0 (xi )

d du
p(x) · ≈ h
dx dx i 2

ui+1 −ui
p(xi+ 12 ) · h − p(xi ) · u0 (xi )
≈ h
2
which gives
2 · [p(xi+ 21 ) · (ui+1 − ui ) − h · p(xi ) · u0 (xi )]

d du
p(x) · = + error term
dx dx
i h2
(2.47)
where
h
error term = − [3p00 (xi ) · u0 (xi ) + 6p0 (xi ) · u00 (xi ) + 4p(xi ) · u000 (xi )] + O(h2 )
12
(2.48)
We have kept the term u0 (xi ) since (2.47) usually is used at the boundary, and
u0 (xi ) may be prescribed there. For p(x) = 1 = constant we get the expression
2 · [ui+1 − ui − h · u0 (xi )] h 000

u00i = − u (xi ) + O(h2 ) (2.49)
h2 3
Backward Differences: We start with

p(xi ) · u0 (xi ) − p(xi ) · u0 (xi− 21 ) · u0 (xi− 21 )

d du
p(x) ≈ h
dx dx i 2

p(xi ) · u (xi ) − p(xi− 21 ) ui −u
0
h
i−1
≈ h
2
which gives
2 · [h · p(xi )u0 (xi ) − p(xi− 21 ) · (ui − ui−1 )]

d du
p(x) = + error term
dx dx
i h2
(2.50)
where
h
error term = [3p00 (xi )·u0 (xi )+6p0 (xi )·u00 (xi )+4p(xi )·u000 (xi )]+O(h2 ) (2.51)
12
This is the same error term as in (2.48) except from the sign. Also here we have
kept the term u0 (xi ) since (2.51) usually is used at the boundary where u0 (xi )
may be prescribed. For p(x) = 1 = constant we get the expression
2 · [h · u0 (xi ) − (ui − ui−1 )] h 000

u00i = + u (xi ) + O(h2 ) (2.52)
h2 3
2.6 Euler’s method

The ODE is given as
dy
= y 0 (x) = f (x, y) (2.53)
dx
y(x0 ) =y0 (2.54)
By using a first order forward approximation (2.29) of the derivative in (2.53)

we obtain:
y(xn+1 ) = y(xn ) + h · f (xn , y(xn )) + O(h2 )
or
yn+1 = yn + h · f (xn , yn ) (2.55)
(2.55) is a difference equation and the scheme is called Euler’s method (1768).
The scheme is illustrated graphically in Figure 2.5. Euler’s method is a first
order method, since the expression for y 0 (x) is first order of h. The method has
a global error of order h, and a local of order h2 .
y
y(xn+2)
y(x) numerical
solution
yn+2
y(xn+1)
yn+1
yn h∙f(xn,yn)
xn xn+1 xn+2 x
Figure 2.5: Graphical illustration of Euler’s method.
2.6.1 Example: Eulers method on a simple ODE

# src-ch1/euler_simple.py
import numpy as np
import matplotlib.pylab as plt
""" example using eulers method for solving the ODE

y’(x) = f(x, y) = y
y(0) = 1
Eulers method:
y^(n + 1) = y^(n) + h*f(x, y^(n)), h = dx
"""
N = 30
x = np.linspace(0, 1, N + 1)
h = x[1] - x[0] # steplength
y_0 = 1 # initial condition
Y = np.zeros_like(x) # vector for storing y values
Y[0] = y_0 # first element of y = y(0)
for n in range(N):
f = Y[n]
Y[n + 1] = Y[n] + h*f
Y_analytic = np.exp(x)
# change default values of plot to make it more readable

LNWDT=2; FNT=15
plt.rcParams[’lines.linewidth’] = LNWDT
plt.rcParams[’font.size’] = FNT
plt.figure()
plt.plot(x, Y_analytic, ’b’, linewidth=2.0)
plt.plot(x, Y, ’r--’, linewidth=2.0)
plt.legend([’$e^x$’, ’euler’], loc=’best’, frameon=False)
plt.xlabel(’x’)
plt.ylabel(’y’)
#plt.savefig(’../fig-ch1/euler_simple.png’, transparent=True)
plt.show()
2.8
ex
2.6
euler
2.4
2.2
2.0
y
1.8
1.6
1.4
1.2
1.0
0.0 0.2 0.4 0.6 0.8 1.0
x
Figure 2.6: result from the code above
2.6.2 Example: Eulers method on the mathematical pen-

dulum
# src-ch1/euler_pendulum.py
import numpy as np
from math import pi
""" example using eulers method for solving the ODE:

theta’’(t) + thetha(t) = 0
thetha(0) = theta_0
thetha’(0) = dtheta_0
Reduction of higher order ODE:
theta = y0
theta’ = y1
theta’’ = - theta = -y0

y0’ = y1
y1’ = -y0
eulers method:
y0^(n + 1) = y0^(n) + h*y1, h = dt
y1^(n + 1) = y1^(n) + h*(-y0), h = dt
"""
N = 100
t = np.linspace(0, 2*pi, N + 1)
h = t[1] - t[0] # steplength
thetha_0 = 0.1
y0_0 = thetha_0 # initial condition
y1_0 = 0
Y = np.zeros((2, N + 1)) # 2D array for storing y values
Y[0, 0] = y0_0 # apply initial conditions
Y[1, 0] = y1_0
for n in range(N):
y0_n = Y[0, n]
y1_n = Y[1, n]
Y[0, n + 1] = y0_n + h*y1_n

Y[1, n + 1] = y1_n - h*y0_n
thetha = Y[0, :]
thetha_analytic = thetha_0*np.cos(t)

LNWDT=2; FNT=15
plt.figure()
plt.plot(t, thetha_analytic, ’b’)
plt.plot(t, thetha, ’r--’)
plt.legend([r’$\theta_0 \cdot cos(t)$’, ’euler’], loc=’best’, frameon=False)

plt.xlabel(’t’)
plt.ylabel(r’$\theta$’)
#plt.savefig(’../fig-ch1/euler_pendulum.png’, transparent=True)
plt.show()
2.6.3 Example: Generic euler implementation on the math-

ematical pendulum
# src-ch1/euler_pendulum_generic.py
import numpy as np
from math import pi
# define Euler solver
def euler(func, y_0, time):
""" Generic implementation of the euler scheme for solution of systems of ODEs:
0.15
0.10
0.05
0.00
−0.05
−0.10
0 cos(t)
euler
−0.15
0 1 2 3 4 5 6 7
t
Figure 2.7: result from the code above
y0’ = y1
y1’ = y2
.
.
yN’ = f(yN-1,..., y1, y0, t)
method:
y0^(n+1) = y0^(n) + h*y1
y1^(n+1) = y1^(n) + h*y2
.
.
yN^(n + 1) = yN^(n) + h*f(yN-1, .. y1, y0, t)
Args:
func(function): func(y, t) that returns y’ at time t; [y1, y2,...,f(yn-1, .. y1, y0, t)]
y_0(array): initial conditions
time(array): array containing the time to be solved for
Returns:
y(array): array/matrix containing solution for y0 -> yN for all timesteps"""
y = np.zeros((np.size(time), np.size(y_0)))
y[0,:] = y_0
for i in range(len(time)-1):
dt = time[i+1] - time[i]
y[i+1,:]=y[i,:] + np.asarray(func(y[i,:], time[i]))*dt
return y
def pendulum_func(y, t):

""" function that returns the RHS of the mathematcal pendulum ODE:
Reduction of higher order ODE:
theta = y0
theta’ = y1
theta’’ = - theta = -y0
y0’ = y1
y1’ = -y0
Args:
y(array): array [y0, y1] at time t
t(float): current time
Returns:
dy(array): [y0’, y1’] = [y1, -y0]
"""
dy = np.zeros_like(y)
dy[:] = [y[1], -y[0]]
return dy
N = 100
time = np.linspace(0, 2*pi, N + 1)
thetha_0 = [0.1, 0]
theta = euler(pendulum_func, thetha_0, time)
thetha = theta[:, 0]
thetha_analytic = thetha_0[0]*np.cos(time)

LNWDT=2; FNT=15
plt.figure()
plt.plot(time, thetha_analytic, ’b’)
plt.plot(time, thetha, ’r--’)
plt.legend([r’$\theta_0 \cdot cos(t)$’, ’euler’], loc=’best’, frameon=False)

plt.xlabel(’t’)
plt.ylabel(r’$\theta$’)
#plt.savefig(’../fig-ch1/euler_pendulum.png’, transparent=True)
plt.show()
2.6.4 Example: Sphere in free fall

Figure 2.8 illustrates a falling sphere in a gravitational field with a diameter
d and mass m that falls vertically in a fluid. Use of Newton’s 2nd law in the
z-direction gives
dv 1 dv 1
m = mg − mf g − mf − ρf v |v| Ak CD , (2.56)
dt 2 dt 2
where the different terms are interpreted as follows: m = ρk V , where ρk is the
density of the sphere and V is the sphere volume. The mass of the displaced fluid
is given by mf = ρf V , where ρf is the density of the fluid, whereas buoyancy and
the drag coefficient are expressed by mf g and CD , respectively. The projected
area of the sphere is given by Ak = π4 d2 and 12 mf is the hydro-dynamical mass
(added mass). The expression for the hydro-dynamical mass is derived in White
[16], page 539-540. To write Equation (2.56) on a more convenient form we
dz
z v=
dt
mg
Figure 2.8: Falling sphere due to gravity.
introduce the following abbreviations:

ρf ρ 3ρ
ρ= , A = 1 + , B = (1 − ρ)g, C = . (2.57)
ρk 2 4d
in addition to the drag coefficient CD which is a function of the Reynolds number
vd
Re = , where ν is the kinematical viscosity. Equation (2.56) may then be
ν
written as
dv 1
= (B − C · v |v| Cd ). (2.58)
dt A
In air we may often neglect the buoyancy term and the hydro-dynamical mass,
whereas this is not the case for a liquid. Introducing v = dz dt in Equation (2.58),
we get a 2nd order ODE as follows
d2 z

1 dz dz
= B − C · Cd (2.59)
dt2 A dt dt
For Equation (2.59) two initial conditions must be specified, e.g. v = v0 and
z = z0 for t = 0.
Figure 2.9 illustrates CD as a function of Re. The values in the plot are not
as accurate as the number of digits in the program might indicate. For example
is the location and the size of the "valley" in the diagram strongly dependent of
the degree of turbulence in the free stream and the roughness of the sphere. As
the drag coefficient CD is a function of the Reynolds number, it is also a function
of the solution v (i.e. the velocity) of the ODE in Equation (2.58). We will use
the function CD (Re) as an example of how functions may be implemented in
Python.
3
10
2
10
1
10
CD 10
0
-1
10
-2
10 -1 0 1 2 3 4 5 6 7
10 10 10 10 10 10 10 10 10
Re
Figure 2.9: Drag coefficient CD as function of the Reynold’s number Re .
2.6.5 Euler’s method for a system

Euler’s method may of course also be used for a system. Let’s look at a
simultaneous system of p equations
y10 = f1 (x, y1 , y2 , . . . yp )
y20 = f2 (x, y1 , y2 , . . . yp )
. (2.60)
.
yp0 = fp (x, y1 , y2 , . . . yp )
with initial values
y1 (x0 ) = a1 , y2 (x0 ) = a2 , . . . , yp (x0 ) = ap (2.61)
Or, in vectorial format as follows,
y0 = f (x, y) (2.62)
y(x0 ) = a
where y0 , f , y and a are column vectors with p components.

The Euler scheme (2.55) used on (2.62) gives
yn+1 = yn + h · f (xn , yn ) (2.63)
For a system of three equations we get
y10 =y2
y20 =y3 (2.64)
y30 = − y1 y3
In this case (2.63) gives
(y1 )n+1 = (y1 )n + h · (y2 )n

(y2 )n+1 = (y2 )n + h · (y3 )n (2.65)
(y3 )n+1 = (y3 )n − h · (y1 )n · (y3 )n
(2.66)
with y1 (x0 ) = a1 , y2 (x0 ) = a2 , and y3 (x0 ) = a3

In section 2.4 we have seen how we can reduce a higher order ODE to a set of
2 2
first order ODEs. In (2.67) and (2.68) we have the equation ddt2z = g − α · dz
dt
which we have reduced to a system as
dz
=v
dt
dv
= g − α · v2
dt
which gives an Euler scheme as follows,
zn+1 = zn + ∆t · vn
vn+1 = nn + ∆t · [g − α(vn )2 ]
med z0 = 0, v0 = 0
2.6.6 Example: Falling sphere with constant and varying

drag
We write (2.58) and (2.59) as a system as follows,
dz
=v (2.67)
dt
dv
= g − αv 2 (2.68)
dt
where
3ρf
α= · CD
4ρk · d
The analytical solution with z(0) = 0 and v(0) = 0 is given by
√
ln(cosh( αg · t))
z(t) = (2.69)
r α
g √
v(t) = · tanh( αg · t) (2.70)
α
r
dv g
The terminal velocity vt is found by = 0 which gives vt = .
dt α
3 3
We use data from a golf ball: d = 41 mm, ρk = 1275 kg/m , ρk = 1.22 kg/m ,
and choose CD = 0.4 which gives α = 7 · 10−3 . The terminal velocity then
becomes r
g
vt = = 37.44
α
If we use Taylor’s method from section 2.3 we get the following expression
by using four terms in the series expansion:
1 1
z(t) = gt2 · (1 − αgt2 ) (2.71)
2 6
1
v(t) =gt · (1 − αgt2 ) (2.72)
3
By applying the Euler scheme (2.55) on (2.67) and (2.68)
zn+1 = zn + ∆t · vn (2.73)
vn+1 = vn + ∆t · (g − α · vn2 ), n = 0, 1, . . . (2.74)
with z(0) = 0 and v(0) = 0.

By adopting the conventions proposed in (2.23) and substituting z0 for z and
z1 for v and we may render the system of equations in (2.67) and (2.68) as:
dz0
= z1
dt
dz1
= g − αz12
dt
One way of implementing the integration scheme is given in the following
function euler():
def euler(func,z0, time):
"""The Euler scheme for solution of systems of ODEs.
z0 is a vector for the initial conditions,
the right hand side of the system is represented by func which returns
a vector with the same size as z0 ."""
z = np.zeros((np.size(time),np.size(z0)))
z[0,:] = z0
dt = time[i+1]-time[i]
z[i+1,:]=z[i,:] + np.asarray(func(z[i,:],time[i]))*dt
return z
The program FallingSphereEuler.py computes the solution for the first 10

seconds, using a time step of ∆t = 0.5 s, and generates the plot in Figure 2.10.
In addition to the case of constant drag coefficient, a solution for the case of
varying CD is included. To find CD as function of velocity we use the function
cd_sphere() that we implemented in (2.6.4). The complete program is as
follows,
# src-ch1/FallingSphereEuler.py;DragCoefficientGeneric.py @ git@lrhgit/tkt4140/src/src-ch1/DragCoeffi
from DragCoefficientGeneric import cd_sphere
from matplotlib.pyplot import *
import numpy as np
# change some default values to make plots more readable
LNWDT=2; FNT=11
rcParams[’lines.linewidth’] = LNWDT; rcParams[’font.size’] = FNT
g = 9.81 # Gravity m/s^2

d = 41.0e-3 # Diameter of the sphere
rho_f = 1.22 # Density of fluid [kg/m^3]
rho_s = 1275 # Density of sphere [kg/m^3]
nu = 1.5e-5 # Kinematical viscosity [m^2/s]
CD = 0.4 # Constant drag coefficient
def f(z, t):

"""2x2 system for sphere with constant drag."""
zout = np.zeros_like(z)
alpha = 3.0*rho_f/(4.0*rho_s*d)*CD
zout[:] = [z[1], g - alpha*z[1]**2]
return zout
def f2(z, t):

"""2x2 system for sphere with Re-dependent drag."""
v = abs(z[1])
Re = v*d/nu
CD = cd_sphere(Re)
zout[:] = [z[1], g - alpha*z[1]**2]
return zout
# define euler scheme
z[0,:] = z0
return z
def v_taylor(t):
# z = np.zeros_like(t)
v = np.zeros_like(t)
v=g*t*(1-alpha*g*t**2)
return v
# main program starts here
T = 10 # end of simulation
N = 20 # no of time steps
time = np.linspace(0, T, N+1)
z0=np.zeros(2)
z0[0] = 2.0
ze = euler(f, z0, time) # compute response with constant CD using Euler’s method
ze2 = euler(f2, z0, time) # compute response with varying CD using Euler’s method
k1 = np.sqrt(g*4*rho_s*d/(3*rho_f*CD))
k2 = np.sqrt(3*rho_f*g*CD/(4*rho_s*d))
v_a = k1*np.tanh(k2*time) # compute response with constant CD using analytical solution
# plotting
legends=[]
line_type=[’-’,’:’,’.’,’-.’,’--’]
plot(time, v_a, line_type[0])
legends.append(’Analytical (constant CD)’)
plot(time, ze[:,1], line_type[1])
legends.append(’Euler (constant CD)’)
plot(time, ze2[:,1], line_type[3])

legends.append(’Euler (varying CD)’)
time_taylor = np.linspace(0, 4, N+1)

plot(time_taylor, v_taylor(time_taylor))
legends.append(’Taylor (constant CD)’)
legend(legends, loc=’best’, frameon=False)

font = {’size’ : 16}
rc(’font’, **font)
xlabel(’Time [s]’)
ylabel(’Velocity [m/s]’)
#savefig(’example_sphere_falling_euler.png’, transparent=True)
show()
Python implementation of the drag coefficient function and how to

plot it. The complete Python program CDsphere.py used to plot the drag
coefficient in the example above is listed below. The program uses a function
cd_sphere which results from a curve fit to the data of Evett and Liu [4]. In
our setting we will use this function for two purposes, namely to demonstrate
how functions and modules are implemented in Python and finally use these
functions in the solution of the ODE in Equations (2.58) and (2.59).
# src-ch1/CDsphere.py
from numpy import logspace, zeros
# Define the function cd_sphere

def cd_sphere(Re):
"This function computes the drag coefficient of a sphere as a function of the Reynolds number Re.
# Curve fitted after fig . A -56 in Evett and Liu: "Fluid Mechanics and Hydraulics"
40
35
30
25
Velocity [m/s]
Analytical (constant CD)
20 Euler (constant CD)
15 Euler (varying CD)
Taylor (constant CD)
10
−5
0 2 4 6 8 10
Time [s]
Figure 2.10: Euler’s method with ∆t = 0.5 s.
from numpy import log10, array, polyval
if Re <= 0.0:
CD = 0.0
elif Re > 8.0e6:
CD = 0.2
elif Re > 0.0 and Re <= 0.5:
CD = 24.0/Re
elif Re > 0.5 and Re <= 100.0:
p = array([4.22, -14.05, 34.87, 0.658])
CD = polyval(p, 1.0/Re)
elif Re > 100.0 and Re <= 1.0e4:
p = array([-30.41, 43.72, -17.08, 2.41])
CD = polyval(p, 1.0/log10(Re))
elif Re > 1.0e4 and Re <= 3.35e5:
p = array([-0.1584, 2.031, -8.472, 11.932])
CD = polyval(p, log10(Re))
elif Re > 3.35e5 and Re <= 5.0e5:
x1 = log10(Re/4.5e5)
CD = 91.08*x1**4 + 0.0764
else:
p = array([-0.06338, 1.1905, -7.332, 14.93])
return CD
# Calculate drag coefficient

Npts = 500
Re = logspace(-1, 7, Npts, True, 10)
CD = zeros(Npts)
i_list = range(0, Npts-1)
for i in i_list:
CD[i] = cd_sphere(Re[i])
# Make plot
from matplotlib import pyplot
LNWDT=2; FNT=11
pyplot.rcParams[’lines.linewidth’] = LNWDT; pyplot.rcParams[’font.size’] = FNT
pyplot.plot(Re, CD, ’-b’)

pyplot.yscale(’log’)
pyplot.xscale(’log’)
pyplot.xlabel(’$Re$’)
pyplot.ylabel(’$C_D$’)
pyplot.grid(’on’, ’both’, ’both’)
#pyplot.savefig(’example_sphere.png’, transparent=True)
pyplot.show()
In the following, we will break up the program and explain the different parts.
In the first code line,
from numpy import logspace, zeros
the functions logspace and zeros are imported from the package numpy.
The numpy package (NumPy is an abbreviation for Numerical Python) enables
the use of array objects. Using numpy a wide range of mathematical operations
can be done directly on complete arrays, thereby removing the need for loops
over array elements. This is commonly called vectorization and may cause a
dramatic increase in computational speed of Python programs. The function
logspace works on a logarithmic scale just as the function linspace works on
a regular scale. The function zeros creates arrays of a certain size filled with
zeros. Several comprehensive guides to the numpy package may be found at
http://www.numpy.org.
In CDsphere.py a function cd_sphere was defined as follows:
def cd_sphere(Re):
"This function computes the drag coefficient of a sphere as a function of the Reynolds number Re.
from numpy import log10, array, polyval
if Re <= 0.0:
CD = 0.0
elif Re > 8.0e6:
CD = 0.2
elif Re > 0.0 and Re <= 0.5:
CD = 24.0/Re
elif Re > 0.5 and Re <= 100.0:
p = array([4.22, -14.05, 34.87, 0.658])
CD = polyval(p, 1.0/Re)
elif Re > 100.0 and Re <= 1.0e4:
p = array([-30.41, 43.72, -17.08, 2.41])
CD = polyval(p, 1.0/log10(Re))
elif Re > 1.0e4 and Re <= 3.35e5:
p = array([-0.1584, 2.031, -8.472, 11.932])
elif Re > 3.35e5 and Re <= 5.0e5:
x1 = log10(Re/4.5e5)
CD = 91.08*x1**4 + 0.0764
else:
p = array([-0.06338, 1.1905, -7.332, 14.93])
return CD
The function takes Re as an argument and returns the value CD. All Python func-
tions begin with def, followed by the function name, and then inside parentheses
a comma-separated list of function arguments, ended with a colon. Here we have
only one argument, Re. This argument acts as a standard variable inside the
function. The statements to perform inside the function must be indented. At
the end of a function it is common to use the return statement to return the
value of the function.
Variables defined inside a function, such as p and x1 above, are local variables
that cannot be accessed outside the function. Variables defined outside functions,
in the "main program", are global variables and may be accessed anywhere, also
inside functions.
Three more functions from the numpy package are imported in the func-
tion. They are not used outside the function and are therefore chosen to be
imported only if the function is called from the main program. We refer to the
documentation of NumPy for details about the different functions.
The function above contains an example of the use of the if-elif-else
block. The block begins with if and a boolean expression. If the boolean
expression evaluates to true the indented statements following the if statement
are carried out. If not, the boolean expression following the elif is evaluated. If
none of the conditions are evaluated to true the statements following the else
are carried out.
In the code block
Npts = 500
Re = logspace(-1, 7, Npts, True, 10)
CD = zeros(Npts)
i_list = range(0, Npts-1)
for i in i_list:
CD[i] = cd_sphere(Re[i])
the function cd_sphere is called. First, the number of data points to be

calculated are stored in the integer variable Npts. Using the logspace function
imported earlier, Re is assigned an array object which has float elements with
values ranging from 10−1 to 107 . The values are uniformly distributed along
a 10-logarithmic scale. CD is first defined as an array with Npts zero elements,
using the zero function. Then, for each element in Re, the drag coefficient is
calculated using our own defined function cd_sphere, in a for loop, which is
explained in the following.
The function range is a built-in function that generates a list containing
arithmetic progressions. The for i in i_list construct creates a loop over all
elements in i_list. In each pass of the loop, the variable i refers to an element
in the list, starting with i_list[0] (0 in this case) and ending with the last
element i_list[Npts-1] (499 in this case). Note that element indices start at
0 in Python. After the colon comes a block of statements which does something
useful with the current element; in this case, the return of the function call
cd_sphere(Re[i]) is assigned to CD[i]. Each statement in the block must be
indented.
Lastly, the drag coefficient is plotted and the figure generated:
from matplotlib import pyplot

LNWDT=2; FNT=11
pyplot.rcParams[’lines.linewidth’] = LNWDT; pyplot.rcParams[’font.size’] = FNT
pyplot.plot(Re, CD, ’-b’)
pyplot.yscale(’log’)
pyplot.xscale(’log’)
pyplot.xlabel(’$Re$’)
pyplot.ylabel(’$C_D$’)
pyplot.grid(’on’, ’both’, ’both’)
#pyplot.savefig(’example_sphere.png’, transparent=True)
pyplot.show()
To generate the plot, the package matplotlib is used. matplotlib is

the standard package for curve plotting in Python. For simple plotting the
matplotlib.pyplot interface provides a Matlab-like interface, which has been
used here. For documentation and explanation of this package, we refer to
http://www.matplotlib.org.
First, the curve is generated using the function plot, which takes the x-values
and y-values as arguments (Re and CD in this case), as well as a string specifying
the line style, like in Matlab. Then changes are made to the figure in order to
make it more readable, very similarly to how it is done in Matlab. For instance,
in this case it makes sense to use logarithmic scales. A png version of the figure
is saved using the savefig function. Lastly, the figure is showed on the screen
with the show function.
To change the font size the function rc is used. This function takes in the
object font, which is a dictionary object. Roughly speaking, a dictionary is a
list where the index can be a text (in lists the index must be an integer). It is
best to think of a dictionary as an unordered set of key:value pairs, with the
requirement that the keys are unique (within one dictionary). A pair of braces
creates an empty dictionary: {}. Placing a comma-separated list of key:value
pairs within the braces adds initial key:value pairs to the dictionary. In this
case the dictionary font contains one key:value pair, namely ’size’ : 16.
Descriptions and explanations of all functions available in pyplot may be
found here.
2.7 Python functions with vector arguments and

modules
For many numerical problems variables are most conveniently expressed by arrays
containing many numbers (i.e. vectors) rather than single numbers (i.e. scalars).
The function cd_sphere above takes a scalar as an argument and returns a scalar
value too. For computationally intensive algorithms where variables are stored
in arrays this is inconvenient and time consuming, as each of the array elements
must be sent to the function independently. In the following, we will therefore
show how to implement functions with vector arguments that also return vectors.
This may be done in various ways. Some possibilities are presented in the
following, and, as we shall see, some are more time consuming than others. We
will also demonstrate how the time consumption (or efficiency) may be tested.
A simple extension of the single-valued function cd_sphere is as follows:
def cd_sphere_py_vector(ReNrs):
CD = zeros_like(ReNrs)
counter = 0
for Re in ReNrs:
CD[counter] = cd_sphere(Re)
counter += 1
return CD
The new function cd_sphere_py_vector takes in an array ReNrs and calcu-

lates the drag coefficient for each element using the previous function cd_sphere.
This does the job, but is not very efficient.
A second version is implemented in the function cd_sphere_vector. This
function takes in the array Re and calculates the drag coefficient of all elements by
multiple calls of the function numpy.where; one call for each condition, similarly
as each if statement in the function cd_sphere. The function is shown here:
def cd_sphere_vector(Re):
"Computes the drag coefficient of a sphere as a function of the Reynolds number Re."
from numpy import log10, array, polyval, where, zeros_like

CD = zeros_like(Re)
CD = where(Re < 0, 0.0, CD) # condition 1
CD = where((Re > 0.0) & (Re <=0.5), 24/Re, CD) # condition 2
p = array([4.22, -14.05, 34.87, 0.658])

CD = where((Re > 0.5) & (Re <=100.0), polyval(p, 1.0/Re), CD) #condition 3
p = array([-30.41, 43.72, -17.08, 2.41])

CD = where((Re > 100.0) & (Re <= 1.0e4), polyval(p, 1.0/log10(Re)), CD) #condition 4
p = array([-0.1584, 2.031, -8.472, 11.932])

CD = where((Re > 1.0e4) & (Re <= 3.35e5), polyval(p, log10(Re)), CD) #condition 5
CD = where((Re > 3.35e5) & (Re <= 5.0e5), 91.08*(log10(Re/4.5e5))**4 + 0.0764, CD) #condition 6
p = array([-0.06338, 1.1905, -7.332, 14.93])

CD = where(Re > 8.0e6, 0.2, CD) # condition 8

return CD
A third approach we will try is using boolean type variables. The 8 variables
condition1 through condition8 in the function cd_sphere_vector_bool are
boolean variables of the same size and shape as Re. The elements of the boolean
variables evaluate to either True or False, depending on if the corresponding
element in Re satisfy the condition the variable is assigned.
def cd_sphere_vector_bool(Re):
from numpy import log10, array, polyval, zeros_like
condition1 = Re < 0
condition2 = logical_and(0 < Re, Re <= 0.5)
condition3 = logical_and(0.5 < Re, Re <= 100.0)
condition4 = logical_and(100.0 < Re, Re <= 1.0e4)
condition5 = logical_and(1.0e4 < Re, Re <= 3.35e5)
condition8 = Re > 8.0e6
CD = zeros_like(Re)
CD[condition1] = 0.0
CD[condition2] = 24/Re[condition2]
p = array([4.22,-14.05,34.87,0.658])
CD[condition3] = polyval(p,1.0/Re[condition3])
p = array([-30.41,43.72,-17.08,2.41])
CD[condition4] = polyval(p,1.0/log10(Re[condition4]))
p = array([-0.1584,2.031,-8.472,11.932])
CD[condition5] = polyval(p,log10(Re[condition5]))
CD[condition6] = 91.08*(log10(Re[condition6]/4.5e5))**4 + 0.0764

p = array([-0.06338,1.1905,-7.332,14.93])
return CD
Lastly, the built-in function vectorize is used to automatically generate a

vector-version of the function cd_sphere, as follows:
cd_sphere_auto_vector = vectorize(cd_sphere)
To provide a convenient and practical means to compare the various imple-

mentations of the drag function, we have collected them all in a file DragCoef-
ficientGeneric.py. This file constitutes a Python module which is a concept
we will discuss in section 2.8.
2.8 How to make a Python-module and some

useful programming features
Python modules A module is a file containing Python definitions and state-
ments and represents a convenient way of collecting useful and related functions,
classes or Python code in a single file. A motivation to implement the drag
coefficient function was that we should be able to import it in other programs to
solve e.g. the problems outlined in (2.6.4). In general, a file containing Python-
code may be executed either as a main program (script), typically with python
filename.py or imported in another script/module with import filename.
A module file should not execute a main program, but rather just define
functions, import other modules, and define global variables. Inside modules,
the standard practice is to only have functions and not any statements outside
functions. The reason is that all statements in the module file are executed from
top to bottom during import in another module/script [10], and thus a desirable
behavior is no output to avoid confusion. However, in many situations it is also
desirable to allow for tests or demonstration of usage inside the module file, and
for such situations the need for a main program arises. To meet these demands
Python allows for a fortunate construction to let a file act both as a module with
function definitions only (i.e. no main program) and as an ordinary program we
can run, with functions and a main program. The latter is possible by letting
the main program follow an if test of the form:
if __name__ ==’__main__’:
<main program statements>
The __name__ variable is automatically defined in any module and equals the
module name if the module is imported in another program/script, but when the
module file is executed as a program, __name__ equals the string ’__main__’.
Consequently, the if test above will only be true whenever the module file
is executed as a program and allow for the execution of the <main program
statements>. The <main program statements> is normally referred to as the
test block of a module.
The module name is the file name without the suffix .py [10], i.e. the module
contained in the module file filename.py has the module name filename. Note
that a module can contain executable statements as well as function definitions.
These statements are intended to initialize the module and are executed only
the first time the module name is encountered in an import statement. They
are also run if the file is executed as a script.
Below we have listed the content of the file DragCoefficientGeneric.py
to illustrate a specific implementation of the module DragCoefficientGeneric
and some other useful programming features in Python. The functions in the
module are the various implementations of the drag coefficient functions from
the previous section.
Python lists and dictionaries
• Lists hold a list of values and are initialized with empty brackets, e.g.
fncnames = []. The values of the list are accessed with an index, start-
ing from zero. The first value of fncnames is fncnames[0], the second
value of fncnames is fncnames[1] and so on. You can remove values
from the list, and add new values to the end by fncnames. Example:
fncnames.append(name) will append name as the last value of the list
fncnames. In case it was empty prior to the append-operation, name will
be the only element in the list.
• Dictionaries are similar to what their name suggests - a dictionary - and

an empty dictionary is initialized with empty braces, e.g. CD = {}. In a
dictionary, you have an ’index’ of words or keys and the values accessed by
their ’key’. Thus, the values in a dictionary aren’t numbered or ordered,
they are only accessed by the key. You can add, remove, and modify
the values in dictionaries. For example with the statement CD[name] =
func(ReNrs) the results of func(ReNrs) are stored in the list CD with key
name.
To illustrate a very powerful feature of Python data structures allowing for lists
of e.g. function objects we put all the function names in a list with the statement:
funcs = [cd_sphere_py_vector, cd_sphere_vector, cd_sphere_vector_bool, \
cd_sphere_auto_vector] # list of functions to test
which allows for convenient looping over all of the functions with the following
construction:
for func in funcs:
Exception handling Python has a very convenient construction for testing

of potential errors with try-except blocks:
try:
<statements>
except ExceptionType1:
<remedy for ExceptionType1 errors>
except ExceptionType2:
<remedy for ExceptionType1 errors>
except:
<remedy for any other errors>
In the DragCoefficientGeneric module, this feature is used to handle the

function name for a function which has been vectorized automatically. For such
a function func.func_name has no value and will return an error, and the name
may be found by the statements in the exception block.
Efficiency and benchmarking The function clock in the module time,
return a time expressed in seconds for the current statement and is frequently
used for benchmarking in Python or timing of functions. By subtracting the time
t0 recored immediately before a function call from the time immediately after
the function call, an estimate of the elapsed cpu-time is obtained. In our module
DragCoefficientGeneric the efficiency is implemented with the codelines:
t0 = time.clock()
CD[name] = func(ReNrs)
exec_times[name] = time.clock() - t0
Sorting of dictionaries The computed execution times are for convenience

stored in the dictionary exec_time to allow for pairing of the names of the
functions and their execution time. The dictionary may be sorted on the values
and the corresponding keys sorted are returned by:
exec_keys_sorted = sorted(exec_times, key=exec_times.get)
Afterwards the results may be printed with name and execution time, ordered
by the latter, with the most efficient function at the top:
for name_key in exec_keys_sorted:
print name_key, ’\t execution time = ’, ’%6.6f’ % exec_times[name_key]
By running the module DragCoefficientGeneric as a script, and with 500

elements in the ReNrs array we got the following output:
cd_sphere_vector_bool execution time = 0.000312
cd_sphere_vector execution time = 0.000641
cd_sphere_auto_vector execution time = 0.009497
cd_sphere_py_vector execution time = 0.010144
Clearly, the function with the boolean variables was fastest, the straight forward
vectorized version cd_sphere_py_vector was slowest and the built-in function
vectorize was nearly as inefficient.
The complete module DragCoefficientGeneric is listed below.
# src-ch1/DragCoefficientGeneric.py
from numpy import linspace,array,append,logspace,zeros_like,where,vectorize,\
logical_and
import numpy as np
from matplotlib.pyplot import loglog,xlabel,ylabel,grid,savefig,show,rc,hold,\
legend, setp
from numpy.core.multiarray import scalar
# single-valued function
def cd_sphere(Re):
from numpy import log10,array,polyval
if Re <= 0.0:
CD = 0.0
elif Re > 8.0e6:
CD = 0.2
elif Re > 0.0 and Re <= 0.5:
CD = 24.0/Re
elif Re > 0.5 and Re <= 100.0:
p = array([4.22,-14.05,34.87,0.658])
CD = polyval(p,1.0/Re)
elif Re > 100.0 and Re <= 1.0e4:
p = array([-30.41,43.72,-17.08,2.41])
CD = polyval(p,1.0/log10(Re))
elif Re > 1.0e4 and Re <= 3.35e5:
p = array([-0.1584,2.031,-8.472,11.932])
CD = polyval(p,log10(Re))
elif Re > 3.35e5 and Re <= 5.0e5:
x1 = log10(Re/4.5e5)
CD = 91.08*x1**4 + 0.0764
else:
p = array([-0.06338,1.1905,-7.332,14.93])
CD = polyval(p,log10(Re))
return CD
# simple extension cd_sphere

def cd_sphere_py_vector(ReNrs):
CD = zeros_like(ReNrs)
counter = 0
for Re in ReNrs:
CD[counter] = cd_sphere(Re)
counter += 1
return CD
# vectorized function
def cd_sphere_vector(Re):
from numpy import log10, array, polyval, where, zeros_like

CD = zeros_like(Re)
CD = where(Re < 0, 0.0, CD) # condition 1
CD = where((Re > 0.0) & (Re <=0.5), 24/Re, CD) # condition 2
p = array([4.22, -14.05, 34.87, 0.658])

CD = where((Re > 0.5) & (Re <=100.0), polyval(p, 1.0/Re), CD) #condition 3
p = array([-30.41, 43.72, -17.08, 2.41])

CD = where((Re > 100.0) & (Re <= 1.0e4), polyval(p, 1.0/log10(Re)), CD) #condition 4
p = array([-0.1584, 2.031, -8.472, 11.932])

CD = where((Re > 3.35e5) & (Re <= 5.0e5), 91.08*(log10(Re/4.5e5))**4 + 0.0764, CD) #condition 6
p = array([-0.06338, 1.1905, -7.332, 14.93])

CD = where(Re > 8.0e6, 0.2, CD) # condition 8

return CD
# vectorized boolean
def cd_sphere_vector_bool(Re):
from numpy import log10, array, polyval, zeros_like
condition1 = Re < 0
condition2 = logical_and(0 < Re, Re <= 0.5)
condition3 = logical_and(0.5 < Re, Re <= 100.0)
condition4 = logical_and(100.0 < Re, Re <= 1.0e4)
condition8 = Re > 8.0e6
CD = zeros_like(Re)
CD[condition2] = 24/Re[condition2]
p = array([4.22,-14.05,34.87,0.658])
CD[condition3] = polyval(p,1.0/Re[condition3])
p = array([-30.41,43.72,-17.08,2.41])
CD[condition4] = polyval(p,1.0/log10(Re[condition4]))
p = array([-0.1584,2.031,-8.472,11.932])
CD[condition6] = 91.08*(log10(Re[condition6]/4.5e5))**4 + 0.0764

p = array([-0.06338,1.1905,-7.332,14.93])
return CD
if __name__ == ’__main__’:
#Check whether this file is executed (name==main) or imported as a module
import time
from numpy import mean
CD = {} # Empty list for all CD computations
ReNrs = logspace(-2,7,num=500)
# make a vectorized version of the function automatically
cd_sphere_auto_vector = vectorize(cd_sphere)
# make a list of all function objects

funcs = [cd_sphere_py_vector, cd_sphere_vector, cd_sphere_vector_bool, \
cd_sphere_auto_vector] # list of functions to test
# Put all exec_times in a dictionary and fncnames in a list

exec_times = {}
fncnames = []
for func in funcs:
try:
name = func.func_name
except:
scalarname = func.__getattribute__(’pyfunc’)
name = scalarname.__name__+’_auto_vector’
fncnames.append(name)
# benchmark
t0 = time.clock()
CD[name] = func(ReNrs)
exec_times[name] = time.clock() - t0
# sort the dictionary exec_times on values and return a list of the corresponding keys
exec_keys_sorted = sorted(exec_times, key=exec_times.get)
# print the exec_times by ascending values

for name_key in exec_keys_sorted:
print name_key, ’\t execution time = ’, ’%6.6f’ % exec_times[name_key]
# set fontsize prms

fnSz = 16; font = {’size’ : fnSz}; rc(’font’,**font)
# set line styles

style = [’v-’, ’8-’, ’*-’, ’o-’]
mrkevry = [30, 35, 40, 45]
# plot the result for all functions

i=0
for name in fncnames:
loglog(ReNrs, CD[name], style[i], markersize=10, markevery=mrkevry[i])
hold(’on’)
i+=1
# use fncnames as plot legend

leg = legend(fncnames)
leg.get_frame().set_alpha(0.)
xlabel(’$Re$’)
ylabel(’$C_D$’)
grid(’on’, ’both’, ’both’)
# # savefig(’example_sphere_generic.png’, transparent=True) # save plot if needed
show()
2.8.1 Example: Numerical error as a function of ∆t

In this example we will assess how the error of our implementation of the Euler
method depends on the time step ∆t in a systematic manner. We will solve a
problem with an analytical solution in a loop, and for each new solution we do
the following:
• Divide the time step by two (or double the number of time steps)
• Compute the error

• Plot the error
Euler’s method is a first order method and we expect the error to be O(h) =
O(∆t). Consequently if the time-step is divided by two, the error should also
be divided by two. As errors normally are small values and are expected to be
smaller and smaller for decreasing time steps, we normally do not plot the error
itself, but rather the logarithm of the absolute value of the error. The latter we
do due to the fact that we are only interested in the order of magnitude of the
error, whereas errors may be both positive and negative. As the initial value is
always correct we discard the first error at time zero to avoid problems with the
logarithm of zero in log_error = np.log2(abs_error[1:]).
# coding: utf-8
# src-ch1/Euler_timestep_ctrl.py;DragCoefficientGeneric.py @ git@lrhgit/tkt4140/src/src-ch1/DragCoeff
import numpy as np

LNWDT=2; FNT=11

def f(z, t):

zout[:] = [z[1], g - alpha*z[1]**2]
return zout
# define euler scheme
z[0,:] = z0
return z
def v_taylor(t):
# z = np.zeros_like(t)
v = np.zeros_like(t)
v=g*t*(1-alpha*g*t**2)
return v

z0=np.zeros(2)
z0[0] = 2.0
# Prms for the analytical solution

Ndts = 4 # Number of times to divide the dt by 2
legends=[]
error_diff = []
for i in range(Ndts+1):
abs_error=np.abs(ze[:,1] - v_a)
log_error = np.log2(abs_error[1:])
max_log_error = np.max(log_error)
plot(time[1:], log_error)
legends.append(’Euler scheme: N ’ + str(N) + ’ timesteps’ )
N*=2
if i > 0:
error_diff.append(previous_max_log_err-max_log_error)
previous_max_log_err = max_log_error
print ’Approximate order of scheme n =’, np.mean(error_diff)

print ’Approximate error reuduction by dt=dt/2:’, 1/2**(np.mean(error_diff))
# plot analytical solution

# plot(time,v_a)
# legends.append(’analytical’)
# fix plot
#ylabel(’Velocity [m/s]’)
ylabel(’log2-error’)
#savefig(’example_euler_timestep_study.png’, transparent=True)
show()
The plot resulting from the code above is shown in Figure (2.11). The
difference or distance between the curves seems to be rather constant after an
initial transient. As we have plotted the logarithm of the absolute value of the
error i , the difference di+1 between two curves is di+1 = log2 i − log2 i+1 =
i
log2 . A rough visual inspection of Figure (2.11) yields di+1 ≈ 1.0, from
i+1
which we may deduce the apparent order of the scheme:
i
n = log2 ≈ 1 ⇒ i+1 ≈ 0.488483620 i (2.75)
i+1
The print statement returns ’n=1.0336179048’ and ’0.488483620794’, thus we

see that the error is reduced even slightly more than the theoretically expected
value for a first order scheme, i.e. ∆ti+1 = ∆ti /2 yields i+1 ≈ i /2.
−2
−4
log2 -error
−6
−8
Euler scheme: N 10 timesteps
−10
−12 Euler scheme: N 40 timesteps
−14
−16
0 2 4 6 8 10
Time [s]
Figure 2.11: Plots for the logarithmic errors for a falling sphere with constant
drag. The timestep ∆t is reduced by a factor two from one curve to the one
immediately below.
2.9 Heun’s method

From (2.29) or (2.33) we have
f (xn + h) − f (xn )
y 00 (xn , yn ) = f 0 (xn , y(xn , yn )) ≈ (2.76)
h
The Taylor series expansion (2.27) gives
h2 00
y(xn + h) = y(xn ) + hy 0 [xn , y(xn )] + y [xn , y(xn )] + O(h3 )
2
which, inserting (2.76), gives
h
yn+1 = yn + · [f (xn , yn ) + f (xn+1 , y(xn+1 ))] (2.77)
2
This formula is called the trapezoidal formula, since it reduces to computing
an integral with the trapezoidal rule if f (x, y) is only a function of x. Since yn+1
appears on both sides of the equation, this is an implicit formula which means
that we need to solve a system of non-linear algebraic equations if the function
f (x, y) is non-linear. One way of making the scheme explicit is to use the Euler
scheme (2.55) to calculate y(xn+1 ) on the right side of (2.77). The resulting
scheme is often denoted Heun’s method.
The scheme for Heun’s method becomes
p
yn+1 = yn + h · f (xn , yn ) (2.78)
h p
yn+1 = yn + · [f (xn , yn ) + f (xn+1 , yn+1 )] (2.79)
2
Index p stands for "predicted". (2.78) is then the predictor and (2.79) is the
corrector. This is a second order method. For more details, see [3]. Figure 2.12
is a graphical illustration of the method.
y
corrected
yn+1 slope
slope
f(xn+1,yn+1)
y(x)
ypn+1
slope
yn
f(xn,yn)
xn xn+1 x
Figure 2.12: Illustration of Heun’s method.
In principle we could make an iteration procedure where we after using the

corrector use the corrected values to correct the corrected values to make a
new predictor and so on. This will likely lead to a more accurate solution of
the difference scheme, but not necessarily of the differential equation. We are
therefore satisfied by using the corrector once. For a system, we get
p
yn+1 = yn + h · f (xn , yn ) (2.80)
h p
yn+1 = yn + · [f (xn , yn ) + f (xn+1 , yn+1 )] (2.81)
2
p
Note that yn+1 is a temporary variable that is not necessary to store.
If we use (2.80) and (2.81) on the example in (2.64) we get
Predictor:
(y1 )pn+1 = (y1 )n + h · (y2 )n
(y2 )pn+1 = (y2 )n + h · (y3 )n
(y3 )pn+1 = (y3 )n − h · (y1 )n · (y3 )n
Corrector:
(y1 )n+1 = (y1 )n + 0.5h · [(y2 )n + (y2 )pn+1 ]
(y2 )n+1 = (y2 )n + 0.5h · [(y3 )n + (y3 )pn+1 ]
(y3 )n+1 = (y3 )n − 0.5h · [(y1 )n · (y3 )n + (y1 )pn+1 · (y3 )pn+1 ]
2.9.1 Example: Newton’s equation

Let’s use Heun’s method to solve Newton’s equation from section 2.1,
y 0 (x) = 1 − 3x + y + x2 + xy, y(0) = 0 (2.82)
with analytical solution
√ √
√

x 2 2
y(x) =3 2πe · exp x 1 + · erf (1 + x) − erf
2 2 2
h x i
+4 · 1 − exp x 1 + −x (2.83)
2
Here we have f (x, y) = 1 − 3x + y + x2 + xy = 1 + x(x − 3) + (1 + x)y
The following program NewtonHeun.py solves this problem using Heun’s
method, and the resulting figure is shown in Figure 2.13.
# src-ch1/NewtonHeun.py
# Program Newton
# Computes the solution of Newton’s 1st order equation (1671):
# dy/dx = 1-3*x + y + x^2 +x*y , y(0) = 0
# using Heun’s method.
import numpy as np
xend = 2
dx = 0.1
steps = np.int(np.round(xend/dx, 0)) + 1
y, x = np.zeros((steps,1), float), np.zeros((steps,1), float)
y[0], x[0] = 0.0, 0.0
for n in range(0,steps-1):
x[n+1] = (n+1)*dx
xn = x[n]
fn = 1 + xn*(xn-3) + y[n]*(1+xn)
yp = y[n] + dx*fn
xnp1 = x[n+1]
fnp1 = 1 + xnp1*(xnp1-3) + yp*(1+xnp1)
y[n+1] = y[n] + 0.5*dx*(fn+fnp1)
# Analytical solution
from scipy.special import erf
a = np.sqrt(2)/2
t1 = np.exp(x*(1+ x/2))
t2 = erf((1+x)*a)-erf(a)
ya = 3*np.sqrt(2*np.pi*np.exp(1))*t1*t2 + 4*(1-t1)-x
# plotting

LNWDT=2; FNT=11
plt.rcParams[’lines.linewidth’] = LNWDT; plt.rcParams[’font.size’] = FNT
plt.plot(x, y, ’-b.’, x, ya, ’-g.’)

plt.xlabel(’x’)
plt.ylabel(’y’)
plt.title(’Solution to Newton\’s equation’)

plt.legend([’Heun’, ’Analytical’], loc=’best’, frameon=False)
plt.grid()
#plt.savefig(’newton_heun.png’, transparent=True)
plt.show()
0.5
0.0
−0.5
−1.0
−1.5
y
−2.0
−2.5
−3.0 Heun
Analytical
−3.5
0.0 0.5 1.0 1.5 2.0
x
Figure 2.13: Velocity of falling sphere using Euler’s and Heun’s methods.
2.9.2 Example: Falling sphere with Heun’s method

Let’s go back to (2.6.6), and implement a new function heun() in the program
FallingSphereEuler.py.
We recall the system of equations as
dz
=v
dt
dv
= g − αv 2
dt
which by use of Heun’s method in (2.80) and (2.81) becomes
Predictor:
p
zn+1 = zn + ∆tvn (2.84)
p
vn+1 = vn + ∆t · (g − αvn2 )
Corrector:
p
zn+1 = zn + 0.5∆t · (vn + vn+1 ) (2.85)
2 p
)2

vn+1 = vn + 0.5∆t · 2g − α[vn + (vn+1
with initial values z0 = z(0) = 0, v0 = v(0) = 0. Note that we don’t use the
p
predictor zn+1 since it doesn’t appear on the right hand side of the equation
system.
One possible way of implementing this scheme is given in the following
function named heun(), in the program ODEschemes.py:
def heun(func, z0, time):
"""The Heun scheme for solution of systems of ODEs.

z = np.zeros((np.size(time), np.size(z0)))
z[0,:] = z0
zp = np.zeros_like(z0)
for i, t in enumerate(time[0:-1]):
zp = z[i,:] + np.asarray(func(z[i,:],t))*dt # Predictor step
z[i+1,:] = z[i,:] + (np.asarray(func(z[i,:],t)) + np.asarray(func(zp,t+dt)))*dt/2.0 # Correct
Using the same time steps as in (2.6.6), we get the response plotted in
Figure 2.14.
40
35
30
25
Velocity [m/s]
20
15 Analytical (constant CD)

Euler (constant CD)
10
Heun (constant CD)
5 Euler (varying CD)
Heun (varying CD)
0
0 2 4 6 8 10
Time [s]
Figure 2.14: Velocity of falling sphere using Euler’s and Heun’s methods.
The complete program FallingSphereEulerHeun.py is listed below. Note

that the solver functions euler and heun are imported from the script ODE-
schemes.py.
# src-ch1/FallingSphereEulerHeun.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch1/ODEschemes.py;
from ODEschemes import euler, heun
import numpy as np
LNWDT=5; FNT=11

def f(z, t):
zout[:] = [z[1], g - alpha*z[1]**2]
return zout
def f2(z, t):

v = abs(z[1])
Re = v*d/nu
CD = cd_sphere(Re)
zout[:] = [z[1], g - alpha*z[1]**2]
return zout
z0=np.zeros(2)
z0[0] = 2.0
zh = heun(f, z0, time) # compute response with constant CD using Heun’s method
zh2 = heun(f2, z0, time) # compute response with varying CD using Heun’s method
# plotting
legends=[]
line_type=[’-’,’:’,’.’,’-.’,’--’]


plot(time, zh[:,1], line_type[2])

legends.append(’Heun (constant CD)’)

plot(time, zh2[:,1], line_type[4])
legends.append(’Heun (varying CD)’)
#savefig(’example_sphere_falling_euler_heun.png’, transparent=True)
show()
2.10 Generic second order Runge-Kutta method

We seek to derive a generic second order Runge-Kutta method for the solution
of a generic ODE (2.2) on the form:
yn+1 = yn + h (a1 K1 + a2 K2 ) (2.86)
where a1 and a2 are some weights to be determined and K1 and K2 are

derivatives on the form:
K1 = f (xn , yn ) and K2 = f (xn + p1 h, yn + p2 K1 h) (2.87)
By substitution of (2.87) in (2.86) we get:
yn+1 = yn + a1 h f (xn , yn ) + a2 h f (xn + p1 h, yn + p2 K1 h) (2.88)
Now, we may find a Taylor-expansion of f (xn + p1 h, yn + p2 K1 h):
f (xn + p1 h, yn + p2 K1 h) = f + p1 h fx + p2 K1 h fy + h.o.t.
= f + p1 h fx + p2 h f fy + h.o.t. (2.89)
where we for convenience have adopted the common notation for partial deriva-
tives
∂f ∂f
fx ≡ and fy ≡ (2.90)
∂x ∂y
By substitution of (2.89) in (2.88) we eliminate the implicit dependency of

yn+1
yn+1 = yn + a1 h f (xn , yn ) + a2 h (f + p1 h fx + p2 h f fy )
= yn + (a1 + a2 ) hf + (a2 p1 fx + a2 p2 f fy ) h2 (2.91)
Further, a second order derivative of the solution to (2.2) may be obtained

by differentiation:
d2 y df dx dy
y 00 = 2
= = ∂f x + ∂f y = fx + f fy (2.92)
dx dx dx dx
which may be used in a second order Taylor expansion of the solution of (2.2):
h2 00
y(xn + h) = yn + hy 0 + y + O(h3 ) (2.93)
2
Substitution of (2.92) and (2.2) into (2.93) yields:
h2
y(xn + h) = yn + hf + (fx + f fy ) + O(h3 ) (2.94)
2
Now the idea of the generic second order Runge-Kutta method is to select
a1 , a2 , K1 , and K2 in such a way that (2.91) approximates (2.94), which will be
true if:
a1 + a2 = 1
1
a2 p1 = (2.95)
2
1
a2 p2 =
2
Note, that since we have 4 unknowns (a1 , a2 , K1 , and K2 ) and only 3
equations in (2.95), several methods of the kind proposed in (2.86) are possible. 2.9
is retrieved by selecting a2 = 1/2, from which we get a1 = 1/2 and p1 = p2 = 1
from (2.95).
2.11 Runge-Kutta of 4th order

Euler’s method and Heun’s method belong to the Runge-Kutta family of explicit
methods, and is respectively Runge-Kutta of 1st and 2nd order, the latter with
one time use of corrector. Explicit Runge-Kutta schemes are single step schemes
that try to copy the Taylor series expansion of the differential equation to a
given order.
The classical Runge-Kutta scheme of 4th order (RK4) is given by
k1 = f (xn , yn )
h h
k2 = f (xn + , yn + k1 )
2 2
h h
k3 = f (xn + , yn + k2 ) (2.96)
2 2
k4 = f (xn + h, yn + hk3 )
h
yn+1 = yn + (k1 + 2k2 + 2k3 + k4 )
6
We see that we are actually using Euler’s method four times and find a
weighted gradient. The local error is of order O(h5 ), while the global is of O(h4 ).
We refer to [3].
Figure 2.15 shows a graphical illustration of the RK4 scheme.
In detail we have
1. In point (xn , yn ) we know the gradient k1 and use this when we go forward
a step h/2 where the gradient k2 is calculated.
2. With this gradient we start again in point (xn , yn ), go forward a step h/2
and find a new gradient k3 .
3. With this gradient we start again in point (xn , yn ), but go forward a
complete step h and find a new gradient k4 .
4. The four gradients are averaged with weights 1/6, 2/6, 2/6 and 1/6. Using
the averaged gradient we calculate the final value yn+1 .
Each of the steps above are Euler steps.
y
y(x)
yn+1 4
3
2
yn 1
xn xn+½ xn+1 x
Figure 2.15: Illustration of the RK4 scheme.
Using (2.96) on the equation system in (2.64) we get
h
(y1 )n+1 = (y1 )n + (k1 + 2k2 + 2k3 + k4 )
6
h
(y2 )n+1 = (y2 )n + (l1 + 2l2 + 2l3 + l4 ) (2.97)
6
h
(y3 )n+1 = (y3 )n + (m1 + 2m2 + 2m3 + m4 )
6
(2.98)
where
k1 = y2
l1 = y3
m1 = −y1 y3
k2 = (y2 + hll /2)

l2 = (y3 + hm1 /2)
m2 = −[(y1 + hk1 /2)(y3 + hm1 /2)]
k3 = (y2 + hl2 /2)

l3 = (y3 + hm2 /2)
m3 = −[(y1 + hk2 /2)(y3 + hm2 /2)]
k4 = (y2 + hl3 )
l4 = (y3 + hm3 )
m4 = −[(y1 + hk3 )(y3 + hm3 )
2.11.1 Example: Falling sphere using RK4

Let’s implement the RK4 scheme and add it to the falling sphere example. The
scheme has been implemented in the function rk4(), and is given below
def rk4(func, z0, time):
"""The Runge-Kutta 4 scheme for solution of systems of ODEs.
z[0,:] = z0
dt2 = dt/2.0
k1 = np.asarray(func(z[i,:], t)) # predictor step 1
k2 = np.asarray(func(z[i,:] + k1*dt2, t + dt2)) # predictor step 2
k4 = np.asarray(func(z[i,:] + k3*dt, t + dt)) # predictor step 4
z[i+1,:] = z[i,:] + dt/6.0*(k1 + 2.0*k2 + 2.0*k3 + k4) # Corrector step
Figure 2.16 shows the results using Euler, Heun and RK4. AS seen, RK4
and Heun are more accurate than Euler. The complete program Falling-
SphereEulerHeunRK4.py is listed below. The functions euler, heun and
rk4 are imported from the program ODEschemes.py.
# src-ch1/FallingSphereEulerHeunRK4.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch1/ODEschemes.py;
40
35
30
25
Velocity [m/s]
20
Analytical (constant CD)
Euler (constant CD)
15 Heun (constant CD)
RK4 (constant CD)
10
Euler (varying CD)
5 Heun (varying CD)
RK4 (varying CD)
0
0 2 4 6 8 10
Time [s]
Figure 2.16: Velocity of falling sphere using Euler, Heun and RK4.

from ODEschemes import euler, heun, rk4
import numpy as np

LNWDT=5; FNT=11

def f(z, t):
zout[:] = [z[1], g - alpha*z[1]**2]
return zout
def f2(z, t):

v = abs(z[1])
Re = v*d/nu
CD = cd_sphere(Re)
zout[:] = [z[1], g - alpha*z[1]**2]
return zout

z0=np.zeros(2)
z0[0] = 2.0
zh = heun(f, z0, time) # compute response with constant CD using Heun’s method
zh2 = heun(f2, z0, time) # compute response with varying CD using Heun’s method
zrk4 = rk4(f, z0, time) # compute response with constant CD using RK4
zrk4_2 = rk4(f2, z0, time) # compute response with varying CD using RK4
# plotting
legends=[]
line_type=[’-’,’:’,’.’,’-.’,’:’,’.’,’-.’]


plot(time, zh[:,1], line_type[2])

legends.append(’Heun (constant CD)’)
plot(time, zrk4[:,1], line_type[3])

legends.append(’RK4 (constant CD)’)
plot(time, zh2[:,1], line_type[5])
legends.append(’Heun (varying CD)’)
plot(time, zrk4_2[:,1], line_type[6])

legends.append(’RK4 (varying CD)’)
#savefig(’example_sphere_falling_euler_heun_rk4.png’, transparent=True)
show()
2.11.2 Example: Particle motion in two dimensions

In this example we will calculate the motion of a particle in two dimensions.
First we will calculate the motion of a smooth ball with drag coefficient given
by the previously defined function cd_sphere() (see (2.6.4)), and then of a golf
ball with drag and lift. The problem is illustrated in the following figure:
y
vf
Fɭ
vr
ϕ
Fd
v0 m
g
Figure 2.17: An illustration of a particle subjected to both gravity and drag in

two dimensions.
where v is the absolute velocity, vf = is the velocity of the fluid, vr = v − vf

is the relative velocity between the fluid and the ball, α is the elevation angle,
v0 is the initial velocity and φ is the angle between the x-axis and vr .
Fl is the lift force stemming from the rotation of the ball (the Magnus-effect)
and is normal to vr . With the given direction the ball rotates counter-clockwise
(backspin). Fd is the fluids resistance against the motion and is parallel to vr .
These forces are given by
1
Fd = ρf ACD vr2 (2.99)
2
1
Fl = ρf ACL vr2 (2.100)
2
CD is the drag coefficient, CL is the lift coefficient, A is the area projected
in the velocity direction and ρF is the density of the fluid.
Newton’s law in x- and y-directions gives
dvx A 2
= −ρf v (CD · cos(φ) + CL sin(φ)) (2.101)
dt 2m r
dvy A 2
= ρf v (CL · cos(φ) − CD sin(φ)) − g (2.102)
dt 2m r
From the figure we have
vrx
cos(φ) =
vr
vry
sin(φ) =
vr
A 3ρ
We assume that the particle is a sphere, such that C = ρf 2m = 4ρkfd as in
(2.6.4). Here d is the diameter of the sphere and ρk the density of the sphere.
Now (2.101) and (2.102) become

dvx
= −C · vr (CD · vrx + CL · vry ) (2.103)
dt
dvy
= C · vr (CL · vrx − CD · vry ) − g (2.104)
dt
dx dy
With dt = vx and dt = vy we get a system of 1st order equations as follows,
dx
= vx
dt
dy
= vy
dt
dvx
= −C · vr (CD · vrx + CL · vry ) (2.105)
dt
dvy
= C · vr (CL · vrx − CD · vry ) − g
dt
Introducing the notation x = y1 , y = y2 , vx = y3 , vy = y4 , we get
dy1
= y3
dt
dy2
= y4
dt
dy3
= −C · vr (CD · vrx + CL · vry ) (2.106)
dt
dy4
= C · vr (CL · vrx − CD · vry ) − g
dt
q we have vrx = vx − vf x = y3 − vf x , vry = vy − vf y = y4 − vf y ,

Here
vr = vrx 2 + v2
ry
Initial conditions for t = 0 are
y1 = y2 = 0
y3 = v0 cos(α)
y4 = v0 sin(α)
—–
Let’s first look at the case of a smooth ball. We use the following data (which
are the data for a golf ball):
6m 3
Diameter d = 41mm, mass m = 46g which gives ρk = = 1275kg/m
πd3
We use the initial velocity v0 = 50 m/s and solve (2.106) using the Runge-Kutta
4 scheme. In this example we have used the Python package Odespy (ODE
Software in Python), which offers a large collection of functions for solving
ODE’s. The RK4 scheme available in Odespy is used herein.
The right hand side in (2.106) is implemented as the following function:
def f(z, t):

"""4x4 system for smooth sphere with drag in two directions."""
C = 3.0*rho_f/(4.0*rho_s*d)
vrx = z[2] - vfx
vry = z[3] - vfy
vr = np.sqrt(vrx**2 + vry**2)
Re = vr*d/nu
CD = cd_sphere(Re) # using the already defined function
zout[:] = [z[2], z[3], -C*vr*(CD*vrx), C*vr*(-CD*vry) - g]
Note that we have used the function cd_sphere() defined in (2.6.4) to

calculate the drag coefficient of the smooth sphere.
The results are shown for some initial angles in Figure 2.18.
25
angle=30.0, smooth ball
20 angle=15.0, smooth ball
15
y [m]
10
00 20 40 60 80 100
x [m]
Figure 2.18: Motion of smooth ball with drag.
—–
Now let’s look at the same case for a golf ball. The dimension and weight
are the same as for the sphere. Now we need to account for the lift force from
the spin of the ball. In addition, the drag data for a golf ball are completely
different from the smooth sphere. We use the data from Bearman and Harvey
[1] who measured the drag and lift of a golf ball for different spin velocities in
a vindtunnel. We choose as an example 3500 rpm, and an initial velocity of
v0 = 50 m/s.
The right hand side in (2.106) is now implemented as the following function:
def f3(z, t):
"""4x4 system for golf ball with drag and lift in two directions."""
C = 3.0*rho_f/(4.0*rho_s*d)
vrx = z[2] - vfx
vry = z[3] - vfy
Re = vr*d/nu
CD, CL = cdcl(vr, nrpm)
zout[:] = [z[2], z[3], -C*vr*(CD*vrx + CL*vry), C*vr*(CL*vrx - CD*vry) - g]
The function cdcl() (may be downloaded here) gives the drag and lift data
for a given velocity and spin.
The results are shown in Figure 2.19. The motion of a golf ball with drag
but without lift is also included. We see that the golf ball goes much farther
than the smooth sphere, due to less drag and the lift.
The complete program ParticleMotion2D.py is listed below.
50
40 angle=15.0, smooth ball
angle=30.0, golf ball
30 angle=20.0, golf ball
y [m]
angle=30.0, golf ball (with lift)
20 angle=20.0, golf ball (with lift)
10
00 50 100 150 200 250

x [m]
Figure 2.19: Motion of golf ball with drag and lift.
# src-ch1/ParticleMotion2D.py;DragCoefficientGeneric.py @ git@lrhgit/tkt4140/src/src-ch1/DragCoeffici

from cdclgolfball import cdcl
import numpy as np
import odespy
g = 9.81 # Gravity [m/s^2]

d = 41.0e-3 # Diameter of the sphere [m]
v0 = 50.0 # Initial velocity [m/s]
vfx = 0.0 # x-component of fluid’s velocity
vfy = 0.0 # y-component of fluid’s velocity
nrpm = 3500 # no of rpm of golf ball
# smooth ball
def f(z, t):
"""4x4 system for smooth sphere with drag in two directions."""
C = 3.0*rho_f/(4.0*rho_s*d)
vrx = z[2] - vfx
vry = z[3] - vfy
Re = vr*d/nu
CD = cd_sphere(Re) # using the already defined function
return zout
# golf ball without lift

def f2(z, t):
"""4x4 system for golf ball with drag in two directions."""
C = 3.0*rho_f/(4.0*rho_s*d)
vrx = z[2] - vfx
vry = z[3] - vfy
Re = vr*d/nu
return zout
# golf ball with lift

def f3(z, t):

"""4x4 system for golf ball with drag and lift in two directions."""
C = 3.0*rho_f/(4.0*rho_s*d)
vrx = z[2] - vfx
vry = z[3] - vfy
Re = vr*d/nu
zout[:] = [z[2], z[3], -C*vr*(CD*vrx + CL*vry), C*vr*(CL*vrx - CD*vry) - g]
return zout
N2 = 4
alfa = np.linspace(30, 15, N2) # Angle of elevation [degrees]
angle = alfa*np.pi/180.0 # convert to radians
legends=[]
line_color=[’k’,’m’,’b’,’r’]
figure(figsize=(20, 8))
hold(’on’)
LNWDT=4; FNT=18
# computing and plotting
# smooth ball with drag

for i in range(0,N2):
z0 = np.zeros(4)
z0[2] = v0*np.cos(angle[i])
z0[3] = v0*np.sin(angle[i])
solver = odespy.RK4(f)
solver.set_initial_condition(z0)
z, t = solver.solve(time)
plot(z[:,0], z[:,1], ’:’, color=line_color[i])
legends.append(’angle=’+str(alfa[i])+’, smooth ball’)
# golf ball with drag

z0 = np.zeros(4)
solver = odespy.RK4(f2)
plot(z[:,0], z[:,1], ’-.’, color=line_color[i])
legends.append(’angle=’+str(alfa[i])+’, golf ball’)
# golf ball with drag and lift

z0 = np.zeros(4)
solver = odespy.RK4(f3)
plot(z[:,0], z[:,1], ’.’, color=line_color[i])
legends.append(’angle=’+str(alfa[i])+’, golf ball (with lift)’)

xlabel(’x [m]’)
ylabel(’y [m]’)
axis([0, 250, 0, 50])
#savefig(’example_particle_motion_2d_2.png’, transparent=True)
show()
2.12 Basic notions on numerical methods for IVPs

In this section we define basic notions about numerical method that are needed
in order to identify their essential properties.
Consider the first order ODE
y 0 = f (x, y), (2.107)
where f (x, y) is the source term. Moreover, the ODE is equipped with initial
conditions
y(x0 ) = y0 . (2.108)
Finally, we define y(xn ) as the solution of IVP defined by (2.107) and (2.108)
evaluated at x = xn , whereas yn is a numerical approximation of y(xn ) at the
same location. We can now define an approximation error as
en = y(xn ) − yn (2.109)
and some useful variants such as the absolute error
|en | = |y(xn ) − yn | (2.110)
and the relative error
|y(xn ) − yn |
rn = . (2.111)
|y(xn )|
We can define a generic explicit numerical scheme for Equation (2.107)

as
yn+1 = yn + hφ(xn , yn , h), (2.112)

where h is the discretization step for x, i.e. h = xn+1 − xn , whereas

φ(xn , yn , h) is the increment function. Note that one can define a generic implicit
or multi-step scheme by changing the arguments of φ.
Having defined a generic numerical scheme, we can say that it is consistent if
lim φ(x, y, h) = φ(x, y, 0) = f (x, y).

h→0
In short, the approximation produced by a consistent numerical scheme will

converge to the original ODE as h → 0.
Two additional useful definitions are the exact differential operator
Le (y) = y 0 − f (x, y) = 0 (2.113)
and the approximate differential operator
La (yn ) = yn+1 − [yn + hφ(xn , yn , h)] = 0. (2.114)
Now we have introduced all necessary concepts to define the local trunca-
tion error (LTE)
1
τn = La (y(xn )). (2.115)
h
In practice, one applies the approximate differential operator, that is defined
by the numerical scheme, to the exact solution of the problem at hand. The
evaluation of the exact solution for different x around xn , as required by La is
performed using a Taylor series expansion.
Finally, we can state that a scheme is p-th order accurate by examining its
LTE and observing its leading term
τn = Chp + H.O.T., (2.116)
where C is a constant, independent of h and H.O.T. are the higher order

terms of the LTE.
Example: LTE for Euler’s scheme
Consider the IVP defined by
y 0 = λy, (2.117)
with initial condition
y(0) = 1. (2.118)
The approximation operator (2.114) for Euler’s scheme is
Leuler
a = yn+1 − [yn + hλyn ], (2.119)
whereas the LTE can be computed by inserting y(x) in (2.119)
1 1
τn = {La (y(xn ))} = {y(xn+1 ) − [y(xn ) + hλy(xn )]} , (2.120)
h h
h2

1 1
= y(xn ) + hy 0 (xn ) + y 00 (xn ) + . . . + hp y (p) (xn ) − y(xn ) − hλy(xn )
h 2 p!
(2.121)
1 00 1 p−1 (p)
= hy (xn ) + . . . + h y (xn ) (2.122)
2 p!
1
≈ hy 00 (xn ). (2.123)
2
2.13 Variable time stepping methods

Recall the local truncation error of the Euler scheme applied to the model
exponential decay equation (2.117), computed in (2.123)
1 00
τneuler = hy (xn ).
2
The error of the numerical approximation depends on h in a linear fashion,
as expected for a first order scheme. However, the approximation error does also
depend on the solution itself, more precisely on its second derivative. This is a
property of the problem and we have no way of intervening on it. If one wants
to set a maximum admissible error, then the only parameter left is the time step
h. The wisest manner to use this freedom is to adopt a variable time-stepping
strategy, in which the time step is adapted according to estimates of the error as
long as one advances in x. Here we briefly present one possible approach, namely
the 5-th order Runge-Kutta-Fehlberg scheme with variable time step (RKF45),
as presented for example in [3]. Schematically, the main steps for implementing
such scheme are listed bellow:
1. Assuming that the solution at xn is available, compute a fourth and a fifth

order RKF solution at time xn+1
RKF 5 RKF 4
2. Compute an estimate of the error: en+1 ≈ |yn+1 − yn+1 |
3. Increase or decrease h accoring to pre-defined maximum and minimum
admissible errors.
An example of how to implement RKF45 is given bellow. Solutions obtained using

this method and a first order Euler fixed-step scheme are shown in Figure 2.20.
Moreover, this figures shows how time step varies as one moves along the x axis.
# src-ch1/fixed_vs_variable.py;
import numpy as np
import matplotlib.pyplot as plt
# Solve
# y’ = beta * y
# y(0) = 1.
# with explicit Euler (fixed h) and RKF45
# ode coefficient
beta = -10.
# ode initial condition
y0 = 1.
# time step for Euler’s scheme
h = 0.02
# domain
xL = 0.
xR = 1.
# number of intervals for Euler’s scheme
N = int((xR-xL)/h)
# correct h for Euler’s scheme
h = (xR-xL)/float(N)
# independent variable array for Euler’s scheme
x = np.linspace(xL,xR,N)
# unknown variable array for Euler’s scheme
y = np.zeros(N)
# x for exact solution
xexact = np.linspace(xL,xR,max(1000.,100*N))
# ode function
def f(x,y):
return beta * y
# exact solution
def exact(x):
return y0*np.exp(beta*x)
# Euler’s method increment

def eulerIncrementFunction(x,yn,h,ode):
return ode(x,yn)
# RKF45 increment
def rkf45step(x,yn,h,ode):
# min and max time step
hmin = 1e-5
hmax = 5e-1
# min and max errors
emin = 1e-7
emax = 1e-5
# max number of iterations
nMax = 100
# match final time
if x+h > xR:
h = xR-x
# loop to find correct error (h)
update = 0
for i in range(nMax):
k1 = ode(x,yn)
k2 = ode(x+h/4.,yn+h/4.*k1)
k3 = ode(x+3./8.*h,yn+3./32.*h*k1+9./32.*h*k2)
k4 = ode(x+12./13.*h,yn+1932./2197.*h*k1-7200./2197.*h*k2+7296./2197.*h*k3)
k5 = ode(x+h,yn+439./216.*h*k1-8.*h*k2+3680./513.*h*k3-845./4104.*h*k4)
k6 = ode(x+h/2.,yn-8./27.*h*k1+2.*h*k2-3544./2565.*h*k3+1859./4140.*h*k4-11./40.*h*k5)
# 4th order solution
y4 = yn + h * (25./216*k1 + 1408./2565.*k3+2197./4104.*k4-1./5.*k5)
# fifth order solution
y5 = yn + h * (16./135.*k1 + 6656./12825.*k3 + 28561./56430.*k4 - 9./50.*k5 +2./55.*k6)
# error estimate
er = np.abs(y5-y4)
if er < emin:
# if error small, enlarge h, but match final simulation time
h = min(2.*h,hmax)
if x+h > xR:
h = xR-x
break
elif er > emax:
# if error big, reduce h
h = max(h/2.,hmin)
else:
# error is ok, take this h and y5
break
if i==nMax-1:
print "max number of iterations reached, check parameters"
return x+h, y5, h , er

# time loop for euler
y[0] = y0
for i in range(N-1):
y[i+1] = y[i] + h * eulerIncrementFunction(x[i],y[i],h,f)
# time loop for RKF45

nMax = 1000
xrk = np.zeros(1)
yrk = y0*np.ones(1)
hrk = np.zeros(1)
h = 0.5
for i in range(nMax):
xloc , yloc, h , er = rkf45step(xrk[-1],yrk[-1],h,f)
xrk = np.append(xrk,xloc)
yrk = np.append(yrk,yloc)
if i==0:
hrk[i] = h
else:
hrk = np.append(hrk,h)
if xrk[-1]==xR:
break
plt.subplot(211)
plt.plot(xexact,exact(xexact),’b-’,label=’Exact’)
plt.plot(x,y,’rx’,label=’Euler’)
plt.plot(xrk,yrk,’o’, markersize=7,markeredgewidth=1,markeredgecolor=’g’,markerfacecolor=’None’,label
plt.xlabel(’x’)
plt.ylabel(’y(x)’)
plt.legend()
plt.subplot(212)
plt.plot(xrk[1:],hrk,’o’, markersize=7,markeredgewidth=1,markeredgecolor=’g’,markerfacecolor=’None’,l
plt.ylabel(’h’)
plt.legend(loc=’best’)
#plt.savefig(’fixed_vs_variable.png’)
plt.show()
Figure 2.20: Numerical solution for the model exponential decay equation
(2.117) using the Euler scheme and the RKF45 variable time step scheme (top
panel) and time step used at each iteration by the variable time stepping scheme
(bottom panel).
2.14 Numerical error as a function of ∆t for ODE-

schemes
To investigate whether the various ODE-schemes in our module ’ODEschemes.py’
have the expected, theoretical order, we proceed in the same manner as outlined
in (2.8.1). The complete code is listed at the end of this section but we will
highlight and explain some details in the following.
To test the numerical order for the schemes we solve a somewhat general
linear ODE:
u0 (t) = a u + b (2.124)
u(t0 ) = u0
which has the analytical solutions:
(
u0 + ab ea t − ab ,

a 6= 0
u= (2.125)
u0 + b t, a=0
The right hand side defining the differential equation has been implemented in
function f3 and the corresponding analytical solution is computed by u_nonlin_analytical:
def f3(z, t, a=2.0, b=-1.0):

""" """
return a*z + b
def u_nonlin_analytical(u0, t, a=2.0, b=-1.0):

from numpy import exp
TOL = 1E-14
if (abs(a)>TOL):
return (u0 + b/a)*exp(a*t)-b/a
else:
return u0 + b*t
—– The basic idea for the convergence test in the function convergence_test is
that we start out by solving numerically an ODE with an analytical solution on
a relatively coarse grid, allowing for direct computations of the error. We then
reduce the timestep by a factor two (or double the grid size), repeatedly, and
compute the error for each grid and compare it with the error of previous grid.
The Euler scheme (2.55) is O(h), whereas the Heun scheme (2.78) is O(h2 ),
and Runge-Kutta (2.96) is O(h4 ), where the h denote a generic step size which
for the current example is the timestep ∆t. The order of a particular scheme is
given exponent n in the error term O(hn ). Consequently, the Euler scheme is a
first oder scheme, Heun is second order, whereas Runge-Kutta is fourth order.
By letting i+1 and i denote the errors on two consecutive grids with
∆ti
corresponding timesteps ∆ti+1 = . The errors i+1 and i for a scheme of
2
order n are then related by:
1
i+1 = n i (2.126)
2
Consequently, whenever i+1 and i are known from consecutive simulations an
estimate of the order of the scheme may be obtained by:
i
n ≈ log2 (2.127)
i+1
The theoretical value of n is thus n = 1 for Euler’s method, n = 2 for Heun’s

method and n = 4 for RK4.
In the function convergence_test the schemes we will subject to a conver-
gence test is ordered in a list scheme_list. This allows for a convenient loop
over all schemes with the clause: for scheme in scheme_list:. Subsequently,
for each scheme we refine the initial grid (N=30) Ndts times in the loop for i in
range(Ndts+1): and solve and compute the order estimate given by (2.127) with
the clause order_approx.append(previous_max_log_err - max_log_err). Note
that we can not compute this for the first iteration (i=0), and that we use a an
initial empty list order_approx to store the approximation of the order n for
each grid refinement. For each grid we plot log2 () as a function of time with:
plot(time[1:], log_error, linestyles[i]+colors[iclr], markevery=N/5)
and for each plot we construct the corresponding legend by appending a new ele-
ment to the legends-list legends.append(scheme.func_name +’: N = ’ + str(N)).
This construct produces a string with both the scheme name and the number of
elements N . The plot is not reproduced below, but you may see the result by
downloading and running the module yourself.
Having completed the given number of refinements Ndts for a specific scheme
we store the order_approx for the scheme in a dictionary using the name of
the scheme as a key by schemes_orders[scheme.func_name] = order_approx.
This allows for an illustrative plot of the order estimate for each scheme with
the clause:
for key in schemes_orders:
plot(N_list, (np.asarray(schemes_orders[key])))
and the resulting plot is shown in Figure 2.21, and we see that our numerical
approximations for the orders of our schemes approach the theoretical values as
the number of timesteps increase (or as the timestep is reduced by a factor two
consecutively).
5
rk4
heun
4
euler
Scheme order approximation
theoretical
3
0
120
240
480
960
192
0
Number of unknowns
Figure 2.21: The convergence rate for the various ODE-solvers a function of
the number of timesteps.
The complete function convergence_test is a part of the module ODEschemes

and is isolated below:
def convergence_test():
""" Test convergence rate of the methods """
from numpy import linspace, size, abs, log10, mean, log2
figure()
tol = 1E-15
T = 8.0 # end of simulation
Ndts = 5 # Number of times to refine timestep in convergence test
z0 = 2
schemes =[euler, heun, rk4]
legends=[]
schemes_order={}
colors = [’r’, ’g’, ’b’, ’m’, ’k’, ’y’, ’c’]

linestyles = [’-’, ’--’, ’-.’, ’:’, ’v--’, ’*-.’]

iclr = 0
for scheme in schemes:
time = linspace(0, T, N+1)
order_approx = []
z = scheme(f3, z0, time)
abs_error = abs(u_nonlin_analytical(z0, time)-z[:,0])
log_error = log2(abs_error[1:]) # Drop 1st elt to avoid log2-problems (1st elt is zer
max_log_err = max(log_error)
legends.append(scheme.func_name +’: N = ’ + str(N))
hold(’on’)
if i > 0: # Compute the log2 error difference

order_approx.append(previous_max_log_err - max_log_err)
previous_max_log_err = max_log_err
N *=2
schemes_order[scheme.func_name] = order_approx
iclr += 1
legend(legends, loc=’best’)
xlabel(’Time’)
ylabel(’log(error)’)
grid()
N = N/2**Ndts
N_list = [N*2**i for i in range(1, Ndts+1)]
N_list = np.asarray(N_list)
figure()
for key in schemes_order:
plot(N_list, (np.asarray(schemes_order[key])))
# Plot theoretical n for 1st, 2nd and 4th order schemes

axhline(1.0, xmin=0, xmax=N, linestyle=’:’, color=’k’)
xticks(N_list, rotation=-70)
legends = schemes_order.keys()
legends.append(’theoretical’)
xlabel(’Number of unknowns’)
ylabel(’Scheme order approximation’)
axis([0, max(N_list), 0, 5])
# savefig(’ConvergenceODEschemes.png’, transparent=True)
def manufactured_solution():
""" Test convergence rate of the methods, by using the Method of Manufactured solutions.
The coefficient function f is chosen to be the normal distribution
f = (1/(sigma*sqrt(2*pi)))*exp(-((t-mu)**2)/(2*sigma**2)).
The ODE to be solved is than chosen to be: f’’’ + f’’*f + f’ = RHS,
leading to to f’’’ = RHS - f’’*f - f
"""
from sympy import exp, symbols, diff, lambdify

from math import sqrt, pi
print "solving equation f’’’ + f’’*f + f’ = RHS"

print "which lead to f’’’ = RHS - f’’*f - f"
t = symbols(’t’)
sigma=0.5 # standard deviation
mu=0.5 # mean value
Domain=[-1.5, 2.5]
t0 = Domain[0]
tend = Domain[1]
f = (1/(sigma*sqrt(2*pi)))*exp(-((t-mu)**2)/(2*sigma**2))
dfdt = diff(f, t)
d2fdt = diff(dfdt, t)
d3fdt = diff(d2fdt, t)
RHS = d3fdt + dfdt*d2fdt + f
f = lambdify([t], f)
dfdt = lambdify([t], dfdt)
d2fdt = lambdify([t], d2fdt)
RHS = lambdify([t], RHS)
def func(y,t):
yout = np.zeros_like(y)
yout[:] = [y[1], y[2], RHS(t) -y[0]- y[1]*y[2]]
return yout
z0 = np.array([f(t0), dfdt(t0), d2fdt(t0)])
figure()
tol = 1E-15
legends=[]
schemes_order={}
colors = [’r’, ’g’, ’b’, ’m’, ’k’, ’y’, ’c’]

linestyles = [’-’, ’--’, ’-.’, ’:’, ’v--’, ’*-.’]
iclr = 0
time = linspace(t0, tend, N+1)
fanalytic = np.zeros_like(time)
k = 0
for tau in time:
fanalytic[k] = f(tau)
k = k + 1
order_approx = []
z = scheme(func, z0, time)
abs_error = abs(fanalytic-z[:,0])
hold(’on’)

N *=2
k = 0
for tau in time:
k = k + 1
iclr += 1
xlabel(’Time’)
grid()
N = N/2**Ndts
figure()

title(’Method of Manufactured Solution’)
# savefig(’MMSODEschemes.png’, transparent=True)
# test using MMS and solving a set of two nonlinear equations to find estimate of order
def manufactured_solution_Nonlinear():
leading to f’’’ = RHS - f’’*f - f
"""
from numpy import linspace, abs
from numpy import log, log2
sigma= 0.5 # standard deviation
mu = 0.5 # mean value
#### Perform needed differentiations based on the differential equation ####
dfdt = diff(f, t)
#### Create Python functions of f, RHS and needed differentiations of f ####
f = lambdify([t], f, np)
dfdt = lambdify([t], dfdt, np)
def func(y,t):
""" Function that returns the dfn/dt of the differential equation f + f’’*f + f’’’ = RHS
as a system of 1st order equations; f = f1
f1’ = f2
f2’ = f3
f3’ = RHS - f1 - f2*f3
Args:
y(array): solutian array [f1, f2, f3] at time t
Returns:
yout(array): differantiation array [f1’, f2’, f3’] at time t
"""
yout[:] = [y[1], y[2], RHS(t) -y[0]- y[1]*y[2]]
return yout
t0, tend = -1.5, 2.5

z0 = np.array([f(t0), dfdt(t0), d2fdt(t0)]) # initial values
schemes = [euler, heun, rk4] # list of schemes; each of which is a function

schemes_error = {} # empty dictionary. to be filled in with lists of error-norms for all sche
h = [] # empty list of time step
Ntds = 4 # number of times to refine dt
fig, ax = subplots(1, len(schemes), sharey = True, squeeze=False)
for k, scheme in enumerate(schemes):

N = 20 # initial number of time steps
error = [] # start of with empty list of errors for all schemes
legendList = []
for i in range(Ntds + 1):

if k==0:
h.append(time[1] - time[0]) # add this iteration’s dt to list h
z = scheme(func, z0, time) # Solve the ODE by calling the scheme with arguments. e.g:
fanalytic = f(time) # call analytic function f to compute analytical solutions at tim
abs_error = abs(z[:,0]- fanalytic) # calculate infinity norm of the error

error.append(max(abs_error))
ax[0][k].plot(time, z[:,0])
legendList.append(’$h$ = ’ + str(h[i]))
N *=2 # refine dt
schemes_error[scheme.func_name] = error # Add a key:value pair to the dictionary. e.g: "e

ax[0][k].plot(time, fanalytic, ’k:’)

legendList.append(’$u_m$’)
ax[0][k].set_title(scheme.func_name)
ax[0][k].set_xlabel(’time’)
ax[0][2].legend(legendList, loc = ’best’, frameon=False)

ax[0][0].set_ylabel(’u’)
setp(ax, xticks=[-1.5, 0.5, 2.5], yticks=[0.0, 0.4 , 0.8, 1.2])
# #savefig(’../figs/normal_distribution_refinement.png’)
def Newton_solver_sympy(error, h, x0):
""" Function that solves for the nonlinear set of equations
error1 = C*h1^p --> f1 = C*h1^p - error1 = 0
error2 = C*h2^p --> f2 = C h2^p - error 2 = 0
where C is a constant h is the step length and p is the order,
with use of a newton rhapson solver. In this case C and p are
the unknowns, whereas h and error are knowns. The newton rhapson
method is an iterative solver which take the form:
xnew = xold - (J^-1)*F, where J is the Jacobi matrix and F is the
residual funcion.
x = [C, p]^T
J = [[df1/dx1 df2/dx2],
[df2/dx1 df2/dx2]]
F = [f1, f2]
This is very neatly done with use of the sympy module
Args:
error(list): list of calculated errors [error(h1), error(h2)]
h(list): list of steplengths corresponding to the list of errors
x0(list): list of starting (guessed) values for x
Returns:
x(array): iterated solution of x = [C, p]
"""
from sympy import Matrix
#### Symbolic computiations: ####
C, p = symbols(’C p’)
f1 = C*h[-2]**p - error[-2]
f2 = C*h[-1]**p - error[-1]
F = [f1, f2]
x = [C, p]
def jacobiElement(i,j):
return diff(F[i], x[j])
Jacobi = Matrix(2, 2, jacobiElement) # neat way of computing the Jacobi Matrix

JacobiInv = Jacobi.inv()
#### Numerical computations: ####
JacobiInvfunc = lambdify([x], JacobiInv)
Ffunc = lambdify([x], F)
x = x0
for n in range(8): #perform 8 iterations

F = np.asarray(Ffunc(x))
Jinv = np.asarray(JacobiInvfunc(x))
xnew = x - np.dot(Jinv, F)
x = xnew
#print "n, x: ", n, x
x[0] = round(x[0], 2)
x[1] = round(x[1], 3)
return x
ht = np.asarray(h)
eulerError = np.asarray(schemes_error["euler"])
heunError = np.asarray(schemes_error["heun"])
rk4Error = np.asarray(schemes_error["rk4"])
[C_euler, p_euler] = Newton_solver_sympy(eulerError, ht, [1,1])

[C_heun, p_heun] = Newton_solver_sympy(heunError, ht, [1,2])
[C_rk4, p_rk4] = Newton_solver_sympy(rk4Error, ht, [1,4])
from sympy import latex
h = symbols(’h’)
epsilon_euler = C_euler*h**p_euler
epsilon_euler_latex = ’$’ + latex(epsilon_euler) + ’$’
epsilon_heun = C_heun*h**p_heun
epsilon_heun_latex = ’$’ + latex(epsilon_heun) + ’$’
epsilon_rk4 = C_rk4*h**p_rk4
epsilon_rk4_latex = ’$’ + latex(epsilon_rk4) + ’$’
print epsilon_euler_latex
print epsilon_heun_latex
print epsilon_rk4_latex
epsilon_euler = lambdify(h, epsilon_euler, np)
epsilon_heun = lambdify(h, epsilon_heun, np)
epsilon_rk4 = lambdify(h, epsilon_rk4, np)
N = N/2**(Ntds + 2)
N_list = [N*2**i for i in range(1, Ntds + 2)]
print len(N_list)
print len(eulerError)
figure()
plot(N_list, log2(eulerError), ’b’)
plot(N_list, log2(epsilon_euler(ht)), ’b--’)
plot(N_list, log2(heunError), ’g’)
plot(N_list, log2(epsilon_heun(ht)), ’g--’)
plot(N_list, log2(rk4Error), ’r’)
plot(N_list, log2(epsilon_rk4(ht)), ’r--’)
LegendList = [’${\epsilon}_{euler}$’, epsilon_euler_latex, ’${\epsilon}_{heun}$’, epsilon_heu
legend(LegendList, loc=’best’, frameon=False)
xlabel(’-log(h)’)
ylabel(’-log($\epsilon$)’)
# #savefig(’../figs/MMS_example2.png’)
The complete module ODEschemes is listed below and may easily be down-
loaded in your Eclipse/LiClipse IDE:
# src-ch1/ODEschemes.py
import numpy as np
from matplotlib.pyplot import plot, show, legend, hold,rcParams,rc, figure, axhline, close,\
xticks, title, xlabel, ylabel, savefig, axis, grid, subplots, setp

LNWDT=3; FNT=10
# define Euler solver

def euler(func, z0, time):
z[0,:] = z0
z[i+1,:]=z[i,:] + np.asarray(func(z[i,:], time[i]))*dt
return z
# define Heun solver

def heun(func, z0, time):
"""The Heun scheme for solution of systems of ODEs.
z[0,:] = z0
zp = z[i,:] + np.asarray(func(z[i,:],t))*dt # Predictor step
z[i+1,:] = z[i,:] + (np.asarray(func(z[i,:],t)) + np.asarray(func(zp,t+dt)))*dt/2.0 # Correct
return z
# define rk4 scheme

def rk4(func, z0, time):
"""The Runge-Kutta 4 scheme for solution of systems of ODEs.
z[0,:] = z0
dt2 = dt/2.0
k1 = np.asarray(func(z[i,:], t)) # predictor step 1
k4 = np.asarray(func(z[i,:] + k3*dt, t + dt)) # predictor step 4
z[i+1,:] = z[i,:] + dt/6.0*(k1 + 2.0*k2 + 2.0*k3 + k4) # Corrector step
return z
if __name__ == ’__main__’:
a = 0.2
b = 3.0
u_exact = lambda t: a*t + b
def f_local(u,t):
"""A function which returns an np.array but less easy to read
than f(z,t) below. """
return np.asarray([a + (u - u_exact(t))**5])
def f(z, t):

"""Simple to read function implementation """
return [a + (z - u_exact(t))**5]
def test_ODEschemes():
"""Use knowledge of an exact numerical solution for testing."""
from numpy import linspace, size
tol = 1E-15
z0 = np.zeros(1)
z0[0] = u_exact(0.0)
schemes = [euler, heun, rk4]

z = scheme(f, z0, time)
max_error = np.max(u_exact(time) - z[:,0])
msg = ’%s failed with error = %g’ % (scheme.func_name, max_error)
assert max_error < tol, msg
# f3 defines an ODE with analytical solution in u_nonlin_analytical
def f3(z, t, a=2.0, b=-1.0):
""" """
return a*z + b
def u_nonlin_analytical(u0, t, a=2.0, b=-1.0):
from numpy import exp
TOL = 1E-14
if (abs(a)>TOL):
return (u0 + b/a)*exp(a*t)-b/a
else:
return u0 + b*t
# Function for convergence test

def convergence_test():
""" Test convergence rate of the methods """
figure()
tol = 1E-15
z0 = 2

legends=[]
schemes_order={}
colors = [’r’, ’g’, ’b’, ’m’, ’k’, ’y’, ’c’]
linestyles = [’-’, ’--’, ’-.’, ’:’, ’v--’, ’*-.’]
iclr = 0
order_approx = []
abs_error = abs(u_nonlin_analytical(z0, time)-z[:,0])
hold(’on’)

N *=2
iclr += 1
xlabel(’Time’)
grid()
N = N/2**Ndts
figure()

# savefig(’ConvergenceODEschemes.png’, transparent=True)
def manufactured_solution():

leading to to f’’’ = RHS - f’’*f - f
"""
print "solving equation f’’’ + f’’*f + f’ = RHS"

print "which lead to f’’’ = RHS - f’’*f - f"
sigma=0.5 # standard deviation
mu=0.5 # mean value
Domain=[-1.5, 2.5]
t0 = Domain[0]
tend = Domain[1]
dfdt = diff(f, t)
f = lambdify([t], f)
dfdt = lambdify([t], dfdt)
def func(y,t):
yout[:] = [y[1], y[2], RHS(t) -y[0]- y[1]*y[2]]
return yout
z0 = np.array([f(t0), dfdt(t0), d2fdt(t0)])

figure()
tol = 1E-15
legends=[]
schemes_order={}
colors = [’r’, ’g’, ’b’, ’m’, ’k’, ’y’, ’c’]

linestyles = [’-’, ’--’, ’-.’, ’:’, ’v--’, ’*-.’]
iclr = 0
k = 0
for tau in time:
k = k + 1
order_approx = []
z = scheme(func, z0, time)
abs_error = abs(fanalytic-z[:,0])
hold(’on’)

N *=2
k = 0
for tau in time:
k = k + 1
iclr += 1
xlabel(’Time’)
grid()
N = N/2**Ndts
figure()
title(’Method of Manufactured Solution’)
# savefig(’MMSODEschemes.png’, transparent=True)
# test using MMS and solving a set of two nonlinear equations to find estimate of order
def manufactured_solution_Nonlinear():
leading to f’’’ = RHS - f’’*f - f
"""
from numpy import linspace, abs
from numpy import log, log2
sigma= 0.5 # standard deviation

mu = 0.5 # mean value
#### Perform needed differentiations based on the differential equation ####
dfdt = diff(f, t)
#### Create Python functions of f, RHS and needed differentiations of f ####
f = lambdify([t], f, np)
dfdt = lambdify([t], dfdt, np)
def func(y,t):
""" Function that returns the dfn/dt of the differential equation f + f’’*f + f’’’ = RHS
as a system of 1st order equations; f = f1
f1’ = f2
f2’ = f3
f3’ = RHS - f1 - f2*f3
Args:
y(array): solutian array [f1, f2, f3] at time t
Returns:
yout(array): differantiation array [f1’, f2’, f3’] at time t
"""
yout[:] = [y[1], y[2], RHS(t) -y[0]- y[1]*y[2]]
return yout
t0, tend = -1.5, 2.5

z0 = np.array([f(t0), dfdt(t0), d2fdt(t0)]) # initial values
schemes = [euler, heun, rk4] # list of schemes; each of which is a function

schemes_error = {} # empty dictionary. to be filled in with lists of error-norms for all sche
h = [] # empty list of time step
Ntds = 4 # number of times to refine dt

fig, ax = subplots(1, len(schemes), sharey = True, squeeze=False)
for k, scheme in enumerate(schemes):

N = 20 # initial number of time steps
error = [] # start of with empty list of errors for all schemes
legendList = []
for i in range(Ntds + 1):

if k==0:
h.append(time[1] - time[0]) # add this iteration’s dt to list h
z = scheme(func, z0, time) # Solve the ODE by calling the scheme with arguments. e.g:
fanalytic = f(time) # call analytic function f to compute analytical solutions at tim
abs_error = abs(z[:,0]- fanalytic) # calculate infinity norm of the error

error.append(max(abs_error))
ax[0][k].plot(time, z[:,0])
legendList.append(’$h$ = ’ + str(h[i]))
N *=2 # refine dt
schemes_error[scheme.func_name] = error # Add a key:value pair to the dictionary. e.g: "e
ax[0][k].plot(time, fanalytic, ’k:’)

legendList.append(’$u_m$’)
ax[0][k].set_title(scheme.func_name)
ax[0][k].set_xlabel(’time’)
ax[0][2].legend(legendList, loc = ’best’, frameon=False)

ax[0][0].set_ylabel(’u’)
setp(ax, xticks=[-1.5, 0.5, 2.5], yticks=[0.0, 0.4 , 0.8, 1.2])
# #savefig(’../figs/normal_distribution_refinement.png’)
def Newton_solver_sympy(error, h, x0):
""" Function that solves for the nonlinear set of equations
error1 = C*h1^p --> f1 = C*h1^p - error1 = 0
error2 = C*h2^p --> f2 = C h2^p - error 2 = 0
where C is a constant h is the step length and p is the order,
with use of a newton rhapson solver. In this case C and p are
the unknowns, whereas h and error are knowns. The newton rhapson
method is an iterative solver which take the form:
xnew = xold - (J^-1)*F, where J is the Jacobi matrix and F is the
residual funcion.
x = [C, p]^T
J = [[df1/dx1 df2/dx2],
[df2/dx1 df2/dx2]]
F = [f1, f2]
This is very neatly done with use of the sympy module
Args:
error(list): list of calculated errors [error(h1), error(h2)]
h(list): list of steplengths corresponding to the list of errors
x0(list): list of starting (guessed) values for x
Returns:
x(array): iterated solution of x = [C, p]
"""
from sympy import Matrix
#### Symbolic computiations: ####
C, p = symbols(’C p’)
f1 = C*h[-2]**p - error[-2]
f2 = C*h[-1]**p - error[-1]
F = [f1, f2]
x = [C, p]
def jacobiElement(i,j):
return diff(F[i], x[j])
Jacobi = Matrix(2, 2, jacobiElement) # neat way of computing the Jacobi Matrix
JacobiInv = Jacobi.inv()
#### Numerical computations: ####
JacobiInvfunc = lambdify([x], JacobiInv)
Ffunc = lambdify([x], F)
x = x0
for n in range(8): #perform 8 iterations

F = np.asarray(Ffunc(x))
Jinv = np.asarray(JacobiInvfunc(x))
xnew = x - np.dot(Jinv, F)
x = xnew
#print "n, x: ", n, x
x[0] = round(x[0], 2)
x[1] = round(x[1], 3)
return x
ht = np.asarray(h)
eulerError = np.asarray(schemes_error["euler"])
heunError = np.asarray(schemes_error["heun"])
rk4Error = np.asarray(schemes_error["rk4"])
[C_euler, p_euler] = Newton_solver_sympy(eulerError, ht, [1,1])

[C_heun, p_heun] = Newton_solver_sympy(heunError, ht, [1,2])
[C_rk4, p_rk4] = Newton_solver_sympy(rk4Error, ht, [1,4])
from sympy import latex

h = symbols(’h’)
epsilon_euler = C_euler*h**p_euler
epsilon_euler_latex = ’$’ + latex(epsilon_euler) + ’$’
epsilon_heun = C_heun*h**p_heun
epsilon_heun_latex = ’$’ + latex(epsilon_heun) + ’$’
epsilon_rk4 = C_rk4*h**p_rk4
epsilon_rk4_latex = ’$’ + latex(epsilon_rk4) + ’$’
print epsilon_euler_latex
print epsilon_heun_latex
print epsilon_rk4_latex
epsilon_euler = lambdify(h, epsilon_euler, np)

epsilon_heun = lambdify(h, epsilon_heun, np)
epsilon_rk4 = lambdify(h, epsilon_rk4, np)
N = N/2**(Ntds + 2)
N_list = [N*2**i for i in range(1, Ntds + 2)]
print len(N_list)
print len(eulerError)
figure()
plot(N_list, log2(eulerError), ’b’)
plot(N_list, log2(epsilon_euler(ht)), ’b--’)
plot(N_list, log2(heunError), ’g’)
plot(N_list, log2(epsilon_heun(ht)), ’g--’)
plot(N_list, log2(rk4Error), ’r’)
plot(N_list, log2(epsilon_rk4(ht)), ’r--’)
LegendList = [’${\epsilon}_{euler}$’, epsilon_euler_latex, ’${\epsilon}_{heun}$’, epsilon_heu
legend(LegendList, loc=’best’, frameon=False)
xlabel(’-log(h)’)
ylabel(’-log($\epsilon$)’)
# #savefig(’../figs/MMS_example2.png’)
def plot_ODEschemes_solutions():
"""Plot the solutions for the test schemes in schemes"""
from numpy import linspace
figure()
z0 = 2.0
schemes = [euler, heun, rk4]

legends = []

plot(time, z[:,-1])
legends.append(scheme.func_name)
plot(time, u_nonlin_analytical(z0, time))

legends.append(’analytical’)
manufactured_solution_Nonlinear()
#test_ODEschemes()
#convergence_test()
#plot_ODEschemes_solutions()
#manufactured_solution()
show()
2.15 Absolute stability of numerical methods for

ODE IVPs
To investigate stability of numerical methods for the solution of intital value
ODEs on the generic form (2.2), let us consider the following model problem:
y 0 (x) = λ · y(x) (2.128)

y(0) = 1 (2.129)
which has the analytical solution:
y(x) = eλx , λ , (2.130)

with λ being a constant.
The solution y(x) of (2.128) is illustrated in Figure 2.22 for positive (λ = 1)
and negative (λ = −2) growth factors.
We illustrate the stability concept for the numerical solution of ODEs by two
examples below, namely, the Euler scheme and the Heun scheme. In the latter
example we will also allude to how the analysis may be applied for higher order
RK-methods.
2.15.1 Example: Stability of Euler’s method

By applying the generic 2.6 On our particular model problem given by (2.128)
we obtain the following generic scheme:
yn+1 = (1 + λh) yn , h = ∆x, n = 0, 1, 2, . . . (2.131)

3.0
=1
2.5 = -2
2.0
x
1.5
e
1.0
0.5
0.0
0.0 0.2 0.4 0.6 0.8 1.0
x
Figure 2.22: Solution of ODE for exponential growth with positive and negative
growth factor λ.
The numerical soltion yn at the n-th iteration may be related with the initial
solution y0 by applying the scheme (2.131) iteratively:
y1 = (1 + λh) y0
y2 = (1 + λh) y1 = (1 + λh)2 y0
..
.
which yields:
yn = (1 + λh)n y0 , n = 1, 2, 3, . . . (2.132)
To investigate stability we first introduce the analytic amplification factor
Ga as
y(xn+1 )
Ga = (2.133)
y(xn )
Note that y(xn ) represents the analytical solution of (2.128) at xn , whereas
yn denotes the numerical approximation. For the model problem at hand we see
that Ga reduces to:
y(xn+1 )
Ga = = exp[λ(xn+1 − xn )] = eλh = 1 + λh + (λh)2 /2 . . . (2.134)
y(xn )
Exponential growth λ > 0.

Consider first the case of a positive λ > 0, corresponding to an exponential
growth, as given by (2.130). We observe that our scheme will also exhibit
exponential growth (2.133), and will thus have no concerns in terms of stability.
Naturally, the choice of h will affect the accuracy and numerical error of the
solution.
Exponential decay λ < 0.

In case of exponential decay with a negative λ < 0, we adopt the following
convention for convenience:
λ = −α, α>0 (2.135)
and (2.132) may be recasted to:
yn = (1 − αh)n y0 (2.136)
If yn is to decrease as n increases we must have
|1 − αh| < 1 ⇒ −1 < 1 − αh < 1 (2.137)
which yields the following criterion for selection of h:
0 < αh < 2, α > 0 (2.138)
The criterion may also be formulated by introducing the numerical amplifica-

tion factor G as:
yn+1
G= (2.139)
yn
which for the current example (2.131) reduces to:
yn+1
G= = 1 + λh = 1 − αh (2.140)
yn
For the current example G > 1 for λ > 0 and G < 1 for λ < 0. Compare with
the expression for the analytical amplification factor Ga (2.134).
Numerical amplification factor Consider IVP (2.128). Then, the nu-

merical amplification factor is defined as
yn+1
G(z) = , (2.141)
yn
with z = hλ.
As we shall see later on, the numerical amplification factor G(z) is some
function of z = hλ, commonly a polynomial for an explicit method and a
rational function for implicit methods. For consistent methods G(z) will be an
approximation to ez near z = 0, and if the method is p-th order accurate, then
G(z) − ez = O(z p+1 ) as z → 0.
Absolute stability The numerical scheme is said to be stable when:
|G(z)| ≤ 1 (2.142)
Region of absolute stability The region of absolute stability for a

one-step method is given by
S = {z ∈ C : |G(z)| ≤ 1}. (2.143)
For example, in the case under examination, when α = 100 we must choose
h < 0.02 to produce an stable numerical solution.
By the introduction of the G in (2.140) we may rewrite (2.132) as:
yn = Gn y0 , n = 0, 1, 2, . . . (2.144)
From (2.144) we observe that a stable, exponentially decaying solution (λ < 0

) the solution will oscillate if G < 0, as Gn < 0 when n is odd and Gn > 0 when
n is even. As this behavior is qualitatively different from the behaviour of the
analytical solution, we will seek to avoid such behavior.
The Euler scheme is conditionally stable for our model equation when
0 < αh < 2, α > 0 (2.145)
but oscillations will occur when
1 < αh < 2, α > 0 (2.146)
Consequently, the Euler scheme will be stable and free of spurious oscillations
when
0 < αh < 1, α>0 (2.147)
For example the numerical solution with α = 10 will be stable in the intervall
0 < h < 0.2, but exhibit oscillations in the interval 0.1 < h < 0.2. Note, however,
that the latter restriction on avoiding spurious oscillations will not be of major
concerns in pratice, as the demand for descent accuracy will dictate a far smaller
h.
2.15.2 Example: Stability of Heun’s method

In this example we will analyze the stability of 2.9 when applied on our model
ODE (2.128) with λ = −α, α > 0.
The predictor step reduces to:

p
yn+1 = yn + h · fn = (1 − αh) · yn (2.148)
p
for convenience the predictor yn+1 may be eliminated in the corrector step:
h p 1
yn+1 = yn + · (fn + fn+1 ) = (1 − αh + (αh)2 ) · yn (2.149)
2 2
which shows that Heun’s method may be represented as a one-step method
for this particular and simple model problem. From (2.149) we find the numerical
amplification factor G (ref<eq:1618a);
yn+1 1
G= = 1 − αh + (αh)2 (2.150)
yn 2
Our taks is now to investigate for which values of αh the condition for stability
(2.142) may be satisfied.
First, we may realize that (2.150) is a second order polynomial in αh and
that G = 1 for αh = 0 and αh = 2. The numerical amplification factor has an
extremum where:
dG
= αh − 1 = 0 ⇒ αh = 1
dαh
and since
d2 G
>0
d(αh)2
this extremum is a minimum and Gmin = G(αh = 1) = 1/2. Consequently we
may conclude that 2.9 will be conditionally stable for (2.128) in the following
αh-range (stability range):
0 < αh < 2 (2.151)

Note that this condition is the same as for the 2.15.1 above, but as G > 0
always, the scheme will not produce spurious oscillations.
The 2.6 and 2.9 are RK-methods of first and second order, respectively. Even
though the αh-range was not expanded by using a RK-method of 2 order, the
quality of the numerical predictions were improved as spurios oscillations will
vanish.
2.15.3 Stability of higher order RK-methods

A natural questions to raise is whether the stability range may be expanded for
higher order RK-methods, so let us now investigate their the stability range.
The analytical solution of our model ODE (2.128) is:
(λx)2 (λx)3 (λx)n

y(x) = eλx = 1 + λx + + + ··· + + ··· (2.152)
2! 3! n!
By substitution of λx with λh, we get the solution of the difference equation

corresponding to Euler’s method (RK1) with the two first terms, Heun’s method
(RK2) the three first terms RK3 the four first terms, and finally RK4 with the
first five terms:
(λh)2 (λh)3 (λh)n

G = 1 + λh + + + ··· + . (2.153)
2! 3! n!
We focus on decaying solutions with λ < 0 and we have previously show from
(2.138) that 0 < αh < 2, α > 2 or:
−2 < λh < 0 (2.154)

for RK1 and RK3, but what about RK3? In this case:
(λh)2 (λh)3
G = 1 + λh + + (2.155)
2! 3!
The limiting values for the G-polynomial are found for G = ±1 which for
real roots are:
−2.5127 < λh < 0 (2.156)
For RK4 we get correspondingly
(λh)2 (λh)3 (λh)4

G = 1 + λh + + + (2.157)
2! 3! 4!
which for G = 1 has the real roots:
−2.7853 < λh < 0 (2.158)

In conclusion, we may argue that with respect to stability there is not much
to gain by increasing the order for RK-methods. We have assumed λ to be real
in the above analysis. However, λ may me complex for higer order ODEs and
for systems of first order ODEs. If complex roots are included the regions of
stability for RK-methods become as illstrated in figure 2.23.
Naturally, we want numerical schemes which are stable in the whole left
half-plane. Such a scheme would be denoted strictly absolutely stable or simply
L-stable (where L alludes to left half-plane). Regretfully, there are no existing,
explicit methods which are L-stable. For L-stable methods one has to restort to
implicit methods which are to be discussed in 2.15.4.
2.15.4 Stiff equations

While there is no precise definition of stiff ODEs in the literature, these are
some generic characteristics for ODEs/ODE systems that are often called as
such:
• ODE solution has different time scales. For example, the ODE solution
can be written as the sum of a fast and a slow component as:
3 RK4
RK3
2
RK2
1 RK1
imaginary part of h
3
4 3 2 1 0 1 2 3 4
real part of h
Figure 2.23: Regions of stability for RK-methods order 1-4 for complex roots.
y(x) = yslow (x) + yf ast (x).
• For a first ODE system
y 0 (x) = f (x, y),

the above statement is equivalent to the case in which the eigenvalues of the
Jacobian ∂f
∂y differ greatly in magnitude.
• The definition of h for stiff ODEs is limited by stability concerns rather

than by accuracy.
This last point is related to the definition of absolute stability regions for explicit
schemes, which is inversely proportional to λ. Note that λ is in fact the (only)
eigenvalue of the Jacobian for ODE (2.128)!
Example for stiff ODE: Consider the IVP
)
y 0 = λy − λsin(x) + cos(x), λ < 0,
(2.159)
y(0) = 1.
The analytical solution of problem (2.15.4) is given by

y(x) = sin(x) + eλx ,

from which we can identify a slow component
yslow (x) = sin(x)

and a fast component
yf ast (x) = eλx ,

for |λ| sufficiently large.
1.00 y(x)
yfast(x)
0.75 yslow(x)
0.50
0.25
0.00
y(x)
0.25
0.50
0.75
1.00
0 1 2 3 4 5 6
x
Figure 2.24: Solution (2.15.4) of stiff ODE IVP (2.15.4) for λ = −25.
A way to avoid having to use excessively small time steps, which can be even
harmful for the accuracy of the approximation, due to error accumulation, is
to use numerical schemes which have a larger region of absolute stability than
explicit schemes.
2.15.5 Example: Stability of implicit Euler’s method

The most natural, and widely used, candidate as a replacement of an explicit
first order scheme is the implicit Euler’s scheme. Equation (2.128) discretized
using this scheme reads
yn+1 = yn + hfn+1 = yn + hλyn+1 ,
and after solving for yn+1

1
yn+1 = yn .
1 − hλ
Therefore, the amplification factor for this scheme is
1 1
G= = , λ = −α, α > 0.
1 − hλ 1 + hα
It is evident that G < 1 for any h (recall that h is positive by definition).
This means that the interval of absolute stability for the implicit Euler’s scheme
includes R− . If this analysis was carried out allowing for λ to be a complex
number with negative real part, then the region of absolute stability for the
implicit Euler’s scheme would have included the entire complex half-plane with
negative real part C− .
2.15.6 Example: Stability of trapezoidal rule method

Equation (2.128) discretized by the trapezoidal rule scheme reads
h h
yn+1 = yn + (fn + fn+1 ) = yn + (λyn + λyn+1 ),
2 2
and after solving for yn+1

λh
1+ 2
yn+1 = λh
yn .
1− 2
Therefore, the amplification factor for this scheme is
λh
1+ 2 2 − αh
G= λh
= , λ = −α, α > 0.
1− 2
2 + αh
As for the implicit Euler’s scheme, also in this case we have that G < 1 for
any h. Therefore, interval of absolute stability for the implicit Euler’s scheme
includes R− . In this case, the analysis for λ complex number with negative real
part, would have shown that the region of absolute stability for this scheme is
precisely the entire complex half-plane with negative real part C− .
The above results motivate the following definition:
A-stable scheme A numerical scheme for (2.128) is said to be A-stable

when its region of absolute stability includes the entire complex half-plane
with negative real part C− .
This important definition was introduced in the ’60s by Dahlquist, who

also proved that A-stable schemes can be most second order accurate. A-
stable schemes are unconditionally stable and therefore robust, since no stability
problems will be observed. However, the user must always be aware of the fact
that even if one is allowed to take a large value for h, the error must be contained
within acceptable limits.
2.16 Exercises
Exercise 1: Solving Newton’s first differential equation us-
ing euler’s method
One of the earliest known differential equations, which Newton solved with series
expansion in 1671 is:
y 0 (x) = 1 − 3x + y + x2 + xy, y(0) = 0 (2.160)
Newton gave the following solution:
x3 x4 x5 x6
y(x) ≈ x − x2 + − + − (2.161)
3 6 30 45
Today it is possible to give the solution on closed form with known functions
as follows,
√ √
√

h x i 2 2 h x i
y(x) =3 2πe · exp x 1 + · erf (1 + x) − erf + 4 · 1 − exp[x 1 + −x
2 2 2 2
(2.162)
√
Note the combination 2πe.
a) Solve Eq. (2.160) using Euler’s method. Plot and compare with Newton’s
solution Eq. (2.161) and the analytical solution Eq. (2.162). Plot between x = 0
and x = 1.5
Hint 1. The function scipy.special.erf will be needed. See http://docs.

scipy.org/doc/scipy-0.15.1/reference/generated/scipy.special.erf.html.
Hint 2. Figure should look something like this:
Exercise 2: Solitary wave

This exercise focuses on a solitary wave propagating on water.
The differential equation for a solitary wave can be written as:
d2 Y

3Y A 3
2
= 2 − Y (2.163)
dX D D 2D
0.4
0.2
0.0
y −0.2
−0.4 analytical
euler
Newton
−0.6
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6
x
Figure 2.25: Problem 1 with 101 samplepoints between x = 0 and x = 1.5 .
A
Y
O X
Figure 2.26: Solitary wave.
where D is the middle depth, Y (X) is the wave height above middle depth
and A is the wave height at X = 0. The wave is symmetric with respect to
X = 0. See Figure 2.26. The coordinate system follows the wave.
By using dimensionless variables: x = X A Y
D , a = D , y = A , Eq. (2.163) can be
written as:

3
y 00 (x) = a 3 y(x) 1 − y(x) (2.164)
2
initial conditions: y(0) = 1, y 0 (0) = 0. Use a = 32

Pen and paper
The following problems should be done using pen and paper:
a) Calculate y(0.6), and y’(0.6) using euler’s method with ∆x = 0.2
b) Solve a) using Heuns’s method.

c) Perform a taylor expansion series around x = 0 (include 3 parts) on Eq.
(2.164), and compare results with a) and b).
d) Solve Eq. (2.164) analytically for a = 23 , given:
Z
dy p
√ = −2 arctanh 1−y
y 1−y
Compare with solutions in a), b) and c).

Programing: Write a program that solve a), b) and c) numerically, and
compare with the analytical solution found in d). Solve first with ∆x = 0.2, and
experiment with different values.
Hint 1. Solutions:
a) y(0.6) = 0.88, y 0 (0.6) = −0.569.
b) y(0.6) = 0.8337, y 0 (0.6) = −0.4858.
2 4
c) y ≈ 1 − x2 + x6 , y 0 ≈ −x + 23 x3
d y = cosh2 1x /√2 = 1+cosh1 √2·x
( ) ( )
1.00
0.98
0.96
0.94
0.92
0.90
0.88
euler, y(0.6)=0.88
0.86 heun, y(0.6)=0.8336
taylor, y(0.6)=0.8416
0.84
analytic, y(0.6)=0.8396
0.82
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
Figure 2.27: Plot should look something like this.
Hint 2. If you want you can use this template and fill in the lines where it’s
indicated.
#import matplotlib; matplotlib.use(’Qt4Agg’)
#plt.get_current_fig_manager().window.raise_()
import numpy as np
#### set default plot values: ####

LNWDT=3; FNT=15
""" This script solves the problem with the solitary wave:
y’’ = a*3*y*(1-y*3/2)
y(0) = 1, y’(0) = 0
or as a system of first order differential equations (y0 = y, y1 = y’):
y0’ = y’
y1’ = a*3*y0*(1-y0*3/2)
y0(0) = 1, y1(0) = 0
"""
a = 2./3
h = 0.2 # steplength dx
x_0, x_end = 0, 0.6
x = np.arange(x_0, x_end + h, h) # allocate x values

#### solution vectors: ####
Y0_euler = np.zeros_like(x) # array to store y values
Y1_euler = np.zeros_like(x) # array to store y’ values
Y0_heun = np.zeros_like(x)
Y1_heun = np.zeros_like(x)
#### initial conditions: ####

Y0_euler[0] = 1 # y(0) = 1
Y1_euler[0] = 0 # y’(0) = 0
Y0_heun[0] = 1
Y1_heun[0] = 0
#### solve with euler’s method ####
for n in range(len(x) - 1):

y0_n = Y0_euler[n] # y at this timestep
y1_n = Y1_euler[n] # y’ at this timestep
"Fill in lines below"
f0 =
f1 =
"Fill in lines above"
Y0_euler[n + 1] = y0_n + h*f0

Y1_euler[n + 1] = y1_n + h*f1
#### solve with heun’s method: ####
for n in range(len(x) - 1):

y0_n = Y0_heun[n] # y0 at this timestep (y_n)
y1_n = Y1_heun[n] # y1 at this timestep (y’_n)
"Fill in lines below"

f0 =
f1 =
y0_p =
y1_p =
f0_p =
f1_p =
"Fill in lines above"
Y0_heun[n + 1] = y0_n + 0.5*h*(f0 + f0_p)

Y1_heun[n + 1] = y1_n + 0.5*h*(f1 + f1_p)
Y0_taylor = 1 - x**2/2 + x**4/6

Y1_taylor = -x + (2./3)*x**3
Y0_analytic = 1./(np.cosh(x/np.sqrt(2))**2)
Exercise 3: Mathematical pendulum
g
θ
θ0
Figure 2.28: Mathematical pendulum.
Figure 2.28 shows a mathematical pendulum where the motion is described

by the following ODE:
d2 θ g
+ sin θ = 0 (2.165)
dτ 2 l
with initial conditions
θ(0) = θ0 (2.166)
dθ
(0) = θ̇0 (2.167)
dτ
We introduce a dimensionless time t = gl t such that (2.165) may be written
p
as
θ̈(t) + sin θ(t) = 0 (2.168)
with initial conditions
θ(0) = θ0 (2.169)
θ̇(0) = θ̇0 (2.170)
Assume that the pendulum wire is a massless rod, such that −π ≤ θ0 ≤ π.

The total energy (potential + kinetic), which is constant, may be written in
dimensionless form as
1 2 1
(θ̇) − cos θ = (θ̇0 )2 − cos θ0 (2.171)
2 2
We define an energy function E(θ) from (2.171) as
1
E(θ) = [(θ̇)2 − (θ̇0 )2 ] + cos θ0 − cos θ (2.172)
2
We see that this function should be identically equal to zero at all times. But,
when it is computed numerically, it will deviate from zero due to numerical
errors.
Movie 1: mov-ch1/pendulum.mp4
a) Write a Python program that solves the ODE in (2.168) with the specified
initial conditions using Heun’s method, for given values of θ0 and θ̇0 . Set for
instance θ0 = 85o and θ̇0 = 0. (remember to convert use rad in ODE) Experiment
with different time steps ∆t, and carry out the computation for at least a whole
period. Plot both the amplitude and the energy function in (2.172) as functions
of t. Plot in separate figures.
b) Solve a) using Euler’s method.
c) Solve the linearized version of the ODE in (2.168):
θ̈(t) + θ(t) = 0 (2.173)

θ(0) = θ0 , θ̇(0) = 0 (2.174)
using both euler and Heuns method. Plot all four solutions (Problem 2, 3a
and b) in the same figure. Experiment with different timesteps and values of θ0 .
Hint 1. Euler implementations of the linearized version of this problem may be

found in the digital compendium. See http://folk.ntnu.no/leifh/teaching/
tkt4140/._main007.html.
Hint 2. Figure should look something like this:

2.0
EulerLin
1.5
HeunLin
EulerNonlin
1.0
HeunNonlin
Amplitude [rad]
0.5
0.0
−0.5
−1.0
−1.5
−2.0
0 2 4 6 8 10 12 14 16
Dimensionless time [-]
Figure 2.29: Problem 3 with 2500 samplepoints and θ0 = 85o .
Exercise 4: Comparison of 2nd order RK-methods

In this exercise we seek to compare several 2nd order RK-methods as outlined
in 2.10.
a) Derive the Midpoint method a2 = 1 and Ralston’s method a2 = 2/3 from
the generic second order RK-method in (2.86).
b) Implement the Midpoint method a2 = 1 and Ralston’s method a2 = 2/3
and compare their numerical predictions with the predictions of Heun’s method
for a model problem.
Chapter 3
Shooting Methods for

Boundary Value Problems
3.1 Shooting methods for boundary value prob-

lems with linear ODEs
Shooting methods are developed to transform boundary value probelms (BVPs)
for ordinary differential equations to an equivalent initial value problem (IVP).
The term "shooting method" is inspired by the problem illustrated in Figure 3.1,
where the problem is to "shoot" a ballistic object in the field of gravity, aming hit
a target at a given length L. The initial angle α is then changed in an repeated
fasion, based on observed lengths, until the target at L is hit.
y
v0
m
v
α
mg
x
L
Figure 3.1: The trajectory of a ballistic object launched with an inital angle α.
The problem is a boundary value problem with one condition given at x = 0

and another at x = L. It might seems obvious that the problem of varying α
until the condition at x = L is satisfied has a solution. We will use this approach
on problems which has nothing to do with ballistic trajectories.
108
CHAPTER 3. SHOOTING METHODS 109
Let us consider the following example:
y 00 = y(x) (3.1)
which is a second order, linear ODE with initial conditions:
y(0) = 0, y 0 (0) = s (3.2)
and consequently an initial value problem which can be shown to have the
following analytical solution:
y(x) = s · sinh(x) (3.3)

For each choice of the initial value y 0 (0) = s we will get a new solution y(x),
and in Figure 3.2 we see the solutions for s = 0.2 og 0.7.
The problem we really is (3.1) with the following boundary conditions:
y(0) = 0, y(1) = 1 (3.4)
which is what we call a boundary value problem.

From (3.3) we realize that the boundary problem may be solve by selecting
s = s∗ such that y(1) = s∗ · sinh(1) or
1
s∗ = (3.5)
sinh(1)
For this particular case we are able to find the analytical solution of both
the initial problem and the boundary value problem. When this is not the case
our method of approach is a shooting method where we select values of s until
the condition y(1) = 1 is fulfilled. For arbitrary values of s the boundary value
y(1) becomes a function of s, which is linear if the ODE is linear and nonlinear
if the ODE is nonlinear.
To illustrate the shooting method, consider a somewhat general boundary
problem for a second order linear ODE:
y 00 (x) = p(x) · y 0 (x) + q(x) · y(x) + r(x) (3.6)

where p(x), q(x) are arbritary functions and r(x) may be considered as a
source term. The boundary values may be prescribed as:
y(a) = α, y(b) = β (3.7)

The second order ODE (3.6) may be reduced to a system of two ODEs
(see 2.4):
y 0 (x) = g(x)
g 0 (x) = p(x) · g(x) + q(x) · y(x) + r(x) (3.8)
with the boundary conditions given in (3.7).

We start the shooting method by chosing the given boundary value y(a) = α
as an initial condition. As we need an initial condition for y 0 (α) ≡ g(α) to solve
(3.8) as an initial value problem, we have to guess and initial value in a way
such that the boundary value y(b) = β is satisfied. As (3.6) is a linear ODE it
suffices to guess two values of s = g(α) ≡ y 0 (a). The correct value for the initial
value of s which gives the correct value of the solution y(b) = β, may then be
found by linear interpolation. Note that y(x) is always proportional to s when
the ODE is linear.
To quantify the goodness of how well the boundary value resulting from our
initial guess s = g(a) ≡ y 0 (a), we introduce a boundary value error function φ:
φ(s) = y(b; s) − β (3.9)

∗
The correct initial guess s = s is found when the boundary value error
function is zero:
φ(s∗ ) = 0 (3.10)
The procedure may be outlined as follows:
The shooting method for linear boundary value problems.
1. Guess two initial values s0 og s1
2. Compute the corresponding values for the error function φ0 og φ1 by

solving the initial value problem in (3.8)
3. Find the correct initial value s∗ by linear interpolation.
4. Compute the solution to the boundary value problem by solving the
initial value problem in (3.8) with g(a) = y 0 (a) = s∗ .
To see how we find s∗ from lineariztion, consider first φ as a linear fuction of

s:
φ = ka · s + kb (3.11)
where ka is the slope and kb is the offset when s = 0. We easily see from
kb
(3.11) that φ = 0 for s∗ = − . When we have two samples φ0 og φ1 of φ at s0
ka
og s1 , both ka and kb may be found from (3.11) to be:
φ1 − φ0 s1 · φ0 − φ1 · s0
ka = , kb = (3.12)
s1 − s0 s1 − s0
and concequently we may find s∗ as:
φ1 s0 − φ0 s1
s∗ = (3.13)
φ1 − φ0
which also may be presented on an incremental form as:
1
s − s0

s∗ = s1 + ∆s, ∆s = −φ1 · (3.14)
φ1 − φ0
Let us go back to our model boundary value example (3.1) with boundary
values as prescribed in (3.4) which may be presented as reduced system of first
order ODEs:
y 0 (x) = g(x)
(3.15)
g 0 (x) = y(x)
For the given values of b and β and s = y 0 (0) = g(0) the boundary value
error function in (3.9) takes the form:
φ(s) = y(1; s) − 1 (3.16)
In accordance with the shooting method we choose two initial values s0 = 0.2
og s1 = 0.7 and solve (3.15) as an initial value problem with e.g. and RK4-solver
with ∆x = 0.1 and obtain the following results:.
m sm φ(sm )
0 0.2 -0.7650
1 0.7 -0.1774
Note that we use m as a superindex for iterations, which will be used in case of
nonlinear equations. Substitution into (3.13) gives the s∗ = 0.8510. Subsequent
use of the s∗ -value in the RK4-solver yields φ(0.8510) = 0.0001. We observe a
good approximation to the correct value for the initial value may be found from
1
the analytical solution as: y 0 (0) = = 0.8509
sinh(1)
The presentation above with the shooting method is chosen as it can easily
be generalized to the solution of nonlinear ODEs.
Alternative approach for linear second order ODEs. For linear second
order ODEs the solution may be found in somewhat simpler fashion, by solving
the following sub-problems:
Sub-problem 1
y000 (x) = p(x) · y00 (x) + q(x) · y0 (x) + r(x), y0 (a) = α, y00 (a) = 0 (3.17)
and
Sub-problem 2
y100 (x) = p(x) · y10 (x) + q(x) · y1 (x), y1 (a) = 0, y10 (a) = 1 (3.18)
1.0
y(x)
y(x,s1 )
0.8
y(x,s0 )
0.6
y 0.4
0.2
0.0
0.0 0.2 0.4 0.6 0.8 1.0
x
Figure 3.2: The solution y(x) of a boundary value problem resulting from two
initial guesses y(x; s0 ) and y(x; s1 ).
That is, the two sub-problems differ only by the source term r(x) and the
boundary conditions. Notice that the condition y 0 (α) = 0 in (3.17) corresponds
to s0 = 0 and the condition y 0 (α) = 1 i (3.18) corresponds to s1 = 1.
Let y0 (x) represent the solution to (3.17) and y1 (x) be the solution to (3.18).
The complete solution of the boundary value problem in (3.6) with the boundary
conditions (3.7) may then be shown to be:
φ0

β − y0 (b)
y(x) = y0 (x) + · y1 (x) = y0 (x) − 1 · y1 (x) (3.19)
y1 (b) φ +β
with s0 = 0 og s1 = 1. Note that the solution of (3.17) corresponds to a

particular solution of (3.1) and (3.4), and that the solution of (3.18) corresponds
to the homogenous solution of (3.1) and (3.4) as the source term is missing.
From general theory on ODEs we know that the full solution may be found by
the sum of a particular solution and the homogenous solution, given that the
boundary conditions are satisfied.
3.1.1 Example: Couette-Poiseuille flow

In a Couette-Poiseuille flow, we consider fluid flow constrained between two walls,
where the upper wall is moving at a prescribed velocity U0 and at a distance
Y = L from the bottom wall. Additionally, the flow is subjected to a prescribed
∂p
pressure gradient ∂x .
We assume that the vertical velocity component V = 0 and concequently we
∂U
get from continuity: ∂X = 0 which implies that U = U (Y ), i.e. the velocity in
the streamwise X-direction depends on the cross-wise Y-direction only.
y
U0
p(x) U
Figure 3.3: An illustration of Couette flow driven by a pressure gradient and a

moving upper wall.
∂p
The equation of motion in the Y-direction simplifies to ∂Y = −ρg, whereas
the equation of motion in the X-direction has the form:
2
∂2U

∂U ∂p ∂ U
ρU · =− +µ +
∂X ∂X ∂X 2 ∂Y 2
which due to the above assumptions and ramifications reduces to
d2 U 1 dp
= (3.20)
dY 2 µ dX
with no-slip boundary conditions: U (0) = 0, U (L) = U0 . To render equation

(3.20) on a more appropriate and generic form we introduce dimesionless variables:
dp L2
u = UU0 , y = YL , P = − U10 ( dX ) µ , which yields:
d2 u
= −P (3.21)
dy 2
with corresponding boundary conditions:
u = 0 for y = 0, u = 1 for y = 1 (3.22)
An analytical solution of equation (3.21) with the corresponding boundary

conditions (3.22) may be found to be:

P
u = y · 1 + (1 − y) (3.23)
2
Observe that for P ≤ −2 we will get negative velocities for some values of
y. In Figure 3.4 velocity profiles are illustrated for a range of non-dimensional
pressure gradients P.
To solve (3.21) numerically, we represent is as a system of equations:
u0 (y) = u1 (y)
(3.24)
u01 (y) = −P
1.0
P = -8
P = -4
0.8 P = -2
P=0
P=4
0.6
P=8
y 0.4
0.2
0.01.0 0.5 0.0 0.5 1.0 1.5 2.0

u
Figure 3.4: Velocity profiles for Couette-Poiseuille flow with various pressure
gradients.
u(0) = 0, u(1) = 1 (3.25)

To solve this boundary value problem with a shooting method, we must find
s = u0 (0) = u1 (0) such that the boundary condition u(1) = 1 is satisfied. We
can express this condition in Dette kan uttrykkes på følgende måte:
φ(s) = u(1; s) − 1, such that φ(s) = 0 when s = s∗
We guess two values s0 og s1 and compute the correct s by linear interpolation

due to the linearity of system of ODEs (3.24). For the linear interpolation see
equation (3.14).
The shooting method is implemented in the python-code Poiseuille_shoot.py
and results are computed and plotted for a range of non-dimensional pressure
gradients and along with the analytical solution.
# src-ch2/Couette_Poiseuille_shoot.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;
import numpy as np

LNWDT=5; FNT=11
N=200
L = 1.0
y = np.linspace(0,L,N+1)
def f(z, t):

"""RHS for Couette-Posieulle flow"""
zout[:] = [z[1], -dpdx]
return zout
def u_a(y,dpdx):
return y*(1.0 + dpdx*(1.0-y)/2.0);
beta=1.0 # Boundary value at y = L
# Guessed values
s=[1.0, 1.5]
z0=np.zeros(2)
dpdx_list=[-5.0, -2.5, -1.0, 0.0, 1.0,2.5, 5.0]

legends=[]
for dpdx in dpdx_list:

phi = []
for svalue in s:
z0[1] = svalue
z = rk4(f, z0, y)
phi.append(z[-1,0] - beta)
# Compute correct initial guess

s_star = (s[0]*phi[1]-s[1]*phi[0])/(phi[1]-phi[0])
z0[1] = s_star
# Solve the initial value problem which is a solution to the boundary value problem
z = rk4(f, z0, y)
plot(z[:,0],y,’-.’)
legends.append(’rk4: dp=’+str(dpdx))
# Plot the analytical solution
plot(u_a(y, dpdx),y,’:’)
legends.append(’exa: dp=’+str(dpdx))
# Add the labels

legend(legends,loc=’best’,frameon=False) # Add the legends
xlabel(’u/U0’)
ylabel(’y/L’)
show()
3.1.2 Example: Simply supported beam with constant cross-

sectional area
Figure 3.5 illustrates a simply supported beam subjected to an evenly distributed
load q per length unit and a horizontal force P .
The differential equation for the deflection U (X) is given by:
d2 U P q
2
+ U =− (L2 − X 2 ), P > 0 (3.26)
dX EI 2EI
q
A B P
U(X)
X
L L
Figure 3.5: Simply supported beam with an evenly distributed load q per length
unit.
U (−L) = U (L) = 0 (3.27)

where EI denotes the flexural rigidity. The equation (3.26) was linearized by
dU
assuming small deflections. Alternatively we may assume dX (0) = 0 due to the
symmetry.
For convenience we introduce dimensionless variables:
x P 2 P L2
x= , u= · U, θ = (3.28)
L qL2 EI
which by substitution in (3.26) and (3.27) yield:
d2 u (1 − x2 )
2
+ θ2 · u = θ2 , −1 < x < 1 (3.29)
dx 2
with the corresponding boundary conditions:
u(−1) = 0, u(1) = 0 (3.30)

Equation (3.29) with boundary conditions (3.30) have the analytixcal solution:
(1 − x2 )

1 cos(θx)
u(x) = 2 · −1 − (3.31)
θ cos(θ) 2
πEI
The buckling load for this case is given by Pk = such that
4L2
π
0≤θ≤ (3.32)
2
Numerical solution. We wish to solve the second order linear ODE in (3.29)
by means of shooting methods and choose the alternative approach as outlined
in equation (3.19).
Two similar sub-problems are then two be solved; they differ only with the
source term and the boundary conditions.
Sub-problem 1
(1 − x2 )
u000 (x) = −θ2 u0 (x) + θ2 , u0 (−1) = 0, u00 (−1) = 0 (3.33)
2
(3.34)
Sub-problem 2
u001 (x) = −θ2 · u1 (x) u1 (−1) = 0, u01 (−1) = 1 (3.35)

(3.36)
From the superposition principle for linear equation given in equation n (3.19)
a complete solution is obtained as a combination of the solutions to the two
sub-problems:
u0 (1)
u(x) = u0 (x) − · u1 (x) (3.37)
u1 (1)
Now, to solve the equation numerically we write (3.33) and (3.35) as a system
of first order ODEs:
Sub-problem 1 as a system of first order ODEs
For convenience we introduce the notation u0 = y1 and u0o = y2 :
y10 =y2 (3.38)

(1 − x2 )

y20 = − θ2 · y1 +
2
with the corresponding initial conditions
y1 (−1) = 0, y2 (−1) = 0 (3.39)

Sub-problem 2 as a system of first order ODEs
With u1 = y1 og u01 = y2 :
y10 =y2 (3.40)

y20 2
= − θ y1
and initial conditions
y1 (−1) = 0, y2 (−1) = 1 (3.41)

Both sub-problems must be integrated from x = −1 to x = 1 such that u0 (1)
and u1 (1) may be found and the solution to the original problem in equation (3.29)
may be constructed according to (3.37). A python-code beam_deflect_shoot_constant.py
for the problem is listed below. Here the resluting solution is compared with the
analytical solution too.
# src-ch2/beam_deflect_shoot_constant.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py
from numpy import cos, sin
import numpy as np

LNWDT=2; FNT=15; FMLY=’helvetica’;
rcParams[’lines.linewidth’] = LNWDT
rcParams[’font.size’] = FNT
rcParams[’font.family’] = FMLY
def SubProblem1(y, x):

""" system 1 of Governing differential equation on beam bending with constant cross section
Args:
y(array): an array containg y and its derivatives up to second order. (RHS)
x(array): an array
Returns:
yout(array): dydx RHS of reduced system of 1st order equations
"""
yout[:] = [y[1],-theta2*(y[0]+0.5*(1-x**2))]
return yout
def SubProblem2(y, x):

""" system 2 of Governing differential equation on beam bending with constant cross section
Args:
x(array): an array
Returns:
yout(array): dydx RHS of reduced system of 1st order equations
"""
yout[:] = [y[1], -theta2*y[0]]
return yout
# === main program starts here ===
N = 20 # number of elements
L = 1.0 # half the length of the beam
x = np.linspace(-L, L, N + 1) # allocate space
theta = 1 # PL**2/EI
theta2 = theta**2
solverList = [euler, heun, rk4]

solver = solverList[2]
s = [0, 1] # guessed values

# === shoot ===
y0Sys1 = [0, s[0]] # initial values of u and u’
y0Sys2 = [0, s[1]]
u0 = solver(SubProblem1, y0Sys1,x)
u1 = solver(SubProblem2, y0Sys2,x)
u0 = u0[:,0] # extract deflection from solution data

u1 = u1[:,0]
u = u0 -(u0[-1]/u1[-1])*u1 # interpolate to find correct solution

ua = (1/theta2)*(cos(theta*x)/cos(theta) - 1)-(1 - x**2)/2 # analytical solution
legendList=[] # empty list to append legends as plots are generated
plot(x,u,’y’,)
plot(x,ua,’r:’)
legendList.append(’shooting technique’)
legendList.append(’analytical solution’)
## Add the labels
legend(legendList,loc=’best’,frameon=False)
ylabel(’u’)
xlabel(’x’)
grid(b=True, which=’both’, color=’0.65’,linestyle=’-’)
#savefig(’../fig-ch2/beam_deflect_shoot_constant.png’, transparent=True)
#savefig(’../fig-ch2/beam_deflect_shoot_constant.pdf’, transparent=True)
#sh
show()
The output of beam_deflect_shoot_constant.py is shown in Figure 3.6.
0.40
0.35
0.30
0.25
0.20
u
0.15
0.10
0.05 shooting technique
analytical solution
0.001.0 0.5 0.0 0.5 1.0
x
Figure 3.6: The anyalytical and numerical solutions for simply supported beam
with an evenly distributed load q per length unit. The results are produced with
a shooting method implemented in beam_deflect_shoot_constant.py.
3.1.3 Example: Simply supported beam with varying cross-

sectional area
In this example we will study a simply supported beam with varying cross-
sectional area and refer to the figure in our previous example (3.1.2). The
difference with the previous example is that the second area momentis a function
of X due to the varying cross-sectional area.
We denote the second area moment at X = 0 as I0 , and let the second area
moment I vary in the following manner:
I0
I(X) = , n = 2, 4, 6, . . . (3.42)
1 + (X/L)n
Consider a rectangular cross-sectional area with a constant width b and a
varying height h:
h0
h(X) = (3.43)
[1 + (X/L)n ]1/3
where h0 denotes h at X = 0. From equation (3.26) in example (3.1.2) we
have:
d2 U P q
+ U =− (L2 − X 2 ) (3.44)
dX 2 EI 2EI
dU (0)
U (−L) = U (L) = 0, =0 (3.45)
dX
We start by computing the moment distribution M (X) in the beam, which
is related with the displaceent by the following expression:
d2 U M
2
=− (3.46)
dX EI
By substitution of equation (3.46) in (3.44) and two esubsequent differentia-
tions we get:
d2 M P
2
+ @, M = −q (3.47)
dX EI
To represent the resulting second order ODE on a more convenient and
generic form we introduce the dimensionless variables:
X M
x= , m= (3.48)
L qL2
and let
EI0
P = (3.49)
L2
such that equation (3.47) becomes:
m00 (x) + (1 + xn ) · m(x) = −1 (3.50)

with the boundary conditions:
m(−1) = m(1) = 0, m0 (0) = 0 (3.51)

To our knowledge, no analytical solution exists for equation (3.50), even if it
is linear and analytical solutions exists for the previous example in (3.1.2).
Once the moment distribution has been computed, the dimensionless dis-
placement u(x) may retrieved from:
1
u(x) = m(x) − (1 − x2 ) (3.52)
2
The displacement in physical dimensions U may subsequently computed from
the dimensionless u:
qL4
U= u (3.53)
EI0
Further, the differential equaiton for u(x) may be found by substitution of
equation (3.52) into (3.50):
1
u00 (x) + (1 + xn ) · u(x) = − (1 − x2 ) · (1 + xn ) (3.54)
2
By expoliting the symmetry in the problem, we solve equation (3.50) numeri-
cally with a shooting method. The boundary conditions are:
m(1) = 0, m0 (0) = 0 (3.55)
The dimensionless displacement u(x) is subsequently computed from equation
(3.52). We have implemented the outlined approach in beam_deflect_shoot_varying.py
in such a way that one easily may choose between various RK-solvers (euler,
heun, og rk4) from the module ODEschemes.
We represent the second order ODE (3.50) as a system of first order ODEs
by introducing the following conventions m(x) = y1 (x) og m0 (x) = y2 (x):
y10 = y2 (3.56)
y20 n
= − (1 + (1 + x ) · y1 )
and the boundary conditions become:
y2 (0) = y1 (1) = 0 (3.57)
In this case y2 (0) ≡ m0 (0) is given, which leaves m(0) ≡ y1 (0) to be guessed
such that the boundary condition at the other end m(1) ≡ y1 (1) = 0 is fulfilled.
To express the approach algorithmically as previously, we let s = y1 (0) and let
the bondary value error function be φ(s) = y1 (1; s).
Simplified shooting method for linear ODEs
1. Let the two initial values s0 = 0 og s1 = 1 for simplicity

2. Compute the corresponding values for the error function φ0 og φ1 by
solving the initial value problem (in the current example (3.56)).
3. Find the correct initial value s∗ by linear interpolation:
φ0
s∗ = (3.58)
φ0 − φ1
The simplfied expression for s in equation (3.58) may be found from (3.14)
by setting s0 = 0 og s1 = 1.
One should be aware of that the choice in (3.58) is not necessarily always a
good choice even though the ODE is linear. If the solution is of the kind eαx
when both α and x are large, we may end up outside the allowable range even
for double precision which is approximately 10308 . In our example above this is
not a problem and we may use (3.58) and we have implemented the approach
in beam_deflect_shoot_varying.py as shown below where both the moment
and deflection is computed for n = 2.
# src-ch2/beam_deflect_shoot_varying.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

import numpy as np

LNWDT=3; FNT=11
def f(y, x):

"""Governing differential equation on beam bending as a system of 1. order equations.
Args:
x(array): space array
Returns:
dydx(array): dydx. RHS of reduced system of 1st order equations
"""
yout[:] = [y[1],-(1+(1+x**n)*y[0])]
return yout
# === main program starts here ===
N = 10 # number of elements
L = 1.0 # half the length of the beam
x = np.linspace(0,L,N+1) # vector of
theta = 1 # PL**2/EI
theta2 = theta**2
h0=1 # height of beam at x = 0
n=2 # polynomial order to go into h(x)
h=h0/(1+(x/L)**n)**(1/3.) # height of beam assuming constant width
solvers = [euler, heun, rk4]
solver = solvers[2]
s = [0, 1] # guessed values

# === shoot ===
y01 = [s[0], 0] # initial values
y02 = [s[1], 0]
m0 = solver(f, y01, x)
m1 = solver(f, y02, x)
phi0 = m0[-1,0]
phi1 = m1[-1,0]
sfinal = phi0/(phi0 - phi1) # find correct initial value of moment m(x=0) using secant method
y03 = [sfinal, 0]
mfinal = solver(f,y03,x)
m = mfinal[:, 0] # extract moment from solution data
u = m -0.5*(1 - x**2) # final solution of dimensionless deflection

ua = (1/theta2)*(cos(theta*x)/cos(theta)-1)-(1-x**2)/2 #analytical solution with constant stifness
legendList=[] # empty list to append legends as plots are generated
plot(x, m, ’b’,)
legendList.append(’m’)
plot(x, u, ’r’,)
legendList.append(’u’)
## Add the labels
ylabel(’m, u’)
xlabel(’x’)
grid(b=True, which=’both’, axis=’both’,linestyle=’-’)
show()
1.0
m
0.8 u
0.6
m, u
0.4
0.2
0.0
−0.2
0.0 0.2 0.4 0.6 0.8 1.0
x
Figure 3.7: Moment m and deflection u .

3.2 Shooting methods for boundary value prob-

lems with nonlinear ODEs
As model examaple for a non-linear boundary value problems we will investigate:
y 00 (x) = 32 y 2
(3.59)
y(0) = 4, y(1) = 1
Our model problem (3.59) may be proven to have two solutions.
Solution I:
4
yI = (3.60)
(1 + x)2
Solution II: Can be expressed by elliptic Jacobi-functions ( see G.3 in Nu-

meriske Beregninger)
Both solutions are illustrated in Figure 3.8 .
−2
−4
y
−6
−8
−10 yI
yII
−12
0.0 0.2 0.4 0.6 0.8 1.0
x
Figure 3.8: The two solutions yI og yII of the non-linear ODE (3.59).
Our model equation (3.59) may be reduced to a system of ODEs in by

introducing the common notation y → y0 og y 0 = y00 = y1 and following the
procedure in 2.4:
y00 (x) = y1 (x)

0 (3.61)
y1 (x) = 32 [y0 (x)]2
y0 (0) = 4, y0 (1) = 1 (3.62)

We will use the shooting method to find s = y 0 (0) = y1 (0) such that the
boundary condition y0 (1) = 1 is satisfied. Our boundary value error function
becomes:
φ(sm ) = y0 (1; sm ) − 1, m = 0, 1, . . . such that y0 (1) → 1 for φ(sm ) → 0, m → ∞

(3.63)
As our model equation is non-linear our error function φ(s) will also be
non-linear, and we will have to conduct an iteration process as outlined in (3.63).
Our task will then be to find the values s∗ so that our boundary value error
function φ(s) becomes zero φ(s∗ ) = 0. For this purpose we choose the secant
method (See [2], section 3.3).
Two initial guesses s0 and s1 are needed to start the iteration process. Eq.
(3.61) is then solved twice (using s0 and s1 ). From these solutions two values
for the error function (3.63) may be calculated, namely φ0 and φ1 . The next
value of s (s2 ) is then found at the intersection of the gradient (calculated by
the secant method) with the s − axis.
For an arbitrary iteration, illustrated in Figure 3.9 we find it more convenient
to introduce the ∆s:
sm+1 = sm + ∆s (3.64)
where
sm − sm−1

∆s = sm+1 − sm = −φ(sm ) · , m = 1, 2, . . .
φ(sm ) − φ(sm−1 )
ϕ
ϕ(sm-1)
ϕ(sm)
ϕ(sm+1)
sm-1 sm sm+1 s* s
Figure 3.9: An illustration of the usage of the secant method label to find the
zero for a non-linear boundary value error function.
Given that sm−1 og sm may be taken as known quantities the iteration

process process may be outlined as follows:
Iteration process
1. Compute φ(sm−1 ) og φ(sm ) from numerical solution of (3.61) and

(3.63).
2. Compute ∆s and sm+1 from (3.64)
3. Update
• sm−1 ← sm
• sm ← sm+1
• φ(sm−1 ) ← φ(sm )
4. Repeat 1-3 until convergence
Examples on convergence criteria:
Control of absolute error:
|∆s| < ε1 (3.65)
Control of relative error:

∆s
sm+1 < ε2 (3.66)

The crietria (3.65) and (3.66) are frequently used in combination with:
|φ(sm+1 )| < ε3 (3.67)

We will now use this iteration process to solve (3.61). However, a frequent
problem is to find appropriate initial guesses, in particular in when φ is a non-
linear function and may have multiple solutions. A simple way to assess φ is to
make implement a program which plots φ for a wide range of s-values and then
vary this range until the zeros are in the range. An example for such a code is
given below for our specific model problem:
# src-ch2/phi_plot_non_lin_ode.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

import numpy as np

LNWDT=2; FNT=11

font = {’size’ : 16}; rc(’font’, **font)
N=20
L = 1.0
x = np.linspace(0,L,N+1)
def f(z, t):

zout[:] = [z[1],3.0*z[0]**2/2.0]
return zout
beta=1.0 # Boundary value at x = L
solvers = [euler, heun, rk4] #list of solvers

solver=solvers[2] # select specific solver
smin=-45.0
smax=1.0
s_guesses = np.linspace(smin,smax,20)
# Guessed values
#s=[-5.0, 5.0]
z0=np.zeros(2)
z0[0] = 4.0
z = solver(f,z0,x)
phi0 = z[-1,0] - beta
nmax=10
eps = 1.0e-3
phi = []
for s in s_guesses:
z0[1] = s
z = solver(f,z0,x)
legends=[] # empty list to append legends as plots are generated

plot(s_guesses,phi)
ylabel(’phi’)
xlabel(’s’)
show()
close()
Based on our plots of φ we may now provided qualified inital guesses for
and choose s0 = −3.0 og s1 = −6.0. The complete shooting method for the our
boundary value problem may be found here:
# src-ch2/non_lin_ode.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

import numpy as np

LNWDT=3; FNT=16
N=40
L = 1.0
x = np.linspace(0,L,N+1)
def dsfunction(phi0,phi1,s0,s1):
if (abs(phi1-phi0)>0.0):
return -phi1 *(s1 - s0)/float(phi1 - phi0)
else:
return 0.0
def f(z, t):

zout[:] = [z[1],3.0*z[0]**2/2.0]
return zout
def y_analytical(x):
return 4.0/(1.0+x)**2
beta=1.0 # Boundary value at x = L

# Guessed values
# s=[-3.0,-9]
s=[-40.0,-10.0]
z0=np.zeros(2)
z0[0] = 4.0
z0[1] = s[0]
z = solver(f,z0,x)
phi0 = z[-1,0] - beta
nmax=10
eps = 1.0e-3
for n in range(nmax):
z0[1] = s[1]
z = solver(f,z0,x)
phi1 = z[-1,0] - beta
ds = dsfunction(phi0,phi1,s[0],s[1])
s[0] = s[1]
s[1] += ds
phi0 = phi1
print ’n = {} s1 = {} and ds = {}’.format(n,s[1],ds)
if (abs(ds)<=eps):
print ’Solution converged for eps = {} and s1 ={} and ds = {}. \n’.format(eps,s[1],ds)
break

plot(x,z[:,0])
legends.append(’y’)
plot(x,y_analytical(x),’:^’)
legends.append(’y analytical’)
# Add the labels

ylabel(’y’)
xlabel(’x/L’)
show()
After selection ∆x = 0.1 and using the RK4-solver, the following iteration
output may be obtained:
m sm−1 φ(sm−1 ) sm φ(sm ) sm+1 φ(sm+1 )

1 -3.0 26.8131 -6.0 6.2161 -6.9054 2.9395
2 -6.0 6.2161 -6.9054 2.9395 -7.7177 0.6697
3 -6.9054 2.9395 -7.7177 0.6697 -7.9574 0.09875
4 -7.7177 0.6697 -7.9574 0.09875 -7.9989 0.0004
After four iterations s = y 0 (0) = −7.9989 , while the analytical value is −8.0. The
code non_lin_ode.py illustrates how our non-linear boundary value problem
may be solve with a shooting method and offer graphical comparison of the
numerical and analytical solution.
The secant method is simple and efficient and does not require any knowledge
of the analytical expressions for the derivatives for the function for which the
zeros are sought, like e.g. for Newton-Raphsons method. Clearly a drawback is
that two initial guesses are mandatory to start the itertion process. However, by
using some physical insight for the problemn and ploting the φ-function for a
wide range of s-values, the problem is normally possible to deal with.
3.2.1 Example: Large deflection of a cantilever
A
θ y
P δ
ɭ
C B
ɭh
Figure 3.10: Large deflection of a beam subjected to a vertical load.
Consider a cantilever (see figure 3.10) which is anchored at one end A to

a vertical support from which it is protruding. The cantilever is subjected to
a structural load P at the other end B. When subjected to a structural load,

the cantilever carries the load to the support where it is forced against by a
moment and shear stress [8]. In this example we will allow for large deflections
and for that reason we introduce the arc length l and the angle of inclination θ
as variables.
All lengths are rendered dimensionless by division with the cantilever length
L. The arch length l ranges consequently from l = 0 in A to l = 1 in B. Without
further ado we present the differential equation for the elastic cantilever as:
dθ M
κ= = (3.68)
L · dl EI
where κ denotes the curvature, M the moment and EI flexural rigidity. The
balance of moments in C may by computed as C: M = P · L(lh − x) which by
substitution in (3.68) results in:
dθ P L2
= (lh − x) (3.69)
dl EI
From figure 3.10 we may deduce the following geometrical relations:
dx dy
= cos θ, = sin θ (3.70)
dl dl
Further, differentiation of (3.69) with respect to the arch length l and substi-
tution of (3.70) yields:
d2 θ P L2
+ cos θ = 0 (3.71)
dl2 EI
For convenience we introduce the parameter α which is defined as:
P L2
α2 = (3.72)
EI
As a result we must solve the following differential equations:
d2 θ
+ α2 cos θ = 0 (3.73)
dl2
dy
= sin θ (3.74)
dl
dθ
y(0) = 0, θ(0) = 0, (1) = 0 (3.75)
dl
The first two boundary conditions are due the anchoring in A, whereas the
latter is due to a vanishing moment in B. The analytical solution of this problem
may be found in appendix G, section G.3 in Numeriske Beregninger.
Numerical solution
First, we need to represent (3.73) and (3.74) as a system of first order

differntial equations. By introducing the conventions θ = z0 , θ0 = z1 and y = z2
we get:
z00 = z1
z10 = −α2 cos z0 (3.76)
z20 = sin z0
with the boundary conditions
z0 (0) = 0, z1 (1) = 0, z2 (0) = 0 (3.77)

We have to guess the initial value θ0 (0) such that the condition dθ
dl (1) = 0 is
satisfied.
To do so, we let s = θ0 (0) = z1 (0) and φ(s) = θ0 (1; s) − 0 = z1 (1). Conse-
quently, we have to find s = s∗ such that φ(s∗ ) = 0, which with z-variables takes
the form: s = z1 (0).
With the introduction of these convendtions the task at hand becomes to

find s = s∗ such that:
φ(s∗ ) = z1 (1; s∗ ) = 0 (3.78)
Additionally we find:
s∗
lh = (3.79)
α2
Due to the nonlinear nature of (3.76) the function (3.78) will be nonlinear
too and we will use the secant method to find s∗ . To start the secant iterations
we need two intial guesses s0 og s1 . Suitable initial guesses may be found by
first looking at φ(s) graphically. The python-code phi_plot_beam_shoot.py
produces Figure 3.11.
# src-ch2/phi_plot_beam_shoot.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;
import numpy as np

LNWDT=3; FNT=11
N=20
L = 1.0
def f(z, t):

"""RHS for deflection of beam"""
zout[:] = [z[1],-alpha2*cos(z[0]),sin(z[0])]
return zout

alpha2 = 5.0
N_guess = 30
s_guesses=np.linspace(1,5,N_guess)
z0=np.zeros(3)
phi = []
for s_guess in s_guesses:
z0[1] = s_guess
z = solver(f,z0,y)

plot(s_guesses,phi)
# Add the labels

title(’alpha2 = ’ + str(alpha2))
ylabel(’phi’)
xlabel(’s’)
grid(b=True, which=’both’)
show()
0
(s)
−1
−2
−3
−4
1.0 1.5 2.0 2.5 3.0 3.5
s
Figure 3.11: A plot of φ(s) for α2 = 5 for identification of zeros.
From visual inspection of Figure 3.11 we find that φ as a zero at approximately

s∗ ≈ 3.05. Note, that the zeros depend on the values of α.
Given this approximation of the zero we may provide two initial guesses
s0 = 2.5 and s1 = 5.0 which are ’close enough’ for the secant method to
converge. An example for how the solution may be found and presented is given
in beam_deflect_shoot.py.
# src-ch2/beam_deflect_shoot.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;
import numpy as np

LNWDT=3; FNT=11
N=20
L = 1.0
else:
return 0.0
def f(z, t):

"""RHS for deflection of beam"""
zout[:] = [z[1],-alpha2*cos(z[0]),sin(z[0])]
return zout
alpha2 = 5.0

# Guessed values
s=[2.5, 5.0]
z0=np.zeros(3)
z0[1] = s[0]
z = solver(f,z0,y)
phi0 = z[-1,1] - beta
nmax=10
eps = 1.0e-10
z0[1] = s[1]
z = solver(f,z0,y)
phi1 = z[-1,1] - beta
s[0] = s[1]
s[1] += ds
phi0 = phi1

if (abs(ds)<=eps):
break
plot(y,z[:,0])
legends.append(r’$\theta$’)
plot(y,z[:,1])
legends.append(r’$d\theta/dl$’)
plot(y,z[:,2])
legends.append(r’$y$’)
# Add the labels

ylabel(r’$\theta, d\theta/dl, y$’)
xlabel(’y/L’)
#grid(b=True, which=’both’, axis=’both’,linestyle=’-’)
show()
And the resulting output of solutions is illustrated in Figure 3.12
3.5
3.0 d /dl
2.5 y
=5.0
2.0
2
1.5
,d /dl,y for
1.0
0.5
0.0
0.50.0 0.2 0.4 0.6 0.8 1.0
y/L
Figure 3.12: Solutions generated with a shooting method for large deflections
of a cantilever subjected to a point load.
3.3 Notes on similarity solutions

The transient one dimensional heat equation may be represented:
∂T ∂2T
=α (3.80)
∂τ ∂X 2
where τ and X denote time and spatial coordinates, respectively. The
temperature T is a function of time and space T = T (X, τ ), and α is the thermal
diffusivity. (See appendix B in Numeriske Beregninger for a derivation)
X
Ts
T0
Figure 3.13: Beam in the right half-space in one-dimension.
In Figure 3.13 a one-dimensional beam in the right half-space (0 ≤ X < ∞)

is illustrated. The beam has initially a temperature Ts , but at time τ = 0, the
temperature at the left end X = 0 is abruptly set to T0 , and kept constant
thereafter.
We wish to compute the temperature distribution in the beam as a function
of time τ . The partial differential equation describing this problem is given by
equation (3.80), and to model the time evolution of the temperature we provide
the following initial condition:
T (X, τ ) = Ts , τ < 0 (3.81)

along with the boundary conditions which do not change in time:
T (0, τ ) = T0 , T (∞, τ ) = Ts (3.82)

Before we solve the problem numerically, we scale equation (3.80) by the
introduction of the following dimensionless variables:
T − T0 X τ ·α
u= , x= , t= (3.83)
Ts − T0 L L2
where L is a characteristic length. By substitution of the dimensonless

variables in equation (3.83) in (3.80), we get the following:
∂u ∂2u
= , 0<x<∞ (3.84)
∂t ∂x2
accompanied by the dimensionless initial condition:
u(x, t) = 1, t<0 (3.85)

and dimensionless boundary conditions:
u(0, t) = 0, u(∞, t) = 1 (3.86)

The particular choice of the time scale in t (3.83) has been made to make the
thermal diffusivity vanish and to present the governing partial on a canonical
form (3.84) which has many analytical solutions and a wide range of applications.
The dimensionless time in (3.83) is a dimensionless number, which is commonly
referred to as the Fourier-number.
We will now try to transform the partial differential equation (3.84) with
boundary conditions (3.85) to a simpler ordinary differential equations. We will
do so by introducing some appropriate scales for the time and space coordinates:
x̄ = a x and t̄ = b t (3.87)
where a and b are some positive constants. Substitution of equation (3.87)
into equation (3.84) yields the following equation:
∂u a2 ∂ 2 u
= (3.88)
∂ t̄ b ∂ x̄2
We chose b = a2 to bring the scaled equation (3.88) on the canonical,
dimensionless form of equation (3.84) with the boundary conditions:
u(x, t) = u(x̄, t̄) = u(ax, a2 t), with b = a2 (3.89)

For (3.89) to be independent of a > 0, the solution u(x, t) has to be on the
form:
2
x x
u(x, t) = f √ , g , etc. (3.90)
t t
and for convenience we choose the first alternative:

x
u(x, t) = f √ = f (η) (3.91)
t
where we have introduced a new similarity variable η defined as:
x
η= √ (3.92)
2 t
and the factor 2 has been introduced to obtain a simpler end result only. By
introducing the similariy variable η in equation (3.91), we transfor the solution
(and the differential equation) from being a function of x and t, to only depend
on one variable, namely η. A consequence of this transformation is that one
profile u(η), will define the solution for all x and t, i.e. the solutions will be
similar and is denoted a similarity solution for that reason and (3.92) a similarity
transformation.
The Transformation in equation (3.92) is often referred to as the Boltzmann-
transformation.
The original PDE in equation (3.84) has been transformed to an ODE,
which will become clearer from the following. We have introduced the following
variables:
τ ·α x X
t= , η= √ = √ (3.93)
L2 2 t 2 τα
Let us now solve equation (3.84) analytically and introduce:
u = f (η) (3.94)
f (0) = 0, f (∞) = 1 (3.95)

Based on the mathematical representation of the solution in equation (3.94)
we may express the partial derivatives occuring in equation (3.84) as:

∂u ∂u ∂n x η
= = f (η) · − √ = −f 0 (η)
0
∂t ∂η ∂t 4t t 2t
2

∂u ∂u ∂η 0 1 ∂ u ∂ 1 1
= = f (η) √ , 2
= f (η) √ = f 00 (η)
0
∂x ∂η ∂x 2 t ∂x ∂x 2 t 4t
∂u ∂2u
By substituting the expressions for and in equation (3.84) we
∂t ∂x2
transform the original PDE in equation (3.84) to an ODE in equation (3.96) as
stated above:
1 η
f 00 (η) + f 0 (η) = 0
4t 2t
or equivalently:
f 00 (η) + 2ηf 0 (η) = 0 (3.96)

The ODE in equation (3.96) may be solved directely by integration. First,
we rewrite equation (3.96) as:
f 00 (η)
= −2η
f 0 (η)
which may be integrated to yield:

ln f 0 (η) = −η 2 + ln C1 (3.97)
which may be simplified by and exponential transformation on both sides:
2
f 0 (η) = C1 e−η
and integrated once again to yield:
Z η
2
f (η) = C1 e−t dt (3.98)
0
where we have used the boundary condition f (0) = 0 from (3.95).

The integral in (3.98) is related with the error function:
Z x √
2 π
e−t dt = erf(x)
0 2
where the error function erf (x) is defined as:
Z x
2 2
√ e−t dt (3.99)
π 0
Substitution of equation (3.99) in equation (3.98) yields:
√
π
f (η) = C1 erf(η) (3.100)
2
As the error function has the property erf(η) → 1 for η → ∞ we get:
2
C1 = √ , and subsequent substitution of equation (3.100) yields:
π
A similarity solution for the one-dimensional heat equation

x
u(x, t) = erf(η) = erf √ (3.101)
2 t
If we wish to express u(x, t) by means of the original variables we have from

equation (3.83):

T (X, τ ) − T0 X
= erf √ (3.102)
Ts − T0 2 τ ·α
In Figure 3.14 the error function erf(η), which is a solution of the one-
dimensional heat equation (3.84), is plotted along with the complementary error
function erfc(η) defined as:
Z ∞
2 2
erfc(η) = 1 − erf(η) = √ e−t dt (3.103)
π η
1.0
0.8
0.6
erf
erf , erfc
erfc
0.4
0.2
0.00.0 0.5 1.0 1.5 2.0
Figure 3.14: The error function is a similarity solution of the one-dimensional

heat equation expressed by the similarity variable η.
3.3.1 Example: Freezing of a waterpipe

A dry layer of soil at initial temperature 20◦ C, is in a cold period exposed to a
suface temperature of −15◦ C in 30 days and nigths. The question of concern is
how deep the waterpipe must be located in order to avoid that the water in the
pipe starts freezing?
The thermal diffusivity is α = 5 · 10−4 m2 /hour. With reference to (3.81)
we have T0 = −15◦ C and Ts = 20◦ C, and τ = 30 days and nigths = 720 hours,
and water freezes at 0◦ C.
From (3.83):
T − T0 0 − (−15) 3
u= = = = 0.4286
Ts − T0 20 − (−15) 7
Some values for erf(η) are tabulated below :
x erf(η) η erf(η)
0.00 0.00000 1.00 0.84270
0.20 0.22270 1.20 0.91031
0.40 0.42839 1.40 0.95229
0.60 0.60386 1.60 0.97635
0.80 0.74210 1.80 0.98909
From the tabulated values for erf (x) and (3.101) we find erf(η) = 0.4286 →
η ≈ 0.4 which according to (3.93) yields:
√ √
X = 0.4 · 2 τ · α = 0.8 · 720 · 5 · 10−4 = 0.48m
While we have used a constant value for the diffusivity α, in can vary in
the range α ∈ [3 10−4 . . . 10−3 ] m2 /s, and the soil will normaly contain some
humidity.
3.3.2 Example: Stokes’ first problem: flow over a sud-

denly started plate
Analytical solutions of the Navier-Stokes equations may be found only for
situations for which some simplifying assumptions are made for the flow field,
geometry etc. Several solutions are known for laminar flow due to moving
boundaries and these are often used to illustrate the viscous boundary layer
behaviour of such flow regimes.
In this example we illustrate Stokes’ first problem flow over a suddenly started
plate. Consider a quiescent fluid at time τ < 0 resting on a plate parallel to the
X-axis (see 3.15).
Y U=0
U(Y,τ)
U = U0
Figure 3.15: Stokes’ first problem: flow over a suddenly started plate.
At time τ = 0 the plate is accelerated up to a constant velocity U = U0 ,

which allows for a parallel-flow assumption of V = 0, W = 0, for the velocity
components orthogonal to U .
2
∂2U

∂U ∂U ∂U 1 ∂p ∂ U
+U +V =− +ν + (3.104)
∂τ ∂X ∂Y ρ ∂X ∂X 2 ∂Y 2
A more detailed derivation is provided in appendix B in Numeriske Bereg-
ninger). Sufficiently far downstream one may further assume that the flow field
is independent of the streamwise coordinate X too, ie. U = U (Y, τ ), and the
(3.104) reduces to:
∂U ∂2U
=ν , 0<Y <∞ (3.105)
∂τ ∂Y 2
The assumption of a suddenly started plate implies the initial condition:
U (U, 0) = 0 (3.106)
and the assumptions of no slip at the plate and a quiescent fluid far away
from the wall implies the following boundary conditions:
U (0, τ ) = U0 U (∞, τ ) = 0 (3.107)

To present the problem given by (3.105), (3.106), and (3.107) in an even

more convenient manner we introduce the following dimensionless variables:
U Y τ ·ν
u= , y= , t= 2 (3.108)
U0 L L
where U0 , L are characteristic velocity and length, respectively. Further,
substitution of (3.108) in (3.105) yields the following equation:
∂u ∂2u
= , 0<y<∞ (3.109)
∂t ∂y 2
with corresponding initial condition:
u(y, 0) = 0 (3.110)
and boundary conditions
u(0, t) = 1
(3.111)
u(∞, t) = 0
We now realize that the Stokes’ first problems is the same as the on given by
(3.84) and (3.85), save for a substitution of 0 and 1 in the boundary conditions. .
The solution of (3.109) and (3.110) and the appropriate boundary conditions
(3.111) is:

y
u(y, t) = 1 − erf(η) = erfc(η) = erfc √ (3.112)
2 t
which has the following expression in physical variables:

Y
U (Y, τ ) = U0 erfc √ (3.113)
2 τ ·ν
3.3.3 Example: The Blasius equation

The Blasius equation is a nonlinear ODE which may be derived as a simplification
of the Navier-Stokes equations for the particular case of stationary, incompressible
boundary layer flows.
Detailed derivations of the Blasius equation may be found in numerous text in
fluid mechanics concerned with boundary layer flow (appendix C, section C.2 in
Numeriske Beregninger). However, for the sake of completeness we provide a brief
derivation of the Blasius equation in the follwing. A stationary, incompressible
boundary layer flow is governed by a simplifed version of the Navier-Stokes
equations (see equation (C.1.10), appendix C in Numeriske Beregninger).
For this particular flow regime, conservation of mass corresponds to:
∂u ∂v
+ =0 (3.114)
∂x ∂y
U = constant
y
v δ
Figure 3.16: Boundary layer development over a horizontal plate. fig:216
whereas balance of linear momentum is ensured by:
∂u ∂u 1 dp ∂2u
u +v =− +ν 2 (3.115)
∂x ∂y ρ dx ∂y
For the case of a constant free stream velocity U = U0 we may show that
dp
the streamwise pressure gradient dx = 0. (see equation (C.1.8) in appendix C in
Numeriske Beregninger), and the equation eq:23028) reduces to:
∂u ∂u ∂2u
u +v =ν 2 (3.116)
∂x ∂y ∂y
u(0) = 0 (3.117)
u → U0 for y → δ (3.118)
In two dimensions, the mass conservation (3.114) may conveniently be satisfied

by the introduction of a stream function ψ(x, y), defined as:
∂ψ ∂ψ
u= , v=− (3.119)
∂y ∂x
The introduction of a similarity variable η and the non-dimensional stream
function f
r
U0 ψ
η= ·y og f (η) = √ (3.120)
2νx 2Uo νx
we may transform the PDE in equation (3.120) to an ODE, which is the
famous Blasius equation, for f :
f 000 (η) + f (η) · f 00 (η) = 0 (3.121)

df
where we use the conventional notation dη ≡ f 0 (η) and similar for higher
order derivatives.
The physical velocity components may be derived from the stream function
by:
u v η · f 0 (η) − f (η)
= f 0 (η), = √ (3.122)
U0 U0 2Rex
where the Reynolds number has been introduced as
U0 x
Rex = (3.123)
ν
The no slip condition u = 0 for y = 0 will concequently correspond to
f 0 (0) = 0 ad f 0 (η) = Uu0 in equation (3.122).
In abscence of suction and blowing at the boundary for η = 0, the other no
slip condition, i.e v = 0 for η = 0, correponds to f (0) = 0 from equation (3.122).
Further, the condition for the free stream of u → U0 from equation (3.118),
corresponds to f 0 (η) → 1 for η → ∞.
Consequently, the boundary conditions for the boundary layer reduce to:
f (0) = f 0 (0) = 0, f 0 (η∞ ) = 1 (3.124)

The shear stress at the wall may be computed from: Skjærspenningen:
r
00 U0
τxy = µU0 f (η) (3.125)
2νx
Numerical solution
The ODE in equation (3.121) with the boundary conditions in (3.124) repre-
sent a boundary value proble which we intend to solve with a shooting technique.
We start by writing the third order, nonlinear ODE as a set of three, first order
ODEs:
f00 = f1
f10 = f2 (3.126)
f20 = −f0 f2
where f0 = f and the corresponding boundary values are represented as
f0 (0) = f1 (0) = 0 (3.127)

f1 (η∞ ) = 1
As f 00 (0) is unknown we must find the correct value with the shooting method,
00
f (0) = f2 (0) = s. Note that s, which for the Blasius equaiton corresponds to the
wall shear stress, must be chosen such that the boundary condition f1 (η∞ ) = 1
is satisfied. We formulate this condition mathematically by introducing the
boundary value error function φ:
φ(s) = f1 (η∞ ; s) − 1 (3.128)

Vi velger å bruke sekantmetoden til nullpunktsbestemmelsen i

As the system of ODEs in equation (3.126) is nonlinear, we follow in the
procedure as outlined in 3.2, and use the secant method to find the correct initial
value s = s∗ such that the boundary value error function is zero φ(s∗ ) = 0.
Iteration proceduce
The boundary value error function φ in (3.128) is a nonlinear function of s
(due to the nonlinear ODE it is derived from), and therefore, we must itereate
to find the correct value.
Two initial guesses s0 and s1 are needed to start the iteration process, where
the superscript is an iteration counter. Consequently, sm referst to to s-value
after the m-th iteration. In accordance with the generic procedure in (3.64) we
find it more convenient to introduce the ∆s:
sm − sm−1

sm+1 = sm + ∆s, ∆s = −φ(sm ) · , m = 1, 2. . . .
φ(sm ) − φ(sm−1 )
(3.129)
To formulate the iteration procedure we assume two values sm−1 sm to be
know, which intitially correspond to s0 and s1 . The interation procedure the
becomes:
1. Compute φ(sm−1 ) and φ(sm ) by solving (3.126).

2. Compute ∆s og sm+1 from (3.129)
3. Update
• sm−1 ← sm
• sm ← sm+1
• φ(sm−1 ) ← φ(sm )
4. Repeat 1-3 until convergence
For this particular problem, the Blasisus equation, the correct value is s∗ =
0.46960....
Note that the convergence criteria is ∆s < ε, i.e. a test on the absolute value
∆s
of ∆s, whereas a relative metric would be .
s
For some problems it may be difficult to guess appropriate initial guesses s0
og s1 to start the iteration procedure.
A very simple procedure to get an overview of the zeros for a generic function
is to plot it graphically. In Figure 3.17 we have plotted φ(s) for a very wide
range s ∈ [0.05, 5.0].
The computations of φ were performeed with the code phi_plot_blasius_shoot_v2.py,
and by visual inspection we observe that the zero is in the range [0.45, 0.5],
i.e. much more narrow that our initial guess.
Figure 3.17: Exploration of the error function φ as a function of s for the

Blasius equation.
# src-ch2/phi_plot_blasius_shoot_v2.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

# Change some default values to make plots more readable on the screen
LNWDT=3; FNT=20
matplotlib.rcParams[’lines.linewidth’] = LNWDT; matplotlib.rcParams[’font.size’] = FNT
def fblasius(y, x):

"""ODE-system for the Blasius-equation"""
return [y[1],y[2], -y[0]*y[2]]
from numpy import linspace, exp, abs

xmin = 0
xmax = 5.75
N = 50 # no x-values
x = linspace(xmin, xmax, N+1)
# Guessed values
#s=[0.1,0.8]
s_guesses=np.linspace(0.01,5.0)
z0=np.zeros(3)
beta=1.0 #Boundary value for eta=infty
phi = []
for s_guess in s_guesses:
z0[2] = s_guess
u = solver(fblasius, z0, x)
phi.append(u[-1,1] - beta)
plot(s_guesses,phi)
title(’Phi-function for the Blasius equation’)

ylabel(’phi’)
xlabel(’s’)
grid(b=True, which=’both’)
show()
Once approriate values for s0 and s1 have been found we are ready to find
the solution of the Blasius equation with the shooting method as outlined in
blasius_shoot_v2.py.
# src-ch2/blasius_shoot_v2.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

LNWDT=3; FNT=20
def fblasius(y, x):

"""ODE-system for the Blasius-equation"""
return [y[1],y[2], -y[0]*y[2]]
else:
return 0.0

from numpy import linspace, exp, abs

xmin = 0
xmax = 5.750
N = 400 # no x-values
x = linspace(xmin, xmax, N+1)
# Guessed values
s=[0.1,0.8]
z0=np.zeros(3)
z0[2] = s[0]
beta=1.0 #Boundary value for eta=infty
## Compute phi0
phi0 = u[-1,1] - beta
nmax=10
eps = 1.0e-3
z0[2] = s[1]
phi1 = u[-1,1] - beta
s[0] = s[1]
s[1] += ds
phi0 = phi1
if (abs(ds)<=eps):
break
plot(u[:,1],x,u[:,2],x)
xlabel(’u og u\’’)
ylabel(’eta’)
legends=[]
legends.append(’velocity’)
legends.append(’wall shear stress’)
legend(legends,loc=’best’,frameon=False)
title(’Solution of the Blaisus eqn with ’+str(solver.func_name)+’-shoot’)
show()
close() #Close the window opened by show()
6
f'( ) - velocity
5 f''( ) - wall shear stress
0
0.0 0.2 0.4 0.6 0.8 1.0 1.2
Figure 3.18: Solutions of the Blasius equation: dimensionles velocity f 0 (η) and
shear stress f 00 (η).
In Figure 3.18 the dimensionless velocity f 0 (η) and shear stress f 00 (η) are
plotted. The maxial f 00 (η) is inntreffer for η = 0 , which means that the maximal
shear stress is at the wall.
Notice further that f 000 (0) = 0 means f 00 (η) has a vertical tanget at the wall.
3.4 Shooting method for linear ODEs with two

unknown initial conditions
In the follwing we will extend the shoothing methods we we derived in (3.1) to
problems with two initial conditions.
For such problemsn we must guess to initial values r and s with the corre-
sponding boundary value error functions φ og ψ, respectively.
ϕ,Ψ
ϕ = A∙s + B∙r + C
Ψ = D∙s + E∙r + F
r
Ψ=0
(s*,r*)
ϕ=0
s
Figure 3.19: The boundary value error functions for boundary value problems
with two unknown initial conditions.
As the ODE to be solved is linear, we know that the resulting boundary

value error functions also will be linear. As φ and ψ will be functions of the two
initial guesses r and s, the will express two planes which may be represented:
φ=A·s+B·r+C
(3.130)
ψ =D·s+E·r+F
where A, B, C, · · · , F are constants to be determined. The correct values r∗
og s∗ satisfy φ(r∗ , s∗ ) = 0 and ψ(r∗ , s∗ ) = 0. (See Figure 3.19.)
Consequently we have six constants which may be found from the equations;
three equations for φ:
φ0 = A · s0 + B · r0 + C
φ1 = A · s1 + B · r1 + C (3.131)
φ(2) = A · s(2) + B · r(2) + C
and three equations for ψ:
ψ 0 = D · s0 + E · r0 + F
ψ 1 = D · s1 + E · r1 + F (3.132)
(2) (2) (2)
ψ =D·s +E·r +F
where s0 , s1 , s(2) , r0 , r1 og r(2) are initial guesses for the unknown initial
values. For convenience we choose the following pairs of initial guesses:
s0 = 0, r0 = 0
s1 = 0, r1 = 1 (3.133)
(2) (2)
s = 1, r =0
which results in the following expressions for the constants:
A = φ(2) − φ0 , B = φ 1 − φ0 , C = φ0 ,
(3.134)
D = ψ (2) − ψ 0 , E = ψ1 − ψ0 , F = ψ0
The correct values r∗ og s∗ which satisfies φ(r∗ , s∗ ) = 0 and ψ(r∗ , s∗ ) = 0,
i.e. A · s∗ + B · r∗ + C = D · s∗ + E · r∗ + C = 0 may be expressed by the
coefficients:
E·C −B·F A·F −D·C
s∗ = , r∗ = (3.135)
D·B−A·E D·B−A·E
which after substitution of A, B, C, . . . , F results in:
ψ 1 · φ0 − φ1 · ψ 0
s∗ =
(ψ (2) − ψ 0 ) · (φ1 − φ0 ) − (φ(2) − φ0 ) · (ψ 1 − ψ 0 )
(3.136)
(2) 0 (2) 0
φ ·ψ −ψ ·φ
r∗ =
(ψ (2) − ψ 0 ) · (φ1 − φ0 ) − (φ(2) − φ0 ) · (ψ 1 − ψ 0 )
The shooting method solution procedure for a linear ODE boundary value
problem with two initial guesses then becomes:
Shooting method for two unknown initial values
1. Solve the ODE for the three pairs of initial guesses r og s as given in
(3.133)
2. Find the correspondning values of φ and ψ.
3. Compute the correct initial values r∗ og s∗ from equation (3.136)).
We will illustrate the usage of this procedure in example (3.4.1).
3.4.1 Example: Liquid in a cylindrical container

In this example we consider a cylindrical container filled with a liquid to the
height H for two different geometries, namely constant container wall thickness
and with a container with at wall thickness which varies linearly (See Figure 3.20).
We will look at the two geometries separately below.
Constant wall thickness. We denote radial displacement with W . In the

zoomed in part of Figure 3.20, V denotes shear force per unit length while M
denotes moment per unit length.
R
g
t
∇
M X
V W
V H
M
Figure 3.20: A cylindrical container with a liquid of heigth H. The arrows

denote positive direction.
The differential equation for the displacement W is given by:
d4 W H −X
+ B · W = −γ (3.137)
dX 4 D
where:
12(1 − ν 2 ) Et3
B= , D= , γ = ρg
R 2 t2 12(1 − ν 2 )
and ν is Poisson’s ratio, E the E-modulus and ρ the density of the fluid. For
convenience we introduce dimensionless variables:
2
X E t
x= and w= W (3.138)
H γHt R
which after substitution in equation (3.137) results in the following dimen-
sionless, fourth order ODE:
d4 w 3(1 − ν 2 )H 4
+ 4 β 4 w = −4 β 4 (1 − x) where β4 = (3.139)
dx4 R 2 t2
Boundary conditions
To solve the ODE in equation (3.139) we need to supply some boundary
conditions at both ends. At x = 0:
dW
W = 0, and = 0 (fixed beam)
dX
At the other end, for X = H both the moment and the shear force must be
zero, which mathematically corresponds to:
d2 W d3 W
M = −D 2
= 0, V = −D =0
dX dX 3
d2 w
The dimensionless expression for the moment is m(x) = − , while the
dx2
d3 w
dimemsionless shear force has the expression: v(x) = − 3 .
dx
In dimensionless variables the boundary conditions takes the form:
dw
w = 0, =0 for x = 0 (3.140)
dx
d2 w d3 w
= 0, =0 for x = 1 (3.141)
dx2 dx3
For our example we select a container with the following dimensions R = 8.5m,
H = 7.95m, t = 0.35m, γ = 9810N/m3 (water), ν = 0.2, and the E-modulus
E = 2 · 104 MPa. For these values the nondimensional parameter β = 6.0044.
Numerical solution with the shooting method for two unknown
initial values
First we need to reformulate the fourth order ODE in (3.139) as a system of
first order ODEs by following the procedure outlined in 2.4.
By using the following auxilliary variables w = y0 , w0 = y00 = y1 , w00 = y10 =
y2 , w000 = y20 = y3 we may write (3.137) as a system of four, first order ODEs:
y00 = y1
y10 = y2
y20 = y3 (3.142)
y30 4
= −4β (y0 + 1 − x)
y0 (0) = 0, y1 (0) = 0, y2 (1) = 0, y3 (1) = 0 (3.143)

We intend to solve the boundary value problem defined by equations (3.142)
and (3.143) with a shooting method approach, i.e. we will use and intital
value solver and guess the unknown initital values. The intention with the
shooting method is to find the unknown initial values s = w00 (0) = y2 (0) and
r = w000 (0) = y3 (0) such that the boundary values y2 (1) = 0 and y3 (1) = 0.
We introduce the boundary value error function as φ(r, s) = y2 (1; r, s) and
ψ(r, s) = y3 (1; r, s) and the correct values for our intitial guesses r∗ og s∗ are
found when φ(r∗ , s∗ ) = 0.
To find the correct initial guesses s = w00 (0) = y2 (0) and r = w000 (0) we do
the following:
• Solve (3.142) three times, always with y0 (0) and y1 (0) = 0. The values of
r and s for each solution is taken from (3.133).
• Compute the correct values for r∗ and s∗ from (3.136).
Below you find a python implementation of the procedure. Rune the code and
check for yourself if the boundary conditions are satisfied.
# src-ch2/tank1.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

import numpy as np

LNWDT=3; FNT=11
def tank1(y, x):

"""Differential equation for the displacement w in a cylindrical tank with constant wall-thicknes
Args:
y(array): an array containg w and its derivatives up to third order.
x(array): independent variable
Returns:
dydx(array): RHS of the system of first order differential equations
"""
dydx = np.zeros_like(y)
dydx[0] = y[1]
dydx[1] = y[2]
dydx[2] = y[3]
dydx[3] = -4*beta4*(y[0]+1-x)
return dydx
# === main program ===

R = 8.5 # Radius [m]
H = 7.95 # height [m]
t = 0.35 # thickness [m]
ny = 0.2 # poissons number
beta = H*(3*(1 - ny**2)/(R*t)**2)**0.25 # angle
beta4 = beta**4
N = 100
X = 1.0
x = np.linspace(0,X,N + 1)
solverList = [euler, heun, rk4]
solver = solverList[2]
#shoot:
s = np.array([0, 0, 1])
r = np.array([0, 1, 0])
phi = np.zeros(3)
psi = np.zeros(3)
# evaluate the boundary value error functions for the initial guesses in s and r
for k in range(3):
y0 = np.array([0, 0, s[k], r[k]])
y = solver(tank1, y0, x)
phi[k] = y[-1, 2]
psi[k]=y[-1, 3]
# calculate correct r and s
denominator = (psi[2] - psi[0])*(phi[1] - phi[0]) - (phi[2] - phi[0])*(psi[1] - psi[0])
rstar = (phi[2]*psi[0] - psi[2]*phi[0])/denominator
sstar = (psi[1]*phi[0] - phi[1]*psi[0])/denominator
print ’rstar’, rstar, ’sstar’, sstar
# compute the correct solution with the correct initial guesses

y0 = np.array([0, 0, sstar, rstar])
y = solver(tank1, y0, x)
legendList=[]
plot(x,-y[:,3]/beta**2)
plot(x,-y[:,2]/beta)
legendList.append(r’$v(x)/\beta^2$ ’)
legendList.append(r’$m(x)/\beta$ ’)
# Add the labels
ylabel(’v, m’)
xlabel(’x’)
#savefig(’../figs/tank1.pdf’)
show()
Note that the ODE in equation (3.139) may be classified as singular as

the highest order derivative is multiplied with a potentially very small number
ε = 4β1 4 , when rearranged. We observe that ε → 0 as β → ∞ and that the nature
of equation (3.139) will go from an ODE to an algebraic equation. The analytical
solution may be shown to have terms of the kind eβx sin βx og eβx cos βx (see
equation (D.1.6) and (D.1.8), part D.1, appendix D in Numeriske Beregninger.
Varying wall thickness. In the following we will modify the example above
of liquid in a cylindrical container by allowing for a wall thickness which varies
linearly from t0 at the bottom to t1 at the top.
R
g t1
∇
M X W
H
V
V
M
t0
Figure 3.21: A cylindrical container with a varying wall thickenss containing a

liquid of heigth H.
t0 − t1
Based on t0 and t1 , we introduce α = the steepness of the wall
t0
thickness, which allows for the thickness t to be represented as:

X
t= 1−α · t0 , 0 ≤ X ≤ H (3.144)
H
The differential equaiton for the displacement W for this situation is given
by:
d2 d2 W

tW
D +E = −γ(H − X) (3.145)
dX 2 dX 2 R2
Et3
where D = , γ = ρg, with the same meaning as in the previous
12(1 − ν 2 )
part of the example.
Dimensionless form
2
3(1 − ν 2 ) 4

X E t0
x= , w= W, β4 = H (3.146)
H γHt0 R R2 t20
By substitution of (3.146) in (3.145) we get the following nondimensional

ODE:
d2 2

3d w
(1 − αx) + 4β 4 (1 − αx) · w = −4β 4 (1 − x) (3.147)
dx2 dx2
which may be differentiated to yield:
d4 w(x) 6α d5 w(x) 6α2 d2 w(x) 4β 4 4β 4 (1 − x)

4
− 5
+ 2 2
+ 2
w(x) = −
dx (1 − αx) dx (1 − αx) dx (1 − αx) (1 − αx)5
(3.148)
This problem has also an analytical, but more complicated, solution (see
appendix D, part D.2 in Numeriske Beregninger) expressed by means of Kelvin
functions, a particular kind of Bessel functions. However, the numerical solution
remains the same as for the case with the constant wall thickness.
We choose physical parameters as for the previous example:
R = 8.5m, H = 7.95m, t0 = 0.35m, t1 = 0.1m

(3.149)
γ = 9810N/m3 (vann), ν = 0.2, E = 2 · 104 MPa
and α = t0t−t
0
1
= 57 and as previously we get β = 6.0044.
Numerical solution
We proceed as before with w = y0 , w0 = y00 = y1 , w00 = y10 = y2 , w000 = y20 =
y3 , z = 1 − αx and write (3.144) as a system of four first order ODEs:
y00 = y1
y10 = y2
y20 = y3 (3.150)
6α 6α2 4β 4 4β 4 (1 − x)
y30 = y3 − 2 y2 − 2 y0 −
z z z z3
y0 (0) = 0, y1 (0) = 0, y2 (1) = 0, y3 (1) = 0 (3.151)

Only the last equation in the system of ODEs (3.150) differs from the previous
part of the example 3.4.1).
The procedure will be as for 3.4.1, except for the we now are introduced the
additional parameter α.
# src-ch2/tank2.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

import numpy as np

LNWDT=3; FNT=11
def tank2(y, x):

"""Differential equation for the displacement w in a cylindrical tank with linearly varying wall-
Args:
y(array): an array containg w and its derivatives up to third order.
x(array): independent variable
Returns:
dydx(array): RHS of the system of first order differential equations
"""
z = 1-alpha*x
dydx = np.zeros_like(y)
dydx[0] = y[1]
dydx[1] = y[2]
dydx[2] = y[3]
temp = (6*alpha/z)*y[3]- (6*alpha**2/z**2)*y[2]
dydx[3] = temp - 4*beta4*y[0]/z**2 - 4*beta4*(1-x)/z**3
return dydx
R = 8.5 # radius [m]

H = 7.95 # height [m]
t0 = 0.35 # thickness [m]
t1 = 0.1 # thickness [m]
ny = 0.2 # poissons number
beta = H*(3*(1-ny**2)/(R*t0)**2)**0.25
beta4 = beta**4
alpha = (t0-t1)/t0
N = 100
print "beta: ", beta, "alpha", alpha
X = 1.0
x = np.linspace(0,X,N + 1)
solverList = [euler, heun, rk4] #list of solvers
solver = solverList[2] # select specific solver
# shoot:
s = np.array([0, 0, 1])
r = np.array([0, 1, 0])
phi = np.zeros(3)
psi = np.zeros(3)
for k in range(3):
y0 = np.array([0,0,s[k],r[k]])
y = solver(tank2,y0,x)
phi[k] = y[-1, 2]
psi[k]=y[-1, 3]
# calculate correct r and s

denominator = (psi[2] - psi[0])*(phi[1] - phi[0]) - (phi[2] - phi[0])*(psi[1] - psi[0])
rstar = (phi[2]*psi[0] - psi[2]*phi[0])/denominator
sstar = (psi[1]*phi[0] - phi[1]*psi[0])/denominator
print ’rstar’, rstar, ’sstar’, sstar
# compute the correct solution with the correct initial guesses

y0 = np.array([0, 0, sstar, rstar])
y = solver(tank2,y0,x)

plot(x,-y[:,3]/beta**2)
plot(x,-y[:,2]/beta)
legends.append(r’$v(x)/\beta^2$’)
legends.append(r’$m(x)/\beta$’)
# Add the labels

ylabel(’v, m’)
xlabel(’x’)
show()
3.5 Exercises
Exercise 5: Stokes first problem for a non-Newtonian fluid
y
u = U0
u(y,t)
Figure 3.22: Stokes first problem for a non-Newtonian fluid: schematic repre-
sentation of the velocity profile.
A non-Newtonian fluid flows in the plane with constant velocity U0 , paralell

with the x-axis. At time t = 0 we insert a thin plate into the flow, paralell to
the x-axis. See Figure 3.22 (The plate must be concidered as infinitesimal thin,
with infinite extent in th x-directions.) We are interested in finding the velocity
field u(y, t) which developes because of the adherence between the fluid and the
plate. The equation of motion for the problem may be stated as:
∂u ∂τxy
ρ = (3.152)
∂t ∂y
∂u
The relation between the shear stress τxy and the velocitygradient ∂y is given
by:
21
∂u ∂u
τxy = K · (3.153)
∂y ∂y
where K is a positive constant.

We introduce the following transformation which reduces the system to an
ordinary differential equation:
y u
η=C· 2 , f (η) = (3.154)
t5 U0
where C is a positive constant, y ≥ 0 and t ≥ 0. By inserting Eq. (3.153)
into Eq. (3.152) and using (3.154), we get:
p
f 00 (η) + 2 η f 0 (η) = 0 (3.155)
With boundaryconditions:
f (0) = 0, f (δ) = 1 (3.156)
where δ is the boundary-layer thickness.
Movie 2: mov-ch2/stokes.mp4
Pen and paper: The following problems should be done using pen and
paper:
a) We are first gonna solve Eq. (3.155) as an initial-value problem. In this

problem we thus set f 0 (0) = 1.25. Calculate f , and f 0 for η = 0.6 using Euler’s
method. Use ∆η = 0.2
b) Write Eq (3.155) as a system of equations. Explain how you could solve the
problem using shooting technique, when f (0) = 0, f (δ) = 1, and δ is considered
known.
c) During the solution of the system above we find that f 0 (δ) = 0. This
information makes it possible to determine δ as a part of the solution of the
system in b). Explain how.
d) Solve Eq. (3.155) when f1 (0) = f 0 (0) is given. Show that the boundary
condition f (δ) = 1 give f1 (0) = 225
128 = 1.25311 . . .
Programming:
Write a python program that solves the problem defined in b. Use the
information about δ obtained in c.
Hint 1. Pen and paper:

a) Solutions: f (0.6) = 0.732, f 0 (0.6)
hp= 0.988. i
d
c) Perform the differentiation dη f 0 (η) , and integrate Eq. (3.155) once.
q p
Solution: δ = 2 f 0 (0)
Hint 2. Programming:
You may use the template script below:
# src-ch2/stokes.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch2/ODEschemes.py;

import numpy as np

from math import sqrt
from scipy.interpolate import splev, splrep
LNWDT=2; FNT=15
#### a: ####
def func(f, eta):
f0 = f[0]
f1 = f[1]
if f1<0:
# f1 should not be negative
f1=0
df0 = ?
df1 = ?
dF = np.array([df0, df1])
return dF
N = 5
Eta = np.linspace(0, 1, N + 1)
d_eta = Eta[1] - Eta[0]
f_0 = ?
df_0 = ?
F_0 = [f_0, df_0]
F = euler(func, F_0, Eta)
print "f(0.6) = {0}, f’(0.6) = {1}".format(F[N*3/5, 0], F[N*3/5, 1])
#### b/c: ####

N = 100 # using more gridpoints
d_eta = Eta[1] - Eta[0]
f_0 = 0
s0, s1 = 1., 1.1

delta = ?
Eta = np.linspace(0, delta, N + 1)

F_0 = [?, ?]
phi0 = ?
# Solve using shooting technique:

for n in range(4):
F_0 = [?, ?]
delta = ?
Eta = np.linspace(0, delta, N + 1)
phi1 = ?
s = ?
s1, s0 = s, s1
print "n = {0}, s = {1}, ds = {2}, delta = {3} ".format(n, s0, s1 - s0, delta)
# Transform to time-space variables:
C = 1.
U0 = 1
dt = 0.05
t0, tend = dt, 2.
dy = 0.01
y0, yend = 0, 2.
time = np.arange(dt, tend + dt, dt)
Y = np.arange(y0, yend + dy, dy)

Usolutions = np.zeros((len(time), len(Y)))
f = F[:, 0]*U0
tck = splrep(Eta, f) # create interpolation splines
for n, t in enumerate(time):
Eta_n = C*Y/(t**(2./5))
Eta_n = np.where(Eta_n>delta, delta, Eta_n) # if eta>delta set eta to delta
Usolutions[n, :] = splev(Eta_n, tck)

from Visualization import myAnimation
myAnimation(Y, Usolutions, time)
You also need this file in order to run simulation:
# src-ch2/Visualization.py

import numpy as np
from matplotlib import animation
def myAnimation(Y, Usolutions, time):

fig = plt.figure()
ax = plt.axes(xlim=(0, 1.1), ylim=(0, Y[-1]))
lines=[] # list for plot lines for solvers and analytical solutions
legends=[] # list for legends for solvers and analytical solutions
line, = ax.plot([], [])

time_text = ax.text(0.1, 0.95, ’’, transform=ax.transAxes)
dt = time[1]-time[0]
plt.xlabel(’u’)
plt.ylabel(’y’)
# initialization function: plot the background of each frame

def init():
time_text.set_text(’’)
line.set_data([], [])
return lines,
# animation function. This is called sequentially

def animate(i):
time = i*dt
time_text.set_text(’time = %.4f’ % time)
U = Usolutions[i, :]
line.set_data(U, Y)
return lines,
# call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.FuncAnimation(fig, animate, frames=len(time), init_func=init, blit=False)
# Writer = animation.writers[’ffmpeg’]
# writer = Writer(fps=15, metadata=dict(artist=’Me’), bitrate=1800)
# anim.save(’../mov/stokes.mov’, writer=writer)
plt.show()
Exercise 6: The Falkner-Skan equation

Blasius equation is a special case of a more general equation called the Falkner-
Skan equation. The Falkner-Skan Transformation (1930) allows to write the
original equation in a similarity form given by:
f 000 + f f 00 + β · [1 − (f 0 )2 ] = 0. (3.157)
It can be shown that the velocity U (x) is defined as:
U (x) = U0 xm , m = constant. (3.158)

(3.157) is sometimes called the wedge flow equation because parameter β has
the geometric meaning as shown in Figure 3.23:
We see that half the wedge angle is β · π2 . For β = 0, we get a flat plate and
(3.157) becomes Blasius equation.
A detailed derivation can be found in Appendix C.3 in the Numeriske Bereg-
ninger. Here we reproduce some of the derivations for the sake of completeness.
U0
η
π
β
2
π
β
2
Figure 3.23: Schematic representation of the Falkner-Skan equation reference

frame and problem setup.
2m β
Connection between m and β: β = m+1 and m = 2−β . We see that β = 0
results in U (x) = U0 .
Falkner-Skan transformation is given by:
s
U ψ
η= · y, f (η) = p , (3.159)
(2 − β)νx (2 − β)U νx
where ψ is a potential function such that
∂ψ ∂ψ
u= , v=
∂y ∂x
If we choose the same boundary conditions as in the example (3.3.3), we get:
f 000 + f f 00 + β · [1 − (f 0 )2 ] = 0, (3.160)
with boundary conditions:
f (0) = f 0 (0) = 0,
(3.161)
f 0 (η∞ ) = 1.
Chapter 4
Finite difference methods

for ordinary differential
equtions
4.1 Introduction
Consider the following boundary value problem
y 00 (x) = f (x) , for 0 < x < 1 , (4.1)
with boundary conditions
y(0) = α , y(1) = β . (4.2)
Our goal is to solve this problem using finite differences. First we discretize
the independent variable xi = ih, with h = 1/(N + 1). Having defined a grid,
automatically defines our unknowns, i.e. y0 , y1 , . . . yN , yN +1 . From boundary
conditions we know that y0 = α and yN +1 = β, so that the number of true
unknowns is N . Next we approximate y 00 (x) by a centered finite difference
approximation
1
y 00 (xi ) ≈ (yi−1 − 2yi + yi+1 ) , (4.3)
h2
obtaining the following algebraic equations
1
(yi−1 − 2yi + yi+1 ) = f (xi ) (4.4)
h2
for i = 1, 2, . . . , N . It is important to note that the first equation (i = 1) involves
the boundary value y0 = α and that the last equation (i = N ) involves the value
163
CHAPTER 4. FINITE DIFFERENCES FOR ODES 164
yN +1 = β. It can be easily noted that the set of equations (4.4) is as a linear

system of N equations for N unknowns, which can be written as
AY = F , (4.5)
where Y is the vector of unknowns Y = [y1 , y2 , . . . , yN ]T and
f (x1 ) − α/h2
   
−2 1
 1 −2 1   f (x2 ) 
   
 · · ·   · 
1    
A= 2 · · ·  ,F =  ·  (4.6)
h    

 · · · 


 · 

 1 −2 1   f (xN −1 ) 
1 −2 f (xN ) − β/h2
This is a nonsingular tridiagonal system, that can be solved for Y from any
right-hand side F .
4.2 Errors and stability

In what follows we refer to the numerical method introduced in the previous
section. This section is based in [12], the reader is referred to that book for
further details.
In order to discuss about the approximation error introduced by a numerical
scheme, the first step is to define what one means by error. Let us denote Ŷ the
vector of exact solutions at grid points, i.e.
Ŷ = [y(x1 ), y(x2 ), . . . , y(xN )]T (4.7)
and the error function E
E = Y − Ŷ . (4.8)
Once the error has been define we will concentrate in determining how big
this error is. We can do that by using some norm for vector E. For example, we
can use the max-norm
kEk∞ = max |ei | = max |yi − y(xi )| , (4.9)

1≤i≤N 1≤i≤N
the 1-norm
N
X
kEk1 = h |ei | (4.10)
i=1
or the 2-norm
N
!1/2
X
2
kEk2 = h |ei | . (4.11)
i=1
Now our goal is to obtain a bound on the magnitude of the error vector E,
so that, for example, if we can show that kEk∞ = O(h2 ), then it follows that
the error in each point of the grid must be O(h2 ) as well. In order to achive
this goal, we will first define the local truncation error (LTE) of the numerical
scheme and then, via stability, show that the global error can be bounded by
the LTE.
4.2.1 Local truncation error

We have already study the LTE for initial value problems in Section 2.12. The
LTE for the finite difference scheme (4.4) is obtained by replacing yi with the
true solution y(xi ). In general, the exact solution y(xi ) will not satisfy the finite
difference equation. This difference is called LTE and denoted as
1
τi = (y(xi−1 ) − 2y(xi ) + y(xi+1 )) − f (xi ) (4.12)
h2
for i = 1, 2, . . . , N . Since we in practice do not know the exact solution. However,
if we assume that it is smooth, we can then use Taylor series expansions, which
results in
1 2 0000
τi = (y 00 (xi ) + h y (xi ) + O(h4 )) − f (xi ) . (4.13)
12
Furthermore, by using the original differential equation (4.1), the LTE becomes
1 2 0000
τi = h y (xi ) + O(h4 ) . (4.14)
12
Finally, we note that even if y 0000 is in general unknown, it is some fixed function
independent of h, so that τi = O(h2 ) as h → 0.
Finally, we note that we can define a vector form of the LTE as
τ = AŶ − F , (4.15)
and so
AŶ = F + τ . (4.16)
4.2.2 Global error

We obtain a relation between LTE and global error by subtracting (4.16) from
(4.5), obtaining
AE = −τ . (4.17)
This is just a matrix form of the system of equations

1
(ei−1 − 2ei + ei+1 ) = −τi for i = 1, 2, . . . , N (4.18)
h2
with boundary conditions e0 = eN +1 = 0, because we are prescribing the solution

at both ends. We can interpret (4.18) as the discretization of tne ODE
e00 (x) = −τ (x) (4.19)
1 2 0000
with boundary conditions e(0) = 0 and e(1) = 0. Since τ (x) ≈ 12 h y (x),
integrating twice shows that the global error should be roughly
1 2 00 1
e(x) ≈ − h y (x) + h2 (y 00 (0) + x(y 00 (1) − y 00 (0))) (4.20)
12 12
and then the error should be O(h2 ).
4.2.3 Stability
In the above made considerations we have assumed that solving the difference
equations results in a decent approximation to the solution of the underlying
differential equations. But since this is exactly what we are trying to prove, so
that the reasoning is rather circular. Let us pursue a different approach and
look at the discrete system
Ah E h = −τ h , (4.21)
where we have used the superscript h to indicate that these quantities depend
on h. If we define (Ah )−1 as the inverse of Ah , then we can write
E h = −(Ah )−1 τ h (4.22)
and taking the norms gives
kE h k = k(Ah )−1 τ h k (4.23)
≤ k(Ah )−1 kkτ h k. (4.24)
We already know that kτ h k = O(h2 ) and we want to verify that this is also
true for kE h k. Therefore, we need that k(Ah )−1 k is bounded by some constant
independent of h as h → 0, in other words
k(Ah )−1 k ≤ C (4.25)
for all h sufficiently small. If that is true, we will then have that
kE h k ≤ Ckτ h k (4.26)
with the consequence that kE h k goes to zero at least as fast as kτ h k.
4.2.4 Consistency
A method is said to be consistent with the differential equation and boundary
conditions if
kτ h k → 0 as h → 0 . (4.27)
4.2.5 Convergence
We are finally in the position of giving a proper definition of convergence. We
say that a method is convergent if kE h k → 0 as h → 0. Combining the concepts
introduced above, we can say that
consistency + stability =⇒ convergence. (4.28)
Lax equivalence theorem A consistent finite difference method for a

well-posed linear BVP is convergent if and only if it is stable.
4.2.6 Stability - Example in 2-norm

We conclude this section with an example on how to verify the stability of a
finite difference scheme for BVP. We consider the scheme used until now as
example, i.e. (4.5).Since A in (4.5) is symmetric, the 2-norm of A is equal to its
spectral radius
kAk2 = ρ(A) = max |λp | , (4.29)

1≤p≤N
where λp refers to the p-th eigenvalue A. We further note that A−1 is also
symmetric, so that its eigenvalues are the inverses of the eigenvalues of A
−1
kA−1 k2 = ρ(A−1 ) = max |(λp |)−1 = min |λp | . (4.30)
1≤p≤N 1≤p≤N
These considerations tell us that in order to verify the stability of the numerical
scheme under investigation we just have to compute the eigenvalues of A and
show that they are bounded away from zero as h → 0. In general this can be
very trick, since we have an infinite set of matrices A (because A changes with
h). However, in this case the structure is so simple that we can obtain a general
expression for the eigenvalues.
We consider a single value of h = 1/(N + 1). Then the N eigenvalues of A
are
2
λp = (cos(pπh) − 1) , (4.31)
h2
for p = 1, 2, . . . , N . Moreover, the eigenvector y p , corresponding to λp has
components yip for i = 1, 2, . . . , N given by
yip = sin(pπih) . (4.32)
You can verify this by checking that Ay p = λp y p .

It can be easily seen that the smallest eigenvalue of A, in magnitude, is

2
λ1 = (cos(πh) − 1) (4.33)
h2
2 1 1
= 2 (− π 2 h2 + π 4 h4 + O(h6 )) (4.34)
h 2 24
= −π 2 + O(h2 ) . (4.35)
This is clearly bounded away from zero for h → 0, which allows us to conclude
that the scheme is stable in the 2-norm.
4.3 Tridiagonal systems of algebraic equations

When finite difference discretization methods are applied on ordinary of partial
differential equations, the resulting equation systems will normally have a distinct
band structure. In particular, a tridiagonal coefficient matrix will normally be the
result when a second order ODE is solved. The tridiagonal matrix has only three
diagonals with non-zero elements (hence the name); one main diagonal with the
two other on each side of this. Such a tridiagonal system may be represented
mathematically as:
b1 x1 + c1 x2 = d1
···
ai xi−1 + bi xi + ci xi+1 = di
(4.36)
···
aN xN −1 + bN xN = dN
i = 1, 2, ...N , a1 = cN = 0
or more convenient in a matrix representation:
     
b1 c1 x1 d1
a2 b2 c2   x2
   d2
     

 · · ·  
  ·
 
 

 ·

 · · · ·
  ·
=
 

 · (4.37)

 · · ·  
  ·
 
 

 ·
 aN −1 bN −1 cN −1  xN −1  dN −1 
aN bN xN dN
The linear algebraic equation system (4.36) may be solved by Gauss-elimination,
which has a particular simple form for tridiagonal matrices and is often referred
to as the Thomas-algorithm. More details on the derivation may be found in
appendix I of Numeriske beregninger (in Norwegian).
The Thomas-algorithm has two steps; an elimination step and a substitution
step. In the elimination step, sub-diagonal elements are eliminated by summation
of two consecutive appropriate scaled rows in the matrix, starting at the first
row at the top. By completion of the elimination step, the last unknown xN
may the be computed as xN := dbN N
.
In the equation for xN −1 immediately above the last, there are only two
unknows, namely xN and xN −1 . But since we already have calculated xN in
the elimination step, we substitute its value in this equation and may thereby
find xN −1 . By moving on to the equation for xN −2 , there will be only two
unknowns as well, xN −2 and xN −1 , and since we have already calculated xN −1 ,
it may be substituted into this equation such that xN −2 may be found. This
procedure of back-substitution may be repeated until all the unknows for the
tridiagolnal equation system is known. This step of the procedure is known as
back substitution or simply substitution.
The algorithms for the two steps are listed outlined below in mathematical
terms:
Elimination:
aj
qj := for j = 2, 3 · · · , N
bj−1
bj := bj − qj · cj−1 (4.38)
dj := dj − qj · dj−1
Back substitution:
dN
xN :=
bN
dj − cj · xj+1
xj := for j = N − 1, N − 2, · · · , 1 (4.39)
bj
(4.40)
The Python-code for (4.38) and (4.39) is implemented in tdma, which

corresponds to tri in [2] , section 6.3 (see below).
def tdma(a, b, c, d):
"""Solution of a linear system of algebraic equations with a
tri-diagonal matrix of coefficients using the Thomas-algorithm.
Args:
a(array): an array containing lower diagonal (a[0] is not used)
b(array): an array containing main diagonal
c(array): an array containing lower diagonal (c[-1] is not used)
d(array): right hand side of the system
Returns:
x(array): solution array of the system
"""
n = len(b)
x = np.zeros(n)
# elimination:
for k in range(1,n):
q = a[k]/b[k-1]
b[k] = b[k] - c[k-1]*q
d[k] = d[k] - d[k-1]*q
# backsubstitution:
q = d[n-1]/b[n-1]
x[n-1] = q
for k in range(n-2,-1,-1):
q = (d[k]-c[k]*q)/b[k]
x[k] = q
return x
Stability of (4.38) and (4.39) is satisfied subject to the following conditions:
| b1 |>| c1 |> 0
| bi |≥| ai | + | ci |, ai · ci 6= 0 , i = 2, 3, · · · , N − 1 (4.41)
| bn |>| aN |> 0
Matrices satisfying (4.41) are called diagonally dominant for obvious reasons,
and strictly diagonally in case ≥ may be substitued with > in (4.41). Pivoting
during Gauss-elimination is not necessary for diagonally dominant matrices,
and thus the band structure is preserved and the algorithm becomes less CPU-
intensive. See appendix I in Numeriske beregninger for a proof of (4.41).
Notice that all coefficients in each row of (4.37) has the same index. However,
the linear algebraic equation system (4.36) may also be presented:
b1 x1 + c2 x2 = d1
···
ai−1 xi−1 + bi xi + ci+1 xi+1 = di
··· (4.42)
aN −1 xN −1 + bN xN = dN
i = 1, 2, ...N , a1 = cN = 0
or in matrix form:
     
b1 c2 x1 d1
a1 b2 c3   x2
   d2
     

 · · ·  
  ·
 
 

 ·

 · · · ·
  ·
=
 

 · (4.43)

 · · ·  
  ·
 
 

 ·
 aN −2 bN −1 cN  xN −1  dN −1 
aN −1 bN xN dN
We notice that in this notation the coefficients of each column in the matrix
(4.43) has the same index.
The version in (4.43) may be deduced from (4.37) by subtraction of 1 from

the a-inddices and addition of 1 for the c-indices. Commercial codes like Matlab
store tridiagonal matrices on the form given in (4.43). We have implemented
(4.43) in tridiag.
def tripiv(a, b, c, d):
"""Solution of a linear system of algebraic equations with a
tri-diagonal matrix of coefficients using the Thomas-algorithm with pivoting.
Args:
a(array): an array containing lower diagonal (a[0] is not used)
b(array): an array containing main diagonal
c(array): an array containing lower diagonal (c[-1] is not used)
d(array): right hand side of the system
Returns:
x(array): solution array of the system
"""
n = len(b)
x = np.zeros(n)
fail = 0
# reordering
a[0] = b[0]
b[0] = c[0]
c[0] = 0
# elimination:
l = 0
for k in range(0,n):
q = a[k]
i = k
if l < n-1:
l = l + 1
for j in range(k+1,l+1):
q1 = a[j]
if (np.abs(q1) > np.abs(q)):
q = q1
i = j
if q == 0:
fail = -1
if i != k:
q = d[k]
d[k] = d[i]
d[i] = q
q = a[k]
a[k] = a[i]
a[i] = q
q = b[k]
b[k] = b[i]
b[i] = q
q = c[k]
c[k] =c[i]
c[i] = q
for i in range(k+1,l+1):
q = a[i]/a[k]
d[i] = d[i]-q*d[k]
a[i] = b[i]-q*b[k]
b[i] = c[i]-q*c[k]
c[i] = 0
# backsubstitution
x[n-1] = d[n-1]/a[n-1]
x[n-2] = (d[n-2]-b[n-2]*x[n-1])/a[n-2]
for i in range(n-3,-1,-1):
q = d[i] - b[i]*x[i+1]
x[i] = (q - c[i]*x[i+2])/a[i]
return x
emfs /src-ch3/ #python #package TRIdiagonalSolvers.py @ git@lrhgit/tkt4140/src/src-ch3/TRIdiagonalSol
4.4 Examples
4.4.1 Example: Heat exchanger with constant cross-section
T∞ D
X
Tr
Figure 4.1: Schematic illustration of a heat exchanger.
A circular cylindrical rod of length L, environmental temperature T∞ , and

a constant temperature Tr at X = L is illustrated in Figure 4.1. The physical
paramters relevant for the heat conduction are the heat transfer coefficient
h̄ = 100W/m2 ◦ C and the thermal conductivity k = 200W/m/◦ C.
From equation (B.0.22), appendix B in Numeriske Beregninger we find that
the governing equation for the heat transfer in the circular cylindrical rod in
Figure 4.1 is governed by:

d d(T − T∞ h̄P
A(X) = (T − T∞ ) (4.44)
dX dX k
which for rod with constant cross-section, i.e. A(x) = A reduces to:
d2 h̄P
2
(T − T∞ ) = (T − T∞ ) (4.45)
dX kA
For convenience we introduce dimensonless variables:
X T − T∞ h̄P 2
x= , θ= , β2 = L (4.46)
L Tr − T∞ kA
where L is a characteristic length (see Figure B.5 i Numeriske Beregninger).
A relative, dimensionless temperature may be constructed by using a reference
temperature Tr and temperature and the temperature T∞ in combination. The
parameteren β 2 is frequently referred to as the Biot-number, Bi = β 2 .
Given the assumptions and prerequesites above (4.44) may be presented in
the more convenient and generic form:
d2 θ
− β2θ = 0 (4.47)
dx2
which is a second order ODE with the following generic analytical solution:
θ(x) = A sinh(βx) = −B cosh(βx) (4.48)

where the constants A and B must be determined from the boundary cond-
tions.
In the following we will investigate the numerical solution to two different
boundary conditions commonly denoted as prescribed boundary conditions (which
for this particular corresponds to prescribed temperatures) and mixed boundary
conditions.
Prescribed temperatures:
T = T∞ for X = 0
T = Tr for X = L
which may be presented on dimensionless form as:
θ=0 for x = 0 (4.49)

θ=1 for x = 1
The analytical solution (4.48) for these particular boundary conditions reduces
to:
sinh(βx) dθ cosh(βx)
θ(x) = , =β (4.50)
sinh(β) dx sinh(β)
With D = 0.02m and L = 0.2m the Biot-number β 2 = 4. By doubling the
length the Biot-number quadurples. For eksemplet ovenfor blir Biot-tallet:
h̄P 2 2 2
β2 = L = L (4.51)
kA D
Mixed boundary conditions:
dT
Qx = 0 = for X = 0
dX
T = Tr for X = L
A zero temperature gradient at X = 0 corresponds to an isolated rod at that

location. The dimensionless representation of this boundary conditions is:
dθ
=0 for x = 0 (4.52)
dx
θ=1 for x = 1 (4.53)
The analytical solution (4.48) reduces for the mixed boundary conditions to:
cosh(βx) dθ sinh(βx)
θ(x) = , =β (4.54)
cosh(β) dx cosh(β)
d2 θ
Numerical solution. We will use central differences for the term and
dx2
2

d θ θi−1 − 2θi + θi+1
with ≈ we get the following difference equation:
dx2 i h2
θi−1 − (2 + β 2 h2 ) θi + θi+1 = 0 (4.55)
Prescribed temperatures
We enumerate the nodes for the unkowns as shown in Figure 4.2:
0.0 1.0
X
0 1 2 3 i N-1 N N+1
Figure 4.2: Enumeration of nodes for prescribed boundary temperatures.
The x-coordinates can be denoted in a compact manner by:: xi = i h ,

i = 0, 1, ..., N + 1 where h = N1+1 , i.e. the prescribed temperatures at the
boundaries are denoted θ0 and θN +1 , respectively.
A good practice is to apply the generic scheme (4.55) for nodes in the
immediate vincinity of the boundaries, and first we take a look at i = 1
θ0 − (2 + β 2 h2 ) θ1 + θ2 = 0 → (2 + β 2 h2 ) θ1 + θ2 = 0
which may be simplified by substitution of the prescribed boundary value
θ(0) = θ0 = 0
(2 + β 2 h2 ) θ1 + θ2 = 0
For the other boundary at i = N the generic scheme (4.55) is:
θN −1 − (2 + β 2 h2 ) θN + θN +1 = 0
which by substitution of the prescribed value for θN +1 = 1 yields:
θN −1 − (2 + β 2 h2 ) θN = −1
A complete system of equations may finally be obtained from (4.4.1), (4.55),
and (4.4.1):
i = 1 : − (2 + β 2 h2 ) θ1 + θ2 = 0
i = 2, 3, ..., N − 1 : θi−1 − (2 + β 2 h2 ) θi + θi+1 = 0 (4.56)
2 2
i = N : θN −1 − (2 + β h ) θN = −1
See appendix A, section A.5, example A.15 in Numeriske Beregninger, this

example is treated in more detail. The following system of coefficents may be
obtained by comparison with (4.36):
ai = 1 , i = 2, 3, ..., N
2 2
bi = −(2 + β h ) , i = 1, 2, ..., N (4.57)
ci = 1 , i = 1, 2, ..., N − 1
di = 0 , i = 1, 2, ..., N − 1
dN = −1
In the program ribbe1.py below the linear, tri-diagonal equation system

(4.57) resulting from the discretization in (4.55) is solved by using two SciPy mod-
ules, the generic scipy.linalg.solve and the computationally more efficient
scipy.sparse.linalg.spsolve.
SciPy is built using the optimized ATLAS LAPACK and BLAS libraries and
has has very fast linear algebra capabilities. Allegedly, all the raw lapack and
blas libraries are available for even greater speed. However, the sparse linear
algebra module of scipy offers easy-to-use python interfaces to these routines.
Numpy has also a linalg-module, however, an advantage of using scipy.linalg over
numpy.linalg is that it is always compiled with BLAS/LAPACK support, while
this is optional for numpy. Note that the scipy.linalg algorithms do not exploit
the sparse nature of the matrix as they use direct solvers for dense matrices.
However, SciPy offers a sparse matrix package scipy.sparse. The spdiags
function may be used to construct a sparse matrix from diagonals. Note that all
the diagonals must have the same length as the dimension of their sparse matrix
- consequently some elements of the diagonals are not used. The first k elements
are not used of the k super-diagonal, whereas the last k elements are not used
of the −k sub-diagonal. For a quick tutorial of the usage of these sparse solvers
see SciPy sparse examples.
We have implemented a simple means to compare the two solution procedures
with a tic-toc statements. You may experiment with various problems sizes
(by varying the element size h) to assess the impact on the computational speed
of the two procedures. The analytical solution is given by (4.50).
# src-ch3/section321/ribbe1.py
import numpy as np
import scipy as sc
import scipy.linalg
import scipy.sparse
import scipy.sparse.linalg
import time
from math import sinh
#import matplotlib.pyplot as plt

LNWDT=3; FNT=20
# Set simulation parameters
beta = 5.0
h = 0.001 # element size
L =1.0 # length of domain
n = int(round(L/h)) -1 # number of unknowns, assuming known boundary values
x=np.arange(n+2)*h # x includes min and max at boundaries were bc are imposed.
#Define useful functions
def tri_diag_setup(a, b, c, k1=-1, k2=0, k3=1):

return np.diag(a, k1) + np.diag(b, k2) + np.diag(c, k3)
def theta_analytical(beta,x):
return np.sinh(beta*x)/np.sinh(beta)
#Create matrix for linalg solver

a=np.ones(n-1)
b=-np.ones(n)*(2+(beta*h)**2)
c=a
A=tri_diag_setup(a,b,c)
#Create matrix for sparse solver
diagonals=np.zeros((3,n))
diagonals[0,:]= 1 #all elts in first row is set to 1
diagonals[1,:]= -(2+(beta*h)**2)
diagonals[2,:]= 1
A_sparse = sc.sparse.spdiags(diagonals, [-1,0,1], n, n,format=’csc’) #sparse matrix instance
#Crete rhs array

d=np.zeros(n)
d[n-1]=-1
#Solve linear problems

tic=time.clock()
theta = sc.sparse.linalg.spsolve(A_sparse,d) #theta=sc.linalg.solve_triangular(A,d)

toc=time.clock()
print ’sparse solver time:’,toc-tic
tic=time.clock()
theta2=sc.linalg.solve(A,d,)
toc=time.clock()
print ’linalg solver time:’,toc-tic
# Plot solutions
plot(x[1:-1],theta,x[1:-1],theta2,’-.’,x,theta_analytical(beta,x),’:’)
legend([’sparse’,’linalg’,’analytical’])
show()
close()
print ’done’
The numerical predictions are presented and compared with analytical values
in the table below:
x numerical analytical rel.err
0.0 0.00000 0.00000
0.1 0.05561 0.05551 0.00176
0.2 0.11344 0.11325 0.00170
0.3 0.17582 0.17554 0.00159
0.4 0.24522 0.24487 0.00144
0.5 0.32444 0.32403 0.00126
0.6 0.41663 0.41619 0.00105
0.7 0.52548 0.52506 0.00082
0.8 0.65536 0.65499 0.00056
0.9 0.81145 0.81122 0.00029
1.0 1.00000 1.00000 0.00000
Mixed boundary conditions. Version 1

For mixed boundary conditions the nodes with uknown temperatures are
enumerated as shown in Fig. (4.3), to make the first unknown temperature we
compute have index 1. The x-coordinates are given by: xi = (i − 1) h, i =
1, 2, ..., N + 1 der h = N1 and we wil use second order central differences as
dθ
an approximation of the zero-gradient boundary condition dx = 0 at the left
boundary where x = 0.
0.0 1.0
X
0 1 2 i N-2 N-1 N
Figure 4.3: Enumeration of nodes for mixed boundary conditions.

For a generic node ’i’ the central difference approximation may be denoted:

dθ θi+1 − θi−1
≈ (4.58)
dx i 2h
which for node 1 takes the form:

θ2 − θ0
=0 (4.59)
2h
and we observe that this strategy requires a value of the temperature θ0 at a
node 0 which is not included in Figure 4.3. However, this is not a problem since
θ0 may be eliminated using the identity obtained from (4.59):
θ0 = θ2 (4.60)
Due to the zero gradient boundary condition, the first of the N equations,
represented on generic form by (4.55), takes the particular form:
−(2 + β 2 h2 ) θ1 + 2θ2 = 0 (4.61)
This first equation (4.61) is the only equation which differs from the resulting
equation system for prescibed boundary conditions in (4.56). All the coefficients
ai , bi , and di are the same as in for the prescribed temperature version in (4.57),
except for c1 :
c1 = 2, ci = 1, i = 2, ...N − 1 (4.62)

An alternative version 2 for implementation of the zero-gradient boundary
condition may be obtained by using a forward approximation for the gradient as
given by (4.55) for a generic node i:

dθ −3θi + 4θi+1 − θi+2
≈ (4.63)
dx i 2h
which takes the following form for node 1 where is should evaluate to zero:

dθ −3θ1 + 4θ2 − θ3
≈ =0 (4.64)
dx 1 2h
From equation (4.64) we see that θ3 may be eliminated by:
θ3 = 4θ2 − 3θ1 (4.65)
The first difference equation (4.55) in which θ3 occurs, is the one for node 2
θ1 − (2 + β 2 h2 ) θ2 + θ3 = 0 (4.66)
and we may eliminate θ3 from equation (4.66) by substitution of (4.65):
2 θ1 − (2 + β 2 h2 ) θ2 = 0 (4.67)
This is the first equation in the system of equations and the only one which
differs from (4.56). Rather than (4.57), we get the following equations for the
coefficients:
b1 = 2 bi = (2 + β 2 h2 ) i = 2, ..., N (4.68)
2 2
c1 = −(2 − β h ) ci = 1 i = 2, ..., N − 1 (4.69)
For convenience we summarise the resulting system of equations by:
2 θ1 − (2 − β 2 h2 ) θ2 = 0, i=1
2 2
θi−1 − (2 + β h ) θi + θi+1 = 0, i = 2, 3, ..., N − 1 (4.70)
2 2
θN −1 − (2 + β h ) θN = −1, i=N
Note that we in this case, with a forward difference apporoximation of a

gradient boundary condition, had to combine the approximation of the gradient
with the difference equation to get a resulting tri-diagonal system. The the
program ribbe2 we solve the mixed boundary conditions with both Version 1
and Version 2.
# src-ch3/section321/ribbe2.py
import numpy as np
import scipy as sc
import scipy.linalg
import scipy.sparse
import time
from numpy import cosh

LNWDT=3; FNT=20

beta = 3.0
h = 0.001 # element size
L =1.0 # length of domain
n = int(round(L/h)) # # of unknowns, assuming known bndry values at outlet
x=np.arange(n+1)*h # x includes min and max at boundaries were bc are imposed.
#Define useful functions
def tri_diag_setup(a, b, c, k1=-1, k2=0, k3=1):

return np.diag(a, k1) + np.diag(b, k2) + np.diag(c, k3)
def theta_analytical(beta,x):
return np.cosh(beta*x)/np.cosh(beta)
#Create matrix for linalg solver

a=np.ones(n-1) # sub-diagonal
b=-np.ones(n)*(2+(beta*h)**2) # diagonal
c=np.ones(n-1) # sub-diagonal
#c=a.copy() # super-diagl, copy as elts are modified later
#c=a
# particular diagonal values due to derivative bc
version=2
if (version==1):
c[0]=2.0
else:
b[0]=2.0
c[0]=-(2-(beta*h)**2)
print ’version 2’
A=tri_diag_setup(a,b,c)
#Create matrix for sparse solver
diagonals=np.zeros((3,n))
diagonals[0,:]= 1.0 # all elts in first row is set to 1
diagonals[0,0]= 1.0
diagonals[1,:]= -(2+(beta*h)**2)
diagonals[2,:]=1.0
if (version==1):
diagonals[2,1]= 2.0 # index 1 as the superdiagonal of spdiags is not used,
else:
diagonals[1,0]=2.0 # Sets the first element in the main diagonal
diagonals[2,1]= -(2+(beta*h)**2) # index 1 as the superdiagonal of spdiags is not used,
super-diagonal
A_sparse = sc.sparse.spdiags(diagonals, [-1,0,1], n, n,format=’csc’) #sparse matrix instance

#Crete rhs array
d=np.zeros(n)
d[-1]=-1
#Solve linear problems

tic=time.clock()
theta = sc.sparse.linalg.spsolve(A_sparse,d) #theta=sc.linalg.solve_triangular(A,d)
toc=time.clock()
tic=time.clock()
theta2=sc.linalg.solve(A,d,)
toc=time.clock()
# Plot solutions
plot(x[0:-1],theta,x[0:-1],theta2,’-.’,x,theta_analytical(beta,x),’:’)
xlabel(’x’)
ylabel(r’Dimensionless temperature $\mathregular{\theta}$’)
legend([’sparse’,’linalg’,’analytical’])
show()
close()
print ’done’
The relative error is computed from εrel = |(θnum − θanalyt )/θanalyt |, and
the results of the computations are given in the table below:
x Anlytical Ctr.diff Rel.err Fwd.diff Rel. err

0.0 0.26580 0.26665 0.00320 0.26613 0.00124
0.1 0.27114 0.27199 0.00314 0.27156 0.00158
0.2 0.28735 0.28820 0.00295 0.28786 0.00176
0.3 0.31510 0.31594 0.00267 0.31567 0.00180
0.4 0.35549 0.35632 0.00232 0.35610 0.00171
0.5 0.41015 0.41095 0.00194 0.41078 0.00153
0.6 0.48128 0.48202 0.00154 0.48189 0.00128
0.7 0.57171 0.57237 0.00114 0.57228 0.00098
0.8 0.68510 0.68561 0.00075 0.68555 0.00067
0.9 0.82597 0.82628 0.00037 0.82625 0.00034
1.0 1.00000 1.00000 0.00000 1.00000 0.00000
We observe that the two versions for zero-gradient boundary condition yields
approximately the same result except close to x = 0, where the forward difference
is somewhat better. In section (6.3.1) we will take a closer look at the accuracy
of gradient boundary conditions.
Rather than the enumeration in Figure (4.3), we may use the enumeration in
Fig. (4.2) with xi = i h , i = 0, 1, ..., N + 1 where h = N1+1 such that we must
take i = 1 in (4.55) to get:
θ0 − (2 + β 2 h2 ) θ1 + θ2 = 0
The boundary condition in (4.63) the becomes:

dθ −3θ0 + 4θ1 − θ2
= =0
dx 0 2h
from which an elimination equation for θ0 may be obtained:
θ0 = 4(θ1 − θ2 )/3 (4.71)

Concequently, equation (4.71) may be used to eliminate θ0 from the first difference
equation:
−(2 + 3β 2 h2 ) θ1 + 2θ2 = 0 (4.72)

With approach θ0 is not solved with the equation system, but retrieved
subsequently from equation (4.71).
4.4.2 Cooling rib with variable cross-section

In Figure 4.4 trapezoidal cooling rib with length L and width b is illustrated.
The thickness vaires from d at X = 0 to D at X = L. The cooling rib has a
D
A(X)
b
L
Figure 4.4: A cooling rib of length L, width b and a varying cross-sectional

area A(X) (trapezoidal).
heat loss at X = 0 and a prescribed temperature TL at XL , with a constant

surounding temperature T∞ .
In Figure 4.5 the upper half of the cooling rib is illustrated for a better
quantitative demonstration of the geometry, where d and D denote the diameters
of the cooling rib at the inlet and outlet, respectively, whereas L denotes the
length of the rib.
T = TL
X
½D
½L
ϕ ½d
X
X=0 X=L
Figure 4.5: A cooling rib with a trapezoidal cross-section.
The following geometrical relations may be deducted:
D/2 − d/2 d∗ /2 − d/2

tan(φ) = =
L X
Further, the spatial dependent diameter d∗ may be isolated and presented as:

D−d
d∗ (X) = d + X (4.73)
L
We assume that the temperature mostly varies in the X-direction such that the
derivation in appendix B in Numeriske metoder is valid. We assume accordingly
d
that D L and that 0 < D <1.
In Numeriske metoder the quasi one-dimensional differential equation for
heat conduction is derived, and for stationary conditions is reduces to equation
(B.0.22):

d d(T − T∞ ) h̄P (X)
A(X) = (T − T∞ ) (4.74)
dX dX k
dT (0) h̄0
= [T (0) − T∞ ] , T (L) = TL (4.75)
dX k
For the trapezoidal cross-section in Fig. (4.4):
P (X) = 2(b + d∗ ) ≈ 2b for d∗ b andA(X) = b d∗

For convenience we introduce dimensionless quantities for temperature θ =
T − T∞ X d
, length x = , and the diameter ratio α = , 0 < α < 1 and the
TL − T∞ L D
Biot numbers:
2h̄ L2 h̄0
β2 = and β02 = L (4.76)
Dk k
Equipped with these dimensionless quantities we may render equation (4.74)
as:

d dθ(x)
{α + (1 − α) x} − β 2 θ(x) = 0 (4.77)
dx dx
and corresponding boundary conditions become:
dθ
(0) = β02 θ(0) , θ(1) = 1 (4.78)
dx
As d → 0 the trapezoidal profile approaches a triangular profile and α → 0,
and consequently equation (4.77) takes the form:
d2 θ dθ
x 2
+ − β 2 θ(x) = 0 (4.79)
dx dx
For the left boundary at x = 0 we get:
dθ
(0) − β 2 θ(0) = 0 (4.80)
dx
An analytical solution of equation (4.77) and (4.79) is outlined in G.5 of
Numeriske Beregninger.
4.4.3 Example: Numerical solution for specific cooling rib

In the current example we will look at the solution for a specific cooling rib based
on the generic representation a bove. We consider the a rib with the following
geometry:
L = 0.1m, D = 0.01m, d = 0.005m, (4.81)

and physical constants:
h̄ = 80W/m2 /◦ C, h̄0 = 200W/m2 /◦ C, og k = 40W/m/◦ C (4.82)
Together, these particular values of geometrical and physical variables corre-

spond to β 2 = 4.0, α = 12 and β02 = 0.5 and equation (4.77) will consequently
take the form:

d dθ
(1 + x) − 8 θ(x) = 0 (4.83)
dx dx
Analytical solution An analytical solution of the resulting differential equa-

tion (4.83) for this example may be found in appendix G.5 in Numeriske metoder
and a corresponding python snippet is shown below along with the resulting
plot:
# coding: utf-8
# src-ch3/trapes.py;
import numpy as np
from numpy import sqrt as sqrt
from scipy.special import kv as besselk
from scipy.special import iv as besseli

LNWDT=3; FNT=11
def trapes_analytical(h):
N=int(round(1.0/h)) # number of unknowns, assuming the RHS boundary value is known
x=np.arange(N+1)*h
# x = np.linspace(0, 1, N)
z0 = 4*sqrt(2)
z1 = 8
g = sqrt(2)/8
k1z0, i0z1 = besselk(1, z0), besseli(0, z1)

J = k1z0*i0z1 + k0z1*i1z0 + g*(k0z0*i0z1 - k0z1*i0z0)
A = (k1z0 + g*k0z0)/J
B = (i1z0 - g*i0z0)/J
z = sqrt(32.*(1 + x))
theta = A* besseli(0, z) + B* besselk(0, z)
dtheta = 16*(A*besseli(1, z) - B*besselk(1, z))/z
return x, theta, dtheta
if __name__ == ’__main__’:
h=0.2
x, theta, dtheta = trapes_analytical(h)
plot(x,theta,’b’,)
plot(x,dtheta,’r’)
legends.append(r’$\theta^{\prime}$’)
## Add the labels
ylabel(r’$\theta$ and $\theta^{\prime}$’)
xlabel(’x’)
show()
close()
Figure 4.6: The dimensionless temperature θ and the spatial derivative θ0 as a

function of of the spatial variable x.
Numerical solution
We will solve equation (4.83) numerically by central differences, and first

rewrite it as:
d2 θ dθ
(1 + x)
2
+ − 8 θ(x) = 0 (4.84)
dx dx
dθ θ(0)
(0) = β02 θ(0) = , θ(1) = 1 (4.85)
dx 2
The procedure for this example will follow the along the lines of the procedure
in section (4.4.1) for mixed boundary conditions.
0.0 1.0
X
0 1 2 i N-2 N-1 N
Figure 4.7: Discretization of a bar of unit length.
The x-coordinates are discretized by: xi = i h , i = 0, 1, 2, . . . , N where

h = N1 and a central difference approximation of equation (4.84) leads to the
following discretized equation:
θi+1 − 2θi + θi−1 θi+1 − θi−1
(1 + xi ) + − 8θi = 0
h2 2h
which after collection of terms may be presented as for generic point xi along
the unit length bar in Figure 4.7:
−(1 − γi ) θi−1 + 2 (1 + 8h γi ) θi − (1 + γi ) θi+1 = 0 (4.86)

where
h h
γi = = , i = 1, . . . , N − 2 (4.87)
2 (1 + xi ) 2 (1 + i h)
We observe that equation (4.86) form a tridiagonal equation system as only
the immediate neigbors of θi , namely θi−1 and θi−1 , are involved for the i − th
equation.
At the left boundary we need equation (4.85) to be satisfied, which has the
−θ0
second order discretized equivalent θ22h = θ21 which yields θ0 = θ2 − θ1 h and
after substitution in equation (4.86) for i = 0 we get the following discretized
version at the boundary:
[2 + h (1 + 15 γ0 )] θ0 − 2 θ1 = 0 (4.88)
Note that the discretization θ0 = θ2 − θ1 h is equally valid, it is just that the

implementation of the boundary condtion would then violate the nice tridiagonal
structure of the discretized equation system, and thus we prefer equation (4.88)
which fits very nicely in the tri-diagonal structure.
For i = N − 1 we make use of the fact that θN = 1 is given :
−(1 − γN −1 ) θN −2 + 2 (1 + 8h γN −1 ) θN −1 = (1 + γN −1 ) θN = 1 + γN −1 (4.89)
Together, the equations (4.86), (4.88),and (4.89) form a tridiagonal, linear

equation system which we will solve using the sparse library scipy.sparse
from SciPy. An example of how to use the sparse library may be found in our
Introdcution to Scientific Python programming.
The python code for the implementation of the algorithm is shown below.
We compare the solution with the analytical solution and again we observe
the striking difference between the very complex analytical solution (involving
Bessel-functions of various kinds) and the relatively simple numerical solution
involving only the solution of a tridiagonal linear equation system.
As we have discretized equation (4.84) using central differences, we expect the
numerical solution to be second order accurate and have added a code segment
to approximate the order of the scheme. You may test the code an see whether
our implementation has the expected order. Optionally, we have also provided
the possibility to plot the solution as a function of the mesh size.
# coding: utf-8
# src-ch3/trapes_numerical.py;
import numpy as np
import scipy as sc
import scipy.sparse
import time
from trapes import trapes_analytical
from Carbon.Aliases import false

LNWDT=3; FNT=11
def solve_trapes_numerically(h):
L=1.0 # length of domain
n=int(round(L/h)) # number of unknowns, assuming the RHS boundary value is known
x=np.arange(n+1)*h # x includes xmin and xmax at boundaries
gamma=h/(2*(1+x[0:-1])) # uses all but the last x-value for gamma
subdiag=gamma[:-1]-1.0 #the subdiagonal has one less element than the diagonal
diag=2.0*(1+8.0*h*gamma)
diag[0]= 2.0+h*(1+15.0*gamma[0])
superdiag=np.zeros(n-1)
superdiag[0]=-2
superdiag[1:]=-(gamma[0:-2]+1.0)
A_diags=[subdiag,diag,superdiag] #note the offdiags are one elt shorter than the diag
A = sc.sparse.diags(A_diags, [-1,0,1], format=’csc’)
#Create rhs
d=np.zeros(n)
d[n-1]=1+gamma[-1]
#Solve linear problem

tic=time.clock()
theta = sc.sparse.linalg.spsolve(A,d) #theta=sc.linalg.solve_triangular(A,d)
toc=time.clock()
return x,theta
if __name__ == ’__main__’:
h=0.2
xa, atheta, adtheta = trapes_analytical(h)
x, theta=solve_trapes_numerically(h)
# Plot solutions
figure(’Solutions’)
plot(xa,atheta,)
plot(x[:-1],theta,’-.’)
legends.append(r’$\theta$-analytical’)
## Add the labels

ylabel(r’$\theta$’)
xlabel(’x’)
# Approximate the order of the scheme

h=0.2
h_range=[h/2.**(d) for d in np.arange(0,6)]
error=[]
for h in h_range:
xa, atheta, adtheta = trapes_analytical(h)
x,theta=solve_trapes_numerically(h)
error.append(np.sqrt(np.sum(((atheta[:-1]-theta)/atheta[:-1])**2)))
log_error = np.log2(error)
dlog_err=log_error[0:-1]-log_error[1:]
print(’Approximate order:{0:.3f}’.format(1./np.mean(dlog_err)))
plot_error=0
if (plot_error):
figure(’error’)
ax = plot(h_range,error)
yscale(’log’,basey=2)
xlabel(r’meshsize $h$’)
ylabel(r’$\frac{\epsilon}{\theta_a}$’)
show()
close()
4.5 Linearization of nonlinear algebraic equations

In this section we will present methods for linearization of nonlinear algebraic
equations and will use the problem in section (4.4.2) as an illustrative example:
3 2
y 00 (x) =
y (4.90)
2
y(0) = 4, y(1) = 1 (4.91)

From (4.4.2) we know that one of the two analytical solutions is given by:
4
y= (4.92)
(1 + x)2
We use central differences for the term y 00 (x) to discretize equation (4.90):
yi+1 − 2yi + yi−1 3
2
= yi2
h 2
which after collection of terms may be presented as:

3 2
yi−1 − 2 + h yi yi + yi+1 = 0 (4.93)
2
We have discretized the unit interval [0, 1] in N parts with a corresponding
mesh size h = 1/N and discrete coordinates denoted by xi = h · i for i =
0, 1, . . . , N + 1 (see Fig 4.8)
x0 x1 xi xN-1 xN xN+1
0.0 1.0
Figure 4.8: Discretization one-dimensional boundary value problem on a unit

interval.
By substituting the known boundary values y0 = 4 og yN +1 = 1 in equation

(4.93) we get the following system of equations:

3
− 2 + h2 y1 y1 + y2 = −4
2
..
.

3
yi−1 − 2 + h2 yi yi + yi+1 = 0 (4.94)
2
..
.

3
yN −1 − 2 + h2 yN yN = −1
2
where i = 1, 2, . . . , N −1. The coefficient matrix is tridiagonal, but the system

is nonlinear. And as we have no direct analytical solutions for such nonlinear
algebraic systems, we must linearize the system and develop an iteration strategy
which hopefully will converge to a solution to the nonlinear problem. In the
following we will present two methods for linearization.
4.5.1 Picard linearization

Due to the nonlinearities in equation (4.93), we must develop an iteration strategy
and to do so we let yim+1 og yim be the solution of the discretized equation (4.94)
at iterations m + 1 and m, respectively.
With Picard linearization we linearize the nonlinear terms simply by replace-
ment of dependent terms at iteration m + 1 with corresponding terms at iteration
m until only linear terms are left. In equation (4.94) the nonlinearity is caused
by the y 2 -term and the correpondings terms need to be linearized.
By using the iteration nomenclature, equation (4.94) takes the form:

3
− 2 + y1m+1 h2 y1m+1 + y2m+1 = −4
2
..
.

m+1 3
yi−1 − 2 + yim+1 h2 yim+1 + yi+1
m+1
=0 (4.95)
2
..
.

m+1 3 m+1 2 m+1
yN −1 − 2 + yN h yN = −1
2
where i = 2, 3, ...N − 1, and the iteration counter m = 0, 1, 2, . . .. The

linearization is performed by replacing 32 h2 y1m+1 , 32 h2 yim+1 and 32 h2 yN
m+1
by
3 2 m 3 2 m 3 2 m
2 h y1 , 2 h y i and 2 h yN such that the following linear system is obtained:

3
− 2 + y1m h2 y1m+1 + y2m+1 = −4
2
..
.

m+1 3
yi−1 m+1
− 2 + yim h2 yim+1 + yi+1 =0 (4.96)
2
..
.

m+1 3 m 2 m+1
yN −1 − 2 + yN h yN = −1
2
We have with the procedure above discretized our analytical problem in

equation (4.93) to a linear, tridiagonal system which may readly be solved with
sparse solvers like the ones we introduced in 4.4.1.
Note that all values at iteration level m, i.e. yim are known and the only
unknowns in (4.96) are yim−1 . Thus, to start the iteration procedure we need
an initial value for yi0 . Unless a well informed initial guess is available, it is
common to start with yi0 = 0, i = 1, 2, ...N . Naturally, initial values close to the
numerical solution will result in faster convergence.
Additionally, we must test for diagonal dominance of the iteration scheme,
which for equation (4.96) means:

2 + 3 yim h2 ≥ 2 , i = 1, 2, ..., N

(4.97)
2
We observe from equation (4.97) that we have a diagonally dominant iteration

matrix when yim > 0. By inspection the analytical solutions in Figure 3.8, we
observe that this condition is fulfilled for only one of the two solutions. In cases
when the condition of diagonal dominance is not fulfilled, make sure your solver
allows for pivotation, e.g. like scipy.sparse.linalg as demonstrated in 4.4.1.
However, for demonstration purposes we have in this example implemented
a version of the classical tri-diagonal solver tdma along with a version with
pivotation tripiv in the module TRIdiagonalSolvers.
In cases where the one has no experience with the itereation process, e.g. while
impelmenting, one may find it beneficial to use the number of iterations as a
stopping critera, rather than an error/residual based stop criterion. However,
after you have gained some experience with the particular problem at hand you
may use other stop critera as demonstrated in section 4.5.4.
In the python-code delay34.py below we have implemented the procedure
of Picard linearization as outlined above.
# src-ch3/delay34.py;TRIdiagonalSolvers.py @ git@lrhgit/tkt4140/src/src-ch3/TRIdiagonalSolvers.py;
import numpy as np
from TRIdiagonalSolvers import tdma

from TRIdiagonalSolvers import tripiv

LNWDT=3; FNT=11
h = 0.05 # steplength
n = int(round(1./h)-1) # number of equations
fac = (3./2.)*h**2
Nmax = 30 # max number of iterations
# iterative solution with Nmax iterations

#initilizing:
ym = np.zeros(n) # initial guess of y. ym = np.zeros(n) will converge to y1. ym = -20*x*(1-x) for ins
for m in range(Nmax):
"""iteration process of linearized system of equations using delay method"""
# need to set a, b, c and d vector inside for loop since indices are changed in tdma solver
a = np.ones(n)
c = np.ones(n)
b = -(np.ones(n)*2. + fac*ym)
d = np.zeros(n)
d[n-1] = -1.
d[0] = -4.
ym1 = tdma(a,b,c,d) # solution
if(m>0):
max_dym = np.max(np.abs((ym1 - ym)/ym))
print(’m = {} \t max dym = {:6.4g} ’.format(m, max_dym))
ym = ym1
# compute analytical solution

xv = np.linspace(h, 1 - h, n)
ya = 4./(1 + xv)**2
error = np.abs((ym1 - ya)/ya)
print(’\nThe max relative error after {} iterations is: {:4.3g}’.format(Nmax,np.max(error)))
The maximal relative deviation between consecutive iterations is stored in

the variable max_dym. We observe that max_dym decreases for each iteration as
4
the solution slowly converges to the analytical solution yI = which was
(1 + x)2
demonstrated in (3.8) in section (3.2).
After Nmax iterations we compute the error by :
Note that error is a vector by construction, since ym1 and ya are vectors.
Thus, to print a scalar measure of the error after Nmax iterations we issue the
following command:
print(’\nThe max relative error after {} iterations is: {:4.3g}’.format(Nmax,np.max(error)))
The method of Picard linearization.

The advantage with the method of Picard linearization, is that the lin-
earization process is very simple. However, the simplicity comes at a prize,
since it converges slowly and often requires a relatively good initial guess.
Once we have gained some experience on how the iteration procedure develops,
we may make use of a stop criterion based on relative changes in the iterative
solutions as shown in the python-snippet below:
# iterative solution with max_dym stop criterion
m=0; RelTol=1.0e-8
max_dym=1.0
a = np.ones(n)
c = np.ones(n)
while (m<Nmax and max_dym>RelTol):
d = np.zeros(n)
d[n-1] = -1.
d[0] = -4.
if(m>0):
max_dym = np.max(np.abs((ym1 - ym)/ym))
print(’m = {} \t max dym = {:6.4g} ’.format(m, max_dym))
m+=1
ym = ym1

print(’\nThe max relative after {} iterations is: {:4.3g}’.format(m,np.max(error)))
# print results nicely with pandas

print_results=0
if (print_results):
import pandas as pd
data=np.column_stack((xv,ym1,ya))
data_labeled = pd.DataFrame(data, columns=[’xv’,’ym’,’ya’])
print(data_labeled.round(3).to_string(index=False))
# plotting:
# Add the labels
plot(xv,ya,"r") # plot analytical solution
legends.append(’analytic’)
plot(xv,ym1,"b--") # plot numerical solution
legends.append(’delayed linearization’)

ylabel(’y’)
xlabel(’x’)
show()
Notice that we must set max_dym>RelTol prior to the while-loop to start

the iterative process.
You may experiment with the code above to find that for a fixed mesh size
h=0.05, the relative error will not be smaller than ≈ 5.6 · 10−4 , regardless of how
low you set RelTol, i.e. after a converged solution is obtained for the given mesh
size. Naturally, the error will be reduced if you reduce the mesh size h. Observe
also that the number of iterations needed to obtain a converged solution for a
given mesh size of h=0.05 is smaller than the Nmax=30, which was our original
upper limit.
4.5.2 Newton linearization

Motivating example of linearization Let us look at a motivating example
which is simple to use for nonlinearities in products. For convenience we introduce
δyi as the the increment between two consecutive yi -values:
yim+1 = yim + δyi (4.98)

By assuming δyi to be small, the nonlinear term in equation (4.93) may then
be linearized using (4.98) in the following way:
2 2
yim+1 = (yim + δyi ) = (yim )2 +2yim δyi +(δyim )2 ≈ (yim )2 +2 yim δyi = yim 2yim+1 − yim

(4.99)
That is, quadratic expression in equation (4.99) is linearized by neglecting
the quadratic terms (δy)2 which are small compared to the other terms in (4.99).
Substitution of the lineariztion (4.99) in (4.94) results in the following system
of equations:
−(2 + 3y1m h2 ) y1m+1 + y2m+1 = − 32 (yim h)2 − 4

..
.
m+1
yi−1 − (2 + 3yim h2 ) yim+1 + yi+1
m+1
= − 23 (yim h)2 (4.100)
..
.
m+1 m+1
yN m 2
−1 − (2 + 3yN h ) yN = − 23 (yN
m 2
h) − 1
where i = 1, 2, . . . , N − 1 and m = 0, 1, . . ..
The resulting equation system (4.100), is a tridiagonal system which may be
solved with the tmda-solver as for the previous example. An implementation is
given below:
# src-ch3/taylor34.py;TRIdiagonalSolvers.py @ git@lrhgit/tkt4140/src/src-ch3/TRIdiagonalSolvers.py;
import numpy as np
LNWDT=3; FNT=11
fac = (3.)*h**2
#initilizing:
x = np.linspace(h,1-h,n)
ym = -20*x*(1-x) # initial guess of y.
#ym = np.zeros(n) # will converge to y1. ym = -20*x*(1-x) for instance will converge to y2
it, itmax, dymax, RelTol = 0, 15, 1., 10**-10

legends=[]
while (dymax > RelTol) and (it < itmax):
"""iteration process of linearized system of equations using taylor
"""
plot(x,ym) # plot ym for iteration No. it
legends.append(str(it))
it = it + 1
a = np.ones(n)
c = np.ones(n)
d = -(fac*0.5*ym**2)
d[n-1] = d[n-1]-1
d[0] = d[0]-4

dymax = np.max(np.abs((ym1-ym)/ym))
ym = ym1
print(’it = {}, dymax = {} ’.format(it, dymax))
legend(legends,loc=’best’,frameon=False)
ya = 4./(1+x)**2
feil = np.abs((ym1-ya)/ya)
print "\n"
for l in range(len(x)):
print ’x = {}, y = {}, ya = {}’.format(x[l], ym1[l], ya[l])
figure()
# plotting:
# Add the labels
plot(x,ya,"r") # plot analytical solution
legends.append(’y1 analytic’)
plot(x,ym1,"b--") # plot numerical solution
legends.append(’y2 taylor linearization’)

ylabel(’y’)
xlabel(’x’)
show()
You may experiment with by changing the initial, guessed solution, and
observe what happens with the resulting converged solution. Which solution
does the numerical solution converge to?
Observe that the iteration process converges much faster than for the Picard
linearization approach. Notably, somewhat slow in the first iterations, but after
some iterations quadratic convergence is obtained.
Generalization: Newton linearization The linearization in (4.98) and

(4.99) may seen as truncated Taylor expansions of the dependent variable at
iteration m, where only the first two terms are retained.
To generalize, we let F denote the nonlinear term to be linearized and assume
F to be a function of the dependent variable z at the discrete point xi (or i for
short) at iteration m.
m+1
We will linearize F (zi ) ≡ F (zi )m+1 , and use the latter notation in the
following. A Taylor expansion may then be expressed at the point i and iteration
m

∂F
F (zi )m+1 ≈ F (zi )m + δzi (4.101)
dzi m
where δzi = zim+1 − zim .
Our simple example in equation (4.99) may be rendered in this slightly more
generic representation:
δzi → δyi , zim+1 → yim+1 , zim → yim , F (yi )m+1 = (yim+1 )2 , F (yi )m = (yim )2
which after substitution in (4.101) yields: (yim+1 )2 ≈ (yim )2 + 2yim · δyi which
is identical to what was achieved in equaiton (4.99) .
The linearization in equation (4.101) is often referred to as Newton-linearization.
In many applications, the nonlinear term is composed z at various discrete,
spatial locations and this has to be accounted for in the linearization. Assume to
begin with that we have terms involving both zi og zi+1 . The Taylor expansion
will then take the form:

∂F ∂F
F (zi , zi+1 )m+1 ≈ F (zi , zi+1 )m + δzi + δzi+1 (4.102)
∂zi m ∂zi+1 m
where:
δzi = zim+1 − zim , m+1

δzi+1 = zi+1 m
− zi+1
When three spatial indices, e.g. zi−1 , zi og zi+1 , are invoved, the Taylor
expansion becomes:

∂F ∂F ∂F
F (zi−1 , zi , zi+1 )m+1 ≈ F (zi−1 , zi , zi+1 )m + δzi−1 + δzi + δzi+1
∂zi−1 m ∂zi m ∂zi+1 m
(4.103)
where
m+1
δzi−1 = zi−1 m
− zi−1 , δzi = zim+1 − zim , m+1
δzi+1 = zi+1 m
− zi+1
Naturally, the same procedure may be expaned whenever the nonlinear term
is composed of instances of the dependent variable at more spatial locations.
4.5.3 Example: Newton linearization of various nonlinear

ODEs
Nonlinear ODE 1 In this example of a nonlinear ODE we consider:
p
y 00 (x) + y(x) y(x) = 0 (4.104)
which after discretization with central differences may be presented:
q
m+1
yi+1 − 2yim+1 + yi−1
m+1
+ h2 yim+1 yim+1 = 0 (4.105)
q
In equation (4.105) it is the nonlinear term yim+1 yim+1 which must be
linearized. As we in this example only have one index we make use of the simplest
3
Taylor expansion (4.101) with zi → yi and F (yi ) = yi2 :

∂F 3 3 m 1
F (yi )m+1 ≈ F (yi )m + δyi = (yim ) 2 + δyi (y ) 2 (4.106)
∂yi m 2 i
where δyi = yim+1 − yim . Equivalently by using the squareroot-symbols:
3p m
q
yim+1 yim+1 ≈ yim yim +
p
yi δyi (4.107)
2
Observe that we have succeeded in the linearization process as all occurences
of yim+1 in the two equivalent discretization in equation (4.106) and (4.107) are
of power one.
After substition in equaiton (4.105) the following linear difference equation
results:
h2 m p m

m+1 3 2p m
yi−1 + h yi − 2 yim+1 + yi+1 m+1
= y yi (4.108)
2 2 i
Nonlinear ODE 2 As a second example of a nonlinear ODE consider:
y 00 (x) + sin (y(x)) = 0 (4.109)

which after discretization by central differences may be rendered:
m+1 m+1
− 2yim+1 + yi−1 + h2 sin yim+1 = 0

yi+1 (4.110)
In equation (4.110) we identify sin yim+1 as the nonlinear term in need of

linearization. Againg, we have only one index and may make use of (4.101) with
zi → yi and F (yi ) = sin(yi ):

∂F
F (yi )m+1 = sin(yim+1 ) ≈ F (yi )m + δyi = sin(yim ) + δyi cos(yim )
∂yi m
which after substitution in equation (4.110) leads to the following difference

equation:
m+1
yi−1 + (h2 cos(yim ) − 2)yim+1 + yi+1
m+1
= h2 (yim cos(yim ) − sin(yim )) (4.111)
p
Nonlinear ODE 3 Consider the ODE y 00 (x) + y(x) y 0 (x) = 0 which after
discretization with central differences may be presented:
q p
m+1
yi+1 − 2yim+1 + yi−1
m+1
+ α yim+1 m+1
yi+1 m+1
− yi−1 = 0 , der α = h h/2 (4.112)
q
The term yim+1 yi+1 m+1 m+1
− yi−1 is nonlinear and must be linearized. As three
indices are involved in the term we utilize (4.103) with zi−1 → yi−1 , zi → yi ,
√
zi+1 → yi+1 , and finally F (yi−1 , yi , yi+1 )m = yi yi+1 − yi−1 .
We present the individual terms in equation(4.103) :
ym

∂F
F (yi−1 , yi , yi+1 )m = yim yi+1 =− p mi
p m m ,
− yi−1 m
∂yi−1 m 2 yi+1 − yi−1
m

∂F ∂F y
= p mi
p m m ,
= yi+1 − yi−1 m
∂yi m ∂yi+1 m 2 yi+1 − yi−1
By collecting terms we get:
yim yim
q
yim+1 m+1 m+1
≈ yim
p m − ym − p
p m m δy + p
yi+1 − yi−1 yi+1 i−1 m − ym
δyi−1 + yi+1 − yi−1 i m − ym
δyi+1
2 yi+1 i−1 2 yi+1 i−1
(4.113)
m+1
Note that equation (4.113) is linear as yi−1 , yim+1 and yi+1
m+1
are only in
first power. By substitution in equaiton (4.112) we get the linear disicretized
equation:
ym ym

m+1
1 − α 2gim yi−1 − (2 − αg m ) yim+1 + 1 + α 2gim yi+1
m+1
ym m m (4.114)
= α 2gim (yi+1 − yi−1 )
der g m =
p m m
yi+1 − yi−1
In the next section we will show how we can operate diretly with the incre-
m+1
mental quantities δyi−1 , δyim+1 and δyi+1
m+1
.
Comparison with the method of Picard linearization
How would the examples above be presented with the method of Picard
linearization?
Equation (4.105) takes the following form the Picard linearization:
q
m+1
yi−1 + (h2 yim+1 − 2) yim+1 + yi−1
m+1
=0 (4.115)
q
The coefficient for the yim+1 -term contains yim+1 which will be replaced by
p m
yi such that:
m+1
+ (h2 yim − 2) yim+1 + yi−1
m+1
p
yi−1 =0 (4.116)
For Picard linearization equation (4.110) become:
m+1
yi−1 − 2yim+1 + yi+1
m+1
= −h2 sin(yim ) = 0 (4.117)
whereas equation (4.112) looks like:
q
m+1 m+1 m+1
yi−1 + (α yi+1 − yi−1 − 2) yim+1 + yi+1
m+1
=0 (4.118)
with
q Picard linearization. The coefficient in front of the yim+1 -term con-
m+1 m+1 p m − y m such that the equation
tains yi+1 − yi−1 which is replaced by yi+1 i−1
becomes:
m+1 m − 2) y m+1 + y m+1 = 0
p m
yi−1 + (α yi+1 − yi−1 i i+1
When we use the method of Picard linearization, the system first has to be
presented in a form which identifies the coefficients in the equation system. If
the coefficients have dependent variable at iteration m + 1, the are to be replaced
with the corresponding value at iteration m. Often this procedure is identical to
a Taylor expansion truncated after the first term.
Thus, we may expect that the method of Picard linearization will converge
more slowly compared to methods which use more terms in the Taylor expansion.
The method of Picard linearization is most frequently used for partial differential
equations where e.g. the material parameters are functions of the dependent
variables, such as temperature dependent heat conduction.
4.5.4 Various stop criteria

We let δyi denote the increment between two consecutive iterations:
δyi = yim+1 − yim , i = 1, 2, ..., N, m = 0, 1, ... (4.119)

where N denotes the number of physical grid points. Two classes of stop
criteria based on absolute and relative metrics, may then be formulated as
outlined in the following.
Stop criteria based on absolute values.
max(|δyi |) < εa (4.120)
N
1X
|δyi | < εa (4.121)
n i=1
v
uN
1u X
t (δy )2 < ε
i a (4.122)
n i=1
where εa is a suitable small number.

Stop criteria based on relative values

δyi
max m+1 < εr ,
yim+1 6= 0 (4.123)
yi
PN
i=1 |δyi |
PN m+1 < εr (4.124)
i=1 yi
max(|δyi |)
< εr (4.125)
max(|yim+1 |)
Note that if the quantity of interest, i.e. yi , is of order 1, the absolute and
the relative criteria are equivalent. Normally we will prefer a relative criterion
as these comply with the expression number of correct digits. If cases where the
values of yi are small in the whole computational domain (i.e. for all i) we may
resort to a absolute value crterion
4.5.5 Example: Usage of the various stop criteria

In this example we will demonstrate the use of the various stop criteria outlined
in the previous section on the problem in section (3.2), where we will compute
the second solution yII ) as illustrated in Figure (3.8).
For convenience we reformulate the ODE (4.90) :
3 2
y 00 (x) = y (4.126)
2
with the boundary condtions:
y(0) = 4, y(1) = 1 (4.127)

We let our initial guess be expressed by the parabola ys = 20(x − x2 ) ,
0 < x < 1, which is plotted along with the solution yII , in Figure (4.9).
4
ys
2 y II
0
2
y s , y II
4
6
8
10
120.0 0.2 0.4 0.6 0.8 1.0
x
Figure 4.9: Intial parabolic guess and solution yII .
For this example it is natural to use a relative stop criterion as we know

that the solution has values in the range [4, −10.68]. After nine iterations the
following table results: tabell:
Iter.nr. tr2 tr3

1 7.25 · 10−1 7.55 · 10−1
2 8.70 · 10−1 8.20 · 10−1
3 8.49 · 10−1 6.65 · 10−1
4 4.96 · 10−1 5.85 · 10−1
5 2.10 · 10−1 2.80 · 10−1
6 4.62 · 10−2 6.07 · 10−2
7 2.24 · 10−3 2.70 · 10−3
8 4.68 · 10−6 5.79 · 10−6
9 2.26 · 10−11 2.55 · 10−11
The iteration evolution is representative for Newton-algorithms as the initial

values are far away from the correct solution. The errors decreases slowly for
the first six iterations, but afterwards we observe quadratic convergence. The
two criteria do not differ significantly for this case.
Nedenfor vises listing av programmet avvikr.
A python code using a relative criteria is shown here:
# src-ch3/avvikr.py;TRIdiagonalSolvers.py @ git@lrhgit/tkt4140/src/src-ch3/TRIdiagonalSolvers.py;
import numpy as np
LNWDT=3; FNT=11
# numerical parameters
h = 0.05 # steplength.
n = round(1./h)-1 # number of equations. h should be set so that n is an integer
fac = (3.)*h**2
#initilizing:
ym = -20*x*(1-x) # initial guess of y. ym = np.zeros(n) will converge to y1. ym = -20*x*(1-x) for ins
itmax = 15
for it in range(itmax):
"""iteration process of linearized system of equations"""
a = np.ones(n)
c = np.ones(n)
d = -(np.ones(n)*fac*0.5*ym**2)
d[n-1] = -1.
d[0] = -4.

dy = np.abs(ym1-ym)
tr2 = np.max(dy)/np.max(np.abs(ym1))
tr3 = np.sum(dy)/np.sum(np.abs(ym1))
ym = ym1
print ’it = {}, tr2 = {}, tr3 = {} ’.format(it, tr2, tr3)
ya = 4./(1+x)**2
feil = np.abs((ym1-ya)/ya)
# plotting:
# Add the labels
plot(x,ym1,"b") # plot numerical solution
legends.append(’y2 numerical’)
plot(x,ya,"r") # plot analytical solution

ylabel(’y’)
xlabel(’x’)
show()
A code where the absolute criteria is used is almost identical, except for the
substitution of tr2 og tr3
ta1 = max(dy);
ta2 = sum(dy)/n;
ta3 = sqrt(dot(dy,dy))/n;
# src-ch3/avvika.py;TRIdiagonalSolvers.py @ git@lrhgit/tkt4140/src/src-ch3/TRIdiagonalSolvers.py;
import numpy as np
LNWDT=3; FNT=11
# numerical parameters
h = 0.05 # steplength.
n = round(1./h)-1 # number of equations. h should be set so that n is an integer
fac = (3.)*h**2
#initilizing:
ym = -20*x*(1-x) # initial guess of y. ym = np.zeros(n) will converge to y1. ym = -20*x*(1-x) for ins
itmax = 15
for it in range(itmax):
a = np.ones(n)
c = np.ones(n)
d = -(np.ones(n)*fac*0.5*ym**2)
d[n-1] = -1.
d[0] = -4.

dy = np.abs(ym1-ym)
ta1 = np.max(dy) # abs stop critertia 1
ta2 = np.sum(dy)/n # abs stop critertia 2
ta3 = np.sqrt(np.dot(dy,dy))/n # abs stop critertia 3
ym = ym1
print ’it = {}, ta2 = {}, ta3 = {} ’.format(it, ta2, ta3)
xv = np.linspace(h,1-h,n)
ya = 4./(1+xv)**2
# plotting:
# Add the labels
plot(xv,ym1,"b")
legends.append(’y2 numerical’)
#plot(xv,ya,"r")
#legends.append(’y1 analytic’)
ylabel(’y’)
xlabel(’x’)
show()
4.5.6 Linerarizations of nonlinear equations on incremen-

tal form
As seen previously in equation (4.99), we may introduce the increment δyi =
yim+1 − yim in a nonlinear term as examplified by:
(yim+1 )2 = (yim + δyi )2 = (yim )2 + 2yim δyi + (δyi )2 (4.128)

≈ (yim )2 + 2yim δyi = yim (2yim+1 − yim ) (4.129)
i.e. we linearize by ingnoring higher order terms of the increments, and solve
equation (4.129) with respect to yim+1 .
Alternatively, we may think of δyi as the unknown quantity and solve the
resulting equation system with respect to the increments. For example we may
substitute ykm+1 = ykm +δyk for k = i−1, i, i+1 and (yim+1 )2 ≈ (yim )2 +2 yim δyi
into equation (4.95) (or equivalently (4.100)), the following equation system
results:
3 m 2
−(2 + 3y1m h2 ) δy1 + δy2 = −(y2m − 2y1m + 4) + (y h)
2 1
3
δyi−1 − (2 + 3yim h2 ) δyi + δyi+1 m
= −(yi+1 − 2yim + yi−1
m
) + (yim h)2 (4.130)
2
m 2 m m 3 m 2
δyN −1 − (2 + 3yN h ) δyN = −(1 − 2yN + yN −1 ) + (yN h)
2
where i = 2, 3, . . . , N − 1 adn m = 0, 1, 2, . . .. In the equation system above
we have incorporated the boundary conditions y0 = 4, δy0 = 0, yN +1 = 1, and
δyN +1 = 0. For each iteration we update the y-values by:
yim+1 = yim + δyi , i = 1, 2, . . . , N (4.131)

After having gained some experience with how the iteration proceed, we may
use stop criteria such ad max |δyi | < ε1 or max |δyi /yim+1 | < ε2 , i = 1, 2, . . . , N .
We have implemented the incremental formulation in the code below.
# src-ch3/delta34.py;ODEschemes.py @ git@lrhgit/tkt4140/src/src-ch3/ODEschemes.py;
import numpy as np
LNWDT=3; FNT=11
fac = (3.)*h**2
nitr = 6 # number of iterations
#initilizing:
it, itmax, dymax, relTol = 0, 10, 1., 10**-5 # numerical tollerance limits
while (dymax > relTol) and (it < itmax):

it = it + 1
a = np.ones(n)
c = np.ones(n)
d = (np.ones(n)*fac*0.5*ym**2)
for j in range(1,n-1):
d[j] = d[j] - (ym[j+1] - 2*ym[j] +ym[j-1])
d[n-1] = d[n-1]-(1 - 2*ym[n-1] + ym[n-2])

d[0] = d[0] -(ym[1] - 2*ym[0] + 4)
dy = tdma(a,b,c,d) # solution
ym = ym + dy
dymax = np.max(np.abs((dy)/ym))
print ’it = {}, dymax = {} ’.format(it, dymax)
xv = np.linspace(h,1-h,n)
ya = 4./(1+xv)**2
print "\n"
for l in range(len(xv)):
print ’x = {}, y = {}, ya = {}’.format(xv[l], ym[l], ya[l])
plot(xv,ya,"r") # plot analytical solution

plot(xv,ym,"b--") # plot numerical solution
legends.append(’delta linearization’)
# Add the labels
ylabel(’y’)
xlabel(’x’)
show()
4.5.7 Quasi-linearization
In previous examples, we have always linearized the discretization of the original
problem. However, it is possible to first linearize the differential equation and
only succesively discretize it. This method is normally called quasi-linearization.
Let us now look at an example given by a second order non linear equation:
y 00 (x) = f (x, y, y 0 ) (4.132)

We now write (4.132) as:
g(x, y, y 0 , y 00 ) ≡ y 00 (x) − f (x, y, y 0 ) = 0 (4.133)

and setting
δy = ym+1 − ym , δy 0 = ym+1
0 0
− ym , δy 00 = ym+1
00 00
− ym (4.134)
where m stands for the iteration number (in this section we will be using
sub-scripts for iteration number)
By expanding (4.133) around iteration m we get:

0 ∂g ∂g ∂g
g(x, ym+1 , ym+1 , y 00 m+1 ) ≈ g(x, ym , ym
0
, y 00 m )+ δy+ δy 0 + 00 δy 00
∂y m ∂y 0 m ∂y m
(4.135)
Suppose that we have iterated so many times that:
00 00
0 0
g(x, ym+1 , ym+1 , ym+1 ) ≈ g(x, ym , ym , ym ) ≈ 0
which inserted in (4.135) gives:

∂g ∂g 0 ∂g 00
δy + δy + δy = 0 (4.136)
∂y m ∂y 0 m ∂y 00 m
After derivation of (4.133):
∂g ∂f ∂g ∂f ∂g
=− , =− 0 , =1 (4.137)
∂y ∂y ∂y 0 ∂y ∂y 00

00 ∂f ∂f
δy = δy + δy 0 (4.138)
∂y m ∂y 0 m
By inserting (4.134) in (4.138) and making use of (4.132), we get:

00 ∂f 0 ∂f
ym+1 − 0
ym+1 − ym+1
∂y m ∂y m
(4.139)
0 ∂f ∂f 0
= f (x, ym , ym )− ym − ym
∂y m ∂y 0 m
Finally we re-write (4.139) with our regular notation, i.e. using super-scripts
for iteration number:

00 m+1 ∂f 0 m+1 ∂f
(y ) − (y ) − y m+1
∂y 0 m ∂y m
(4.140)
∂f ∂f
= f (x, y m , (y 0 )m ) − (y 0 )m − ym
∂y 0 m ∂y m
We have used a second order equation as an example, but (4.138) - (4.140)
allows for generalization to n-th order equations. Take for example a third order
equation:

000 ∂f ∂f ∂f
(y )m+1 − (y 00 )m+1 − (y 0 )m+1 − y m+1
∂y 00 ∂y 0 m ∂y m

∂f
= f (x, y m , (y 0 )m , (y 00 )m ) − (y 00 )m (4.141)
∂y 00 m

∂f ∂f
− (y 0 )m − (y)m
∂y 0 m ∂y m
4.5.8 Example:
1)
Consider the following equation
3 2
y 00 (x) = y
2
∂f ∂f
One can easily check that ∂y 0 = 0 and ∂y = 3y. Inserting these terms in
(4.140) gives:
3
(y 00 )m+1 − 3y m y m+1 = − (y m )2
2
yi+1 −2yi +yi−1
By discretizing with central differences yi00 ≈ h2 we get the follow-
ing algebraic relation:
m+1 3
yi−1 − (2 + 3h2 yim )yim+1 + yi+1
m+1
= − (hyim )2
2
which is in agreement with (4.100).

2)
Falkner-Skan equation reads:
y 00 + y y 00 + β [1 − (y 0 )2 ] = 0
We re-write the equation as:
y 00 = − y y 00 + β [1 − (y 0 )2 ] = f (x, y, y 0 , y 00 )

∂f ∂f ∂f
Noting that in this case ∂y 00 = −y , ∂y 0 = 2βy 0 and ∂y = −y 00 , equation
(4.141) gives:
(y 000 )m+1 + y m (y 00

)m+1 − 2β(y 0)m (y 0 )m+1 + (y 00 )m y m+1
2
= −β 1 + [(y 0 )m ] + y m (y 00 )m
Using central differences one can verify that the result is in agreement with
equation (F.O.16) in Appendix F of the Numeriske Beregninger.
4.6 Exercises
Exercise 7: Circular clamped plate with concentrated single
load
R R0
P
W ϕ
Figure 4.10: Schematic representation of a circular clamped plate with concen-

trated single load. The plate is clamped at R = R0 and the load P is applied at
R = 0.
Figure 4.10 show a rigidly clamped plate with radius R0 . The plate is
loaded with a single load P in the plate centre. The differential equation for the
deflection W (R) is given by:
d3 W 1 d2 W 1 dW P
3
+ − 2 = (4.142)
dR R dR2 R dR 2πD · R
3
Et
The plate stifness D is given by: D = 12(1−ν) , where E is the modulus of
elasticity, ν is the Poissons ratio and t is the plate thickness. The boundary
conditions are given by:
dW dW
W (R0 ) = (R0 ) = 0, (0) = 0 (4.143)
dR dR
In which the two first are because it is rigidly clamped, and the last due to
the symmetry. We introduce dimmensionless variables: r = RR0 , ω(R) = 16πDW P R02
,
so that Eq. (4.142) can be written:
d3 ω 1 d2 ω 1 dω 8
3
+ 2
− 2 = , 0<r<1 (4.144)
dr r dr r dr r
and Eq. (4.143):
dω dω
ω(1) = (1) = 0, (0) = 0 (4.145)
dr dr
The analytical solution is given by:
dω
ω(r) = r2 [2 ln(r) − 1] + 1, (r) = 4 r ln(r) (4.146)
dr
Pen and paper:
The following problems should be done using pen and paper:
a) We introduce the inclination φ(r) = − dω
dr and insert into Eq. (4.144):
d2 φ 1 dφ φ 8
2
+ − 2 =− , (4.147)
dr r dr r r
φ(0) = φ(1) = 0 (4.148)
discretize Eq. (4.147) with central differences. Partion the interval [0, 1] into
N segments, so that h = ∆r = N1 , which gives r = h · i, i = 0, 1, . . . , N . The
coefficient matrix should be tridiagonal. Write out the discretized equation for
i = 1, i and i = N − 1. Write the expressions for the diagonals, and the right
hand side.
b) Choose N = 4 (h = 14 ) in a) and solve the corresponding system of
equations.
c) We will now find ω(r) by integrating the equation dω dr = −φ(r) using
Heuns method. Since the right hand side is independenpt of ω , Heuns method
reduces to the trapes method. (The predictor is not necessary).
Find the ω− values in the points as in b).

d) Eq. (4.147) can also be written:

d 1 d
[r · φ(r)] (4.149)
dr r dr
Discretize Eq. (4.149) using Eq. (2.45). The coefficient matrix should be
tridiagonal. Write out the discretized equation for i = 1, i and i = N − 1. Write
the expressions for the diagonals, and the right hand side.
e) Solve Eq. (4.144) and (4.145) directly by introducing a new independant
variable z = ln(r) and show that Eq. (4.144) can be written: ω 000 (z) − 2ω 00 (z) =
8r2 ≡ 8e2z . Next guess a particular solution on the form ωp (z) = k · z · e2z ,
where k is a constant. Lastly decide the constants using (4.145).
Programing:
a) Write a python program that solves b) and c) from the pen and paper
exercise, numerically. Experiment with finer segmentation. If you want you can
download the python scelleton clampedPlate.py and fill in where applicable.
Hint 1. pen and paper:

142 48
b) Solutions: φ1 = 105 = 1.3524, φ2 = 35 = 1.3714, φ3 = 67 = 0.8571
c) Since ω0 is unknown, but ω4 = φ4 = 0 is known, first find ω3 , then ω2 , ω1
and ω0 . Use the φ-values obtained in a).
94
Solutions: ω0 = 105 = 0.8953, ω1 = 61 27
84 = 0.7262, ω2 = 70 = 0.3857,
3
ω2 = 28 = 0.1071
e) Solution given in Eq. (4.146)
Hint 2. programming:
# src-ch2/clampedPlate.py
import scipy
import scipy.linalg
import scipy.sparse

import numpy as np

LNWDT=2; FNT=15
def solve_phi(N, h):
i_vector = np.linspace(0, N, N + 1) # vector containing the indice number of all nodes

i_mid = i_vector[1:-1] # vector containing the indice number of all interior nodes nodes
# mainDiag = ?
# subDiag = ?
# superDiag = ?
# RHS = ?
# A = scipy.sparse.diags([?, ?, ?], [?, ?, ?], format=’csc’)

# Phi = scipy.sparse.linalg.spsolve(A, RHS)
# Phi = np.append(0, Phi)
# Phi = np.append(Phi, 0)
return Phi
def solve_omega(Phi, N, h):

Omega = np.zeros_like(Phi)
Omega[-1] = 0
#for i in range(N - 1, -1, -1):
# i takes the values N-2, N-3, .... , 1, 0
#Omega[i] = ?
return Omega
N = 3
r = np.linspace(0, 1, N + 1)
h = 1./N
Phi = solve_phi(N, h)
Omega = solve_omega(Phi, N, h)
Omega_analytic = r[1:]**2*(2*np.log(r[1:]) - 1) + 1
Omega_analytic = np.append(1, Omega_analytic) # log(0) not possible
Phi_analytic = - 4*r[1:]*np.log(r[1:])
Phi_analytic = np.append(0, Phi_analytic) # log(0) not possible
fig, ax = plt.subplots(2, 1, sharex=True, squeeze=False)
ax[0][0].plot(r, Phi, ’r’)

ax[0][0].plot(r, Phi_analytic, ’k--’)
ax[0][0].set_ylabel(r’$\phi$’)
ax[1][0].plot(r, Omega, ’r’)

ax[1][0].plot(r, Omega_analytic, ’k--’)
ax[1][0].legend([’numerical’, ’analytical’], frameon=False)
ax[1][0].set_ylabel(r’$\omega$’)
ax[1][0].set_xlabel(’r’)
plt.show()
Exercise 8: Heat conduction in a triangle of two different

materials
In this exercise we will look at a slightly more involved version of the trapezoidal
cooling rib in 4.4.2 which is illustrated in Figure 4.11. This cooling rib is
composed av a triangle (part a) and a trapezoidal part (part b) which are

made of materials of different heat properties. The purpose of this exercise is to
illustrated how discontinuties in heat properties may be treated. We will find that
that the temperature varies continuously along the rib, whereas the temperature
gradient will have a discontinuity at the junction of the two materials.
0.1m 0.1m
d D
Part a Part b
Figure 4.11: A triangle of two different materials.
In appendix B of Numeriske metoder the quasi one-dimensional differential

equation for heat conduction is derived:
dT
−dQx = P h [T (X) − T∞ ] dX where Qx = −kA
dX
As dX → 0 so will dQ → 0 and consequently ⇒ Q = constant which means
that at the cross-section between the two different materials we must satisfy the
following relation:

dT dT
Qa = Qb ⇒ ka Aa = kb Ab (4.150)
dX a dX b
The the junction the cross-sectional
areas
ofthe two materials are the same
dT ka dT
Aa = Ab and there fore = , which may be rendered on
dX b kb dX a
dimensionless form by using (4.76):

dθ ka dθ
= (4.151)
dx b kb dX a
Assume the following cooling rib geometry
L = 0.2m, D = 0.02m, d = 0.01m (4.152)

and let the thermodynamical properties be given by:
Leif Rune 1: ??????? the thermodynamical properties were lost
With these geometrical and thermodynamical properties equation junction
condition in equation (4.151) is transformed to:

dθ dθ
=4 (4.153)
dx b dx a
whereas the differential equation for quasi onedimensional heat conduction

remains:

d x dθ
− θ(x) = 0 (4.154)
dx β 2 dx
the expressions for β are:
2h̄ L2
β2 = (4.155)
Dk
which in our example corresponds to
βa2 = 2.0 and βb2 = 8.0 (4.156)

By rendering equation (4.154) in this way we satisfy the continuity condition
in equation (4.150).
The exercise is to solve the heat equation (4.154) numerically for the triangle
in Figure 4.11 with the geometry and thermodynamical parameters given above.
0.0 0.5 1.0

X
1 2 M-1 M M+1 N N+1
Figure 4.12: Spatial discretization for the example of heat conduction in a

triangle composed by two different materials with material interface at point M .
Hint. Referring to Figure 4.12, we use the following numbering:
xi = (i − 1) h , i = 1, 2, ..., N + 1 with h = N1
(4.157)
M = N2 + 1 with N being an even number and xM = 0.5
With k = ka for i = M we get that β = βa for i = 1, 2, ..., M and β = βb for

i > M . For i 6= M we can write (4.154)

d dθ
x − β 2 θ(x) = 0 (4.158)
dx dx
Discretizing (4.158) using (2.45) in Chapter (2):
−xi− 21 θi−1 + (xi+ 12 + xi− 21 + β 2 h2 ) θi − xi+ 21 θi+1 = 0
Noting that xi− 21 = (i − 32 ) h and xi+ 12 = (i − 21 ) h we can write:


3 2 1
i− θi−1 + [2(i − 1) + β h] θi − i − θi+1 = 0
2 2
or divided by i:
β2h

3 1 1
− 1− θi−1 + 2 1 − + θi − 1 − θi+1 = 0 (4.159)
2i i i 2i
(4.159) is used for i = 2, 3, ..., N , however, except for i = M = N/2 + 1.

For i = 1
From (4.80) we get the boundary condition for x = 0:
dθ −3θ1 + 4θ2 − θ3
(0) = βa2 θ(0) → = βa2 θ1 , (4.160)
dx 2h
where we have used forward differences (See 2.5).
This results in:
θ3 = 4θ2 − (3 + 2hβa2 ) θ1 (4.161)

Writing (4.159) for i = 2:
β2h

3 1 1
− 1− θ1 + 2 1 − + a θ2 − 1 − θ3
4 2 2 4
and considering (4.155) and (4.161) results in:

3h h
1+ θ1 − 1 − θ2 = 0 (4.162)
2 2
This is the first equation of our system.
For i = M.
0.5
M-1 M-½ M M+½ M+1
Figure 4.13: Spatial discretization for the example of heat conduction in a

triangle composed by two different materials with material interface at point M .
Detail of the discretization for the use of centered finite differences for point M ,
According to the discretization illustrated in Figure 4.13, here we use an

equation of the form given in (4.154), which discretized using (2.45) from Chapter
(2), results in:
2
βM − 12
= βa2 = 2.0 , βM
2
+ 21
= βb2 = 8.0
h (1 − h) h (1 + h)
xM − 12 = (N − 1) = , xM + 12 = (N + 1) =
2 2 2 2
which provides:
−4 (1 − h) θM −1 + [5 − 3h + 16h2 ] θM − (1 + h) θM +1 = 0 (4.163)
For i = N we can use (4.159) with β 2 = βb2 = 8.0:

3h 2 h
− 1− θN −1 + 2 [(1 − h) + 4h ] θN = 1 − (4.164)
2 2
Equations (4.162)–(4.164) constitute a linear system of equations with a
tridiagonal matrix and can be solved as such using the Thomas algorithm. In
addition to the temperature, we also want to calculate the temperature gradient
θ0 (x) . For x = 0 we use (4.80) and interface condition dx dθ dθ
(0.5+) ≡ dx b
found
from (4.153). The other values are computed using standard central differences,
besides for x = 0.5 og x = 1.0, where second order backward differences are used.
Let us look at the use of the differential equation as an alternative.
We integrate (4.158):
β 2 xi
Z
dθi
= θ(x)dx , i = 2, 3, ..., N + 1 (4.165)
dx xi x1
Rx
Integral I = x1i θ(x)dx can be calculated using the trapezium method, for
example . The computation of (4.165) is shown below in pseudo-code form:
θ10 := βa2 θ1
s := 0
Utfør for i := 2, ..., N + 1
x := h (i − 1)
s := s + 0.5h (θi + θi−1 )
βa2 s β2 s
Dersom (i ≤ M ) sett θi0 := ellers θi0 := b
x x
In the following table we have used (4.165) and the trapezium method. In
this case, the accuracy of the two methods for calculating θ0 (x) is quite similar
as both θ and θ0 are smooth functions (except at x = 0.5), but generally the
integration method will be more accurate if there are discontinuities..
Solution of equation (4.154) with h = 0.01
x θ(x) rel. feil θ0 (x) rel. feil

0.00 0.08007 1.371E-4 0.16014 1.312E-4
0.10 0.09690 1.341E-4 0.17670 1.302E-4
0.20 0.11545 1.386E-4 0.19438 1.286E-4
0.30 0.13582 1.252E-4 0.21324 1.360E-4
0.40 0.15814 1.265E-4 0.23333 1.329E-4
0.45 0.17007 1.294E-4 0.24387 1.312E-4
0.5- 0.18253 1.260E-4 0.25472 1.021E-4
0.5+ 0.18253 1.260E-4 1.01889 1.021E-4
0.55 0.23482 5.962E-5 1.07785 1.067E-4
0.60 0.29073 5.848E-5 1.16297 9.202E-5
0.70 0.41815 3.348E-5 1.39964 7.288E-5
0.80 0.57334 1.744E-5 1.71778 5.938E-5
0.90 0.76442 7.849E-6 2.11851 4.862E-5
1.00 1.00000 0.0 2.60867 1.526E-4
Chapter 5
Mathematical properties of
partial differential equations
5.1 Model equations

In Appendix 2 in the Numeriske Beregninger Navier-Stokes equations for an
incompressible fluid:
2
∂ u ∂2u

Du ∂u ∂u ∂u 1 ∂p
≡ +u +v =− +ν + 2 (5.1)
Dt ∂t ∂x ∂y ρ ∂x ∂x2 ∂y
2
∂2v

Dv ∂v ∂v ∂v 1 ∂p ∂ v
≡ +u +v =− +ν + 2 (5.2)
Dt ∂t ∂x ∂y ρ ∂y ∂x2 ∂y
2
∂2T

DT ∂T ∂T ∂T ∂ T
≡ +u +v =α + (5.3)
Dt ∂t ∂x ∂y ∂x2 ∂y 2
The left-hand side terms represent transport (advection/convection), while

the right-hand side stands for diffusive processes. It follows that transport is
expressed by first order terms and are, in this case, non-linear, while diffusive
terms are given by second order derivatives, which in this case are linear. Many
important engineering problems can be described by special cases of these
equations, such as boundary layer problems. Since such special cases are, of
course, easier to solve and analyze, these will be often used when we are going
to derive a numerical scheme. These are usually one-dimensional, non-stationary
cases.
5.1.1 List of some model equations

Poisson equation.
∂2u ∂2
2
+ 2 = f (x, y) (5.4)
∂x ∂y
217
CHAPTER 5. MATHEMATICAL PROPERTIES OF PDES 218
Laplace for f (x, y) = 0 Potential flow theory, stationary heat conduction, etc.
One-dimensional diffusion equation.
∂u ∂2u
=α 2 (5.5)
∂t ∂x
Wave equation. Fundamental equation for acoustics and other applications.
∂2u 2
2∂ u
= α0 (5.6)
∂t2 ∂x2
1. order linear advection equation.
∂u ∂u
+ α0 =0 (5.7)
∂t ∂x
Inviscid Burger’s equation. Model for Euler’s equation of gas dynamics.
∂u ∂u
+u =0 (5.8)
∂t ∂x
Burger’s equation. Model for incompressible Navier-Stokes equations.
∂u ∂u ∂2u
+u =ν 2 (5.9)
∂t ∂x ∂x
Tricomi equation. Model for transonic flow.
∂2u ∂2u
y + 2 =0 (5.10)
∂x2 ∂y
Convection-diffusion equation. Linear advection and diffusion.
∂u ∂u ∂2u
+ u0 =α 2 (5.11)
∂t ∂x ∂x
The expressions transport, convection and advection are equivalent. (5.7)
above is known as the advection equation.
5.2 First order partial differential equations

The left-hand side of the model equation, related to transport phenomena,
D()
consists of the term , which is a 1st order partial differential equation (PDE).
Dt
Let us look in detail at this term by considering the following equation:
∂u ∂u
a +b +c=0 (5.12)
∂x ∂y
If a or b are functions of x, y and u, the equation is quasi-linear. When a and b
are functions of x and/or y, or constant, the function is linear.
∂u
The equation is non-linear if a or b are functions of either u, and/or
∂x
∂u
. For example, the transport terms of the Navier-Stokes equations, i.e. the
∂y
left-hand side of (5.1), are quasi-linear 1st order PDEs, even if we normally call
them non-linear.
Writing (5.12) as:

∂u b ∂u
a + +c=0 (5.13)
∂x a ∂y
Assuming that u is continuously differentiable:
∂u ∂u du ∂u dy ∂u
du = dx + dy → = +
∂x ∂y dx ∂x dx ∂y
Defines the characteristic of (5.12) by:
dy b
= (5.14)
dx a
(5.14) inserted above provides:

∂u b ∂u du
a + =a
∂x a ∂y dx
which further provides:
a · du + c · dx = 0 (5.15)
dy
Along the characteristic curve defined by dx = ab , (5.12) reduces to an ODE
given by (5.15). (5.15) is called the compatibility equation for (5.12). If (5.12)
is linear, we can in principle first find the characteristic and then solve (5.15).
This is normally not possible when (5.12) is quasi-linear or non-linear because in
dy
that case dx is a function of u, ie: of the solution itself.
Example 5.2.1. .
The advection equation (5.7) given by ∂u ∂u
∂t + a0 ∂x has the solution u(x, t) =
f (x − a0 t), which is a wave propagating with constant profile and velocity along
x. The wave moves to the right for a0 > 0. The characteristic is given here by:
dx
dt = a0 .
This equation can be integrated directly, yielding: x = a0 t + C1 . Constant
C1 can be determined, for example, by noting that: x = x0 for t = 0, which
implies that C1 = x0 . The characteristic curve equation becomes x = a0 t + x0 .
(5.15) with c = 0 and a = 1 results in du = 0 . This means that u is here the
particle velocity, which is constant along the characteristic.
Let us see an example with initial conditions, choose:
(
1 − x2 for |x| ≤ 1
u(x, 0) =
0for |x| > 1
This is a parabola. From u(x, t) = f (x − a0 t) one has that u(x, 0) = f (x).

With ζ = x − a0 t, we can write the solution:
(
1 − ζ 2 for |ζ| ≤ 1
u(x, t) =
0 for |ζ| > 1
When we use ζ as variable, the solution is stationary: we follow the wave.
Figure 5.1 summarizes the example.
u
u(x,0) = 1 - x2 1
-1 1
x
t1
dx = a > 0
0
dt
t
Figure 5.1: Propagation of initial condition by the linear advection equation.
Example 5.2.2. .
The inviscid Burger’s equation (5.8) is given by:
∂u ∂u
+u =0
∂t ∂x
This has the solution u(x, t) = f (x − u · t) and characteristic curve given by:
dx
dt = u. This is the same solution as in the previous example, with the important
difference that now a0 is replaced with u. The slope of characteristics varies now
with x, as indicated in Figure 5.2.
The same initial condition as in the previous example provide the following
solution:
(
1 − ζ 2 for |ζ| ≤ 1
u(x, t) = der ζ = x − u · t
0 for |ζ| > 1
Again, we see that the only difference from the previous example is that a0
is now replaced by u. Explicit solution, noting that u(x, t) = 0 for |ζ| > 1:
t
dx = u
dt
Figure 5.2: Characteristic curves for the Burger’s equation (5.2.2) at different
locations along x.
1
u(x, t) = 2 [(2xt − 1) ± g(x, t)],
2t
p
g(x, t) = 1 − 4xt + 4t2 , |ζ| ≤ 1 (5.16)
∂u g(x, t) ± 1
= (5.17)
∂x t · g(x, t)
By this method of solution uniqueness is lost when characteristic intersect

(Figure 5.3). In this case one has that u(x, t) takes two possible values for
x ≥ 1, when ∂u ∂x (1, t) > 0. The critical value tcrit when this occur takes
place when ∂u∂x (1, t) → ∞, as one can conclude from (5.17), which implies that
g(1, t) = 0 = 1 − 2t → tcrit = 21 .
For t ≤ 21 one uses (5.16) and (5.17) with positive sign for g(x, t) for all
2
values of x. For t > 12 the critical value arises at x∗ = 1+4t
4t with u∗ = 1 − 4t12 .
∗
For −1 ≤ x ≤ x one can still use (5.16) and (5.17) with positive sign for g(x, t),
but for 1 ≤ x ≤ x∗ the lower part of the solution is given by using (5.16) and
(5.17) with negative sign in front of g(x, t). The maximum value umaks = 1 is
always given at x = t.
In order to obtain a unique solution, one must introduce the concept of a
moving discontinuity: in gas dynamics this is called a shock, in hydraulics a
hydraulic jump.
The characteristic curves shown in Figure 5.4 are given by:
x = (1 − x20 ) · t + x0 , der x = x0 for t = 0

3.0
2.5
2.0
1.5
t
1.0
0.5
0.0
−2 −1 0 1 2 3
x
Figure 5.3: Illustration of intersecting characteristic curves.
1.2
t=0
1.0 t = 12
t =1
0.8
0.6
u
0.4
0.2
0.0
−1.0 −0.5 0.0 0.5 1.0 1.5
x
Figure 5.4: Inviscid Burger’s equation solution. Note that solution for time
t = 1 is multiply. After tcrit additional mathematical notions have to be applied
to identify a unique solution.
5.3 Second order partial differenatial equations

A seond order PDE for two independent variables x and y can be written as:
∂2φ ∂2φ ∂2φ

A + B + C +f =0 (5.18)
∂x2 ∂x∂y ∂y 2
If A, B, C and f are functions of x, y, φ, ∂φ ∂φ
∂x and ∂y , (5.18) is said to be quasi-
linear. If A, B and C are only functions of x and y, (5.18) is semi-linear. If f is
only function of x and y, (5.18) is linear.
Example 5.3.1. .
∂ 2 φ ∂φ

∂φ
− − exy sin φ = 0 quasi-linear (5.19)
∂x ∂x2 ∂x
∂ 2 φ ∂φ
x 2 − − exy sin φ = 0 semi-linear (5.20)
∂x ∂x
2 2 2
∂ φ ∂ φ
= 0 non-linear (5.21)
∂x2 ∂y 2
A PDE that can be written in the following form:
∂2φ ∂2φ ∂2φ ∂φ ∂φ

A 2
+B +C 2 +D +E + Fφ + G = 0 (5.22)
∂x ∂x∂y ∂x ∂x ∂y
where A, B, C, D, E, F , and G are only functions of x and y, is a 2nd order linear
PDE. (5.22) is consequently a special case of (5.18). Notice that usually we will
use the term non-linear as opposed to linear. Let us now investigate whether
(5.18) has characteristics or not.
Consider a function φ such that u = ∂φ ∂φ
∂x and v = ∂y , which in turn implies
∂u ∂v
that ∂y = ∂x .
(5.18) can then be written as a system of two 1st order PDEs:
∂u ∂u ∂v
A +B +C +f =0 (5.23)
∂x ∂y ∂y
∂u ∂v
− =0 (5.24)
∂y ∂x
It can be shown that a higher order (quasi-linear) PDE can always be written
as a system of 1st order PDEs. Moreover, the resulting first order system
can have many forms. If, for example, (5.18) is the potential equation in gas
dynamics, we can denote φ as the potential speed. (5.24) is the condition of
irrotational flow.
We will now try to write (5.23) and (5.24) as a total differential (see (5.13) –
(5.15)). Multiply (5.24) with any scalar σ and add to (5.23):

∂u B + σ ∂u ∂v C ∂v
A + −σ − +f =0 (5.25)
∂x A ∂y ∂x σ ∂y
Now we have:
du ∂u dy ∂u dv ∂v dy ∂v
= + , = +
dx ∂x dx ∂y dx ∂x dx ∂y
Hence:
du ∂u ∂u dv ∂v ∂v
= +λ , = +λ (5.26)
dx ∂x ∂y dx ∂x ∂y
where we have defined λ as:

dy
λ= (5.27)
dx
By comparing (5.25) and (5.26):
dy B+σ C
=λ= =− (5.28)
dx A σ
(5.28) inserted in (5.25) gives the compatibility equation
du dv
A −σ +f =0 (5.29)
dx dx
If the characteristics are real, i.e. λ is real, we have transformed the original
PDE into an ODE given by (5.29) along the directions defined by (5.27).
From (5.28):
C
σ=− (5.30)
λ
which also gives:
B − Cλ
λ= (5.31)
A
The following 2nd order equation can be obtained from (5.31) to determine
λ:
Aλ2 − Bλ + C = 0 (5.32)
or by using (5.27):
A · (dy)2 − B · dy · dx + C · (dx)2 = 0 (5.33)

After λ is found from (5.32) and (5.33), σ can be found from (5.30) and
(5.31) so that the compatibility equation in (5.29) can be determined. Instead of
using (5.29), we can insert σ from (5.28) in (5.29) so that we get the following
compatibility equation:
du C dv
A + +f =0 (5.34)
dx λ dx
2nd degree equations (5.32) and (5.33) have the roots λ1 and λ2
√
B ± B 2 − 4AC
λ1,2 = (5.35)
2A
We have three possibilities for the roots in (5.35):
• B 2 − 4AC > 0 ⇒ λ1 and λ2 are real

• B 2 − 4AC < 0 ⇒ λ1 and λ2 are complex
• B 2 − 4AC = 0 ⇒ λ1 = λ2 and real
The quasi-linear PDE (5.18) is called
• Hyperbolic if B 2 − 4AC > 0: λ1 and λ are real
• Elliptic if B 2 − 4AC < 0: λ1 and λ are complex

• Parabolic if B 2 − 4AC = 0: λ1 = λ2 are real
The above terms come from the analogy with the cone cross-section equation
of the expression Ax2 + Bxy + Cy 2 + Dx + F = 0. For example, x2 − y 2 = 1
represents a hyperbola if B 2 − 4AC > 0.
The roots λ1 and λ2 are characteristics. Hence:
• Hyperbolic: Two real characteristics.

• Elliptic: No real characteristics.
• Parabolic: One real characteristic.
Example 5.3.2. Examples of classification of various PDEs.

∂2u ∂2u
The wave equation − 2 = 0 is hyperbolic since B 2 − 4AC = 4 > 0
∂x2 ∂y
dy
Characteristics are given by λ2 = 1 → = ±1
dx
2 2
∂ u ∂ u
Laplace equation + 2 = 0 is elliptic since B 2 − 4AC = −4 < 0
∂x2 ∂y
∂u ∂2u
The diffusion equation = is parabolic since B 2 − 4AC = 0
∂y ∂x2
Linearisert potensial-ligning for kompressibel strømning:
∂2φ ∂2φ
(1 − M 2 ) + 2 = 0, M = Mach-number. (5.36)
∂x2 ∂y
• M = 1: Parabolic (degenerate).
• M < 1: Elliptic. Subsonic flow.
• M > 1: Hyperbolic. Supersonic flow.

5.4 Boundary conditions for 2nd order PDEs
∂2u ∂2u
+ 2 = 0 : Elliptic . No real characteristics. (5.37)
∂x2 ∂y
∂ u ∂2u
2
− 2 = 0 : Hyperbolic. Two rel characteristics. (5.38)
∂x2 ∂y
∂u ∂2u
= : Parabolic. One real characteristic. (5.39)
∂y ∂x2
The model equations shown above differ by the number of real characteristics.
This directly affects the boundary conditions that are possible in the three cases,
as physical information propagates along characteristics.
5.4.1 Hyberbolic equations

As example we use the wave equation:
∂2u 2
2∂ u
= α 0 (5.40)
∂t2 ∂x2
dx
(5.40) has the characteristic = ±a0 , where a0 is the wave propagation
dt
dx dx
speed. We denote = +a0 with C + and = −a0 with C − . The area of
dt dt
influence for (5.40) is shown in Figure 5.5.
Open boundary Marching direction

t
Region of
influence
C+ C-
Region of
dependency
a b x
Figure 5.5: Regions of dependence and influence with respect to point P for
the wave equation(5.40).
The solution at point P depends only on the solution in the region of depen-
dence, while the value at P only affects the solution within the region of influence.
Boundary values for domain borders can not be prescribed independently of
initial conditions at t = 0. In order to solve this equation as an initial value

∂u
problem, (5.40) must also have an initial condition for at t = 0.
∂t
5.4.2 Elliptic equations

Model equation:
∂2u ∂2u
+ 2 =0 (5.41)
∂x2 ∂y
The solution domain is shown in Figure 5.6.
y
n
C
Region of influence
and region of dependency
Figure 5.6: Regions of dependence and influence for the model elliptic equation
(5.41).
We have no real characteristics. The entire domain Ω, including its boundary

C, coincide with the regions of dependence and influence for P : Any change of
a value in Ω or C will affect the solution in P . (5.41) is a purely boundary value
problem and the following boundary conditions are admissible:
• u is prescribed at C: Dirichlet boundary condition.

∂u
• is prescribed at C: Neumann boundary condition.
∂n
∂u
• a weighted combination of u and are prescribed at C: Robin boundary
∂n
condition.
• a combination of the above conditions in different portions of C.
5.4.3 Parabolic equations

Model equation:
∂u ∂2u
= (5.42)
∂t ∂x2
The solution domain for (5.42) is shown in Figure 5.7.
Open boundary Marching direction

t
Region of influence
P
Characteristic
Region of dependency
a b x
Figure 5.7: Regions of dependence and influence for the model parabolic
equation (5.42).
From the classification equations (5.33) we find that:

dt = 0 ⇒ t = constant is the characteristic curve in this case. Hence:
a0 = dxdt = ∞, which means that the propagation velocity along characteristic
t = constant is infinitely large. The solution at P depends on the value in all
points in the physical space for past time t, including the present. (5.42) behaves
like an elliptical equation for each value of t = constant.
15
Chapter 6
Elliptic partial differential

equations
6.1 Introduction
Important and frequently occuring practical problems are governed by elliptic
PDEs, including steady-state temperature distribution in solids.
Famous elliptic PDEs.

Some famous elliptic PDEs are better know by their given names:
• Laplace:
∇2 u = 0 (6.1)
• Poisson:
∇2 u = q (6.2)
• Helmholtz:
∇2 u + c · u = q (6.3)
∂2 ∂2 ∂2
where ∇2 = + + in 3D
∂x2 ∂y 2 ∂z 2
229
CHAPTER 6. ELLIPTIC PDES 230
and
∂2 ∂2
where ∇2 = + in 2D
∂x2 ∂y 2
The 2D versions of the famous equations above are special cases of the a
generic elliptic PDE may be represented by:

∂ ∂u ∂ ∂u
a + b +c·u=q (6.4)
∂x ∂x ∂y ∂y
where a, b, c og q may be functions of x and y and a and b have the same sign.
• Transient heat conduction is governed by:
∂2T ∂2T ∂2T

∂T
=α + + (6.5)
∂t ∂x2 ∂y 2 ∂z 2

∂T
which is a parabolic PDE, but at steady state when = 0 , we get:
∂t
2
∂2T ∂2T

∂ T
+ + =0 (6.6)
∂x2 ∂y 2 ∂z 2
which is an elliptic PDE.
From the classification approach in Chapter 5.3 (see also Classification of
linear 2nd order PDEs) we see that (6.1) to (6.3) are elliptic, meaning no real
characteristics. Therefore, elliptic PDEs must be treated as boundary value
problems (see 5.4) for a discussion about the conditions for a well posed problem).
For the solution of a generic elliptic PDE like (6.4), in a domain in two-
dimensions bounded by the boundary curve C (6.1), we reiterate the most
common boundary value conditions from 5.4:
• Dirichlet-condition: u(x, y) = G1 (x, y) on the boundary cure C
∂u
• Neumann-condition: = G2 (x, y) on C
∂n
∂u(x, y)
• Robin-condition: a · u(x, y) + b · = G3 (x, y) on C
∂n
For the Neuman-problem, at least one value of u(x, y) must be specified on

C for the solution to be unique. Additionally, the contour integral of G2 (x, y)
on C must vanish.
Several physical problems may be modeled by the Poisson equation (6.2)
Possion problems.
• The stress equation for torsion of an elastic rod
• The displacement of an membrane under constant pressure.
C
n
u(x,y)
Figure 6.1: A solution domain in two-dimensions bounded by a boundary curve

C.
• Stokes flow (low Reynolds number flow)

• Numerous analytical solutions (also in polar coordinantes) may be
found in [15] section 3-3.
6.2 Finite differences. Notation

In Section 2.5 we used Taylor expansions to discretize differential equations for
functions of one independent variable. A similar procedure can be applied to
functions of more than one independent variable In particular, if the expressions
to be discretized have no cross-derivatives, we can apply the formulas derived in
section 2.5, keeping the index for the second variable constant.
Since we are dealing with elliptic equations, we will, for convenience, consider
x and y as the independent variables. Moreover, its discretization is given by:
xi = x0 + i · ∆x, i = 0, 1, 2, . . .
yj = y0 + j · ∆y, j = 0, 1, 2, . . .
As before, we assume that ∆x and ∆y are constant, unless otherwise specified.

Figure 6.2 shows how we discretize a 2D domain using a Cartesian grid.
We now recall several finite difference formulas from Section 2.5.
Forward difference:

∂u ui+1,j − ui,j
= + O(∆x) (6.7)
∂x i,j ∆x
i-1 i i+1
j+1
y
i, j
j
Δy
j-1
x
Δx
Figure 6.2: Cartesian grid discretization of a 2D domain.
Backward difference:

∂u ui,j − ui−1,j
= + O(∆x) (6.8)
∂x i,j ∆x

∂u ui+1,j − ui−1,j
+ O (∆x)2

= (6.9)
∂x i,j 2∆x
∂ 2 u

ui+1,j − 2ui,j + ui−1,j
+ O (∆x)2

2
= 2
(6.10)
∂x i,j (∆x)
∂u ∂2u
Similar formulas for ∂y and ∂y 2 follow directly:
Forward difference:

∂u ui,j+1 − ui,j
= (6.11)
∂y i,j
∆y

∂u ui,j − ui,j−1
= (6.12)
∂y i,j
∆y

∂u ui,j+1 − ui,j−1
= (6.13)
∂y i,j 2∆y
∂ 2 u

ui,j+1 − 2ui,j + ui,j−1
= (6.14)
∂y 2 i,j (∆y)2
6.2.1 Example: Discretization of the Laplace equation

2 2
We discretize the Laplace equation ∂∂yu2 + ∂∂yu2 = 0 by using (6.10) and (6.14) and
setting ∆x = ∆y.
The resulting discrete version of the original equations is:
ui+1,j + ui−1,j + ui,j−1 + ui,j+1 − 4ui,j = 0

The resulting numerical stencil is shown in Figure 6.3.
j+1
j-1
i-1 i i+1
Figure 6.3: 5-point numerical stencil for the discretization of Laplace equations
using central differences.
It is easy to note that in (6.2.1), the value for the central point is the mean
of the values of surrounding points. Equation (6.2.1) is very well-known and is
usually called the 5-point formula (used in Chapter (6) ).
A problem arises at boundaries when using central differences. At those
locations the stencil might fall outside of the computational domain and special
strategies must be adopted. For example, one can replace central differences
with backward or forward differences, depending on the case. Here we report
some of these formulas with 2nd order accuracy.
Forward difference:

∂u −3ui,j + 4ui+1,j − ui+2,j
= (6.15)
∂x i,j
2∆x

∂u 3ui,j − 4ui−1,j + ui−2,j
= (6.16)
∂x i,j 2∆x
The formulas for 2.5 , 2.5 and 2.5 differences given in Chapter 2 can be used
now if equipped with two indexes. A rich collection of difference formulas can
be found in Anderson [14]. Detailed derivations are given in Hirsch [7].
6.3 Direct numerical solution

In situations where the elliptic PDE correspons to the stationary solution of a
parabolic problem (6.5), one may naturally solve the parabolic equation until
stationary conditions occurs. Normally, this will be time consuming task and
one may encounter limitations to ensure a stable solution. By disregarding such
a timestepping approach one does not have to worry about stability. Apart from
seeking a fast solution, we are also looking for schemes with efficient storage
management a reasonable programming effort.
Let us start by discretizing the stationary heat equation in a rectangular
plated with dimension as given in Figure 6.4:
∂2T ∂2T
+ =0 (6.17)
∂x2 ∂y 2
y
100 100 100 100
100
T1,5 T2,5 T3,5

0.0 0.0
T1,4 T2,4 T3,4

0.0 0.0
T1,3 T2,3 T3,3

1.5 0.0 0.0
T1,2 T2,2 T3,2

0.0 0.0
T1,1 T2,1 T3,1

0.0 0.0
0.0 0.0 0.0 0.0 0.0 x
1.0
Figure 6.4: Rectangular domain with prescribed values at the boundaries

(Dirichlet).
We adopt the following notation:
xi = x0 + i · h, i = 0, 1, 2, . . .
yj = y0 + j · h, j = 0, 1, 2, . . .
For convenience we assume ∆x = ∆y = h. The ordering of the unkown

temperatures is illsustrated in (6.5).
By approximation the second order differentials in (6.17) by central differences
we get the following numerical stencil:
Ti+1,j + Ti−1,j + Ti,j+1 + Ti,j−1 − 4Ti,j = 0 (6.18)

j+1
j-1
i-1 i i+1
Figure 6.5: Illustration of the numerical stencil.
which states that the temperature Ti,j in at the location (i, j) depends on the
values of its neighbors to the left, right, up and down. Frequently, the neighbors
are denoted in compass notation, i.e. west = i − 1, east = i + 1, south = j − 1,
and north = j + 1. By referring to the compass directions with their first letters,
and equivalent representation of the stencil in (6.18) reads:
Te + Tw + Tn + Ts − 4Tm = 0 (6.19)
Tn
Tw Tm Te
Ts
Figure 6.6: Illustration of the numerical stencil with compass notation.
The smoothing nature of elliptic problems may be seen even more clearly by
isolating the Ti,j in (6.19) on the left hand side:
Te + Tw + Tn + Ts
Tm = (6.20)
4
showing that the temperature Tm in each point is the average temperature of
the neighbors (to the east, west, north, and south).
The temperature is prescribed at the bondaries (i.e. Dirichlet boundary
conditions) and are given by:
T = 0.0 at y = 0
T = 0.0 at x = 0 and x=1 for 0 ≤ y < 1.5 (6.21)
T = 100.0 at y = 1.5
Our mission is now to find the temperature distribution over the plate by
using (6.18) and (6.21) with ∆x = ∆y = 0.25. In each discretized point in (6.5)
the temperatures need to satisfy (6.18), meaning that we have to satisfy as many
equations as we have unknown temperatures. As the temperatures in each point
depends on their neighbors, we end up with a system of algebraic equations.
To set up the system of equations we traverse our our unknows one by one
in a systematic manner and make use of use of (6.18) and (6.21) in each. All
unknown temperatures close to any of the boundaries (left, right, top, bottom)
in Figure 6.4 will be influenced by the prescribed and known temperatures the
wall via the 5-point stencil (6.18). Prescribed values do not have to be calculated
an can therefore be moved to the right hand side of the equation, and by doing
so we modify the numerical stencil in that specific discretized point. In fact,
inspection of Figure 6.4, reveals that only three unknown temperatures are not
explicitly influenced by the presences of the wall (T2,2 , T2,3 , and T2,3 ). The four
temperatures in the corners (T1,1 ,T1,5 , T3,1 , and T3,5 ) have two precribed values
to be accounted for on the right hand side of their specific version of the generic
numerical stencil (6.18). All other unknown temperatures close to the wall have
only one prescribed value to be accounted for in their specific numerical stenscil.
By starting at the lower left corner and traversing in the y-direction first,
and subsequently in the x-direction we get the following system of equations:
−4 · T11 + T12 + T21 = 0

T11 − 4 · T12 + T13 + T22 = 0
T12 − 4 · T13 + T14 + T23 = 0
T13 − 4 · T14 + T15 + T24 = 0
T14 − 4 · T15 + T25 = −100
T11 − 4 · T21 + T22 + T31 = 0
T12 + T21 − 4 · T22 + T23 + T32 = 0
T13 + T22 − 4 · T23 + T24 + T33 = 0 (6.22)
T14 + T23 − 4 · T24 + T25 + T34 = 0
T15 + T24 − 4 · T25 + T35 = 100
T21 − 4 · T31 + T32 = 0,
T22 + T31 − 4 · T32 + T33 = 0
T23 + T32 − 4 · T33 + T34 = 0
T24 + T33 − 4 · T34 + T35 = 0
T25 + T34 − 4 · T35 = −100
The equations in (6.22) represent a linear, algebraic system of equations with

5 × 3 = 15 unknowns, which has the more convenient and condensed symbolic
representation:
A·T=b (6.23)
where A denotes the coefficient matrix, T holds the unknown tempeartures,

and b the prescribed boundary temperatures. Notice that the structure of the
coefficient matrix A is completely dictated by the way the unknown temperatures
are ordered. The non-zero elements in the coefficient matrix are markers for
which unknow temperatures are coupled with each other. Below we will show an
example where we order the temperatures in y-direction first and then x-direction.
In this case, the components of the coefficient matrix A and the temperature
vector T are given by:
    
−4 1 0 0 0 1 0 0 0 0 0 0 0 0 0 T11 0

 1 −4 1 0 0 0 1 0 0 0 0 0 0 0 0 
 T12  
  0 


 0 1 −4 1 0 0 0 1 0 0 0 0 0 0 0 
 T13  
  0 


 0 0 1 −4 1 0 0 0 1 0 0 0 0 0 0 
 T14  
  0 


 0 0 0 1 −4 0 0 0 0 1 0 0 0 0 0 
 T15  
  −100 


 1 0 0 0 0 −4 1 0 0 0 1 0 0 0 0 
 T21  
  0 


 0 1 0 0 0 1 −4 1 0 0 0 1 0 0 0 
 T22  
  0 


 0 0 1 0 0 0 1 −4 1 0 0 0 1 0 0 ·
 T23 =
  0 


 0 0 0 1 0 0 0 1 −4 1 0 0 0 1 0 
 T24  
  0 


 0 0 0 0 1 0 0 0 1 −4 0 0 0 0 1 
 T25  
  −100 


 0 0 0 0 0 1 0 0 0 0 −4 1 0 0 0 
 T31  
  0 


 0 0 0 0 0 0 1 0 0 0 1 −4 1 0 0 
 T32  
  0 


 0 0 0 0 0 0 0 1 0 0 0 1 −4 1 0 
 T33  
  0 

 0 0 0 0 0 0 0 0 1 0 0 0 1 −4 1  T34   0 
0 0 0 0 0 0 0 0 0 1 0 0 0 1 −4 T35 −100
(6.24)
The analytical solution of (6.17) and (6.21) may be found to be:
∞
X
T (x, y) = 100 · An sinh(λn y) · sin(λn x) (6.25)
n=1
4
where λn = π · n and An =
λn sinh( 23 λn )
The analytical solution of the temperature field T (x, y) in (6.25) may be

proven to be symmetric around x = 0.5 (see 9).
We immediately realize that it may be inefficient to solve (6.24) as it is,
due to the presence of all the zero-elements. The coefficient matrix A has
15 × 15 = 225 elementes, out of which on 59 are non-zero. Further, a symmetric,
band structure of A is evident from (6.24). Clearly, these properties may be
exploited to construct an efficent scheme which does not need to store all non-zero
elements of A.
SciPy offers a sparse matrix package scipy.sparse, which has been used
previously (see section 4.4.1).
# src-ch7/laplace_Diriclhet1.py
import numpy as np
import scipy
import scipy.linalg
import scipy.sparse
import matplotlib; matplotlib.use(’Qt4Agg’)
import time

LNWDT=2; FNT=15
n = 15
d = np.ones(n) # diagonals
b = np.zeros(n) #RHS
d0 = d*-4
d1 = d[0:-1]
d5 = d[0:10]
A = scipy.sparse.diags([d0, d1, d1, d5, d5], [0, 1, -1, 5, -5], format=’csc’)
#alternatively (scalar broadcasting version:)

#A = scipy.sparse.diags([1, 1, -4, 1, 1], [-5, -1, 0, 1, 5], shape=(15, 15)).toarray()
# update A matrix
A[4, 5], A[5, 4], A[10, 9], A[9, 10] = 0, 0, 0, 0
# update RHS:
b[4], b[9], b[14] = -100, -100, -100
#print A.toarray()
tic=time.clock()
theta = scipy.sparse.linalg.spsolve(A,b) #theta=sc.linalg.solve_triangular(A,d)
toc=time.clock()
tic=time.clock()
theta2=scipy.linalg.solve(A.toarray(),b)
toc=time.clock()
# surfaceplot:
x = np.linspace(0, 1, 5)
y = np.linspace(0, 1.5, 7)
X, Y = np.meshgrid(x, y)
T = np.zeros_like(X)
T[-1,:] = 100
for n in range(1,6):
T[n,1] = theta[n-1]
T[n,2] = theta[n+5-1]
T[n,3] = theta[n+10-1]
from mpl_toolkits.mplot3d import Axes3D

from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter

import numpy as np
fig = plt.figure()
ax = fig.gca(projection=’3d’)
surf = ax.plot_surface(X, Y, T, rstride=1, cstride=1, cmap=cm.coolwarm,
linewidth=0, antialiased=False)
ax.set_zlim(0, 110)
ax.zaxis.set_major_locator(LinearLocator(10))
ax.zaxis.set_major_formatter(FormatStrFormatter(’%.02f’))
ax.set_xlabel(’x’)
ax.set_ylabel(’y’)
ax.set_zlabel(’T [$^o$C]’)
ax.set_xticks(x)
ax.set_yticks(y)
fig.colorbar(surf, shrink=0.5, aspect=5)

plt.show()
A somewhat more complicated solution of (6.17) may be found by specifying

the differente temperatures on all four boundaries. However, the code structure
follows the same way of reasoning as for the previous example:
import numpy as np
import scipy
import scipy.linalg
import scipy.sparse
import time
LNWDT=2; FNT=15
# Set temperature at the top
Ttop=100
Tbottom=10
Tleft=10.0
Tright=10.0
xmax=1.0
ymax=1.5

#need hx=(1/nx)=hy=(1.5/ny)
Nx = 20
h=xmax/Nx
Ny = int(ymax/h)
nx = Nx-1
ny = Ny-1
n = (nx)*(ny) #number of unknowns
print n, nx, ny
d0 = d*-4
d1 = d[0:-1]
d5 = d[0:-ny]
A = scipy.sparse.diags([d0, d1, d1, d5, d5], [0, 1, -1, ny, -ny], format=’csc’)
#alternatively (scalar broadcasting version:)

#A = scipy.sparse.diags([1, 1, -4, 1, 1], [-5, -1, 0, 1, 5], shape=(15, 15)).toarray()
# set elements to zero in A matrix where BC are imposed

for k in range(1,nx):
j = k*(ny)
i = j - 1
A[i, j], A[j, i] = 0, 0
b[i] = -Ttop
b[-ny:]+=-Tright #set the last ny elements to -Tright

b[-1]+=-Ttop #set the last element to -Ttop
b[0:ny-1]+=-Tleft #set the first ny elements to -Tleft
b[0::ny]+=-Tbottom #set every ny-th element to -Tbottom
tic=time.clock()
theta = scipy.sparse.linalg.spsolve(A,b) #theta=sc.linalg.solve_triangular(A,d)
toc=time.clock()
tic=time.clock()
theta2=scipy.linalg.solve(A.toarray(),b)
toc=time.clock()
# surfaceplot:
x = np.linspace(0, xmax, Nx + 1)
y = np.linspace(0, ymax, Ny + 1)
# set the imposed boudary values

T[-1,:] = Ttop
T[0,:] = Tbottom
T[:,0] = Tleft
T[:,-1] = Tright
for j in range(1,ny+1):
for i in range(1, nx + 1):
T[j, i] = theta[j + (i-1)*ny - 1]
from mpl_toolkits.mplot3d import Axes3D

from matplotlib import cm
from matplotlib.ticker import LinearLocator, FormatStrFormatter
fig = plt.figure()
ax = fig.gca(projection=’3d’)
surf = ax.plot_surface(X, Y, T, rstride=1, cstride=1, cmap=cm.coolwarm,

linewidth=0, antialiased=False)
ax.set_zlim(0, Ttop+10)
ax.set_xlabel(’x’)
ax.set_ylabel(’y’)
ax.set_zlabel(’T [$^o$C]’)
nx=4
xticks=np.linspace(0.0,xmax,nx+1)
ax.set_xticks(xticks)
ny=8
yticks=np.linspace(0.0,ymax,ny+1)
ax.set_yticks(yticks)
nTicks=5
dT=int(Ttop/nTicks)
Tticklist=range(0,Ttop+1,dT)
ax.set_zticks(Tticklist)
#fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()
Numerical values are listed below with analytical values from (6.25) enclosed
in paranthesis.
T11 = T31 = 1.578 (1.406), T12 = T32 = 4.092 (3.725)

T13 = T33 = 9.057 (8.483), T14 = T34 = 19.620 (18.945)
T15 = T35 = 43.193 (43.483), T21 = 2.222 (1.987), T22 = 5.731 (5.261)
T23 = 12.518 (11.924), T24 = 26.228 (26.049), T25 = 53.154 (54.449)
The structure of the coefficient matrix will not necessarily be so regular in

all cases, e.g. more complicated operators than the Laplace-operator or even
more so for non-linear problems. Even though the matrix will predominantly be
sparse also for these problems, the requirements for fast solutions and efficent
storage will be harder to obtain for the problems. For such problems iterative
methods are appealing, as they often are relatively simple to program and hoffer
effective memory management. However, they are often hapered by convergence
challanges. We will look into iterative methods in a later section.
6.3.1 Neumann boundary conditions

Here we consider a heat conduction problem where we prescribe homogeneous
Neuman boundary conditions, i.e. zero derivatives, at x = 0 and y = 0, as
illustrated in figure 6.7.
Mathematically the problem in Figure 6.7, is specified with the governing
equation (6.17) which we reiterate for convenience:
∂2T ∂2T
2
+ =0 (6.26)
∂x ∂y 2
y
T = 1.0
1.0
∂T
=0 ∇ 2T = 0 T = 0.0
∂x
0.0 ∂T 1.0 x
=0
∂y
Figure 6.7: Laplace-equation for a rectangular domain with homogeneous

Neumann boundary conditions for x = 0 and y = 0.
with homogeneous Neumann boundary conditions at:

∂T
=0 for x = 0, and 0 < y < 1 (6.27)
∂x
∂T
=0 for y = 0, and 0 < x < 1 (6.28)
∂y
and prescribed values (Dirichlet boundary conditions) for:
T =1 for y = 1, and 0 ≤ x ≤ 1
T =0 for x = 1, and 0 ≤ y < 1
By assuming ∆x = ∆y, as for the Dirichlet problem, an identical, generic

difference equation is obtained as in (6.18). However, at the boundaries we need
to take into account the Neumann boundary conditions. We will take a closer
∂T
look at = 0 for x = 0. The other Neumann boundary condition is treated in
∂x
the same manner. Notice that we have discontinuities in the corners (1, 0) og
(1, 1), additionally the corner (0, 0) may cause problems too.
∂T
A central difference approximation (see Figure 6.8) of = 0 at i = 0 yields:
∂x
T1,j − T−1,j
= 0 → T−1,j = T1,j (6.29)
2∆x
where we have introduced ghostcells with negative indices outside the physical
domain to express the derivative at the boundary. Note that the requirement
of a zero derivative relate the value of the ghostcells to the values inside the
physical domain.
∂T
One might also approximate = 0 by forward differences (see (6.15)) for a
∂x
generic discrete point (i, j):
j+1
T0,j T1,j
T-1,j T2,j
j-1
i = -1 i=0 i=1 i=2 x
Figure 6.8: Illustration of how ghostcells with negative indices may be used to
implement Neumann boundary conditions.

∂T −3Ti,j + 4Ti+1,j − Ti+2,j
=
∂x i,j 2∆x
∂T
which for = 0 for x = 0 reduce to:
∂x
4T1,j − T2,j
T0,j = (6.30)
3
For the problem at hand, illustrated in Figure 6.7, central difference ap-
proximation (6.29) and forward difference approximation (6.30) yields fairly
similar results, but (6.30) results in a shorter code when the methods in 6.4 are
employed.
To solve the problem in Figure 6.7 we employ the numerical stencil (6.19)
for the unknows in the field (not influenced by the boundaries)
Tw + Te + Ts + Tn − 4Tm = 0 (6.31)
where we used the same ordering as given in Figure 6.9. For the boundary
conditions we have chosen to implement the by means of (6.29) which is illustrated
in Figure 6.9 by the dashed lines.
Before setting up the complete equation system it normally pays of to look
at the boundaries, like the lowermost boundary, which by systematic usage of
(6.31) along with the boundary conditions in (6.28) yield:
y
1.0 1.0 1.0 1.0 1.0
T14 T13 T14 T15 T16 0.0
T10 T9 T10 T11 T12 0.0
T6 T5 T6 T7 T8 0.0
T2 T1 T2 T3 T4 0.0
x
T5 T6 T7 T8
Figure 6.9: Von Neumann boundary conditions with ghost cells.
2T2 + 2T5 − 4T1 = 0

T1 + 2T6 + T3 − 4T2 = 0
T2 + 2T7 + T4 − 4T3 = 0
T3 + 2T8 − 4T4 = 0
The equations for the upper boundary become:
T9 + 2T14 − 4T13 = −1
T10 + T13 + T15 − 4T14 = −1
T11 + T14 + T16 − 4T15 = −1
T12 + T15 − 4T16 = −1
Notice that for the coarse mesh with only 16 unknown temperatures (see
Figure 6.7) only 4 (T6 , T7 , T10 , and T11 ) are not explicitly influenced by the
boundaries.Finally, the complete discretized equation system for the problem

may be represented:
    
−4 2 0 0 2 0 0 0 0 0 0 0 0 0 0 0 T1 0

 1 −4 1 0 0 2 0 0 0 0 0 0 0 0 0 0 
 T2  
  0 


 0 1 −4 1 0 0 2 0 0 0 0 0 0 0 0 0 
 T3  
  0 


 0 0 1 −4 0 0 0 2 0 0 0 0 0 0 0 0 
 T4  
  0 


 1 0 0 0 −4 2 0 0 1 0 0 0 0 0 0 0 
 T5  
  0 


 0 1 0 0 1 −4 1 0 0 1 0 0 0 0 0 0 
 T6  
  0 


 0 0 1 0 0 1 −4 1 0 0 1 0 0 0 0 0 
 T7  
  0 


 0 0 0 1 0 0 1 −4 0 0 0 1 0 0 0 0 
· T8  
= 0 


 0 0 0 0 1 0 0 0 −4 2 0 0 1 0 0 0 
 T9  
  0 


 0 0 0 0 0 1 0 0 1 −4 1 0 0 1 0 0 
 T10  
  0 


 0 0 0 0 0 0 1 0 0 1 −4 1 0 0 1 0 
 T11  
  0 


 0 0 0 0 0 0 0 1 0 0 1 −4 0 0 0 1 
 T12  
  0 


 0 0 0 0 0 0 0 0 1 0 0 0 −4 2 0 0 
 T13  
  −1 


 0 0 0 0 0 0 0 0 0 1 0 0 1 −4 1 0 
 T14  
  −1 

 0 0 0 0 0 0 0 0 0 0 1 0 0 1 −4 1  T15   −1 
0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 −4 T16 −1
(6.32)
# src-ch7/laplace_Neumann.py; Visualization.py @ git@lrhgit/tkt4140/src/src-ch7/Visualization.py;
import numpy as np
import scipy
import scipy.linalg
import scipy.sparse
import time
LNWDT=2; FNT=15
def setup_LaplaceNeumann_xy(Ttop, Tright, nx, ny):

""" Function that returns A matrix and b vector of the laplace Neumann heat problem A*T=b using
and assuming dx=dy, based on numbering with respect to x-dir, e.g:
1 1 1 -4 2 2 0 0 0 0
T6 T5 T6 0 1 -4 0 2 0 0 0
T4 T3 T4 0 1 0 -4 2 1 0 0
T2 T1 T2 0 --> A = 0 1 1 -4 0 1 ,b = 0
T3 T4 0 0 1 0 -4 2 -1
0 0 0 1 1 -4 -1
T = [T1, T2, T3, T4, T5, T6]^T
Args:
nx(int): number of elements in each row in the grid, nx=2 in the example above
ny(int): number of elements in each column in the grid, ny=3 in the example above
Returns:
A(matrix): Sparse matrix A, in the equation A*T = b
b(array): RHS, of the equation A*t = b
"""
d0 = d.copy()*-4
d1_lower = d.copy()[0:-1]
d1_upper = d1_lower.copy()
dnx_lower = d.copy()[0:-nx]
dnx_upper = dnx_lower.copy()
d1_lower[nx-1::nx] = 0 # every nx element on first diagonal is zero; starting from the nx-th elem
d1_upper[nx-1::nx] = 0
d1_upper[::nx] = 2 # every nx element on first upper diagonal is two; stating from the first elem
# this correspond to all equations on border (x=0, y)
dnx_upper[0:nx] = 2 # the first nx elements in the nx-th upper diagonal is two;
# This correspond to all equations on border (x, y=0)
b[-nx:] = -Ttop
b[nx-1::nx] += -Tright
A = scipy.sparse.diags([d0, d1_upper, d1_lower, dnx_upper, dnx_lower], [0, 1, -1, nx, -nx], forma
return A, b
if __name__ == ’__main__’:
from Visualization import plot_SurfaceNeumann_xy
# Main program
Ttop=1
Tright = 0.0
xmax=1.0
ymax=1.

Nx = 10
h=xmax/Nx
Ny = int(ymax/h)
A, b = setup_LaplaceNeumann_xy(Ttop, Tright, Nx, Ny)
Temp = scipy.sparse.linalg.spsolve(A, b)
plot_SurfaceNeumann_xy(Temp, Ttop, Tright, xmax, ymax, Nx, Ny)

# figfile=’LaPlace_vNeumann.png’
# plt.savefig(figfile, format=’png’,transparent=True)
plt.show()
The analytical solution is given by:

∞
X
T (x, y) = An cosh(λn y) · cos(λn x), (6.33)
n=1
π (−1)n−1
der λn = (2n − 1) · , An = 2 , n = 1, 2, . . . (6.34)
2 λn cosh(λn )
The solution for the problem illustrated Figure 6.7 is computed and visualized
the the python code above. The solution is illustrated in Figure 6.10.
Marit 2: Har ikke endret figuren
Figure 6.10: Solution of the Laplace equation with Neuman boundary conditions.
6.4 Iterative methods for linear algebraic equa-

tion systems
We will in this section seek to illustrate how classical iterative methods for linear
algebraic systems of equations, such as Jacobi, Gauss-Seidel or SOR, may be
applied for the numerical solution of linear, ellipitical PDEs, whereas criteria
for convergence of such iterative schemes can be seen in Section 7.3.2 of the
Discretizations of PDEs, in particular linear elliptic PDEs, will result in
a system of linear, algebraic equations on the form A · x = b, which may be
presented on component for a system of three equations as:
a11 x1 + a12 x2 + a13 x3 = b1 (6.35)
a21 x1 + a22 x2 + a23 x3 = b2 (6.36)
a31 x1 + a32 x2 + a33 x3 = b3 (6.37)
We rewrite (6.37) as an iterative scheme by normally referred to as:
Jacobi’s method
1
xm+1
1 = xm
1 + [b1 − (a11 xm m m
1 + a12 x2 + a13 x3 )] (6.38)
a11
1
xm+1
2 = xm
2 + [b2 − (a21 xm m m
1 + a22 x2 + a23 x3 )] (6.39)
a22
1
xm+1
3 = xm
3 + [b3 − (a31 xm m m
1 + a32 x2 + a33 x3 )] (6.40)
a33
where we have introduced m as an iteration counter. In the case when xm i is

a solution of (6.37), the expression in the brackets of (6.40), will be zero and
thus xm+1
i = xm m
i , i.e. convergence is obtained. Whenever, xi is not a solution
of (6.37), the expression in the brackets of (6.40), will represent a correction to
the previous guess at iteration m.
A more compact representation of (6.40) may be used for a system of n
equations:
xm+1
i = xmi + δxi (6.41)
 
n
1  X
δxi = bi − aij xm
j
 , i = 1, 2, . . . , n, m = 0, 1, . . . (6.42)
aii j=1
To start the iteration process one must chose a value x = x0 at iteration

m = 0.
From (6.40) we see that Jacobi’s method may be improved by substitution
of xm+1
1 in the second equation and for xm+1
1 and xm+1
2 in the third equation,
to yield:
Gauss-Seidel’s method
1
xm+1
1 = xm
1 + [b1 − (a11 xm m m
1 + a12 x2 + a13 x3 )] (6.43)
a11
1
xm+1
2 = xm
2 + [b2 − (a21 xm+1
1 + a22 xm m
2 + a23 x3 )] (6.44)
a22
1
xm+1
3 = xm
3 + [b3 − (a31 xm+1
1 + a32 xm+1
2 + a33 xm3 )] (6.45)
a33
The successively improved Jacobi’s method, is normally referred to as Gauss-

Seidel’s method.
The incremental change may be multiplied with a factor ω, to yield another
variant of the iterative scheme:
SOR method
ω
xm+1
1 = xm
1 + [b1 − (a11 xm m m
1 + a12 x2 + a13 x3 )]
a11
ω
xm+1
2 = xm
2 + [b2 − (a21 xm+1
1 + a22 xm m
2 + a23 x3 )] (6.46)
a22
ω
xm+1
3 = xm
3 + [b3 − (a31 xm+1
1 + a32 xm+1
2 + a33 xm3 )]
a33
A general version of (6.46) may be presented:
xm+1
i = xmi + δxi (6.47)
" i−1 n
!#
ω X
m+1
X
m
δxi = bi − aik xk + aik xk , i = 1, 2, . . . , n, (6.48)
aii
k=1 k=1
The factor ω is denoted the relaxation parameter or the relaxation factor.

The method in (6.48) is commonly referred to as the successive over relaxation
method when ω > 1 or simply abbreviated to the SOR method. With ω = 1
Gauss-Seidel’s method is retrieved.
The relaxation factor ω may be shown to be in the range (0, 2) for Laplace/Poisson
equations, but naturally ω > 1 is most efficient. We will not use the SOR method
is presented in (6.37), but rather use the difference equations directly.
Let us first consider Poisson’s equation in two physical dimensions:
∂2u ∂2u
+ 2 = f (x, y) (6.49)
∂x2 ∂y
which after discretization with central differences, with ∆x = ∆y = h results
in the following difference equation:
ui−1,j + ui+1,j + ui,j+1 + ui,j−1 − 4ui,j − h2 · fi,j = 0 (6.50)

We discretize our two-dimensional domain in the normal manner as:
xi = x0 + i · h, i = 0, 1, 2, . . . , nx
yj = y0 + j · h, j = 0, 1, 2, . . . , ny (6.51)
where nx and ny denote the number of grid cells in the x− and y-direction,
respectively (see 6.3).
By using the general SOR method (6.48) on (6.50) and (6.51) we get the
following iterative scheme:
um+1
i,j = umi,j + δui,j (6.52)
ω m+1
+ um+1 m m m 2

δui,j = u i,j−1 + ui+1,j + ui,j+1 − 4ui,j − h · fi,j (6.53)
4 i−1,j
The scheme in (6.53) may be reformulated by introducing the residual Ri,j

Ri,j = um+1 m+1 m m m 2

i−1,j + ui,j−1 + ui+1,j + ui,j+1 − 4ui,j − h · fi,j (6.54)
Note that the residual Ri,j is what is left when a non correct solution is
plugged into to the difference equation (6.50) in a dicrete point (i, j). By
introducing the residual (6.54) into (6.53), the following iterative scheme may
be obtained:
ω
um+1
i,j = um
i,j + Ri,j (6.55)
4
We will now solve the example in Figure 6.4, 6.3, with the iterative SOR-
scheme in (6.53).
y
T2,7 T3,7 T4,7 T5,7
T1,7 T5,6
m
Ti,j+1
T1,6
m+1
Ti-1,j m m
Ti,j Ti+1,j
T2,3
T1,3
T2,2 T3,2 Tm+1
T1,2 i,j-1
T1,1 T2,1 T3,1 T4,1 T5,1 x
Figure 6.11: Rectangular domain as in Figure 6.4 but boundary nodes are
unknown due to Neumann boundary conditions.
In Figure 6.11 we have introduced a new indexation as compared to the

problem in Figure 6.4, and we do not explicitly account for symmetry.
The boundary values are imposed at the boundaries, whereas the rest of the
unknown values are initialized to zero:
# Initialize T and impose boundary values
T[-1,:] = Ttop
T[0,:] = Tbottom
T[:,0] = Tleft
T[:,-1] = Tright
In this first simple program we use ω = 1.5 and perform a fixed number of
20 iterations as shown in the program laplace_sor.py below with h = 0.25:
import numpy as np
import scipy
import scipy.linalg
import scipy.sparse
import time
from astropy.units import dT

LNWDT=2; FNT=15

Ttop=10
Tbottom=0.0
Tleft=0.0
Tright=0.0
xmax=1.0
ymax=1.5
Nx = 40
h=xmax/Nx
Ny = int(ymax/h)
nx = Nx-1
ny = Ny-1
# surfaceplot:
# Initialize T and impose boundary values

T[-1,:] = Ttop
T[0,:] = Tbottom
T[:,0] = Tleft
T[:,-1] = Tright
tic=time.clock()
omega = 1.5
for iteration in range(20):
R = (T[j,i-1]+T[j-1,i]+T[j,i+1]+T[j+1,i]-4.0*T[j,i])
dT = 0.25*omega*R
T[j,i]+=dT
toc=time.clock()
print ’GS solver time:’,toc-tic
As opposed to the above scheme with a fixed number of iterations, we seek

an iteration scheme with a proper stop criteria, reflecting that our solution
approximates the solution with a certain accuracy. Additionally, we would like to
find and relaxation parameter ω which reduces the number of required iterations
for a given accuracy. These topics will be addressed in the following section.
6.4.1 Stop criteria

Examples of equivalent stop criteria are:
• max(δTi,j ) < εa

δTi,j
• max < εr
Ti,j
• The residual Ri,j (6.54) may also be used
• Other alternatives:

1 XX 1 X X δTi,j
|δTi,j | < tola , < tolr , |Ti,j | =
6 0 (6.56)
N i j N i j Ti,j
where N = nx ny is the total number of unknowns. In (6.56) the residual may

be used rather than δTi,j .
In the first expression we use an abolsute tolerance, whereas a relative
tolerance is suggested in the latter. We choose to use the following alternative
for (6.56):
P P
|δTi,j | max (|δTi,j |)
Pi Pj < tolr , < tolr (6.57)
i j |Ti,j | max (|Ti,j |)
The expression (6.57) represents some kind of an average relative stop criteria,
as we sum over all computational grid points in these formulas.
From Figure 6.12, we observe that the number of iterations is a function of
both the relaxation parameter ω and the grid size h.
A more generic program for the Poisson problem (6.49) with a stop criteria
max(|δTi,j |)
max(|Ti,j |) < tolr and variable grid size h is included below. As for the previous
code with a fixed number of iterations, both a Jacobi-scheme with array-slicing
and a Gauss-Seidel scheme is included for comparison.
import numpy as np
import scipy
import scipy.linalg
import scipy.sparse
300
250
h = 0.03125
200
iterasjoner
150
h = 0.0625
100
h = 0.125
50
h = 0.25
0
1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2
ω
Figure 6.12: Number of iterations as a function of ω and h with tolr = 10−5 .
import time
from astropy.units import dT
LNWDT=2; FNT=15
Ttop=10
Tbottom=0.0
Tleft=0.0
Tright=0.0
xmax=1.0
ymax=1.5

Nx = 20
h=xmax/Nx
Ny = int(ymax/h)
nx = Nx-1
ny = Ny-1
# surfaceplot:
# set the imposed boudary values

T[-1,:] = Ttop
T[0,:] = Tbottom
T[:,0] = Tleft
T[:,-1] = Tright
T2 = T.copy()
reltol=1.0e-3
omega = 1.5
iteration = 0
rel_res=1.0
# Gauss-Seidel iterative solution

tic=time.clock()
while (rel_res > reltol):
dTmax=0.0
R = (T[j,i-1]+T[j-1,i]+T[j,i+1]+T[j+1,i]-4.0*T[j,i])
dT = 0.25*omega*R
T[j,i]+=dT
dTmax=np.max([np.abs(dT),dTmax])
rel_res=dTmax/np.max(np.abs(T))
iteration+=1
toc=time.clock()
print "Gauss-Seidel solver time:\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,iteration)
iteration = 0
rel_res=1.0
# Jacobi iterative solution
tic=time.clock()
R2 = (T2[1:-1,0:-2]+T2[0:-2,1:-1]+T2[1:-1,2:]+T2[2:,1:-1]-4.0*T2[1:-1,1:-1])
dT2 = 0.25*R2
T2[1:-1,1:-1]+=dT2
rel_res=np.max(dT2)/np.max(T2)
iteration+=1
toc=time.clock()
print "Jacobi solver time:\t\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,iteration)
6.4.2 Optimal relaxation parameter

By optimal, we mean the relaxation parameter value that results in the lowest
possible iteration number for ω hold constant throughout the computation. For
Laplace and Poisson equations in a rectangular domain it is possible to calculate
an optimal ω, which we will call as theoretically optimal.
Let Lx and Ly be the extent of the rectangular domain in the x- and y-
directions. Taking the same space discretization h in both directions, we set:
Lx Ly
nx = , ny = , with nx and ny being the number of intervals in the x- and
h h
y-directions, respectively. nx and ny must be integers. The theoretical optimal

ω is then given by:
1
ρ= [cos(π/nx ) + cos(π/ny )] (6.58)
2
2
ω= p (6.59)
1 + 1 − ρ2
If the interval length h is different in x- og y-directions with h = hx in
x-direction and h = hy in y-direction, instead of (6.59), the following holds:
cos(π/nx ) + (hx /hy )2 · cos(π/ny )

ρ= (6.60)
1 + (hx /hy )2
Moreover, if the domain is not rectangular, one can use Garabedian’s estimate:
2
ω= √ , (6.61)
1 + 3.014 · h/ A
where A is the area of the domain.
Let us compute some numerical values for these formulas for the case in
which Lx = 1, Ly = 1.5 and A = Lx · Ly = 1.5.
h (6.59) (6.61)
0.25 1.24 1.24
0.125 1.51 1.53
0.0625 1.72 1.74
0.03125 1.85 1.86
We see that Garabedian’s estimate is well in line with the theoretically exact
values in this case. Results reported in Figure 6.12 are also in good agreement
with the values in this table.
6.4.3 Example using SOR

We now solve the temperature problem shown in Figure 6.7 (se 6.3.1). The
numbering convention used here is depicted in Figure 6.13.
Here we use ghost points as indicated by the dashed lines. For T1,1 we get:
1 1
T1,1 = (T1,2 + T1,2 + T2,1 + T2,1 ) = (T1,2 + T2,1 ) (6.62)
4 2
The calculation is started by iterating along y = 0, starting with T2,1 . Then
iteration along x = 0 follows, starting with T1,2 . Afterwards we iterate in a
double loop over inner points. Finally, T1,1 is computed from (6.62). The
algorithmPis P
shown in code lapsor2 in the next page. We have used the stopping
|δT
i,j |
criteria Pi Pj |T | < tolr and optimal ω from (6.59) and denote T = 0.5
i,j
i j
as initial guess for the entire domain, expect where boundary conditions are
prescribed. Accuracy is as in the print out for lap2v3 in 6.3.1.
1.0 1.0 1.0 1.0 1.0
T2,4 T1,4 T2,4 T3,4 T4,4 0.0
T2,3 T1,3 T2,3 T3,3 T4,3 0.0
T2,2 T1,2 T2,2 T3,2 T4,2 0.0
T2,1 T1,1 T2,1 T3,1 T4,1 0.0

x
T1,2 T2,2 T3,2 T4,2
Figure 6.13: Ghost-cells are used to implement Neumann boundary conditions.
% program lapsor2
clear
net = 1;
h = 0.25;
hn = h/2^(net -1);
nx = 1/hn; ny = nx;
imax = nx + 1; % points in x-direction
jmax = ny + 1; % points in y-direction
T = 0.5*ones(imax,jmax); % temperatures
% --- Compute optimal omega ---
ro = cos(pi/nx);
omega = 2/(1 + sqrt(1 - ro^2));
T(1:imax,jmax) = 1; % boundary values along y = 1
T(imax,1:jmax-1) = 0;% boundary values along x = 1
reltol = 1.0e-5; % relative iteration error
relres = 1.0; it = 0;
% --- Start iteration ---
while relres > reltol
it = it + 1;
Tsum = 0.0; dTsum = 0.0;
% --- boundary values along y = 0 ---
for i = 2: imax - 1
resid = 2*T(i,2) + T(i-1,1) + T(i+1,1) - 4*T(i,1);
dT = 0.25*omega*resid;
dTsum = dTsum + abs(dT);
T(i,1) = T(i,1) + dT;
Tsum = Tsum + abs(T(i,1));
end
% --- boundary values along x = 0 ---
for j = 2: jmax - 1
resid = 2*T(2,j) + T(1,j-1) + T(1,j+1) - 4*T(1,j);

T(1,j) = T(1,j) + dT;
Tsum = Tsum + abs(T(1,j));
end
for i = 2: imax-1
for j = 2: jmax-1
resid = T(i-1,j) + T(i,j-1) + T(i+1,j) + T(i,j+1)-4*T(i,j);
T(i,j) = T(i,j) + dT;
Tsum = Tsum + abs(T(i,j));
end
end
T(1,1) = 0.5*(T(2,1) + T(1,2));
relres = dTsum/Tsum;
end
6.4.4 Initial guess and boundary conditions

We expect faster convergence if we use initial guesses that are close to the
problem’s solution. This is typical of nonlinear equations, while we are more free
to choose initial guesses when we solve linear equations without exceeding the
convergence rate. For example, for the temperature problem in Figure 6.4, there
is little difference in number of iterations if we start the iteration by choosing
T = 0 in the entire domain or if we start with T = 1004. The optimal ω for
this case is also independent of the starting values. The situation is completely
different for the case in Figure 6.7. Here we also solve a linear equation but we
have more complicated boundary conditions, where we prescribe the temperature
along two sides of the domain (Dirichlet conditions) and the derivative of the
temperature along the two other sides (Neumann conditions) . In addition, the
temperature is discontinuous in the corner x = 1, y = 1. In the corner x = 0, y
= 0 the correct solution is T = 0.5, as shown by the analytical solution. If we
exclude T = 0.5 as initial guess throughout the domain, we get fast convergence
with optimal ω equal to the theoretical value obtained using (6.59). If we deviate
slightly from T = 0.5 as the initial guess, then the optimal ω is no longer the
theoretical optimal. The situation is as shown in Figure 6.14 below.
The table below shows the theoretical optimal ω obtained with (6.59) and
(6.61)
h (6.59) (6.61)
0.25 1.17 1.14
0.125 1.45 1.45
0.0625 1.67 1.68
0.03125 1.82 1.83
We see that the values reported in the table match values shown in Figure 6.14
when the initial guess for the iterative process is 0.5.
2
h = 0.03125
1.9
h = 0.0625
1.8
1.7 h = 0.125
1.6
h = 0.25
ωopt
1.5
1.4
1.3
1.2
1.1
1
0.5 0.502 0.504 0.506 0.508 0.51 0.512 0.514 0.516 0.518 0.52
startverdier
Figure 6.14: Effect of initial guesses on the number of iterations for a problem
with sub-optimal ω.
6.4.5 Example: A non-linear elliptic PDE

In this example we will solve a non-linear elliptic Poisson equation for a square
2D-domain (see Figure 6.15) given by:
∂2u ∂2u
+ 2 + u2 = −1 (6.63)
∂x2 ∂y
with Dirichlet boundary conditions, i.e precribed values u = 0 on all bound-
aries (see Figure 6.15).
y
u=0
1.0
u=0 ∇2u + u2 = -1 u=0
0.0 u=0 1.0 x
Figure 6.15: Solution domain and boundary conditions for a non-linear Poisson
equation.
The PDE in (6.63) is only weakly non-linear, so-called semi-linear, but the
approach for how to solve a non-linear elliptic PDE is still illustrated.
We discretize (6.63) with central differences over the domain in Figure 6.15
by a constant stepsize h in both physical directions and get the following system
of equations when the source term on the right hands side has been moved to
the left hand side:
1
[ui+1,j + ui−1,j + ui,j−1 + ui,j‘1 − 4ui,j ] + u2i,j + 1 = 0
h2
or equivalently with by introducing the function fi,j :
fi,j = ui+1,j + ui−1,j + ui,j−1 + ui,j+1 − 4ui,j + h2 (u2i,j + 1) = 0 (6.64)
with xi = h · (i − 1), i = 1, 2, . . . and yj = h · (j − 1), j = 1, 2, . . . . A

computational mesh for h = 0.25 is illustrated in Figure 6.16.
y
u1,5 = 0 u2,5 = 0 u3,5 = 0 u4,5 = 0 u5,5 = 0
u2,4 u3,4 u4,4

u1,4 = 0 u5,4 = 0
u2,3 u3,3 u4,3

u1,3 = 0 u5,3 = 0
u2,2 u3,2 u4,2

u1,2 = 0 u5,2 = 0
u1,1 = 0 u2,1 = 0 u3,1 = 0 u4,1 = 0 u5,1 = 0 x
Figure 6.16: Computational mesh for the problem in Figure 6.15.
We will solve the non-linear function in (6.64) with Newton’s method by

changing each variable one at a time. In this case the iteration process becomes:
um+1
i,j = um
i,j + δui,j (6.65)
f (uk,l )
δui,j = −ω (6.66)
∂f (uk,l )
∂ui,j
where:
uk,l = um+1
k,l for k < i, l < j (6.67)
uk,l = um
k,l ellers (6.68)
and
∂f f
= −4 + 2h2 · ui,j and δui,j = ω (6.69)
∂ui,j 4 − 2h2 ui,j
We have implemented (6.64) and (6.66) to (6.69) in the python code nonlin_poisson
below:
# Python Gauss-Seidel method
tic=time.clock()
du_max=0.0
R = (U[j,i-1]+U[j-1,i]+U[j,i+1]+U[j+1,i]-4.0*U[j,i]) + h**2*(U[j,i]**2+1.0)
df=4-2*h**2*U[j,i]
dU = omega*R/df
U[j,i]+=dU
du_max=np.max([np.abs(dU),du_max])
rel_res=du_max/np.max(np.abs(U))
iteration+=1
toc=time.clock()
print "Python Gauss-Seidel CPU-time:\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,iteration)
In this implementation we have used the following stop criterium:

max δui,j
≤ tolr (6.70)
max ui,j
and initialize the solution in the field (excluding boundaries) with u = 0.

We have also used (6.59) to estimat an optimal ωest . In the table below we
compare the estimated optimal ωest with the real optimal ωopt in case of an
initial field of u = 0.
h ωest ωopt
0.25 1.17 1.19
0.125 1.45 1.46
0.0625 1.67 1.69
0.03125 1.82 1.83
We observe relatively good agreement between the two ω-values for a range of
grid sizes h, even though the PDE is weakly non-linear.
As the Gauss-Seidel algorithm above involves a triple-loop (the iterative

while-construct, pluss one loop in each physical direction), the naive python
implementation above must be expected to be computationally expensive.
For comparison we have also implemented another solution to the problem
by making use of numpy’s array slicing capabilities:
tic=time.clock()
R2 = (U2[1:-1,0:-2]+U2[0:-2,1:-1]+U2[1:-1,2:]+U2[2:,1:-1]-4.0*U2[1:-1,1:-1]) + h**2*(U2[1:-1,1:-1
df=4-2*h**2*U2[1:-1,1:-1]
dU2 = R2/df
U2[1:-1,1:-1]+=dU2
rel_res=np.max(dU2)/np.max(U2)
iteration+=1
toc=time.clock()
print "Jacobi CPU-time:\t\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,iteration)
Note with this implementation all values on the right hand side are at the
previous iteration, and thus the method must me denoted a Jacobian algorithm.
Finally, we have implemented a third method the Gauss-Seidel method (6.66)
with Cython. Cython is an optimising static compiler (based on Pyrex) for
both the Python programming language and the extended Cython programming
language. The ambition is to makes the writing of computationally superior C
extensions for Python as easy as Python itself.
The expensive triple loop is implemented in a typed Cython function which
looks very much like the Python implementation, save for the type declarations.
import cython
cimport cython
import numpy as np
cimport numpy as np
DTYPE = np.float64
ctypedef np.float64_t DTYPE_t
@cython.boundscheck(False)
@cython.wraparound(False)
@cython.nonecheck(False)
cpdef gauss(np.ndarray[DTYPE_t, ndim=2] U, double reltol, double h, double omega):

cdef Py_ssize_t i, j, it
cdef double rel_res, dU_max, df, dU, R
cdef unsigned int rows = U.shape[0]
cdef unsigned int cols = U.shape[1]
it=0
rel_res=1.0
itmax=100
while ((rel_res>reltol) and (it<=itmax)):
dU_max=0.0
for j in range(1,rows-2):
for i in range(1,cols-2):
df=4.0-2*h**2*U[j,i]
dU = omega*R/df
U[j,i]+=dU
dU_max=np.max([np.abs(dU),dU_max])
rel_res=dU_max/np.max(np.abs(U[:,:]))
# print ’rel_res’, rel_res
it+=1
if (it>=itmax): print ’Terminated after max iterations’
return U, rel_res, it
After compilation the Cython module is easily imported into our python-code
which allows for comparison with the methods above as illustrated in the code
below:
# Python Gauss-Seidel method
tic=time.clock()
du_max=0.0
df=4-2*h**2*U[j,i]
dU = omega*R/df
U[j,i]+=dU
du_max=np.max([np.abs(dU),du_max])
rel_res=du_max/np.max(np.abs(U))
iteration+=1
toc=time.clock()
print "Python Gauss-Seidel CPU-time:\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,iteration)
iteration = 0
rel_res=1.0
# Second method
tic=time.clock()
R2 = (U2[1:-1,0:-2]+U2[0:-2,1:-1]+U2[1:-1,2:]+U2[2:,1:-1]-4.0*U2[1:-1,1:-1]) + h**2*(U2[1:-1,1:-1
df=4-2*h**2*U2[1:-1,1:-1]
dU2 = R2/df
U2[1:-1,1:-1]+=dU2
rel_res=np.max(dU2)/np.max(U2)
iteration+=1
toc=time.clock()
print "Jacobi CPU-time:\t\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,iteration)
# Third method
# Cython Gauss-Seidel method
rel_res=1.0
tic=time.clock()
U3, relreturn, itsused=gs.gauss(U3,reltol,h, omega)
toc=time.clock()
print "Cython Gauss-Seidel CPU-time:\t{0:0.2f}. \t Nr. iterations {1}".format(toc-tic,itsused)
By running the code we get the follwing results for comparison:

omega=1.85
Python Gauss-Seidel CPU-time: 1.62. Nr. iterations 56
Jacobi CPU-time: 0.04. Nr. iterations 492
Cython Gauss-Seidel CPU-time: 0.97. Nr. iterations 56
which illstrates the extreme efficiency whenever a numerical scheme may

be expressed by means of numpy-array-slicing. Note that the numpy-array-
slicing Jacobi scheme converge slower than the Gauss-Seidel scheme in terms of
iterations, and need apporximately 10 times as many iterations as the Gauss-
seidel algorithm. But even in this situation the CPU-speed of the Jacobi scheme
with array-slicing is approximately 40 times faster than the Gauss-Seidel scheme
in python. We also observe that the Cython implementation Gauss-Seidel scheme
is approximately 1.7 times faster than the python counter part, but not by far
as fast as the Jacobi scheme with array-slicing, which is approximately 24 times
faster.
emfs /src-ch7/ #python #package nonlin_poisson_sor.py @ git@lrhgit/tkt4140/src/src-ch7/nonlin_poisso
Exercise 9: Symmetric solution

Prove that the analytical solution of the temperature field T (x, y) in (6.25) is
symmetric around x = 0.5.
Exercise 10: Stop criteria for the Poisson equation

Implement the various stop criteria outlined in 6.4.1 for the Possion equation in
two dimensions (6.49).
Chapter 7
Diffusjonsproblemer
7.1 Introduction
A one-dimensional diffusion equation takes the canonical form:
∂u ∂2u
=α 2 (7.1)
∂t ∂x
where t is an evolutionary variable, which might be both a time-coordinaten
and a spatial coordinate. Some classical diffusion problems are listed below:
• Heat conduction
∂T ∂2T
=α 2
∂t ∂x
• Unsteady boundary layers (Stokes’ problem):
∂u ∂2u
=ν 2
∂t ∂y
• Linearized boundary layer equation with x as an evolutionary variable:
∂u ν ∂2u
=
∂x U0 ∂y 2
• Flow in porous media:
∂u ∂2u
=c 2
∂t ∂x
Our model equation (7.1) may be classified according to (5.18):
∂2φ ∂2φ ∂2φ

A + B + C +f =0 (7.2)
∂x2 ∂x∂y ∂y 2
264
CHAPTER 7. DIFFUSJONSPROBLEMER 265
A · (dy)2 − B · dy · dx + C · (dx)2 = 0 (7.3)

√
B 2 − 4AC
B±
λ1,2 = (7.4)
2A
B = C = 0 and A = 1 which by substitution in (5.33) and (5.35) yield:
dt = 0, B 2 − 4AC = 0
And we find that (7.1) is a parabolic PDE with the characteristics given by
t = constant. By dividing dt with dx we get:
dt dx
=0→ =∞ (7.5)
dx dt
which corresponds to an infinite propagation speed along the characteristic
curve t = constant.
7.2 Confined, unsteady Couette flow

The classical versicon of unsteady Couette flow with b = ∞ has been presented
as Stokes first problem and discussed in section (3.3).
Y b
U0
Figure 7.1: Confined, unsteady Couette flow or channel flow. The channel with
is b
In this section we will look at the problem of unsteady Couette flow, confined
by two walls, which has the following governing equation:
∂U ∂2U
=ν , 0<Y <b (7.6)
∂τ ∂Y 2
with the following boundary conditions, representing the presence of the two
walls or channel if you like:

U (0, τ ) = U0
=τ ≥0 (7.7)
U (b, τ ) = 0
Futher, the parabolic problem also needs initial conditions to be solved and
we assume:
U (Y, τ ) = 0, τ < 0 (7.8)

In section 3.3 we have presented several ways to render (7.6)) dimensionless,
and for the current problem we introduce the following dimesionless variables:
Y U τν
y= , u= , t= 2 (7.9)
b U0 b
which allow (7.6) to be written:
∂u ∂2u
= , 0<y<1 (7.10)
∂t ∂y 2

u(0, t) = 1
, t≥0 (7.11)
u(1, t) = 0
and initial conditions:
u(y, t) = 0, t < 0 (7.12)

As for the problemn in section 3.3, this present example may also be formu-
lated as a heat conduction problem. The problem of confined, unsteady Couette
flow has the following analytical solution:
∞
2 X1
u(y, t) = 1 − y − · exp[−(nπ)2 t] sin(nπy) (7.13)
π n=1 n
A detailed derivation of (7.13) may be found in appendix G.6 of Numeriske

beregninger.
We discretize (7.10) by a forward difference for the time t:
un+1
n
∂u j − unj
≈ (7.14)
∂t j ∆t
and central differences for the spatial coordinate y:
∂2u unj+1 − 2unj + unj−1

≈ (7.15)
∂y 2 (∆y)2
where:
tn = n · ∆t, n = 0, 1, 2, . . . , yj = j · ∆y, j = 0, 1, 2, . . .
Substitution of (7.14) in (7.10) results in the following difference equation:
un+1
j = D (unj+1 + unj−1 ) + (1 − 2D) unj (7.16)
where D is a dimesionless group, commonly denoted the diffusion number or

the Fourier number in heat conduction. See section 13.1 in [2])) for a discussion
of (7.16).
∆t ∆τ
D= =ν (7.17)
(∆y)2 (∆Y )2
A scheme with Forward differences for the Time (evolutionary) variable and
Central differences for the Space variable, is commonly referred to as a FTCS
(Forward Time Central Space). For a FTCS-scheme we normally mean a scheme
which is first order in t og and second order in y. Another common name for
this scheme is the Euler-scheme.
In section 7.6 we show that for D = 1/6, the scheme (7.16)) is second order
in t og and fourth order in y. Further, in fluid mechanics it is customary to write
unj rather than uj,n , such that index for the evolutionary variable has a super
index. We seek to adopt this convention in general.
n+1
0.0 j-1 j j+1 1.0
Figure 7.2: Numerical stencil of the FTCS scheme.
In Figure 7.2 we try to illustrate that the scheme is explicitt, meaning that
the unknown value at time n + 1 can be found explicitly from the formula without
having to solve an equation system. In other words, the unknown value at time
n + 1 is not implicitly dependent on other values at other spatial locations at
time n + 1.
The above example is implemented in the code below. Download the code
and experiment using different diffusion numbers. The FTCS-scheme is explicit
and thus has a stability constraint. We will look further into stability in the
next sections, but as we will see in the animations displayed below, the stability
limit in this example is D = 12 .
# src-ch5/couette_FTCS.py;Visualization.py @ git@lrhgit/tkt4140/src/src-ch5/Visualization.py;

import numpy as np
from math import exp, sin, pi
def analyticSolution(y, t, N=100):
""" Method that calculates the analytical solution to the differential equation:
du/dt = d^2(u)/dx^2 , u = u(y,t), 0 < y < 1
Boundary conditions: u(0, t) = 1, u(1, t) = 0
Initial condition: u(t, 0) = 0 t<0, u(t, 0) = 1 t>0
Args:
y(np.array): radial coordinat
t(float): time
N(int): truncation integer. Truncate sumation after N elements
Returns:
w(float): velocity, us - ur
"""
sumValue = 0
for n in range(1,N+1):
temp = np.exp(-t*(n*np.pi)**2)*np.sin(n*np.pi*y)/n
sumValue += temp
u = 1 - y - (2/pi)*sumValue
return u
def solveNextTimestepFTCS(Uold, D, U_b=1, U_t=0):
""" Method that solves the transient couetteflow using the FTCS-scheme..
At time t=t0 the plate starts moving at y=0
The method solves only for the next time-step.
The Governing equation is:
du/dt = d^2(u)/dx^2 , u = u(y,t), 0 < y < 1

Boundary conditions: u(0, t) = 1, u(1, t) = 0
Initial condition: u(t, 0) = 0 t<0, u(t, 0) = 1 t>0
Args:
uold(array): solution from previous iteration
D(float): Numerical diffusion number
Returns:
unew(array): solution at time t^n+1
"""
Unew = np.zeros_like(Uold)
Uold_plus = Uold[2:]
Uold_minus = Uold[:-2]
Uold_mid = Uold[1:-1]
Unew[1:-1] = D*(Uold_plus + Uold_minus) + (1 - 2*D)*Uold_mid

Unew[0] = U_b
Unew[-1] = U_t
return Unew
if __name__ == ’__main__’:
import numpy as np
from Visualization import createAnimation
D = 0.505 # numerical diffusion number
N = 20
y = np.linspace(0, 1, N + 1)
h = y[1] - y[0]
dt = D*h**2
T = 0.4 # simulation time
time = np.arange(0, T + dt, dt)
# Spatial BC
U_bottom = 1.0 # Must be 1 for analytical solution
U_top = 0.0 # Must be 0 for analytical solution
# solution matrices:
U = np.zeros((len(time), N + 1))
U[0, 0] = U_bottom # no slip condition at the plate boundary
U[0,-1] = U_top
#
Uanalytic = np.zeros((len(time), N + 1))
Uanalytic[0, 0] = U[0,0]
for n, t in enumerate(time[1:]):
Uold = U[n, :]
U[n + 1, :] = solveNextTimestepFTCS(Uold, D, U_b=U_bottom, U_t=U_top)
Uanalytic[n + 1, :] = analyticSolution(y,t)
U_Visualization = np.zeros((1, len(time), N + 1))
U_Visualization[0, :, :] = U
createAnimation(U_Visualization, Uanalytic, ["FTCS"], y, time, symmetric=False)
In the following two animations we show numerical solutions obtained with

the explicit FTCS scheme, as well as with two implicit schemes (Crank-Nicolson
and Laasonen schemes), along with the analytical solution. The first animation
shows results for D = 0.5, for which the FTCS scheme is stable. In the second
animation we use D = 0.504, a value for which the FTCS scheme’s stability limit
is exceeded. This observation is confirmed by the oscillatory character of the
numerical solution delivered by this scheme. .
Movie 3: Animation of numerical results obtained using three nu-

merical schemes as well as the analytical solution using D = 0.5.
mov-ch5/couette_0.5.mp4
Movie 4: Animation of numerical results obtained using three numer-

ical schemes as well as the analytical solution using D = 0.504. The
FTCS displays an unstable solution. mov-ch5/couette_0.504.mp4
7.3 Stability: Criterion for positive coefficients.

PC-criterion
Consider the following sum:
s = a1 x1 + a2 x2 + · · · + ak xk (7.18)
where a1 , a2 , . . . , ak are positive coefficients. Now, by introuducing the extrema

of xi as:
xmin = min(x1 , x2 , . . . , xk ) and xmax = max(x1 , x2 , . . . , xk ) (7.19)
we may deduce from (7.18):
xmin · (a1 + a2 + · · · + ak ) ≤ s ≤ xmax · (a1 + a2 + · · · + ak ) (7.20)
Note, that the above is valid only when a1 , a2 , . . . , ak are all positive. In the
following we will consider two cases:
Case 1: a1 + a2 + · · · + ak = 1
In this case (7.20) simplifies to:
xmin ≤ s ≤ xmax (7.21)

Equality in (7.21) is obtained when x1 = x2 = · · · = xk .
Let us now apply (7.21) on the difference equation in (7.16):
un+1
j = D (unj+1 + unj−1 ) + (1 − 2 D) unj
In this case the coefficients are a1 = D, a2 = D, a3 = 1 − 2D such that the
sum of all coefficents is: a1 + a2 + a3 = 1.
From (7.21) we get:
min(unj+1 , unj , unj−1 ) ≤ un+1

j ≤ max(unj+1 , unj , unj−1 )
Meaning that un+1
j is restricted by the extrema of unj , i.e. all the solutions at
the previous timestep, and will thus not have to ability to grow without bounds
and therefore be stable.
The conditions for stability are that all the coefficients a1 , a2 . . . . , ak are
positive. As D > 0, this means that only a3 = 1 − 2D may become negative.
The condition for a3 to be positive becomes: 1 − 2D > 0 which yields D < 12 .
When D = 12 , the coefficient a3 = 0, such that a1 + a2 = 1, which still satisfies
the condition. Thus the condition for stability becomes:
1
D≤ (7.22)
2
which is commonly referred to as the Bender-Schmidt formula.
For explicit, homogenous schemes which has a constant solution u = u0 , the

sum of the coefficients will often be one. (Substitute e.g. u = u0 in (7.16)). This
property is due to the form of the difference equations from the Taylor-expansions.
(See (2.5) , (2.5) and (2.5) differences in (2)).
Case 2: a1 + a2 + · · · + ak < 1
As a remedy to that the sum of the coefficient do not sum to unity we define
b = 1 − (a1 + a2 + · · · + ak ) > 0 such that a1 + a2 + · · · + ak + b = 1.
We may the construct the following sum: a = a1 x1 + a2 x2 + · · · + ak + b · 0
which satisfies the conditions for Case 1 and we get:
min(0, x1 , x2 , . . . , xk ) ≤ s ≤ max(0, x1 , x2 , . . . , xk ) (7.23)

The only difference from (7.21) being that we have introduced 0 for the x-es,
which naturally has concequences for the extremal values.
Let us look at an example:
∂T ∂2T
= α 2 + bT, b = konstant, t < tmaks
∂t ∂x
which may be discretized with the FTCS-scheme to yield:
∆t
Tjn+1 = D (Tj+1
n n
+ Tj−1 ) + (1 − 2D + ∆t · b)Tjn , D=α
(∆x)2
and with the following coefficients a1 = a2 = D og a3 = 1 − 2D + ∆t · b we
get:
a1 + a2 + a3 = 1 + ∆t · b ≤ 1
only for negative b-values.
The condition of positive coefficients then becomes: 1 − 2D + ∆t · b > 0 which
corresponds to
1 ∆t · b
0<D< +
2 2
where b < 0.
In this situation the criterion implies that the T − values from the difference
equation will not increase or be unstable for a negative b. This result agrees well
the physics, as a a negative b corresponds to a heat sink.
Note that (7.21) and (7.23) provide limits within which the solution is
bounded, and provides a sufficient criteria to prevent the occurrence of unstable
ocillations in the solution. This criteria may be far more restrictive that what
is necessary for a stable solution. However, in many situations we may be
satisfied with such a criteria. The PC-criterion is used frequently on difference
equations for which a more exact analysis is difficult to pursue. Note that the
PC-criterion may only be applied for explicit schemes if no extra information is
provided. For parabolic equations we often have such extra information by means
of max/min principles (see (7.5.3)). Further, the criterion must be modified in
case of increasing amplitudes.
One would of course hope for the existence of a necessary and sufficient
condition for numerical stability. However, for general difference equations we
have no such condition, which is hardly surprising. But a method which often
leads to sufficient, and in some cases necessary, conditions for stabily, is von
Neumann’s method. This method involves Fourier-analysis of the linearized
difference equation and may be applied for both explicit and implicit numerical
schemes. We will present this method in 7.4.
7.4 Stability analysis with von Neumann’s method

The von Neumann analysis is commonly used to determine stability criteria as it
is generally easy to apply in a straightforward manner. Unfortunately, it can only
be used to find necessary and sufficient conditions for the numerical stability of
linear initial value problems with constant coefficients. Practical problems may
typically involve variable coefficients, nonlinearities and complicated boundary
conditions. For such cases the von Neumann analysis may only be applied locally
on linearized equations. In such situations the von Neumann analysis provides
sufficient, but not alwasy necessary conditions for stability [5]. Further, due to
the basis on Fourier analysis, the method is strictly valid only for interior points,
i.e. excluding boundary conditions.
In this section we will show how von Neumann stability analysis may be
applied to the parabolic PDE:
∂u ∂2u
= (7.24)
∂t ∂x2
To motivate the rationale for the basis of the von Neumann analysis, we will
start by a revisit on the analytical solution of (7.24) by the method of separation
of variables. The aim of the method is to simplify the PDE to two ODEs which
has analytical solutions. With this approach we assume that the solution may
constructed by means of separation of variables as u(x, t), i.e. as a product of to
f (t) og g(x), which each are only functions of time and space, repsectively:
u(x, t) = f (t) g(x) (7.25)

We may now differentiate (7.25) with respect to time and space:
∂u df (t) ∂2u d2 g(x) 1

= g(x), 2
= f (t) (7.26)
∂t dt ∂x dx2 g(x)
which upon substitution in (7.25) results in the following equation:
df (t) d2 g(x)
g(x) = f (t) (7.27)
dt dx2
or more conveniently:
df (t) 1 d2 g(x) 1
= (7.28)
dt f (t) dx2 g(x)
Observe that the left hand side of (7.28) is a function of t only, whereas the
right hand side is a function of x only. As the both sides of (7.28) must be
satisfied for arbitrary values of t and x, the only possible solution is that both
sides of the equation equals a constant, which we for convenience denote −β 2 ,
thus our original PDE in (7.24) has been transformed to two ODEs:
df (t) 1 df (t)
= −β 2 → + β 2 f (t) = 0 (7.29)
dt f (t) dt
d2 g(x) 1 d2 g(x)
2
= −β 2 → + β 2 g(x) = 0 (7.30)
dx g(x) dx2
The first ODE (7.29) is of first order and has the solution (verify by susbti-
tution):
2
f (t) = e−β t
(7.31)
whereas the second ODE is of second order with solution (7.30):
g(x) = A sin(βx) + B cos(βx) (7.32)
such that a particluar solution to (7.25) is found by the product of (7.31)

and (7.32):
2
u(x, t) = e−β t
[A sin(βx) + B cos(βx)] (7.33)
and since (7.25) is a linear PDE, the sum or superposition of solutions like
(7.33) will also represent a solution:
m=∞
X 2
u(x, t) = e−βm t [Am sin(βm x) + Bm cos(βm x)]
m=0
The coefficients Am , Bm og βm may be determined from the initial conditions

and the boundary conditions as demonstrated in Appendix G.6 of Numeriske
Beregninger.
However, for the current purpose of demonstration of von Neumann analysis,
two particular solutions suffice:
( 2
e−β t sin(βx)
u(x, t) = 2 (7.34)
e−β t cos(βx)
which may be presented in a more compact for by making use of Euler’s
formula:
√
ei x = cos(x) + i sin(x), i= −1 (7.35)
The more compact form of (7.34) is then:

2 2
u(x, t) = e−β t ei βx = e−β t+i βx
(7.36)
Note. Both the real and the imaginary part of the complex (7.36) satisfy
(7.24) and is therefore included in the somewhat general solution. For particular
problem (7.36) will be multiplied with complex cofficients such that the solution
is real.
By adpoting the common notation: xj = j ∆x, j = 0, 1, 2, . . . og tn =
n ∆t, n = 0, 1, 2, . . . for (7.36), the solution at location xj and time tn is:
2 2 2
u(xj , tn ) = e−β tn
ei βxj = e−β n ∆t iβxj
e = (e−β ∆t n iβxj
) e (7.37)
and the solution at location xj at the next timestep n + 1 is:
2 2 2
u(xj , tn+1 ) = e−β tn+1
ei βxj = e−β (n+1) ∆t iβxj
e = (e−β ∆t n+1 iβxj
) e (7.38)
If we divide (7.37) with (7.38), the spatial dependency vanishes and a we get
a spatially independent expression for how the solution is amplified (or damped):
u(xj , tn+1 ) 2
Ga = = e−β ∆t (7.39)
u(xj , tn )
For this reason, Ga is commonly denoted the analytical amplificaiton factor.
(See section 1.6.1 i Numeriske Beregninger). In our particular case we find that
Ga < 1. Having introduced the the amplification factor Ga we may rewrite
(7.37) as:
u(xj , tn ) = (Ga )n ei βxj ≡ Gna ei βxj (7.40)

For the current problem Ga < 1, and consequently → 0 for n → ∞. Gna
As shown previously in (7.16), the following difference equation for the
solution of (7.24) may be obtained:
∆t
un+1
j = D (unj+1 + unj−1 ) + (1 − 2D)unj , D= (7.41)
(∆x)2
Clearly, the solution of the difference equation (7.41) will deviate from the
analytical solution to the differential equation for finite values of ∆x and ∆t.
The deviation/error will increase for increasing values of ∆x and ∆t and we
have seen that when D > 12 , the difference equation becomes unstable, with
constantly increasing amplitudes and alternating signs. As (7.40) is a solution of
the differential equuation, we introuce a simpliar expression for the difference
equation (7.41):
unj → Ejn = Gn ei βxj (7.42)

where we have introduced the numerical ampflicication factor G:
Ejn+1
G= (7.43)
Ejn
which may be complex and is a function og ∆t and β. From (7.43) we may
relate the error Ejn at the n-th timestep with the initial error Ej0 :
Ejn = Gn Ej0 , Ej0 = ei β xj (7.44)

Given that Ga is derived from the analytical solution of the differetial equation,
and that G is derived from the difference equation approximating the same
differential equation, we can not expect perfect agreement between the two
factors. We will discuss this further in 7.4.1 and illustrated in Figure 7.3.
From (7.44) we find a that if Ejn is to be limited and not grow exponentially
the following condition must be satisfied, which is denoted:
The strict von Neumann condition.
|G| ≤ 1 (7.45)
The condition (7.45) is denoted strict as no increase in amplitude is allowed

for. This condition will be relaxed in section (7.5.4).
Even though we have demostrated the von Neumann method for a relatively
simple diffusion equation, the method is applicable for more generic equations.
The method is simple:
The strict von Neumann method.

• Substitute (7.42) in the difference equation in question.
• Test if the condition (7.45) is satisfied.
Some properties of the condition:
1. The linear difference equation must have constant coefficients. In

case of variable coefficients, the condition may be applied on
linearized difference equations with frozen coefficients locally. Based on

experience this results in a necessary condition for stability.
1. The criterion do not consider the effect of boundary conditions as it

is based on periodic initital data. If the impact of boundary
conditions is to be investigated on may use matrix methods for the coefficient

matrix of the difference scheme [3].
7.4.1 Example: Practical usage of the von Neumann con-

dition
In this example we will demonstrate how the von Neumann method may be used
on the simple scheme (7.16) on the following form:
Ejn+1 = D (Ej+1
n n
+ Ej−1 ) + (1 − 2D) Ejn (7.46)
We have previously (see (7.42)) established how the error Ejn is related with
the numerical amplification factor: Ejn = Gn ei β yj which upon substitution in
(7.46) yields:
Gn+1 ei βyj = D (Gn ei βyj+1 + Gn ei βyj−1 ) + (1 − 2D)Gn ei βyj

Note that Gn means G in the n-th power, whereas Ejn denotes the error at
timelevel n and the spatial location yj .
The expression in (7.4.1) is a nonlinear, n + 1-order polynomial in G which
may be simplified by division of Gn ei βyj :
G = D (ei βh + e−i βh ) + (1 − 2D) = D (eiδ + e−iδ ) + (1 − 2D) (7.47)
where we introduce δ as:
δ = βh (7.48)
By using terminology for periodical functions, β may be thought of as a
wavenumber (angular frequency) and δ as a phase angle. (See appendix A.3 in
Numeriske Beregninger)
For further simplification we introduce some standard trigonometric formulas:
2 cos(x) = eix + e−ix

i 2 sin(x) = eix − e−ix (7.49)
cos(x) = 1 − 2 sin2 ( x2 )
By substitution (7.49) in (7.47) we get the simpler expression:

δ
G = 1 − 2D (1 − cos(δ)) = 1 − 4D sin2 (7.50)
2
As G is real, the condition |G| ≤ 1 has the following mathematical interpre-
tation:

δ
2
−1 ≤ G ≤ 1 or − 1 ≤ 1 − 4D sin ≤1 (7.51)
2
The right hand side of (7.4.1) is always true as D ≥ 0. For the left hand side
of (7.4.1) we have
1
D≤
2 sin2 ( 2δ )
which is true for all δ ( −π ≤ δ ≤ π) when D ≤ 21 . A von Neumann condition
for stability of (7.16) may then be presented as:
1
0<D< (7.52)
2
∆t
where D = (∆y) 2 from (7.17).
As (7.16) is a two-level scheme with constant coefficients, the condition in

(7.52) is both sufficient and necessary for numerical stability. This condition
agrees well in the previous condition in (7.21), which is sufficient only.
The stability condition impose a severe limitation on the size of the timestep
∆t, which of course influence the CPU-time.
A rough estimate of the CPU-time for the FTCS-schemes with constant
D-value yields:
3
T2 h1
≈
T1 h2
where T1 and T2 are the CPU-times for ∆y = h1 and ∆y = h2 , respectively.
For example for a reduction in spatial resolution from h1 = 0.1 by a factor 10 to
h2 = 0.01 we get an increase in CPU-time by a factor TT21 = 1000.
Differentiation to find stability conditions. An alternative method to

find extremal values (i.e. max/min) for G as a function of δ, is to compute dG
dδ
and then set it to dG
dδ = 0. A stability condition may then be found from the
criterion |G| < 1. For the current example we have from (7.50):
dG
G = 1 − 2D (1 − cos(δ)) ⇒ = −2D sin(δ)
dδ
which has extremal values for δ = 0,
δ = ±π, δ = 0 gir G = 1, whereas δ = ±π yields the condition G = 1 − 4D.
Finally, the condition |G| ≤ 1 yields:
−1 ≤ 1 − 4D ≤ 1
The right hand side of the inequality will always be satisfied for positive D,
whereas the left hand side yields D ≤ 12 as before.
In many situations we will find that δ = ±π are critical values, and it may
therefore be wise to assess these values, but remember it might not be sufficent
in order to prove stability. On the other hand these values might sufficient to
prove instabilities for those values, as the condition |G| ≤ 1 must be satisfied for
all δ-values in the range [−π, π].
Comparison of the amplification factors. In this section we will compare

the numerical amplification factor G in (7.50) with the analytical amplification
factor Ga in (7.39)
By making use of (7.41) and (7.48) we may present the analytical amplification
factor (7.39) as:
Ga = exp(−δ 2 D) (7.53)
And from (7.50) we have an expression for the numerical amplification factor:

δ
G = 1 − 2D (1 − cos(δ)) = 1 − 4D sin2 (7.54)
2
In figure 7.3 we plot G and Ga as functions of δ ∈ [0, π] for selected values
of D. For small values of δ we observe small differences between G and Ga , with
slightly larger values of G than for Ga , with progressively increasing differences
as a function of δ.
1.0
0.5
G
0.0
Ga (D = 0.25)
G (D = 0.25)
Ga (D = 0.35)
−0.5
G (D = 0.35)
Ga (D = 0.5)
G (D = 0.5)
−1.0
0 20 40 60 80 100 120 140 160 180
[degrees]
Figure 7.3: The amplification factors G (dotted) and Ga (solid) as a function

of δ for specific values of D.
Further, we observe large errors for δ ∈ [90◦ , 180◦ ] and that the amplification
factor G even has the wrong sign, which will lead to unpysical oscillations in the
numerical solution.
The reason why the solution may still be usable is due to that the analytical
solution has an amplitude Ga which diminishes strongly with increasing frequency
(see the analytical solution in (7.13)). Such a smoothing effect is typical for
parbolic PDEs. Yet the effect is noticable as we in the current example have
a discontinuity at y = 0, such that the solution contains many high frequent
components.
Errors in the amplitude is commonly quantified with εD = GGa and denoted

diffusion error or dissipation error. No diffusion error corresponds to εD = 1. The
term diffusive scheme will normally refer to a numerical scheme with decreasing
amplitude with increasing t. For our FTCS-scheme applied for the diffusion
equation we have:
εD = 1 − 4D sin2 (δ/2) exp(δ 2 D)

(7.55)
The expression in (7.55) may be simplified by a Taylor-expansion:
εD = 1 − D2 δ 4 /2 + Dδ 4 /12 + O(δ 6 )
which confirms that the dissipation error is smal for low frequencies when
D ≤ 1/2.
7.5 Some numerical schemes for parabolic equa-

tions
7.5.1 Richardson scheme (1910)
The FTCS scheme is 1st order accurate in time and 2nd order accurate in space.
We would like to have a scheme that is also 2nd order accurate in time. This
can be achieved by using central finite differences for ∂u
∂t :
un+1 − un−1
n
∂u j j
≈
∂t j 2∆t
which results in:
un+1 = un−1 + 2D unj−1 − 2unj + unj+1

j j (7.56)
This is an explicit, 3-level scheme called Richardson scheme; see Figure 7.4.
n+1
n-1
j-1 j j+1
Figure 7.4: 3-level stencil of the Richardson scheme (7.56).
Stability analysis
Let us first try with the criterion of positive coefficients introduced in (7.21).
This condition can not be fulfilled for the coefficient in front of unj for D > 0.
Next, we try to gain insight on the stability of the scheme by using von Neumann’s
method. (7.42) inserted in (7.56) gives:
Gn+1 ei·βyj = Gn−1 ei·βyj + 2D[Gn ei·βyj−1 − 2Gn ei·βyj + Gn ei·βyj+1 ]
Dividing with Gn−1 eiβyj where yj = j · h :
G2 = 1 + 2DG · e−iδ + eiδ − 2 = 1 + 4DG · (cos(δ) − 1)

= 1 − 8GD · sin2 2δ
where we have used (7.48) and (7.49). We have obtained a 2nd order equation
because we are dealing with a 3-level scheme:
G2 + 2bG − 1√= 0 med løsning

δ
G1,2 = −b ± b2 + 1, b = 4D sin2

2 ≥0
|G| = 1 for b = 0. For all other values of b we have that |G2 | > 1. The
scheme is therefore unstable for all actual values of D. Such schemes are said to
be unconditionally unstable. This example also shows that stability and accuracy
can be rather independent concepts depending on how they are defined.
Use of derivation
We can also use derivation here:
G2 = 1 + 4DG
· (cos(δ) −dG
1),
2G dG

dδ = 4D (cos(δ) − 1) dδ − G sin(δ) ,
that with dG
dδ = 0 gives max-min for δ = 0, δ = ±π, as for the FTCS scheme.
For δ = 0 we have G1,2 = ±1, while δ = ±π, resulting in
p
G1,2 = −4D ± 1 + (4D)2
with instability for |G2 | > 1 as before.
7.5.2 Dufort-Frankel scheme (1953)

Richardson scheme in (7.56) can be made stable by the following modification:
1 n+1
unj = (u + un−1 ) (7.57)
2 j j

1
un+1 (1 − 2D)un−1 + 2D(unj+1 + unj−1 )

j = j (7.58)
1 + 2D
This is an explicit 3-level scheme called DuFort-Frankel scheme, see Figure 7.5.
3-level schemes where unj is missing are called Leap-frog-type schemes for
obvious reasons. The sufficient criterion of positive coefficients in (7.21) requires
n+1
n-1
j-1 j j+1
Figure 7.5: 3-level stencil of the DuFort-Frankel scheme (7.58).
that D ≤ 21 for a stable scheme. The stability analysis here is slightly more
complicated as with previous cases because we have to discuss what happens
when G takes complex values.
iδj
Inserting (7.42) in (7.58) and division by Gn−1 ee yields:
1 1
G2 = (1 − 2r) + 2r · G · (eiδ + e−iδ ) =

[(1 − 2r) + 4r · G · cos(δ)]
1 + 2r 1 + 2r
which gives the following 2nd order equation:
(1 + 2D) · G2 − 4D · G cos(δ) − (1 − 2D) = 0 ,

with roots:
p
4D cos(δ) ± (4D cos(δ))2 + 4(1 + 2D) · (1 − 2D)
G1,2 =
p 2(1 + 2D)
2D cos(δ) ± 1 − D2 sin2 (δ)
= .
1 + 2D
For stability, both roots must meet the condition |G| ≤ 1. In general, we
must distinguish between real and complex roots to take care of the case for
which G ≤ 0 when G is real.
1. Real roots: 1 − 4D2 sin(δ) p≥ 0
2D · | cos(δ)| + 1 − 4D2 sin2 (δ) 1 + 2D
|G1,2 | ≤ ≤ ≤1
1 + 2r 1 + 2D
p p
2. Complex roots: 1−4D2 sin2 (δ) < 0 → 1 − 4D2 sin2 (δ) = i· 4D2 sin2 (δ) − 1
2D cos(δ) 2 +4D2 sin2 (δ)−1 4D2 −1 2D−1

|G1,2 |2 = (1+2D)2 4D2 +4D+1 2D+1 < 1
= =
Analysis shows that (7.58) is actually unconditionally stable. The DuFort-Frankel

scheme is the only simple know explicit scheme with 2nd order accuracy in space
and time that has this property. Therefore it has been in part used to solve the
Navier-Stokes equations. In Section (7.6) we shall see that there is however a
severe problem with this scheme. For the first time level one lacks the lowest
time level required by the scheme. One option is to evolve the solution for the
first time step using the FTCS scheme.
7.5.3 Crank-Nicolson scheme. θ-scheme

One of the bad characteristics of the DuFort-Frankel scheme is that one needs
a special procedure at the starting time, since the scheme is a 3-level scheme.
Therefore, we try now to find a second order approximation for ∂u ∂t where only
two time levels are required..
Using central differences centered at the half time interval we obtain:
n+ 1
∂u 2 un+1
j − unj
= + O(∆t)2 (7.59)
∂t j ∆t
∂u n+ 21
The problem now is to approximate without knowing without in-
∂t j

cluding any term evaluate at n + 12 explicitly in the scheme. This was achieved
by the Crank-Nicolson approximation (1947):
n+ 1 " n+1 n+1 n+1

#
∂ 2 u 2 1 uj+1 − 2uj + uj−1 unj+1 − 2unj + unj−1
= + + O(∆x)2 (7.60)
∂x2 j 2 (∆x)2 (∆x)2
The difference equation is now:
1 n+1 1 n
un+1
j−1 − 2(1 + )u + un+1 n
j+1 = −uj−1 + 2(1 − )u − unj+1 (7.61)
D j D j
∆t
with D = .
(δx)2
The stencil of the resulting numerical scheme is shown in Figure 7.6.
From (7.61) and Figure 7.6 we see that the scheme is implicit. This in turn
implies that the algebraic equations resulting from the discretization of the
original differential problem are coupled and, differently from an explicit scheme,
we have to solve them as a system. In this case the system is tridiagonal, so
we can use the Thomas’ algorithm, for which the code 4.3 was introduce in
Section 4.3.
Before proceeding with a stability analysis for (7.61), we will re-write it a
more general form, by introducing the parameter θ:
un+1 = unj +D θ(un+1 n+1

+ un+1 n n n

j j+1 − 2uj j−1 ) + (1 − θ)(uj+1 − 2uj + uj−1 ) (7.62)
n+1
n+½
j-1 j j+1
Figure 7.6: 2-level stencil of the Crank-Nicolson scheme. Note that the scheme
does not require any evaluation of the solution at time level n + 21 .
with 0 ≤ θ ≤ 1. This scheme is a generalization of Crank-Nicolson original

scheme and is called the θ-scheme.
For θ = 0 one recovers the explicit FTCS-scheme. On the other hand,
forθ = 12 one has the original Crank-Nicolson scheme. Finally, ror θ = 1 one
gets an implicit scheme some times called the Laasonen-scheme (1949). In fluid
mechanics this last scheme is often called BTCS-scheme. (Backward in Time
and Central in Space).
Inserting (7.42) in (7.62) and dividing by Gn · ei·βxj gives:
G = 1 + D[Gθ(eiδ + e−i·δ − 2) + (1 − θ)(eiδ + e−iδ − 2)]

= 1 + D(eiδ + e−iδ − 2) · (Gθ + 1 − θ)
with δ = β · h. Moreover, using (7.49) results in:
1 − 4D(1 − θ) sin2 ( 2δ )
G= (7.63)
1 + 4Dθ sin2 ( 2δ )
The stability condition is |G| ≤ 1 or, since G is real: −1 ≤ G ≤ 1. Since we
have that 0 ≤ θ ≤ 1 , the condition of the right side is satisfied for D ≥ 0. For
the left side of the inequality we have that:

δ
2D sin2 (1 − 2θ) ≤ 1 (7.64)
2
or
1 δ
D(1 − 2θ) ≤ da sin2 ≤1 (7.65)
2 2
For 12 ≤ θ ≤ 1, stability is given for all D ≥ 0, ie.: the scheme is uncondition-
ally stable.
For 0 ≤ θ ≤ 12 , the stability condition requires that:

1
D(1 − 2θ) ≤ , 0≤θ≤1 (7.66)
2
Use of derivation
Now we write:
+ e−iδ − 2) · (Gθ + 1 − θ) = 1 + 2D cos(δ) − 1 · (Gθ + 1 − θ)

iδ

G = 1 + D(e

dG d dG
= 2D (Gθ + 1 − θ) cos(δ) − 1 + cos(δ) − 1 θ ·
dδ dδ dδ
dG
With = 0 we get the max-min for δ = 0, δ = ±π (δ = 0 gives G = 1
dδ
1 − 4D(1 − θ)
as expected) δ = ±δ gives G = which is identical to (7.63) for
1 + 4Dθ
δ ◦ 1 − 4D(1 − θ)
2 = 90 . The condition |G| ≤ 1 becomes −1 ≤ 1 + 4Dθ
≤ 1 with the
same results as before.
Accuracy
Let us now look at the accuracy of the scheme presented in (7.62).
To do so, we write a simple Maple-program:
> eq1:= u(x+h,t+k) - 2*u(x,t+k) + u(x-h,t+k):
> eq2:= u(x+h,t) - 2*u(x,t) + u(x-h,t):
> eq:= (u(x,t+k) - u(x,t))/k -(theta*eq1 + (1-theta)*eq2)/h^2:
> Tnj:= mtaylor(eq,[h,k]):
> Tnj:= simplify(Tnj):
> Tnj:= convert(Tnj,diff);
Note that we have used h = ∆x and k = ∆t. T nj is the truncation error,

which is discussed in detail in Section 7.6. Suppose now that u(x, t) is the
∂u ∂2u
analytical solution for the partial differential equation = so that we can
∂t ∂x2
2
∂ ∂
say that () = , etc. If we use these relationships in the Maple-program
∂t ∂x2
for T nj, we will obtain the following result:
4
∂6u

1 1 ∂ u 1
Tjn = − θ · ∆t − (∆x)2 · 4
+ (∆t)2 (1 − 3θ) · + . . . (7.67)
2 12 ∂x 6 ∂x6
We see that the truncation error is Tjn = O(∆t) + O(∆x)2 for θ = 0

and 1, i.e. for Euler-scheme and Laasonen-scheme, while it becomes Tjn =
O(∆t)2 + O(∆x)2 for θ = 12 , which corresponds to the Crank-Nicolson scheme.
If we add the following line
Tnj:= simplify(subs(theta =(1-h^2/(6*k))/2,Tnj));
to the Maple-program, we find that Tjn = O(∆t)2 + O(∆x)4 if we choose

1 1
D= when D ≤ .
6(1 − 2θ) 2(1 − 2θ)
More about stability
We have found that the Crank-Nicolson scheme is unconditionally stable for
1
2 ≤ θ ≤ 1. In practice, the scheme can cause oscillations around discontinuities
for θ = 12 . Values of θ above 0.5 will dampen such oscillations with strongest
attenuation for θ = 1. Let’s now look at an example where we use the heat
equation in dimensional form.
100mm 100mm 100mm
Figure 7.7: Thin aluminum rod for one-dimensional heat equation example.
Figure 7.7 shows a thin aluminum rod of length equal to 300 mm.
The one-dimensional heat equation is given by:
∂T ∂2T
= α 2 , T = T (x, t)
∂t ∂x
Boundary conditions are:
T (0, t) = T (300, t) = 20◦ C,

while initial conditions are:
T (x, 0) = 270◦ C for x ∈ (100, 200)

T (x, 0) = 20◦ C for x ∈ (0, 100) and x ∈ (200, 300)
Thermal diffusivity is:
α = 100mm2 /s
The numerical Fourier number is:
∆t
D=α
(∆x)2
√
Furthermore, we set ∆t = 0.25 s so that ∆x = 5/ D. By selecting D = 4,
we get that ∆x = 2.5 mm. Figures 7.8 and 7.9 show computational results with
θ = 12 and with θ = 1; the Crank-Nicolson and Laasonen schemes, respectively.
We see that the solution obtained using the C-N-scheme contains strong
oscillations at discontinuities present in the initial conditions at x = 100 of
x = 200. These oscillation will be dampen by the numerical scheme, in this
Figure 7.8: Solution for the one-dimensional heat equation problem using
Crank-Nicolson scheme.
Figure 7.9: Solution for the one-dimensional heat equation problem using
Laasonen scheme.
case they tend to disappear for t = 4 s. The solution delivered by the Laasonen-
scheme presents no oscillations, even around t = 0. To explain the behavior of
these solutions we have to go back to equation (7.63):
1 − 4D(1 − θ) sin2 ( 2δ )
G=
1 + 4D sin2 ( 2δ )
1
For θ = 2 we get:
1 − 2D sin2 ( 2δ )
G=
1 + 2D sin2 ( 2δ )
For δ near π, G will have a value close to −1 for large values of D. This can
1 − 2D
be clearly seen if one sets δ = π = β · h, which results in G = .
1 + 2D
From (7.44) we have:
Ejn = Gn · Ej0 , Ej0 = ei·βxj

Therefore, we see that we will get oscillations for δ close to π. These high
wave numbers will be dampened as the simulation proceeds because the initial
condition discontinuities are smoothed by dissipation. If we set D = 4 as
used in this example, after 16 time increments with ∆t = 0.25s we get that
16
7
16
G = − ≈ 0.018, whereas with D = 1 we would obtain G16 = 2.3 · 10−8 .
9
This means that high wave-numbers oscillations dampening slows down for large
values of D. This is very similar to what happens in the case of the Gibbs-
phenomenon for Fourier series of discontinuous functions (See Appendix A.3, in
the Numeriske Beregninger).
On the other hand, for the Laasonen-scheme, i.e. the θ-scheme, we have that:
1
G=
δ
1 + 4D sin2
2
We do not get oscillations for any wave number.
If we use terms from the stability analysis of ordinary differential equations
we can say that the θ-scheme is absolute stable for (A-stable) for θ = 12 and
L-stable for θ = 1 . (See Section 1.6.1 in the Numeriske Beregninger)
If we use the θ-scheme for the heat equation in a problem setting as the one
described above with prescribed temperature at both ends (Dirichlet boundary
conditions), we know that the maximum temperature at any given time must be
between the largest value of the initial condition and the smallest given on the
edge of the domain, or Tmin ≤ Tjn ≤ Tmax . For the example presented above
one has Tmin = 20◦ C and Tmax = 270◦ C.
Writing (7.62) for un+1
j yields:
1
un+1 θD(un+1 n+1 n n

j = j−1 + uj+1 ) + (1 − θ)D(uj−1 + uj+1 ) + 1 − (1 − θ)2D
1 + 2θD
(7.68)
By summing the coefficients on the right-hand side we find that the sum
is equal to 1. Next, we check for which conditions the coefficients are positive,
which is equivalent to require that 0 < θ < 1 and 1 − (1 − θ) · 2D > 0. The last
inequality is met for D · (1 − θ) < 12 . By setting θ = 0 and θ = 1, we find that
the sum is equal to 1 for these values, so that we get the following condition for
fulfilling the PC-criterion:
1
D · (1 − θ) ≤ (7.69)
2
By the von Neumann-analysis we found that the following relation should
hold (see (7.66)):
1
D · (1 − 2θ) ≤ (7.70)
2
We see that (7.69) is significantly stricter than (7.70). While the θ-scheme is
unconditionally stable for θ = 12 according to the von Neumann stability analysis,
we must have D ≤ 1 according to the PC-criterion. The PC-criterion provides
a sufficient condition which ensures that the physical max-min criterion is also
fulfilled by the solution delivered by the numerical scheme. The example shown
above confirms the prediction provided by the PC-criterion. Now, is there a
criterion that is both necessary and sufficient for this simple model equation?
Kraaijevanger found the following necessary and sufficient criterion in 1992:
2−θ
D · (1 − θ) ≤ (7.71)
4(1 − θ)
We see that for θ = 12 this criterion implies the condition D ≤ 32 . For θ = 34
it gives (7.71) D ≤ 5, while the PC-criterion yields D ≤ 2.
The purpose of allowing for large D values is related to the fact that one
might be interested in simulations where a large time step is set to let reach
a steady state. But on must not forget about accuracy. Moreover, we should
remember that this is a simple one-dimensional partial differential equation with
constant coefficients, and as such it can be solver very quickly with a modern
PC with for any reasonable ∆t and ∆x, as long as we stay within the stability
area. However, three-dimensional non-linear problems will in general result in
very computational demanding simulations.
7.5.4 Generalized von Neumann stability analysis

We have referred to the stability condition |G| ≤ 1 as the von Neumann’s strong
stability condition. The reason for this designation is that if |G| ≤ 1 is fulfilled,
then the amplitude of the solution can not increase. However, there are many
physical problems for which amplitude grows in time (in a bounded manner). A
simple example is that of heat conduction with a source such as:
∂T ∂2T
= α 2 + bT, b = const., t < tmax (7.72)
∂t ∂x
b < 0 represents a heat sink, while b > 0 is said to be a heat source. In the
first case we can apply the von Neumann strong criterion, whereas in the later
case it is necessary to allow for |G| > 1.
A particular solution of (7.72) is given by:

2 2
T (x, t) = ebt · e−αβ ·t
cos(βx) = e(b−αβ )·t
cos(βx) (7.73)
Now, let us use (7.73) to determine an analytical growth factor, see (7.39):
T (xj , tn+1 ) 2
= exp (b − αβ 2 ) · tn+1 − (b − αβ 2 ) · tn = eb∆t · e−αβ ∆t

Ga =
T (xj , tn )
(7.74)
We see that the term eb·∆t causes increases of amplitude for positive b.
A series expansion for small ∆t:
b2
eb∆t = 1 + b · ∆t + (∆t)2 + . . . (7.75)
2
If we use the FTCS-scheme we obtain:
Tjn+1 = D(Tj+1
n n
+ Tj−1 ) + (1 − 2D)Tjn + ∆tbTjn (7.76)
with
∆t
D=α (7.77)
(∆x)2
If we use the PC-criterion we find that the sum of coefficients is equal to
1 + b · ∆t, which implies that this criterion can only be used for b < 0.
On the other hand, using the von Neumann’s method yields (7.76):
δ
G = 1 − 4D sin2 + ∆t · b (7.78)
2
By setting D = 12 , we have:
δ
|G| ≤ 1 − 2 sin2

+ |∆t · b| ≤ 1 + ∆t · b, b > 0
2
If we compare this expression with (7.75), we see that the agreement between
the analytical and the numerical growth factor for this case.
We can now introduce the generalized von Neumann’s stability condition:
|G| ≤ 1 + K · ∆t (7.79)
with K being a positive constant.
This means that we allow the amplitude to grow exponentially for t < tmax .
In this case we can use the strong stability condition if we reason this way: since
the source term in (7.72) doe not contain any derivative term, we can ignore
this term in the stability analysis. We have the same kind of problem in Section
1.6 in the Numeriske Beregninger, where stiff ordinary differential equations are
discussed (See for example equation (1.6.8) in Section 1.6.1 of the Numeriske
Beregninger).
For a growing amplitude , we must decrease the step lenth of the independent
variable if we are to achieve a prescribed accuracy. For a decreasing amplitude,
however, we must keep it under a maximum step lenth to get a stable calculation.
Let us look at another example using the FTCS-scheme.
∂u ∂2u ∂u
= α 2 + a0 , α > 0 (7.80)
∂t ∂x ∂x
This equation is called the advection-diffusion equation and is a parabolic
equation according to the classification scheme discussed in Section 4.3 of the
∂u
Using the FTCS scheme for (7.80) with central differences for :
∂x
∆t n ∆t
un+1 = unj + D · (unj+1 − 2unj + unj−1 ) + a0 (u − unj−1 ), D = α
j
2∆x j+1 (∆x)2
Using von Neumann’s method we get that:

δ ∆t
G = 1 − 4D sin2 + i · a0 sin(δ)
2 ∆x
which further provides:
δ 2 ∆t 2 2 a2 · D
2 δ
2
|G| = 1 − 4D sin 2
+ a0 sin(δ) = 1 − 4D sin + 0 ·∆t sin2 (δ)
2 ∆x 2 α
Setting now D = 12 :
s
δ 2
a20 a2
|G| = 1 − 2 sin2 · ∆t · sin2 (δ) ≤ 1 + 0 · ∆t
+
2 2α 2α
p
Here we have used the inequality x2 + y 2 ≤ |x| + |y|
a2
With K = 2α0 , we see that the generalized condition (7.79) is met.
The advection-diffusion equation is discussed in detail in Section6.10 of the
7.6 Truncation error, consistency and convergence

7.6.1 Truncation error
Let U (x, t) be the exact solution of a PDE, written as L(U ) = 0, where L(U)
is the differential operator, and u the numerical solution given by a generic
numerical scheme, also written in operator form as F (u) = 0. In this last case
F (u) is the so-called numerical operator. The exact solution at (xi , tn ) is given
by:
Uin ≡ U (xi , tn ) der xi = i · ∆x = i · h, i = 0, 1, 2, . . . (7.81)

tn = n · ∆t = n · k, n = 0, 1, 2, . . . (7.82)
We define the local truncation error Tin as:
Tin = F (Uin ) − L(Uin ) = F (Uin ) (7.83)

Since in general the exact solution U (x, t) is not available, one finds Tin
by expressing the exact solution at required space-time locations using Taylor
expansions.
Some expansions are given to illustrate the concept:
n n n
n ∂U h2 ∂ 2 U h3 ∂ 3 U
Ui±1 ≡ U (xi±h , tn ) = Uin ± h · + · ± · + . . . (7.84)
∂x i 2 ∂x2 i 6 ∂x3 i
n n n
∂U k 2 ∂ 2 U k 3 ∂ 3 U
Uin±1 ≡ U (xi , tn±k ) = Uin ±k· + · ± · + . . . (7.85)
∂t i 2 ∂t2 i 6 ∂t3 i
For example, let us find the local truncation error Tin for the FTCS scheme
applied to the diffusion equation. In this case the analytical operator is
∂U ∂2U
L(U ) = − =0
∂t ∂x2
Uin+1 − Uin U n − 2Uin + Ui+1
n
Tin = F (Uin ) = − i−1 (7.86)
k h2
Replacing (7.84) in (7.86):
n n n
∂2U k ∂2U h2 ∂ 4 U k 2 ∂ 3 U

∂U
Tin = − + − + + O(k 3 , h4 )
∂t ∂x2 i 2 ∂t2 12 ∂x4 i 6 ∂t3 i
∂U ∂2U
Da − =0
∂t ∂x2
2 n
k∂ U h2 ∂ 4 U
Tin = − + higher order terms (7.87)
2 ∂t2 12 ∂x4 i
(7.87) shows that Tin = O(k) + O(h2 ), as expected.
(7.87) can also be written as:
n
h2 k ∂2U ∂4U

n
Ti = · 6 2 2 − + O(k 2 ) + O(h4 )
12 h ∂t ∂x4 i
k 1
By choosing D = 2 = , we get:
h 6
Tin = O(k 2 ) + O(h4 ) (7.88)
∆t becomes very small for D = 1/6, but with today’s computers this should
not be a problem, except for the possible accumulation of rounding errors.
7.6.2 Consistency
We say that a the discretization of a differential equation, i.e. the numerical
scheme, is consistent with the original differential if the local truncation error
Tin → 0 when ∆x and ∆t → 0 independent of each other.
7.6.3 Example: Consistency of the FTCS-scheme

From (7.87)
n
k ∂2U h2 ∂ 4 U

Tin = 2
− → 0 for h and k → 0
2 ∂t 12 ∂x4 i
This means that the FTCS scheme is consistent with the diffusion equation.
7.6.4 Example: Consistency of the DuFort-Frankel scheme

Let us look at the DuFort-Frankel scheme introduced in Section efch5:sec42:
Uin+1 − Uin−1 − (Uin+1 + Uin−1 )

n n

n Ui−1 + Ui+1
Ti = −
2k h2
Using the series expansions (7.84) and (7.85):
" 2 2 # n 2 3 n
∂2U h2 ∂ 4 U
4
∂U k ∂ U k ∂ U k
Tin = − + + − +O 4
,k ,h 4
∂t ∂x2 h ∂t2 6 ∂t3 12 ∂x4 i h2
i
(7.89)
2
k
Due to the factor it is important to specify how k and h → 0. The
h
scheme is not necessarily consistent with the underlying differential equation.
Such schemes are normally called conditionally consistent.
Case 1
k
Set r0 = → k = r0 · h, and let r0 = be a positive constant.
h
Inserting r0 in (7.89):
n
∂2U 2

n ∂U 2 ∂ U
Ti = − + r0 · + O(h2 )
∂t ∂x2 ∂t2 i
For h → 0, we see that the DuFort-Frankel scheme is consistent with the
∂U ∂2U ∂2U
hyperbolic equation + r02 2 = and not with the original diffusuion
∂t ∂t ∂x2
equation.
Case 2
k
Setting r0 = 2 → k = r0 · h2 . Inserting r0 in (7.89):
h
h in h in
∂U ∂2U 2
k2 ∂ 3 U h2 ∂ 4 U k4
Tin = ∂t − ∂x2 + r02 h2 ∂∂tU2 + 3 − 12 ∂x4+O 4
h2 , k , h
4
h 2
i
2 3
in6 ∂t i
h2 ∂ 4 U
= r02 h2 ∂∂tU2 + k6 ∂∂tU3 −

12 ∂x4 + O r04 h6 , k 4 , h4
i
n
∂2U

∂U
da − =0
∂T ∂x2 i
We see that in this case Tin → 0 for h and t → 0 with Tin = O(k 2 ) + O(h2 ).
The scheme is now consistent with the diffusion equation. Therefore, the
DuFort-Frankel scheme can be used with k = r0 · h2 . However, this poses a
restriction in ∆t, arising from consistency constrains and not from stability
considerations. Non-consistent schemes usually arise when we change the scheme
after we have made the Taylor expansions in the usual way.
7.6.5 Convergence
Lucas 5: this must be considerably improved. No concepts given to understand
convergence properly
It is generally difficult to prove the convergence of a difference scheme.
Therefore, many attempts have been made to replace the above definition with
conditions that are easier to prove individually but which together are sufficient
for convergence.
A very important result in this direction is given by the following theorem
Lax’s equivalence theorem: Given a well-posed linear initial value

problem and a consistent numerical scheme, stability of the same scheme
is a necessary and sufficient condition for convergence.
See Section 4.3 in the Numeriske Beregninger for more details. When we see
all the conditions that must be fulfilled in order for Lax’s theorem to be used,
we understand the difficulties of proving convergence in more general issues.
7.7 Example with radial symmetry

2 2 2
If we write the diffusion equation ∂u ∂ u ∂ u ∂ u
∂t = ∂x2 + ∂y 2 + ∂z 2 for cylindrical and
spherical coordinates and require u to be only a function of time t and radius r,
we have that:
Cylinder:
∂u ∂ 2 u 1 ∂u
= +
∂t ∂r2 r ∂r
Sphere:
∂u ∂ 2 u 2 ∂u
= +
∂t ∂r2 r ∂r
Both equations can be written as:
∂u ∂ 2 u λ ∂u
= + , λ = 0, 1, 2 (7.90)
∂t ∂r2 r ∂r
λ = 0 with r → x gives the well-known Cartesian case.
(7.90) is a partial differential equation with variable coefficients. We will now
perform a von Neumann stability analysis for this equation for the θ-scheme
from Section (7.5.3).
Stability analysis using θ -scheme for radius r > 0
∆t
Setting rj = ∆r · j, j = 0, 1, . . . and introducing D = (∆r) 2
For r > 0:
un+1 = unj + D θ(un+1 n+1

+ un+1
n n n

j j+1 − 2uj j−1 ) + (1 − θ)(uj+1 − 2uj + uj−1 )
n+1 n+1
+ λD n n

2j θ(uj+1 − uj−1 ) + (1 − θ)(uj+1 − uj−1 )
(7.91)
We proceed with the von Neumann analysis by inserting Ejn = Gn · ei·βrj =
Gn ei·δ·j with δ = β · ∆r and using the usual formulas (7.49) and (7.50).
We get:
δ δ (1 − θ)λD
θλD
G· 1 + 4θD sin2 −i· sin(δ) = 1−4(1−θ)D·sin2 +i sin(δ)
2 j 2 j
δ δ
which, using the formula sin(δ) = 2 sin cos and the condition |G| ≤
2 2
1, becomes:
δ 2 (1 − θ)2 λ2 D2 2 θ2 λ2 D2

2 2 2 δ
1 − 4(1 − θ)D sin + sin (δ) ≤ 1 + 4θD sin + sin2 (δ)
2 j2 2 j2
and which further provides:
λ2 λ2

2 δ

D · (1 − 2θ) · sin · 4 − 2 + 2 ≤ 2, j ≥ 1 (7.92)
2 j j
It is not difficult to see that the term in parenthesis has its largest value for
δ
2
sin = 1; i.e. for δ = π. (This can also be found by deriving the term with
2
λ2
respect to δ, which has its maximum for δ = π). Factors 2 fall then out.
j
We get:
D · (1 − 2θ) · 2 ≤ 1 (7.93)
As in Section 7.5.3, we must distinguish between two cases:

1.
1
0≤θ≤
2
∆t 1
D= ≤ (7.94)
(∆r)2 2(1 − 2θ)
2.
1
≤θ≤1
2
Next we multiply (7.93) by −1:
D · (2θ − 1) · 2 ≥ −1
This condition is always satisfied for the specified range of θ, so that in such
cases the scheme is unconditionally stable.
In other words, we have obtain the same stability condition as for the equation
∂u ∂2u
with constant coefficients: = , where we have the FTCS scheme for
∂t ∂r2
θ = 0, the Crank-Nicolson scheme for θ = 1/2, and the Laasonen scheme for
θ = 1.
Note that this analysis is only valid for r > 0.
We now have a look at the equation for r = 0.
λ ∂u
The term must be treated with special care for r = 0.
r ∂r
L’Hopital’s rule:
λ ∂u ∂2u ∂u ∂2u
lim =λ 2 → = (1 + λ) 2 for r = 0 (7.95)
r→0 r ∂r ∂r ∂t ∂r
We have found that using the FTCS scheme requires the usual constrain
D ≤ 1/2. Now we will check if boundary conditions pose some constrains when
using the FTCS scheme.
Version 1
Discretize (7.95) for r = 0 and use the symmetry condition ∂u
∂r (0) = 0:
un+1
0 = [1 − 2(1 + λ)D] · un0 + 2(1 + λ)D · un1 (7.96)
If we use the PC-criterion on (7.96), we get:
1
D≤ (7.97)
2(1 + λ)
For λ = 0 we get the well-known condition D ≤ 1/2, whereas for the cylinder
(λ = 1) we get D ≤ 1/4 and for the sphere, (λ = 2) we have D ≤ 1/6. The
question to be answered now is whether this conditions are for cylinder and
sphere are necessary. We know that D ≤ 1/2 is necessary and sufficient for
λ = 0.
It is difficult to find a necessary and sufficient condition in this case. Therefore,

we concentrate in a special case: flow start-up in a tube, which is described in
Example 7.7.1.
Version 2
∂u
To avoid using a separate equation for r = 0 , we discretize (0) = 0 with
∂r
a second order forward difference:
∂u −3un0 + 4un1 − un2 1
(0) = 0 → → un0 = (4un1 − un2 ) (7.98)
∂r 2∆r 3
For a sphere, there is a detailed analysis by Dennis Eisen in the Journal
Numerische Mathematik vol. 10, 1967, pages 397-409. The author shows that a
necessary and sufficient condition for the solution of (7.91), along with (7.96) (for
λ = 2 and θ = 0) is that D < 1/3. In addition, he shows that by avoiding the
use of (7.96), the stability constrain for the FTCS scheme results to be D < 1/2.
Next we will go through two cases to see which stability requirements are
obtained when one uses the two types of boundary conditions.
7.7.1 Example: Start-up flow in a tube
R0
R
Figure 7.10: Velocity profile in a pipe at a given time.
Figure 7.10 shows the velocity profile of an incompressible fluid in a tube at

a given time. The profile has reached the current configuration after evolving
from a static configuration (velocity equal to 0 over the entire domain) by
the application of a constant pressure gradient dp dz < 0, so that we also know
the velocity profile for the steady state configuration, which is the well-known
parabolic profile for Poiseuille flow.
Under the above mentioned conditions, the momentum equation reads:
2
∂U 1 dp ∂ U 1 ∂U
=− +ν + (7.99)
∂τ ρ dz ∂R2 R ∂R
with U = U (R, τ ). 0 ≤ R ≤ R0 , being the velocity profile and τ the physical
time.
Non-dimensional variables are:
τ R U Us R02 dp
t=ν , r = , u = , us = der k = − (7.100)
R02 R0 k k 4µ dz
which introduced in (7.99) yield:
∂u ∂ 2 u 1 ∂u
=4+ 2 + (7.101)
∂t ∂r r ∂r
Boundary conditions are:
∂u
u(±1, t) = 0,(0, t) = 0 (7.102)
∂r
The later is a symmetry condition. Finding a stationary solution requires
that ∂u
∂t = 0:
d2 us

1 dus 1 d dus
+ = −4 → r = −4 som gir:
dr2 r dr r dr dr
dus C1 dus (0)
= −2r + med C1 = 0 da =0
dr r dr
After a new integration and application of boundary conditions we obtain
the familiar parabolic velocity profile:
us = 1 − r2 (7.103)
We assume now that we have a fully developed profile as given in (7.103)
and that suddenly we remove the pressure gradient. From (7.99) we see that
this results in a simpler equation. The velocity ω(r, t) for this case is given by:
W
ω(r, t) = us − u(r, t) med ω = (7.104)
k
We will now solve the following problem:
∂ω ∂ 2 ω 1 ∂ω
= + (7.105)
∂t ∂r2 r ∂r
∂ω
ω(±1, t) = 0, (0, t) = 0 (7.106)
∂r
and initial conditions:
ω(r, 0) = us = 1 − r2 (7.107)
The original problem now reads:
u(r, t) = 1 − r2 − ω(r, t) (7.108)

The analytical solution of (7.105), (7.108), first reported by Szymanski in

1932, can be found in Appendix G.8 of the Numeriske Beregninger.
Let us now look at the FTCS scheme.
From (7.91), with λ = 1, θ = 0 and j ≥ 0:
D
ωjn+1 = ωjn + D · (ωj+1
n
− 2ωjn + ωj−1
n
)+ n
· (ωj+1 n
− ωj−1 ) (7.109)
2j
For j = 0 (7.96) yields:
ω0n+1 = (1 − 4D) · ω0n + 4D · ω1n (7.110)

1
From (7.94) we obtain that the stability range is 0 < D ≤ for r > 0 and
2
0 < D ≤ 14
from (7.97) for r = 0, which is an adequate condition.
By solving (7.109), we get the following table for stability limits:
∆r D
0.02 0.413
0.05 0.414
0.1 0.413
0.2 0.402
0.25 0.394
0.5 0.341
For satisfactory accuracy, we should have that ∆r ≤ 0.1. From the table
above we see that in terms of stabilty, D < 0.4 is sufficient. In order words, a
sort of mean value between D = 12 og D = 14 .
The trouble causing equation is (7.110). We can avoid this problem by using
the following equation instead of (7.98):
1
ω0n = (4ω1n − ω2n ), n = 0, 1, . . . (7.111)
3
A new stability analysis shows then that the new stability condition for the
entire system is now 0 < D ≤ 12 for the FTCS scheme. Figure 7.11 shows the
velocity profile u for D = 0.45 with ∆r = 0.1 after 60 time increments using
(7.110). The presence of instabilities for r = 0 can be clearly observed.
Marit 6: Har ikke endret noe her
Programming of the θ-scheme:
Boundary conditions given in (7.110)
We set rj = ∆r · (j), ∆r = N1 , j = 0, 1, 2, . . . , N as shown in Fig. 7.12. From
Eq. (7.91) and (7.95) we get:
For j = 0:
(1 + 4D · θ) · ω0n+1 − 4D · θ · ω1n+1 = ω0n + 4D(1 − θ) · (ω1n − ω0n ) (7.112)

Figure 7.11: Velocity profile for start-up flow in a tube for different time step
obtained using the FTCS scheme. Stability development can be observed.
0.0 1.0
r
0 1 2 i N-1 N N+1
Figure 7.12: Spatial discretization for the programing of the θscheme.
For j = 1, . . . , N − 1:

n+1
−Dθ · 1 − 1
· ωj−1
2j + (1 + 2Dθ) · ωjn+1 − Dθ · 1 + 1
2j
n+1
· ωj+1

1 n
= D(1 − θ) · 1 − 2j · ωj−1 + [1 − 2D(1 − θ)] · ωjn (7.113)

1 n
+D(1 − θ) · 1 + 2j · ωj+1
Initial values:
ωj0 = 1 − rj2 = 1 − [∆r · j]2 , j = 0, . . . , N (7.114)

In addition we have that ωN = 0 for every n-values.
The system written in matrix form now becomes:

1 + 4Dθ −4Dθ 0 0 0 0 0
 −Dθ(1 − 0.5 ) 1 + 2Dθ −Dθ(1 + 0.5
1 1 ) 0 0 0 0
0.5 0.5


 0 −Dθ(1 − 2 ) 1 + 2Dθ −Dθ(1 + 2 ) 0 0 0

 0 0 · · · 0 0
 0 0 0 · · · 0
−Dθ(1 − N0.5

 0 0 0 0 −2 ) 1 + 2Dθ −Dθ(1 +
0 0 0 0 0 −Dθ(1 − N0.5
−1 ) 1 + 2D
(7.115)
Note that we can also use a linalg-solver for θ = 0 (FTCS-scheme). In
this case the elements in the first upper- and lower diagonal are all 0. The
determinant of the matrix is now the product of the elements on the main
diagonal. The criterion for non singualar matrix is then that all the elements on
the main diagonal are 6= 0, which is satisfied.
The python function thetaSchemeNumpyV1 show how one could imple-
ment the algorithm for solving wn+1 . The function is part of the script/module
startup.py which may be downloaded in your LiClipse workspace.
# Theta-scheme and using L’hopital for r=0
def thetaSchemeNumpyV1(theta, D, N, wOld):
""" Algorithm for solving w^(n+1) for the startup of pipeflow
using the theta-schemes. L’hopitals method is used on the
governing differential equation for r=0.
Args:
theta(float): number between 0 and 1. 0->FTCS, 1/2->Crank, 1->Laasonen
D(float): Numerical diffusion number [dt/(dr**2)]
N(int): number of parts, or dr-spaces. In this case equal to the number of unknowns
wOld(array): The entire solution vector for the previous timestep, n.
Returns:
wNew(array): solution at timestep n+1
"""
superDiag = np.zeros(N - 1)
subDiag = np.zeros(N - 1)
mainDiag = np.zeros(N)
RHS = np.zeros(N)
j_array = np.linspace(0, N, N + 1)
tmp = D*(1. - theta)
superDiag[1:] = -D*theta*(1 + 0.5/j_array[1:-2])

mainDiag[1:] = np.ones(N - 1)*(1 + 2*D*theta)
subDiag[:] = -D*theta*(1 - 0.5/j_array[1:-1])
a = tmp*(1 - 1./(2*j_array[1:-1]))*wOld[0:-2]
b = (1 - 2*tmp)*wOld[1:-1]
c = tmp*(1 + 1/(2*j_array[1:-1]))*wOld[2:]
RHS[1:] = a + b + c
superDiag[0] = -4*D*theta
mainDiag[0] = 1 + 4*D*theta
RHS[0] = (1 - 4*tmp)*wOld[0] + 4*tmp*wOld[1]
A = scipy.sparse.diags([subDiag, mainDiag, superDiag], [-1, 0, 1], format=’csc’)
wNew = scipy.sparse.linalg.spsolve(A, RHS)

wNew = np.append(wNew, 0)
return wNew
Boundary condition given in (7.98)

Now let us look at the numerical solution when using the second order forward
difference for the bc at r = 0 repeated here for convinience:
1
ω0n = (4ω1n − ω2n ), n = 0, 1, 2 . . . (7.116)
3
The difference equation for the θ-scheme for j = 1, . . . , N − 1 are the same
as in the previous case:

n+1
−Dθ · 1 − 1
2j · ωj−1 + (1 + 2Dθ) · ωjn+1 − Dθ · 1 + 1
2j
n+1
· ωj+1

1 n
= D(1 − θ) · 1 − 2j · ωj−1 + [1 − 2D(1 − θ)] · ωjn (7.117)

1 n
+D(1 − θ) · 1 + 2j · ωj+1
Now inserting Eq. (7.116) into Eq. (7.117) and collecting terms we get the
following difference equation for j = 1:
4 4 4 4
(1 + Dθ) · ω1n+1 − Dθω2n+1 = [1 − D(1 − θ)] · ω1n + D(1 − θ)ω2n (7.118)
3 3 3 3
When ω1 , ω2 , . . . are found, we can calculate ω0 from (7.116).
The initial values are as in Eq. (7.114), repeated here for convonience:
ωj0 = 1 − rj2 = 1 − (∆r · j)2 , j = 0, . . . , N + 1 (7.119)

The system written in matrix form now becomes:
1 + 34 Dθ − 43 Dθ

0 0 0 0 0
0.5
 −Dθ(1 − 2 ) 1 + 2Dθ −Dθ(1 + 0.5
2 ) 0 0 0 0
−Dθ(1 − 0.5 0.5


 0 3 ) 1 + 2Dθ −Dθ(1 + 3 ) 0 0 0

 0 0 · · · 0 0
 0 0 0 · · · 0
−Dθ(1 − N0.5

 0 0 0 0 −2 ) 1 + 2Dθ −Dθ(1 +
0 0 0 0 0 −Dθ(1 − N0.5
−1 ) 1 + 2D
(7.120)
The python function thetaSchemeNumpyV2 show how one could imple-

ment the algorithm for solving wn+1 . The function is part of the script/module
startup.py which may be downloaded in your LiClipse workspace.
In Fig. 7.13 we see that the stability-limit for the two version is different for
the to versions of treating the BC, for FTCS. Using L’hopital’s rule for r=0
give a smaller stability-limit, and we see that unstability arises at r=0, befor the
general stability-limit for the FTSC scheme (D = 1/2).
# Theta-scheme and using 2nd order forward difference for r=0
def thetaScheme_numpy_V2(theta, D, N, wOld):
""" Algorithm for solving w^(n+1) for the startup of pipeflow
using the theta-schemes. 2nd order forward difference is used
on the von-Neumann bc at r=0.
Args:
theta(float): number between 0 and 1. 0->FTCS, 1/2->Crank, 1->Laasonen
D(float): Numerical diffusion number [dt/(dr**2)]
N(int): number of parts, or dr-spaces.
wOld(array): The entire solution vector for the previous timestep, n.
Returns:
wNew(array): solution at timestep n+1
"""
superDiag = np.zeros(N - 2)
subDiag = np.zeros(N - 2)
mainDiag = np.zeros(N-1)
RHS = np.zeros(N - 1)
j_array = np.linspace(0, N, N + 1)
tmp = D*(1. - theta)
superDiag[1:] = -D*theta*(1 + 0.5/j_array[2:-2])

mainDiag[1:] = np.ones(N - 2)*(1 + 2*D*theta)
subDiag[:] = -D*theta*(1 - 0.5/j_array[2:-1])
a = tmp*(1 - 1./(2*j_array[2:-1]))*wOld[1:-2]
b = (1 - 2*tmp)*wOld[2:-1]
c = tmp*(1 + 1/(2*j_array[2:-1]))*wOld[3:]
RHS[1:] = a + b + c
superDiag[0] = -(4./3)*D*theta
mainDiag[0] = 1 + (4./3)*D*theta
RHS[0] = (1 - (4./3)*tmp)*wOld[1] + (4./3)*tmp*wOld[2]
A = scipy.sparse.diags([subDiag, mainDiag, superDiag], [-1, 0, 1], format=’csc’)
wNew = scipy.sparse.linalg.spsolve(A, RHS)

w_0 = (1./3)*(4*wNew[0] - wNew[1])
wNew = np.append(w_0, wNew)

wNew = np.append(wNew, 0)
return wNew
FTCSV1 FTCSV2 Crank-Nicolson Laasonen

1.0
radius [-]
0.8
D = 0.4 0.6
0.4
0.2
0.0
1.0
radius [-]
0.8
D = 0.45 0.6
0.4
0.2
0.0
1.0
radius [-]
0.8
D = 0.5 0.6
0.4
0.2
0.0
1.0
radius [-]
0.8
D = 0.53 0.6
0.4
0.2
0.0
1.0
radius [-]
0.8
D=1 0.6
0.4
0.2
0.0
0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0
Velocity [-] Velocity [-] Velocity [-] Velocity [-]
Figure 7.13: Results for stability tests performed for the two versions of the
FTCS scheme, the Crank-Nicolson and the Laasonen schemes. Note that version
1 of FTCS scheme develops instabilities before D = 0.5, and that instabilities
start at r = 0. For version 2 of FTCS scheme instability occurs for D = 0.5 and
for all values of r, as expected.
Lucas 7: I do not see the point of this video, one should use a spatial
resolution that somehow illustrates differences in accuracy, if all results are mesh
independent you can not make any interesting observation ...
Movie 5: Animation of numerical results obtained using three numeri-

cal schemes as well as the analytical solution. mov-ch5/startup.mp4
emfs /src-ch5/ #python #package startup.py @ git@lrhgit/tkt4140/src/src-ch5/startup_compendium.py; V
7.7.2 Example: Cooling of a sphere

Figure 7.14 shows our problem setup, which consists of a sphere immersed in
water and exchanging heat with it. The sphere radius is b = 5 and its temperature
before it is immersed in water is Tk . Moreover, we assume that the water keeps
a constant temperature Tv throughout the entire process and we neglect heat
exchange with the environment.
Additional data are
• Heat conductivity: k = 0.1W/(cm ·◦ C)

• Heat transfer coefficient: h̄ = 0.2W/(cm ·◦ C)
• Thermal diffusivity: α = 0.04cm2 /s
h̄ · b
We have chosen the above reported coefficients so that = 1, since this
k
choice leads to a simple analytical solution. Moreover, such values correspond to
characteristic values found for nickel alloys.
Tv
b
Tk
Figure 7.14: Problem setup for example on the cooling of a sphere.
We must now solve the following problem with T = T (r, t):

2
∂T ∂ T 2 ∂T
=α + , (7.121)
∂t ∂r2 r ∂r
∂T
(0, t) = 0, (symmetribetingelse) (7.122)
∂r
For r = b :
∂T
k = h̄ · (Tv − Tb ) (7.123)
∂r
Initial conditions are:
T (r, 0) = Tk (7.124)
In this example we use the FTCS scheme, but the derivation can be easily
extended to the θ-scheme as shown in Example 7.7.1. With rj = ∆r · j, ∆r =
1 ∆t
N +1 , j = 0, 1, . . . , N + 1 and D = α (∆r)2 from (7.91) we get that when
u(r, t) → T (r, t), θ = 0 and λ = 2:
Tjn+1 = (1 − 2D) · Tjn + D[(1 − 1/j) · Tj−1

n n
+ (1 + 1/j) · Tj+1 ], j = 1, 2, . . . (7.125)
As in Example 7.7.1, we employ two versions of the symmetry condition for

r = 0. 1)
From (7.96) with λ = 2:
T0n+1 = (1 − 6D) · T0n + 6D · T1n (7.126)

2) From (7.98):
1
T0n = (4T1n − T2n ), alle n (7.127)
3
Boundary conditions for r = b.
Discretizing (7.122) using 2nd order backward differences:
n
3TN +1 − 4TNn + TNn −1

k· = h̄ · (Tv − Tb )
2 · ∆r
solving for TNn +1 yields:
4TNn − TNn −1 + 2δ · Tv
TNn +1 = (7.128)
3 + 2δ
with
∆r · h̄
δ= (7.129)
k
We have previously shown that we must have D < 1/3 when we use boundary
condition (7.126). In Example 7.7.1 for a cylindrical domain, we found that the
stability limit of D increased when we reduced ∆r using the FTCS scheme. On
the other hand, for the spherical case it appears that the condition D < 1/3 is
independent of ∆r. The reason for this can be understood by writing (7.125) for
j = 1:
T1n+1 = (1 − 2D) · T1n + 2D · T2n

n
The term (1 − 1/j) · Tj−1 = (1 − 1) · T0n vanishes for j = 1, so that the
temperature in the center of the ball does not affect any other point. This
explains why the stability limit is independent of ∆r. We can actually solve
for j = 1, 2, . . . , N without bothering about boundary conditions (7.126) and
(7.127) (see (??in [?]) for further details). For boundary condition (7.127) we
only need to find the temperature T0n at the center of the sphere.
von Neumann stability analysis without considering the boundary conditions
showed that (7.125) is stable for D ≤ 1/2. In addition, the analysis showed
that the influence of the variable coefficient disappeared for both, the cylindrical
and the spherical cases (see discussion in connection with (7.92)). We have not
shown that D ≤ 1/2 is an adequate condition for the entire system, since that
requires to include the boundary condition (7.128).
We can summarize the usage of the FTCS scheme for the spherical case as
follows:
The scheme in (7.128) is stable for D < 1/2 for j = 1, 2, . . .
If the temperature at the center of the sphere is also required we must ensure
that D < 1/3 when (7.126) is used.
Using (7.127), the temperature at the center of the sphere can be computed
for D < 1/2.
This is in agreement with Eisen’s analysis, mentioned in connection to (7.98).
Below we show the printout resulting from running program kule for a
computation where we used D = 0.4 and ∆r = 0.1cm. Tk = 300◦ C, Tv = 20◦ C
and a simulation time of 10 minutes, with time steps of 1 second. The results for
the analytical solution are computed using the program function kanalyt. We
see that there is a good agreement between analytical and numerical values (the
analytical solution is derived in Appendix G.9 of the Numeriske Beregninger.)
r(cm) T (◦ C) Ta r(cm) T (◦ C) Ta
0.00 53.38 53.37 2.60 49.79 49.78
0.10 53.37 53.36 2.70 49.52 49.51
0.20 53.36 53.35 2.80 49.24 49.23
0.30 53.33 53.32 2.90 48.95 48.94
0.40 53.29 53.28 3.00 48.65 48.64
0.50 53.24 53.23 3.10 48.35 48.34
0.60 53.18 53.17 3.20 48.03 48.03
0.70 53.11 53.10 3.30 47.71 47.71
0.80 53.03 53.02 3.40 47.38 47.38
0.90 52.93 52.93 3.50 47.05 47.04
1.00 52.83 52.82 3.60 46.70 46.70
1.10 52.72 52.71 3.70 46.35 46.35
1.20 52.59 52.58 3.80 46.00 45.99
1.30 52.46 52.45 3.90 45.63 45.63
1.40 52.31 52.30 4.00 45.26 45.26
1.50 52.16 52.15 4.10 44.89 44.88
1.60 51.99 51.98 4.20 44.50 44.50
1.70 51.81 51.81 4.30 44.11 44.11
1.80 51.63 51.62 4.40 43.72 43.71
1.90 51.43 51.42 4.50 43.32 43.31
2.00 51.22 51.22 4.60 42.92 42.91
2.10 51.01 51.00 4.70 42.51 42.50
2.20 50.78 50.78 4.80 42.09 42.09
2.30 50.55 50.54 4.90 41.67 41.67
2.40 50.30 50.30 5.00 41.25 41.24
2.50 50.05 50.04
Movie 6: Animation for example on the cooling of a sphere using the

FTCS scheme, as well as analytical solution. mov-ch5/sphere.mp4
Chapter 8
Convection problems and

hyperbolic PDEs
8.1 The advection equation

The classical advection equation is very often used as an example of a hyper-
bolic partial differential equation which illustrates many features of convection
problems, while still being linear:
∂u ∂u
+ a0 =0 (8.1)
∂t ∂x
Another convenient feature of the model equation (8.1) is that is has an
analytical solution:
u = u0 f (x − a0 t) (8.2)
and represents a wave propagating with a constant velocity a0 with unchanged
shape. When a0 > 0, the wave propagates in the positive x-direction, whereas
for a0 < 0, the wave propagates in the negative x-direction.
Equation (8.1) may serve as a model-equation for a compressible fluid, e.g if
u denote pressure it represents a pressure wave propagating with the velocity a0 .
The advection equation may also be used to model the propgation of pressure or
flow in a compliant pipe, such as a blood vessel.
To allow for generalization we will also when appropriate write (8.1) on the
following form:
∂u ∂F
+ =0 (8.3)
∂t ∂x
where for the linear advection equation F (u) = a0 u.
307
CHAPTER 8. HYPERBOLIC PDES 308
8.2 Forward in time central in space discretiza-

tion
We may discretize (8.1) with a forward difference in time and a central difference
in space, normally abberviated as the FTCS-scheme:
∂u un+1
j − unj ∂u unj+1 − unj−1
≈ , ≈
∂t ∆t ∂x 2∆x
and we may substitute the approximations (8.2) into the advection equation
(8.1) to yield:
C
un+1
j = unj − (unj+1 − unj−1 ) (8.4)
2
For convenience we have introduced the non-dimensional Courant-Friedrich-Lewy
number (or CFL-number or Courant-number for short):
∆t
C = a0 (8.5)
∆x
The scheme in (8.4) is first order in time and second order in space (i.e. (O(∆t)+
O(∆x2 ))), and explicit in time as can bee seen both from Figure 8.1 and (8.4).
n+1
j-1 j j+1
Figure 8.1: Illustration of the first order in time central in space scheme.
We will try to solve model equation (8.1) with the scheme (8.4) and initial
conditions illustrated in Fig (8.2) with the mathematical representation:
u(x, 0) = 1 for x < 0.5

u(x, 0) = 0 for x > 0.5
Solutions for three CFL-numbers: C=0.25, 0.5 and 1.0 are illustrated in
Figure 8.3. Large oscillations are observed for all values of the CFL-number,
even though they seem to be sligtly reduced for smaller C-values,; thus we
1.0
0.0 0.5 1.0 x
Figure 8.2: Initial values for the advection equaution (8.1).
have indications of an unstable scheme. As a first approach observe that the

coefficient for unj+1 in (8.4) always will be negative, and thus the criterion of
positive coefficients (PC-criterion) may not be satisfied for any value of C.
However, as we know that the PC-criterion may be too strict in some cases, we
proceed with a von Neumann analysis by introducing the numerical amplification
factor Gn for the error Ejn in the numerical scheme to be analyzed
unj → Ejn = Gn · ei·βxj (8.6)
Substitution of (8.6) into (8.4) yields:

C
Gn+1 ei·β·xj = Gn ei·β·xj − Gn ei·βxj+1 − Gn ei·βxj−1

2
which after division with Gn ei·β·xj and introduction of the simplified notation
δ = β · h yields:
C i·βh
− e−i·βh = 1 − i · C sin(δ)

G=1− e
2
where the trigonometric relations:
2 cos(x) = eix + e−ix (8.7)

ix −ix
i · 2 sin(x) = e −e (8.8)
x
cos(x) = 1 − 2 sin2 ( ) (8.9)
2
have been introduced for convenience. Finally, we get the following expression
for the numerical ampliciation factor:
q
|G| = 1 + C 2 sin2 (δ) ≥ 1 for all C and δ
and concequently the FTCS-scheme is unconditionally unstable for the advection
equation and is thus not a viable scheme. Even a very small value of C will not
suffice to dampe the oscillations.
C = 1.0
4
u
0
-2
-4
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
2.5
2 C = 0.5
u1.5
1
0.5
-0.5
-1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
C = 0.25
1.5
u
1
0.5
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x
Figure 8.3: Computed solutions with the (8.4). Dotted line: analytical solution,
solid line: computed soultion.
8.3 Upwind schemes

Upwind differences are also called upstream differences. Numerically, this term
indicates that we use backward differences with respect to the direction from
which information is coming. Upwind differences are used only for convection
terms, never for diffusive terms. Figure 8.4 shows a graphical interpretation
of upwind differences when applied to the advection equation . The resulting
scheme is called the upwind scheme.
Upwind finite difference for the convective term:
∂u unj − unj−1
= + O(∆x) (8.10)
∂x ∆x
a0
n+1
j-1 j
Figure 8.4: Stencil of the upwind scheme for the linear advection equation with
characteristic speed a0 .
∂u a0 ∆t
Replacing (8.10) in (8.1) and using a forward difference for ∂t and C = ∆x
yields:
un+1
j = unj − C · (unj − unj−1 ) = (1 − C)unj + C · unj−1 (8.11)
(8.11) has an a truncation error of order O(∆t) + O(∆x)
If we set C = 1 into (8.11), we get that un+1 j = unj−1 , which in turn is the
exact solution of the partial differential equation ∂t + a0 ∂u
∂u
∂x = 0 ( see Section 5.2).
For C < 1 the wavefront of the numerical solution is smoothed, indicating that
the scheme introduces a so called numerical viscosity for these values of C. We
will come back to the term numerical viscosity later on. Computational results
shown in Figure 8.5 indicate that the scheme should be stable for C ≤ 1. In
fact, the PC criterion shows that the scheme should be stable for that range of
C. Let us use the von Neumann stability analysis to see if the stability range
extends beyond that range. Insert (8.11) from (7.42) in Chapter 7:
Gn+1 ei·β·xj = (1 − C) · Gn ei·β·xj + C · Gn ei·β·xj−1

dividing by Gn ei·β·xj gives:
G = (1−C)+C·e−i·β·h = 1−C+C·(cos(δ)−i·sin(δ)) = 1+C·(cos(δ)−1)−i·C·sin(δ)
q p
|G| = [1 + C cos(δ − 1)]2 + C 2 sin2 (δ) = 1 − 2C(1 − cos(δ)) · (1 − C)
(8.12)
Enforcing the stability criterion |G| ≤ 1 results in the following:
1 − 2C · (1 − cos(δ)) · (1 − C) ≤ 1 ⇒ C · (1 − cos(δ)) · (1 − C) ≥ 0
eller

δ
C · sin2 (1 − C) ≥ 0
2
Stability interval:
0<C≤1 (8.13)
(8.13) also applies when we set C = |a0 | · ∆t/∆x. All stable explicit 2-
level schemes for the advection equation have a restriction on the admissible
Courant-number .
Marit 9: Har ikke endret figur
1.5
C = 1.0
1
u
0.5
-0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
1.5
C = 0.5
1 C = 0.5
u
0.5
-0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
1.5
C = 0.25
1
u
0.5
-0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Figure 8.5: Numerical solution for the linear advection equation using the
upwind scheme and different values of Courant-number C.
8.4 The modified differential equation

In Section 7.6 we have seen how to determine the local truncation error and rthe
consistency of numerical schemes. We will now see how to use a similar method
to gain understanding on the stability of the resulting scheme. We will restrcit
the discussion to the linear advection equation
We recall the FTCS scheme from Section 8.2:
un+1
j − unj unj+1 − unj−1
+ a0 =0 (8.14)
k 2h
Inserting in (8.14) the Taylor expansions introduced in Section 7.6, such as
(7.84), we obtain:
 
i n i n

1 h 2
 a h
k 0 h2 h3
u + kut + utt + · · · − unj + u + hux + 2 uxx + 6 uxxx + ···

k 2  2h
j j
i n

a0 h h2 h3
− u − hux + 2 uxx − 6 uxxx + ··· = 0

2h
j
Reordering:
∂u ∂u k ∂ 2 u a0 h2 ∂ 3 u
+ a0 =− − + ··· (8.15)
∂t ∂x 2 ∂t2 6 ∂x3
∂u
We now use the differential equation ∂t + a0 ∂u
∂x = 0 to transform the term
k ∂2u
− 2 ∂t2 in a spatial derivative.
The resulting differential equation is:
∂u ∂u
= −a0 (8.16)
∂t ∂x
Deriving with respect to t:
∂2u ∂2u
2
= −a0
∂t ∂t∂x
and then we derive (8.16) with respect to x:
∂2u ∂2u ∂2u ∂2u

= −a0 2 → −a0 = a20 2
∂t∂x ∂x ∂t∂x ∂x
so that we obtain:
∂2u 2
2∂ u
= a0 (8.17)
∂t2 ∂x2
Inserting (8.17) in (8.15):
∂u ∂u ka2 ∂ 2 u a0 h2 ∂ 3 u
+ a0 =− 0 2 − + ···
∂t ∂x 2 ∂x 6 ∂x3
a0 k
which, when introducing the Courant-number C = h gives:
∂u ∂u Cha20 ∂ 2 u a0 h2 ∂ 3 u
+ a0 =− − + ··· (8.18)
∂t ∂x 2 ∂x2 6 ∂x3
(8.18) clearly shows why the scheme in (8.14) is unstable. In computational
fluid mechanics the term νN = − Cha 2
0
is often called numerical viscosity in
analogy to the physical viscosity that accompanies the diffusive term of advection-
∂2u
diffusion equations like ∂u
∂t + a0 ∂u∂x = ν ∂x2 . When using the scheme given in
(8.14), this results in solving the advection-diffusion equation with a negative
viscosity coefficient , which is certainly unstable. (8.18) provides information
on why oscillations in Figure 8.3 decrease for decreasing C. Equation (8.18) is
called the modified equation for a given numerical scheme, since it shows which
is the differential problem that the scheme is actually solving and it therefore
provides a deep insight in the behavior that the numerical solution will exhibit.
Now we will perform the same procedure as above but for the upwind scheme
presented in Section 8.3. The scheme can be written as:
un+1
j − unj (unj − unj−1 )
+ a0 =0 (8.19)
k h
After introducing Taylor expansions as before and reordering we get:
∂u ∂u k ∂ 2 u a0 h ∂ 2 u
= a0 =− + + ··· (8.20)
∂t ∂x 2 ∂t2 2 ∂x2
Since the differential equation is the same as for the FTCS scheme, we use
the same procedure to replace time derivative terms with spatial derivative ones,
yielding:
∂u ∂u a0 h ∂2u
+ a0 = (1 − C) 2 + · · · (8.21)
∂t ∂x 2 ∂x
The coefficient νN = a02h (1 − C) is the so called numerical viscosity.
For C > 1 this term becomes negative and this in turn means that the
scheme turns unstable. The necessary condition for stability is then C ≤ 1, in
agreement with what seen before in terms of stability analysis. Note that the
numerical viscosity increases with decreasing C, meaning that sharp fronts will
be smoothed more for lower Courant-numbers, as shown in Figure 8.5.
Remark: We have used the original differential equation for the derivation
2 2
of both modified equations (8.18) og (8.21), i.e. ∂∂t2u = a20 ∂∂xu2 . This procedure
is called the Cauchy-Kowalewsky procedure. The problem is that the modified
equation does not generally corresponds to the original equation, as expected.
This means that instead of deriving the original equation, we have to derive
(8.15) and (8.20). This procedure is called the Warming-Hyett procedure and
must be generally adopted. For the above made computations, where we stop
after one iteration of this procedure, both procedures are equivalent. For more
complex differential equations the use of symbolic calculus programs such as
Maple or Maxima becomes mandatory. The Maple-program below illustrate how
to implement such procedure.
> restart:
> EQ:= (u(x,t+k)-u(x,t))/k + a*(u(x+h,t)-u(x-h,t))/(2*h):
> EQT:= mtaylor(EQ,[h,k]):
> MDE:=EQT:
> ELIM:=proc(i::integer,j::integer)
local DE,uxt,UXT:
global EQT,MDE:
DE:=convert(diff(EQT,x$i,t$j-1,D):
uxt:=convert(diff(u(x,t),x$i,t$j),D):
UXT:=solve(DE = 0,uxt):
subs(uxt = UXT,MDE):
end:
> MDE:=ELIM(0,2):
> MDE:=ELIM(1,1):
> MDE:=ELIM(0,3):
> MDE:=ELIM(1,2):
> MDE:=ELIM(2,1):
> # Substitute the Courant number C = a*k/h
> MDE:=expand(subs(k=C*h/a,MDE)):
> u2x:=convert(diff(u(x,t),x$2),D):
> u3x:=convert(diff(u(x,t),x$3),D):
> collect(MDE,[u2x,u3x]):
> RHSMDE:=-coeff(MDE,u2x)*convert(u2x,diff)
-coeff(MDE,u3x)*convert(u3x,diff);
The resulting right-hand side of the modified equation delivered by the

program is
2 ∂2
∂
RHSMDE:= − 21 Cha ∂x 2 u(x, t) − 1
6 ah2
+ 1 2 2
3 C h a ∂x2 u(x, t)
Note that in this case we have obtained the term 13 C 2 h2 a, which was missing
in (8.18). The technique of the modified equation can be also applied to non-
linear equations. Moreover, it can be shown that the method relates to von
Neumann stability method. Hirsch [7] and Anderson [14] use the modified
equation to study fundamental properties of finite difference schemes.
8.5 Errors due to diffusion and dispersion

We have previously seen that the numerical amplification factor G is complex,
i.e. it has a real part Gr and a imaginary part Gi . Being a complex number G
may be represented in the two traditional ways:
G = Gr + i · Gi = |G| · e−iφ (8.22)
where the latter is normally referred to as the polar form which provides a
means to express G by the amplitude (or absolute value) |G| and the phase φ.

−Gi
q
|G| = G2r + G2i , φ = arctan (8.23)
Gr
I section 7.4 we introduced the diffusion/dissipation error εD (dissipasjons-

feilen) as:
|G|
εD = (8.24)
|Ga |
where |Ga | is the amplituden of analytical amplification factor Ga , i.e. we
have no dissipative error when εD = 1 .
For problems and models related with convection we also need to consider
the error related with timing or phase, and introduce a measure for this kind
error as the dispersion error εφ :
φ
εφ = (8.25)
φa
where φa is the phase for the analytical amplification factor. And again
εφ = 1 corresponds to no dispersion error. Note for parabolic problems with
φa = 0 , it is common to use εφ = φ.
8.5.1 Example: Advection equation

Consider the linear advection equation, given by:
∂u ∂u
+ a0 =0
∂t ∂x
Similarly to what introduced in Section 7.4, we use
u(x, t) = ei(βx−ωt) , (8.26)

with ω = a0 β, where β is the wave number. With the notation introduced above
we get:
u(x, t) = ei(βx−ωt) = e−i·φa , φa = ωt − βx (8.27)

From equation (7.39) in Section 7.4:
u(xj , tn+1 ) ei(βxj −ωtn+1 )

Ga = = i(βx −ωt ) = exp[i(βxj −ωtn+1 )−i(βxj −ωtn )] = exp(−iω·∆t)
u(xj , tn ) e j n
where we have inserted (8.27). Thus, |Ga | = 1 → εD = |G| .

Hence:
∆t
φa = ω · ∆t = a0 β · ∆t = a0β · ∆x = C · δ (8.28)
∆x
C is the Courant-number and δ = β · ∆x as usual.
From (8.28) we also get:
φa
a0 = (8.29)
β · ∆t
Analogously to (8.29), we can define a numerical propagation velocity anum :

φ
anum = (8.30)
β · ∆t
The dispersion error in (8.25) can then be written as:
anum
εφ = (8.31)
a0
When the dispersion error is greater than one we have that the numerical
propagation speed is greater than the physical one. This results in the fact that
the numerical solution will move faster than the physical one. On the other
hand, a εφ < 1 results in slower numerical propagation speed compared to the
physical one, with obvious consequences for the numerical solution.
Let us know have a closer look to the upwind scheme and the Lax-Wendroff
scheme.
8.5.2 Example: Diffusion and dispersion errors for the up-

wind schemes
From equation (8.12) in Section 8.3:
G = 1 + C · (cos(δ) − 1) − i · C sin(δ) = Gr + i · Gi (8.32)

which inserted in (8.23) and (8.25) gives:
p
εD = |G| = [1 + C · (cos(δ) − 1)]2 + [C sin(δ)]2
q (8.33)
= 1 − 4C(1 − C) sin2 ( 2δ )

φ 1 C sin(δ)
εφ = = arctan (8.34)
φa Cδ 1 − C(1 − cos(δ))
Figure 8.6 shows (8.33) and (8.34) as a function of δ for different Courant-
number values. We see that εD decreases strongly for larger frequencies, which
means that the numerical amplitude becomes much smaller than the exact one
when use a large time step. This applies even for C = 0.8 . Remember that the
amplitude factor for the n-th time iteration is |G|n (see also Figure 8.5). The
scheme is performance is rather modest for general use even though it is stable.
For C = 0.5 we have no dispersion errors. For C < 0.5 we have that εφ < 1,
so that the numerical propagation speed is smaller than the physical one a0 . On
the other hand, for C > 0.5 we have that εφ > 1.
Lucas 10: axes labels are wrong, also legend is not consistent with text for
denoting the Courant-number
1.2 1.4
1.2
1.0
1.0
0.8
0.8
|G|
<G
0.6
0.6
0.4
CFL = 0.25 0.4
CFL = 0.5
0.2
CFL = 0.8 0.2
CFL = 1.0
0.0 0.0
0 20 40 60 80 100 120 140 160 180
[deg]
Figure 8.6: Diffusion error εD (left y-axis) and dispersion error εφ (right y-axis)
for the upwind scheme as a function of frequency for the upwind scheme. The
dotted lines of εφ correspond to same CFL-numbers as solid lines of εD with the
same color.
8.5.3 Example: Diffusion and disperision errors for the

Lax-Wendroff scheme
From equation (8.47):
G = 1 + C 2 · (cos(δ) − 1) − i · C sin(δ) = Gr + i · Gi (8.35)

which inserted in (8.23) and (8.25) gives:
p
εD = |G| = [1 + C 2 · (cos(δ) − 1)]2 + (C sin(δ))2
q (8.36)
1 − 4C 2 (1 − C 2 ) sin4 2δ

=

φ 1 Csin(δ)
εφ = = arctan (8.37)
φa Cδ 1 + C 2 (cos(δ) − 1)
Figures 8.7 and 8.9 show (8.36) and (8.37) as a function of δ for different
values of the Courant-number. The range for which εD is close to 1 is larger
than in Figure 8.7 for the upwind scheme. This puts in evidence an important
difference between first and second order schemes Figure 8.8 shows that εφ is
generally less than 1, so that the numerical propagation speed is smaller than
the physical one. This is the reason for the occurrence of oscillations as the ones
shown in animation 7. A von Neumann criterion guarantees stability for |G| ≤ 1,
but does not guarantees that the scheme is monotone. On the other hand, the
PC criterion ensures that no oscillations can occur. In fact, this criterion can not
be verified for the Lax-Wendroff scheme applied to the linear advection equation.
In fact, monotone schemes for the linear advection equation can be at most first
order accurate.
Lucas 11: we could say something about Godunov’s theorem here! Lucas
10: axes labels are wrong, also legend is not consistent with text for denoting
the Courant-number
1.2 1.4
1.2
1.0
1.0
0.8
0.8
|G|
<G
0.6
0.6
0.4
CFL = 0.25 0.4
CFL = 0.5
0.2
CFL = 0.8 0.2
CFL = 1.0
0.0 0.0
0 20 40 60 80 100 120 140 160 180
[deg]
Figure 8.7: Diffusion error εD (left y-axis) and dispersion error εφ (right y-axis)
as a function of frequency for the Lax-Wendroff scheme. The dotted lines of εφ
correspond to same CFL-numbers as solid lines of εD with the same color.

Dispersjonsfeil εφ for Lax-Wendroff skjema

1.4
1.2
C= 1
1
C =0.8
0.8
C = 0.25
C = 0.5
0.6
0.4
0.2
0
0 20 40 60 80 100 120 140 160 180
δ (grader)
Figure 8.8: Dispersion error εφ as a function of frequency for the Lax-Wendroff

scheme.
Amplitude |G| som funksjon av C og δ
0.8
0.6
0.4
0.2
0
0
0.5
1 1
1.5 0.8
0.6
δ(rad.) 2
0.4
2.5
0.2 C
3
0
Figure 8.9: Diffusion error εD as a function of frequency and Courant-number

for the Lax-Wendroff scheme.
8.6 The Lax-Friedrich Scheme

Lax-Friedrichs scheme is an explicit, first order scheme, using forward difference
in time and central difference in space. However, the scheme is stabilized by
averaging unj over the neighbour cells in the in the temporal approximation:
un+1
j − 21 (unj+1 + unj−1 ) n
Fj+1 n
− Fj−1
=− (8.38)
∆t 2∆x
The Lax-Friedrich scheme is the obtained by isolation un+1
j at the right hand
side:
1 ∆t
un+1 = (unj+1 + unj−1 ) − (F n − Fj−1
n
) (8.39)
j
2 2∆x j+1
By assuming a linear flux F = a0 u it may be shown that the Lax-Friedrich
scheme takes the form:
1 n C
un+1 = (u + unj−1 ) − (unj+1 − unj−1 ) (8.40)
j
2 j+1 2
where we have introduced the CFL-number as given by (8.5) and have the simple
python-implementation:
def lax_friedrich(u):
u[1:-1] = (u[:-2] +u[2:])/2.0 - c*(u[2:] - u[:-2])/2.0
return u[1:-1]
whereas a more generic flux implementation is implemented as:

def lax_friedrich_Flux(u):
u[1:-1] = (u[:-2] +u[2:])/2.0 - dt*(F(u[2:])-F(u[:-2]))/(2.0*dx)
return u[1:-1]
8.7 Lax-Wendroff Schemes

These schemes were proposed in 1960 by P.D. Lax and B. Wendroff [11] for
solving, approximately, systems of hyperbolic conservation laws on the generic
form given in (8.3).
A large class of numerical methods for solving (8.3) are the so-called conser-
vative methods:
∆t
un+1 = unj +

j Fj−1/2 − Fj+1/2 (8.41)
∆x
Linear advection. The Lax–Wendroff method belongs to the class of con-

servative schemes (8.3) and can be derived in various ways. For simplicity, we
will derive the method by using a simple model equation for (8.3), namely the
linear advection equation with F (u) = a u as in (8.1), where a is a constant
propagation velocity. The Lax-Wendroff outset is a Taylor approximation of

un+1
j :
n n
2
∂u (∆t) ∂ u
un+1 = unj + ∆t + + ··· (8.42)

j
∂t 2 ∂t2
j j
From the differential equation (8.3) we get by differentiation

n n n n
∂u ∂u ∂ 2 u ∂ 2
u
= −a0 and = a20 2 (8.43)

∂t ∂x ∂t2 ∂x
j j j j
Before substitution of (8.43) in the Taylor expansion (8.42) we approximate the

spatial derivatives by central differences:
n n
∂u unj+1 − unj−1 ∂ 2 u unj+1 − 2unj + unj−1
≈ and ≈ (8.44)
∂x (2∆x) ∂x2 (∆x)2

j j
and then the Lax-Wendroff scheme follows by substitution:
C n C2 n
un+1 = unj − uj+1 − unj−1 + uj+1 − 2unj + unj−1

j (8.45)
2 2
with the local truncation error Tjn :
n
∂3u ∂3u

1
Tjn = · (∆t)2 3 + a0 (∆x)2 3 = O[(∆t)2 , (∆x)2 ] (8.46)
6 ∂t ∂x j
The resulting difference equation in (8.45) may also be formulated as:
C C
un+1
j = (1 + C)unj−1 + (1 − C 2 )unj − (1 − C)unj+1 (8.47)
2 2
The explicit Lax-Wendroff stenticl is illustrated in Figure 8.10
An example of how to implement the Lax-Wendroff scheme is given as
follows:
def lax_wendroff(u):
u[1:-1] = c/2.0*(1+c)*u[:-2] + (1-c**2)*u[1:-1] - c/2.0*(1-c)*u[2:]
return u[1:-1]
8.7.1 Lax-Wendroff for non-linear systems of hyperbolic

PDEs
For non-linear equations (8.3) the Lax–Wendroff method is no longer unique and
naturally various methods have been suggested. The challenge for a non-linear
F (u) is that the substitution of temporal derivatives with spatial derivatives (as
we did in (8.43)) is not straightforward and unique.
un+1
j
n+1
j-1 j j+1
Figure 8.10: Schematic of the Lax-Wendroff scheme.
Ricthmyer Scheme. One of the earliest extensions of the scheme is the

Richtmyer two-step Lax–Wendroff method, which is on the conservative form
(8.41) with the numerical fluxes computed as follows:
n+1/2 1 n 1 ∆t
uj + unj+1 + Fjn − Fj+1
n

uj+1/2 = (8.48)
2 2 ∆x
n+1/2
Fj+1/2 = F (uj+1/2 ) (8.49)
Lax-Wendroff two step. A Lax-Wendroff two step method is outlined in the

following. In the first step u(x, t) is evaluated at half time steps n + 1/2 and
half grid points j + 1/2. In the second step values at the next time step n + 1
are calculated using the data for n and n + 1/2.
First step:
n+1/2 1 n ∆t
+ unj − F (unj+1 ) − F (unj )

uj+1/2 = u (8.50)
2 j+1 2∆x
n+1/2 1 n ∆t
uj + unj−1 − F (unj ) − F (unj−1 )

uj−1/2 = (8.51)
2 2∆x
Second step:
∆t n+1/2 n+1/2

un+1
j = unj − F (uj+1/2 ) − F (uj−1/2 ) (8.52)
∆x
Notice that for a linear flux F = a0 u, the two-step Lax-Wendroff method
((8.51) and (8.52)) may be shown to reduce to the one-step Lax-Wendroff method
outlined in (8.45) or (8.47).
MacCormack Scheme. A simpler and popular extension/variant of Lax-

Wendroff schemes like in the previous section, is the MacCormack scheme [13]:
∆t
upj = unj + Fjn − Fj+1
n

∆x
(8.53)
1 n 1 ∆t
un+1 uj + upj + p
− Fjp

j = Fj−1
2 2 ∆x
where we have introduced the convention Fjp = F (upj ).
Note that in the predictor step we employ the conservative formula (8.41)
n
for a time ∆t with forward differencing, i.e. . Fj+1/2 = Fj+1 = F (unj+1 ). The
corrector step may be interpreted as using (8.41) for a time ∆t/2 with initial
condition 12 unj + upj+1 and backward differencing.
Another MacCormack scheme may be obtained by reversing the predictor
and corrector steps. Note that the MacCormack scheme (8.53) is not written in
conservative form (8.41). However, it easy to express the scheme in conservative
form by expressing the flux in (8.41) as:
1
m
Fjp + Fj+1
n

Fj+1 = (8.54)
2
For a linear flux F (u) = a0 u, one may show that the MacCormack scheme
in (8.53) reduces to a two-step scheme:
upj = unj + C unj − unj+1

(8.55)
1 n C p
un+1 u + upj + − upj

= u (8.56)
j
2 j 2 j−1
and substitution of (8.55) into (8.56) shows that the MacCormack scheme
is identical to the Lax-Wendroff scheme (8.47) for the linear advection flux. A
python implementation is given by:
def macCormack(u):
up = u.copy()
up[:-1] = u[:-1] - c*(u[1:]-u[:-1])
u[1:] = .5*(u[1:]+up[1:] - c*(up[1:]-up[:-1]))
return u[1:-1]
# Constants and parameters

a = 1.0 # wave speed
tmin, tmax = 0.0, 1.0 # start and stop time of simulation
xmin, xmax = 0.0, 2.0 # start and end of spatial domain
Nx = 80 # number of spatial points
c = 0.9 # courant number, need c<=1 for stability
8.7.2 Code example for various schemes for the advection

equation
A complete example showing how a range of hyperbolic schemes are implemented
and applied to a particular example:
# src-ch6/advection_schemes.py
import numpy as np
from matplotlib import animation
from scipy import interpolate
from numpy import where
from math import sin

plt.get_current_fig_manager().window.raise_()
LNWDT=2; FNT=15
init_func=1 # Select stair case function (0) or sin^2 function (1)
# function defining the initial condition

if (init_func==0):
def f(x):
"""Assigning a value of 1.0 for values less than 0.1"""
f = np.zeros_like(x)
f[np.where(x <= 0.1)] = 1.0
return f
elif(init_func==1):
def f(x):
"""A smooth sin^2 function between x_left and x_right"""
f = np.zeros_like(x)
x_left = 0.25
x_right = 0.75
xm = (x_right-x_left)/2.0
f = where((x>x_left) & (x<x_right), np.sin(np.pi*(x-x_left)/(x_right-x_left))**4,f)
return f
def ftbs(u): # forward time backward space

u[1:-1] = (1-c)*u[1:-1] + c*u[:-2]
return u[1:-1]
# Lax-Wendroff
def lax_wendroff(u):
u[1:-1] = c/2.0*(1+c)*u[:-2] + (1-c**2)*u[1:-1] - c/2.0*(1-c)*u[2:]
return u[1:-1]
# Lax-Friedrich Flux formulation
def lax_friedrich_Flux(u):
u[1:-1] = (u[:-2] +u[2:])/2.0 - dt*(F(u[2:])-F(u[:-2]))/(2.0*dx)
return u[1:-1]
# Lax-Friedrich Advection
def lax_friedrich(u):
u[1:-1] = (u[:-2] +u[2:])/2.0 - c*(u[2:] - u[:-2])/2.0
return u[1:-1]
# macCormack for advection quation

def macCormack(u):
up = u.copy()
up[:-1] = u[:-1] - c*(u[1:]-u[:-1])
u[1:] = .5*(u[1:]+up[1:] - c*(up[1:]-up[:-1]))
return u[1:-1]
# Constants and parameters

tmin, tmax = 0.0, 1.0 # start and stop time of simulation
# Discretize
x = np.linspace(xmin, xmax, Nx+1) # discretization of space
dx = float((xmax-xmin)/Nx) # spatial step size
dt = c/a*dx # stable time step calculated from stability requirement
Nt = int((tmax-tmin)/dt) # number of time steps
time = np.linspace(tmin, tmax, Nt) # discretization of time
# solve from tmin to tmax
solvers = [ftbs,lax_wendroff,lax_friedrich,macCormack]
#solvers = [ftbs,lax_wendroff,macCormack]
#solvers = [ftbs,lax_wendroff]
#solvers = [ftbs]
u_solutions=np.zeros((len(solvers),len(time),len(x)))
uanalytical = np.zeros((len(time), len(x))) # holds the analytical solution
for k, solver in enumerate(solvers): # Solve for all solvers in list

u = f(x)
un = np.zeros((len(time), len(x))) # holds the numerical solution
for i, t in enumerate(time[1:]):
if k==0:
uanalytical[i,:] = f(x-a*t) # compute analytical solution for this time step
u_bc = interpolate.interp1d(x[-2:], u[-2:]) # interplate at right bndry
u[1:-1] = solver(u[:]) # calculate numerical solution of interior

u[-1] = u_bc(x[-1] - a*dt) # interpolate along a characteristic to find the boundary value
un[i,:] = u[:] # storing the solution for plotting

u_solutions[k,:,:] = un
### Animation
# First set up the figure, the axis, and the plot element we want to animate
fig = plt.figure()
ax = plt.axes(xlim=(xmin,xmax), ylim=(np.min(un), np.max(un)*1.1))
lines=[] # list for plot lines for solvers and analytical solutions
legends=[] # list for legends for solvers and analytical solutions
for solver in solvers:
line, = ax.plot([], [])
lines.append(line)
legends.append(solver.func_name)
line, = ax.plot([], []) #add extra plot line for analytical solution
lines.append(line)
legends.append(’Analytical’)
plt.xlabel(’x-coordinate [-]’)
plt.ylabel(’Amplitude [-]’)
plt.legend(legends, loc=3, frameon=False)
# initialization function: plot the background of each frame
def init():
for line in lines:
line.set_data([], [])
return lines,
# animation function. This is called sequentially

def animate(i):
for k, line in enumerate(lines):
if (k==0):
line.set_data(x, un[i,:])
else:
line.set_data(x, uanalytical[i,:])
return lines,
def animate_alt(i):
for k, line in enumerate(lines):
if (k==len(lines)-1):
line.set_data(x, uanalytical[i,:])
else:
line.set_data(x, u_solutions[k,i,:])
return lines,
# call the animator. blit=True means only re-draw the parts that have changed.
anim = animation.FuncAnimation(fig, animate_alt, init_func=init, frames=Nt, interval=100, blit=False)
plt.show()
Movie 7: Result from code example above using a step function as

the initial value. mov-ch6/step.mp4
Movie 8: Result from code example above using a sine squared

function as the initial value mov-ch6/sine.mp4
8.8 Order analysis on various schemes for the

advection equation
The schemes presented have different theoretical order;ftbs: O (∆x, ∆t), Lax-
Friedrich: O ∆x2 , ∆t , Lax-Wendroff : O ∆x2 , ∆t2 , MacCormack: O ∆x2 , ∆t2 .
For the linear advection equation the MacCormack and Lax-Wendroff schemes
are identical, and we will thus only consider Lax-Wendroff in this section. Assum-
ing the wavespeed a0 is not very big, nor very small we will have ∆t = O (∆x),
because of the cfl constraint condition. Thus for the advection schemes the
discretization error will be of order min(p, q). Where p is the spatial order,
and q the temporal order. Thus we could say that ftbs, and Lax-Friedrich are
both first order, and Lax-Wendroff is of second order. In order to determine the
observed (dominating) order of accuracy of our schemes we could adapt the same
procedure outlined in Example 2.8.1, where we determined the observed order
of accuracy of ODEschemes. For the schemes solving the Advection equation
the discretization error will be a combination of ∆x and ∆t, however since
∆t = O (∆x), we may still assume the following expression for the discretization
error:
≈ Chp (8.57)
where h is either ∆x or ∆t, and p is the dominating order. Further we can

calculate the discretization error, observed for our schemes by successively refining
h with a ratio r, keeping the cfl-number constant. Thus refining ∆x and ∆t with
the same ratio r. Now dividing n−1 by n and taking the log on both sides and
rearranging lead to the following expression for the observed order of accuracy:

log n−1
n

n−1

p= = logr
log (r) n
To determine the observed discretization error we will use the root mean square
error of all our discrete soultions:
v
u
u1 X N 2
E=t fˆ − f
N i=1
where N is the number of sampling points, fˆ is the numerical-, and f is the

analytical solution. A code example performing this test on selected advections
schemes, is showed below. As can be seen in Figure 8.11, the Lax-Wendroff
scheme quickly converge to it’s theoretical order, whereas the ftbs and Lax
Friedrich scheme converges to their theoretical order more slowly.
def test_convergence_MES():
from numpy import log
from math import sqrt
global c, dt, dx, a
# Change default values on plots

LNWDT=2; FNT=13

tmin, tmax = 0.0, 1 # start and stop time of simulation
# Discretize
x = np.linspace(xmin, xmax, Nx + 1) # discretization of space
−1
¶
2
−2
¢
f-f
¡
−3 ftbs
N
1
µq
lax_friedrich
−4
log10
lax_wendroff
−5
160320 640 1280 2560
Nx
2
log2( n-1
n
)
1, 2 2, 3 3, 4 4, 5
( n-1, n)
Figure 8.11: The root mean square error E for the various advection schemes
as a function of the number of spatial nodes (top), and corresponding observed
convergence rates (bottom).
dx = float((xmax-xmin)/Nx) # spatial step size

dt = c/a*dx # stable time step calculated from stability requirement
time = np.arange(tmin, tmax + dt, dt) # discretization of time
init_funcs = [init_step, init_sine4] # Select stair case function (0) or sin^4 function (1)
f = init_funcs[1]
solvers = [ftbs, lax_friedrich, lax_wendroff]
errorDict = {} # empty dictionary to be filled in with lists of errors

orderDict = {}
for solver in solvers:

errorDict[solver.func_name] = [] # for each solver(key) assign it with value=[], an empt
orderDict[solver.func_name] = []
hx = [] # empty list of spatial step-length

ht = [] # empty list of temporal step-length
Ntds = 5 # number of grid/dt refinements
# iterate Ntds times:
for n in range(Ntds):
hx.append(dx)
ht.append(dt)
for k, solver in enumerate(solvers): # Solve for all solvers in list

u = f(x) # initial value of u is init_func(x)
error = 0
samplePoints = 0
for i, t in enumerate(time[1:]):
u_bc = interpolate.interp1d(x[-2:], u[-2:]) # interplate at right bndry
u[1:-1] = solver(u[:]) # calculate numerical solution of interior
u[-1] = u_bc(x[-1] - a*dt) # interpolate along a characteristic to find the bound
error += np.sum((u - f(x-a*t))**2) # add error from this timestep
samplePoints += len(u)
error = sqrt(error/samplePoints) # calculate rms-error
errorDict[solver.func_name].append(error)
if n>0:
previousError = errorDict[solver.func_name][n-1]
orderDict[solver.func_name].append(log(previousError/error)/log(2))
print " finished iteration {0} of {1}, dx = {2}, dt = {3}, tsample = {4}".format(n+1, Ntd
# refine grid and dt:
Nx *= 2
x = np.linspace(xmin, xmax, Nx+1) # new x-array, twice as big as the previous
dx = float((xmax-xmin)/Nx) # new spatial step size, half the value of the previous
dt = c/a*dx # new stable time step
time = np.arange(tmin, tmax + dt, dt) # discretization of time
# Plot error-values and corresponding order-estimates:

fig , axarr = plt.subplots(2, 1, squeeze=False)
lstyle = [’b’, ’r’, ’g’, ’m’]
legendList = []
N = Nx/2**(Ntds + 1)
N_list = [N*2**i for i in range(1, Ntds+1)]
epsilonN = [i for i in range(1, Ntds)]
epsilonlist = [’$\epsilon_{0} , \epsilon_{1}$’.format(str(i), str(i+1)) for i in range(1, Ntd
for k, solver in enumerate(solvers):
axarr[0][0].plot(N_list, np.log10(np.asarray(errorDict[solver.func_name])),lstyle[k])
axarr[1][0].plot(epsilonN, orderDict[solver.func_name],lstyle[k])
legendList.append(solver.func_name)
axarr[1][0].axhline(1.0, xmin=0, xmax=Ntds-2, linestyle=’:’, color=’k’)

axarr[1][0].axhline(2.0, xmin=0, xmax=Ntds-2, linestyle=’:’, color=’k’)
8.8.1 Separating spatial and temporal discretization error

The test performed in the previous example verify the dominating order of accu-
racy of the advection schemes. However in order to verify our implementations
of the various schemes we would like to ensure that both the observed temporal-
and spatial order are close to the theoretical orders. The expression for the
discretization error in (8.57) is a simplification of the more general expression
≈ Cx hpx + Ct hqt (8.58)
where Cx , and Ct are constants, hx and ht are the spatial- and temporal step
lengths and p and q are the spatial- and temporal orders respectively. We are
interested in confirming that p is close to the theoretical spatial order of the
scheme, and that q is close to the theoretical temporal order of the scheme. The
method of doing this is not necessarily straight forward, especially since ht and hx
are coupled through the cfl-constraint condition. In chapter one we showed that
we were able to verify C and p in the case of only one step-length dependency
(time or space), by solving two nonlinear equations for two unknowns using a
Newton-Rhapson solver. To expand this method would now involve solving four
nonlinear algebraic equations for the four unknowns Cx , Ct , p, q. However since
it is unlikely that the observed discretization error match the expression in (8.58)
exactly, we now sugest a method based on optimization and curve-fitting. From
the previous code example we calculated the root mean square error E(hx , ht )
of the schemes by succecively refining hx and ht by a ratio r. We now assume
that E is related to Cx , Ct , p, q as in (8.58). In other words we would like to find
the parameters Cx , Ct , p, q, so that the difference between our caluclated errors
E(hx, ht), and the function (hx , ht ; Cx , p, Ct , q) = Cx hpx + Ct hqt is as small as
possible. This may be done by minimizing the sum (S) of squared residuals of
E and :
N
X
S= ri2 , ri = Ei − (hx , ht ; Cx , p, Ct , q)i
i=1
The python module scipy.optimize has many methods for parameter optimization
and curve-fitting. In the code example below we use scipy.optimize.curve_fit
which fits a function "f(x;params)" to a set of data "y-data" using a Levenberg-
Marquardt algorithmwith a least square minimization criteria. In the code
example below we start by loading the calculated root mean square errors
E (hx , ht ) of the schemes from "advection_scheme_errors.txt", which where
calculated in the same manner as in the previous example. As can be seen
by Figure 8.11 the ftbs, and Lax Friedriech scheme takes a while before they
are in their asymptotic range (area where they converge at a constant rate).
In "advection_scheme_errors.txt" we have computed E (hx , ht ) up to Nx =
89120 in which all schemes should be close to their asymptotic range. This
procedure is demonstrated in the code example below in which the following
expressions for the errors are obtained:
f tbs → = 1.3 h0.98

x + 6.5 h0.98
t (8.59)
laxF riedrich → = −1484 h1.9
x + 26 h1.0
t (8.60)
2.0
laxW endrof f → = −148 hx + 364. h2.0
t (8.61)
import numpy as np
import matplotlib.pyplot as plt
from scipy.optimize import curve_fit
from sympy import symbols, lambdify, latex
def optimize_error_cxct(errorList, hxList,

htList, p=1.0, q=1.0):
""" Function that optimimze the values Cx and Ct in the expression
E = epsilon = Cx hx^p + Ct ht^q, assuming p and q are known,
using scipy’s optimization tool curve_fit
Args:
errorList(array): array of calculated numerical discretization errors E
hxList(array): array of spatial step lengths corresponding to errorList
htList(array): array of temporal step lengths corresponding to errorList

p (Optional[float]): spatial order. Assumed to be equal to theoretical value
q (Optional[float]): temporal order. Assumed to be equal to theoretical value
Returns:
Cx0(float): the optimized values of Cx
Ct0(float): the optimized values of Ct
"""
def func(h, Cx, Ct):

""" function to be matched with ydata:
The function has to be on the form func(x, parameter1, parameter2,...,parametern)
where where x is the independent variable(s), and parameter1-n are the parameters to be o
"""
return Cx*h[0]**p + Ct*h[1]**q
x0 = np.array([1,2]) # initial guessed values for Cx and Ct

xdata = np.array([hxList, htList])# independent variables
ydata = errorList # data to be matched with expression in func
Cx0, Ct0 = curve_fit(func, xdata, ydata, x0)[0] # call scipy optimization tool curvefit
return Cx0, Ct0
def optimize_error(errorList, hxList, htList,

Cx0, p0, Ct0, q0):
""" Function that optimimze the values Cx, p Ct and q in the expression
E = epsilon = Cx hx^p + Ct ht^q, assuming p and q are known,
using scipy’s optimization tool curve_fit
Args:
errorList(array): array of calculated numerical discretization errors E
hxList(array): array of spatial step lengths corresponding to errorList
htList(array): array of temporal step lengths corresponding to errorList
Cx0 (float): initial guessed value of Cx
p (float): initial guessed value of p
Ct0 (float): initial guessed value of Ct
q (float): initial guessed value of q
Returns:
Cx(float): the optimized values of Cx
p (float): the optimized values of p
Ct(float): the optimized values of Ct
q (float): the optimized values of q
"""
def func(h, gx, p, gt, q):

""" function to be matched with ydata:
The function has to be on the form func(x, parameter1, parameter2,...,parametern)
where where x is the independent variable(s), and parameter1-n are the parameters to be o
"""
return gx*h[0]**p + gt*h[1]**q
x0 = np.array([Cx0, p0, Ct0, q0]) # initial guessed values for Cx, p, Ct and q
xdata = np.array([hxList, htList]) # independent variables
ydata = errorList # data to be matched with expression in func
gx, p, gt, q = curve_fit(func, xdata, ydata, x0)[0] # call scipy optimization tool curvefit
gx, p, gt, q = round(gx,2), round(p, 2), round(gt,2), round(q, 2)
return gx, p, gt, q
# Program starts here:
# empty lists, to be filled in with values from ’advection_scheme_errors.txt’

hxList = []
htList = []
E_ftbs = []
E_lax_friedrich = []
E_lax_wendroff = []
lineNumber = 1
with open(’advection_scheme_errors.txt’, ’r’) as FILENAME:

""" Open advection_scheme_errors.txt for reading.
structure of file:
hx ht E_ftbs E_lax_friedrich E_lax_wendroff
with the first line containing these headers, and the next lines containing
the corresponding values.
"""
# iterate all lines in FILENAME:
for line in FILENAME:
if lineNumber ==1:
# skip first line which contain headers
lineNumber += 1
else:
lineList = line.split() # sort each line in a list: lineList = [hx, ht, E_ftbs, E_lax_fri
# add values from this line to the lists

hxList.append(float(lineList[0]))
htList.append(float(lineList[1]))
E_ftbs.append(float(lineList[2]))
E_lax_friedrich.append(float(lineList[3]))
E_lax_wendroff.append(float(lineList[4]))
lineNumber += 1
FILENAME.close()
# convert lists to numpy arrays:

hxList = np.asarray(hxList)
htList = np.asarray(htList)
E_ftbs = np.asarray(E_ftbs)
E_lax_friedrich = np.asarray(E_lax_friedrich)
E_lax_wendroff = np.asarray(E_lax_wendroff)
ErrorList = [E_ftbs, E_lax_friedrich, E_lax_wendroff]

schemes = [’ftbs’, ’lax_friedrich’, ’lax_wendroff’]
p_theoretical = [1, 2, 2] # theoretical spatial orders

q_theoretical = [1, 1, 2] # theoretical temporal orders
h_x, h_t = symbols(’h_x h_t’)

XtickList = [i for i in range(1, len(hxList)+1)]

Xticknames = [r’$(h_x , h_t)_{0}$’.format(str(i)) for i in range(1, len(hxList)+1)]
lstyle = [’b’, ’r’, ’g’, ’m’]
legendList = []
# Optimize Cx, p, Ct, q for every scheme

for k, E in enumerate(ErrorList):
# optimize using only last 5 values of E and h for the scheme, as the first values may be outside
Cx0, Ct0 = optimize_error_cxct(E[-5:], hxList[-5:], htList[-5:],

p=p_theoretical[k], q=q_theoretical[k]) # Find appropriate initial
Cx, p, Ct, q = optimize_error(E[-5:], hxList[-5:], htList[-5:],

Cx0, p_theoretical[k], Ct0, q_theoretical[k]) # Optimize for all pa
# create sympy expressions of e, ex and et:

errorExpr = Cx*h_x**p + Ct*h_t**q
print errorExpr
errorExprHx = Cx*h_x**p
errorExprHt = Ct*h_t**q
# convert e, ex and et to python functions:

errorFunc = lambdify([h_x, h_t], errorExpr)
errorFuncHx = lambdify([h_x], errorExprHx)
errorFuncHt = lambdify([h_t], errorExprHt)
# plotting:
fig , ax = plt.subplots(2, 1, squeeze=False)
ax[0][0].plot(XtickList, np.log10(E),lstyle[k])
ax[0][0].plot(XtickList, np.log10(errorFunc(hxList, htList)),lstyle[k] + ’--’)
ax[1][0].plot(XtickList[-5:], E[-5:],lstyle[k])
ax[1][0].plot(XtickList[-5:], errorFunc(hxList, htList)[-5:],lstyle[k] + ’--’)
ax[1][0].plot(XtickList[-5:], errorFuncHx(hxList[-5:]), lstyle[k] + ’-.’)
ax[1][0].plot(XtickList[-5:], errorFuncHt(htList[-5:]),lstyle[k] + ’:’)
−1.0
ftbs −1.0
lax_friedrich −1
lax_wendroff
=E =E =E
−1.5 −1.5 −2
0.98
=6.45 hx + 1.32 ht0.98 =-1483.49 hx
1.87
+ 25.85 ht1.0 =364.05 hx
2.0
+ -148.59 ht2.0
−2.0 −3
−2.0
log10( )
log10( )
log10( )
−2.5 −4
−2.5
−3.0 −5
−3.5 −3.0 −6
−4.0 −3.5 −7
1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9 1 2 3 4 5 6 7 8 9
(hx ,ht )n (hx ,ht )n (hx ,ht )n
0.0035 0.00006
=E 0.008 =E 0.00005 =E
0.0030
0.98
0.0025 =6.45 hx + 1.32 ht0.98 0.006 =-1483.49 hx
1.87
+ 25.85 ht1.0 0.00004 =364.05 hx
2.0
+ -148.59 ht2.0
0.98 1.87 0.00003 2.0
0.0020 =6.45 hx =-1483.49 hx =364.05 hx
0.98 0.004 1.0 0.00002 2.0
0.0015 =1.32 ht =25.85 ht =-148.59 ht
0.00001
0.0010 0.002
0.00000
0.0005 −0.00001
0.000
0.0000 −0.00002
5 6 7 8 9 5 6 7 8 9 5 6 7 8 9
(hx ,ht )n (hx ,ht )n (hx ,ht )n
Figure 8.12: Optimized error expressions for all schemes. Test using doconce
combine images
Figure 8.13: The solutions at time t=T=1 for different grids corresponding
to the code-example above. A sin4 wave with period T = 0.4 travelling with
wavespeed a = 1. The solutions were calculated with a cfl-number of 0.8

8.9 Flux limiters

Looking at the previous examples and especially Figure 8.13 we clearly see the
difference between first and second order schemes. Using a first order scheme
require a very high resolution (and/or a cfl number close to one) in order to get
a satisfying solution. What characterizes the first order schemes is that they
are highly diffusive (loss of amplitude). Unless the resolution is very high or
the cfl-number is very close to one, the first order solutions will lose amplitude
as time passes. This is evident in the animation (8). (To see how the different
schemes behave with different combinations of CFL-number, and frequencies try
the sliders app located in the git repo in chapter 6.Fredrik 15: add a link or
something? ) However if the solution contain big gradients or discontinuities,
the second order schemes fail, and introduce nonphysical oscillations due to
their dispersive nature. In other words the second order schemes give a much
higher accuracy on smooth solutions, than the first order schemes, whereas
the first order schemes behave better in regions containing sharp gradients or
discontinuities. First order upwind methods are said to behave monotonically
varying in regions where the solution should be monotone (cite:Second Order
Positive Schemes by means of Flux Limiters for the Advection Equation.pdf).
Figure 8.14 show the behavior of the different advection schemes in a solution
containing discontinuities; oscillations arises near discontinuities for the Lax-
Wendroff scheme, independent of the resolution. The dissipative nature of the
first order schemes are evident in solutions with "low" resolution.
Figure 8.14: Effect of refining grid on a solution containing discontinuities; a

square box moving with wave speed a=1. Solutions are showed at t=1, using a
cfl-number of 0.8.
The idea of Flux limiters is to combine the best features of high and low order
schemes. In general a flux limiter method can be obtained by combining any
low order flux Fl , and high order flux Fh . Further it is convenient/required to
write the schemes in the so the so-called conservative form as in (8.41) repeated
here for convenience
∆t
un+1 = unj +

j Fj−1/2 − Fj+1/2 (8.62)
∆x
Fredrik 17: do we need clarifying more about conservative methods and
Fluxes calculated at midpoints and halftime?
The general flux limiter method solves (8.62) with the following definitions
of the fluxes Fi−1/2 and Fi+1/2

Fj−1/2 = Fl (j − 1/2) + φ(rj−1/2 ) Fh (j − 1/2) − Fl (j − 1/2) (8.63)

Fj+1/2 = Fl (j + 1/2) + φ(rj+1/2 ) Fh (j + 1/2) − Fl (j + 1/2) (8.64)
where φ(r) is the limiter function, and r is a measure of the smoothness of

the solution. The limiter function φ is designed to be equal to one in regions
where the solution is smooth, in which F reduces to Fh , and a pure high order
scheme. In regions where the solution is not smooth (i.e. in regions containing
sharp gradients and discontinuities ) φ is designed to be equal to zero, in which
F reduces to Fl , and a pure low order scheme. As a measure of the smoothness,
r is commonly taken to be the ratio of consecutive gradients
uj−1 − uj−2 uj − uj−1
rj−1/2 = , rj+1/2 = (8.65)
uj − uj−1 uj+1 − uj
In regions where the solution is constant (zero gradients), some special
treatment of r is needed to avoid division by zero. However the choice of this
treatment is not important since in regions where the solution is not changing,
using a high or low order method is irrelevant.
Lax-Wendroff limiters. The Lax-Wendroff scheme for the advection equation

may be written in the form of (8.62) by defining the Lax-Wendroff Flux FLW as:

1 ∆t
FLW (j − 1/2) = FLW (uj−1 , uj ) = a uj−1 + a 1 − a uj − uj−1
2 ∆x
(8.66)

1 ∆t
FLW (j + 1/2) = FLW (uj , uj+1 ) = a uj + a 1 − a uj+1 − uj
2 ∆x
(8.67)
which may be showed to be the Lax-Wendroff two step method condensed

∆t
to a one step method as outlined in (8.7.1). Notice the term ∆x a = c. Now
the Lax-Wendroff flux assumes that of an upwind flux (a u j−1 or a uj ) with an
additional anti diffusive term ( 12 a (1 − c) uj − uj−1 for FLW (j − 1/2)). A flux

limited version of the Lax-Wendroff scheme could thus be obtained by adding a

limiter function φ to the second term
1
F (j − 1/2) = a uj−1 + φ rj−1/2 a (1 − c) uj − uj−1 (8.68)
2
1
F (j + 1/2) = a uj + φ rj+1/2 a (1 − c) uj+1 − uj (8.69)
2
When φ = 0 the scheme is the upwind scheme, and when φ = 1 the scheme is
the Lax-Wendroff scheme. Many different limiter functions have been proposed,
the optimal function however is dependent on the solution. Sweby (Fredrik
18: Cite sweby ) showed that in order for the Flux limiting scheme to possess
the wanted properties of a low order scheme; TVD, or monotonically varying in
regions where the solution should be monotone, the following equality must hold
Fredrik 19: definition of TVD? more theory backing these statements?
φ(r)
0≤ , φ(r) ≤ 2 (8.70)
r
Where we require
φ(r) = 0, r≤0 (8.71)

Hence for the scheme to be TVD the limiter must lie in the shaded region
of Figure 8.15, where the limiter function for the two second order schemes,
Lax-Wendroff and Warming and Beam are also plotted.
3
Warming and Beam, =r
Lax-Wendroff, =1
1
0 1 2 3
r
Figure 8.15: TVD region for flux limiters (shaded), and the limiters for the
second order schemes; Lax-Wendroff, and Warming and Beam
For the scheme Fredrik 20: maybe add only using uj−2 , uj−1 uj uj+1 to
be second order accurate whenever possible, the limiter must be an arithmetic
average of the limiter of Lax-Wendroff (φ = 1) and that of Warming and
Beam(φ = r) Fredrik 21: need citing(Sweby), and maybe more clearification .

With this extra constraint a second order TVD limiter must lie in the shaded
region of Figure 8.16
0 1 2 3
r
Figure 8.16: Second orderTVD region for flux limiters; sweby diagram Fredrik
22: cite sweby?
Note that φ(0) = 0, meaning that second order accuracy must be lost at
extrema. All schemes pass through the point φ(1) = 1, which is a general
requirement for second order schemes. Many limiters that pass the above
constraints have been proposed. Here we will only consider a few:
Superbee : φ(r) = max (0, min(1, 2r), min(2, r)) (8.72)

r + |r|
V an − Leer : φ(r) = (8.73)
1 + |r|
minmod : φ(r) = minmod(1, r) (8.74)
where minmod of two arguments is defined as

 a if a · b > 0 and |a| > |b|
minmod(a, b) = b if a · b > 0 and |b| > |a| > 0 (8.75)
0 if a · b < 0

The limiters given in (8.72) -(8.74) are showed in Figure 8.17. Superbee traverses
the upper bound of the second order TVD region, whereas minmod traverses
the lower bound of the region. Keeping in mind that in our scheme, φ regulates
the anti diffusive flux, superbee is thus the least diffusive scheme of these.
In addition we will consider the two base cases:
upwind : φ(r) = 0 (8.76)
Lax − W endrof f : φ(r) = 1 (8.77)
in which upwind, is TVD, but first order, and Lax-Wendroff is second order, but
not TVD.
Superbee Van-Leer
Min-Mod
0 1 2 3
r
Figure 8.17: Second order TVD limiters
Example of Flux limiter schemes on a solution with continuos and

discontinuous sections. All of the above schemes have been implemented in
the python class Fluxlimiters, and a code example of their application on a
solution containing combination of box, and sine moving with wavespeed a=1,
may be found in the python script advection-schemes-flux-limiters.py. The
result may be seen in the video (9)
Movie 9: The effect of Flux limiting schemes on solutions containing

continuos and discontinuous sections mov-ch6/flux_limiter.mp4
1.0
0.8
0.6
u
0.4
0.2
0.0
0.0 0.2 0.4 0.6 0.8 1.0
x
3.0
2.5
2.0
1.5
r
1.0
0.5
0.0
0.0 0.2 0.4 0.6 0.8 1.0
u
2.0
1.5 superbee
1.0
0.5
van-Leer
0.0 minmod
0.0 0.2 0.4 0.6 0.8 1.0
r
Figure 8.18: Relationship between u and smoothness measure r, and different

limiters φ, on continuos and discontinuous solutions
8.10 Example: Burger’s equation

The 1D Burger’s equation is a simple (if not the simplest) non-linear hyperbolic
equation commonly used as a model equation to illustrate various numerical
schemes for non-linear hyperbolic differential equations. It is normally prestented
as:
∂u ∂u
+u =0 (8.78)
∂t ∂x
To enable us to present schemes for a greater variety of hyperbolic differenctial
equations and to better handle shocks (i.e discontinuities in the solution), we
will present our model equation on conservative form:
∂ u2

∂u
+ =0 (8.79)
∂t ∂x 2
and by introducing a flux function
u2
F (u) = (8.80)
2
the conservative formulation of the Burger’s equation may be represented by
a generic transport equation:
∂u ∂F (u)
+ =0 (8.81)
∂t ∂x
8.10.1 Upwind Scheme

The upwind scheme for the general conservation equation take the form:
∆t
un+1 = unj − F (unj+1 ) − F (unj )

j (8.82)
∆x
where we have assumed a forward propagating wave (a(u) = F 0 (u) > 0, i.e. u > 0
for the burger equation). In the opposite case ∂F∂x(u) will be approximated by
1 n n

∆x F (uj ) − F (uj−1 )
8.10.2 Lax-Friedrich
The Lax-Friedrich conservation equation take the form as given in (8.38), repeated
here for convinience:
n n
∆t n Fj+1 − Fj−1
un+1
j = (uj+1 + unj−1 ) − (8.83)
2 2∆x
8.10.3 Lax-Wendroff
As outlined in ((8.7.1)), the general Lax-Wendroff two step method takes the
form as given in (8.51) and (8.52) repeated here for convinience:
First step:
n+1/2 1 n ∆t
uj+1 + unj − F (unj+1 ) − F (unj )

uj+1/2 = (8.84)
2 2∆x
n+1/2 1 n ∆t
uj + unj−1 − F (unj ) − F (unj−1 )

uj−1/2 = (8.85)
2 2∆x
∆t n+1/2 n+1/2

un+1
j = unj − F (uj+1/2 ) − F (uj−1/2 ) (8.86)
∆x
In the previous section ((8.7.1)) we showed how the two step Lax-Wendroff
method could be condensed to a one step method. The same procedure may be
applied to a the general transport equation given by (8.81). However for the
nonlinear case (8.43) no longer holds. This may be overcome be rewriting (8.43):

∂ ∂F∂t(u) n
n n n n
∂u ∂F ∂ 2 u ∂ 2 F (u)
=− and =− =−
∂t ∂x ∂t2 ∂t∂x ∂x

j j j j j

∂F (u) ∂u n n (8.87)
∂ ∂F

∂u ∂t
∂ a(u) ∂x
=− =
∂x ∂x

j j
Now inserting into the Taylor series we get
∂F (u) ∆t2 ∂ a(u) ∂F

un+1
j = unj
− ∆t + ∂x
(8.88)
∂x 2 ∂x
and further we may obtain the general Lax-Wendroff one-step method for a
generic transport equation
∆t ∆t2 h i
un+1
j = unj − (Fj+1 − Fj−1 )+ aj+1/2 (Fj+1 − Fj )−aj−1/2 (Fj − Fj−1 )
2∆x 2∆x2
(8.89)
where a(u) is the wavespeed, or the Jacobian of F, F 0 (u), which is u for
the burger equation. As indicated a(u) has to be approximated at the indice
(j + 1/2) and (j − 1/2). This may simply be done by averaging the neighboring
values:
1 n
uj + unj+1

aj+1/2 = (8.90)
2
for the burger equation. Another method that assure conservation is to use the
following approximation
( F n −F n
j+1 j
un −un if uj+1 6= uj
aj+1/2 = j+1 j (8.91)
uj otherwise
8.10.4 MacCormack
The MacCormack scheme was discussed in ((8.7.1)) and is given by (8.94)
repeated her for convinience
∆t
upj = unj + Fjn − Fj+1
n

(8.92)
∆x
1 n 1 ∆t
un+1 uj + upj + p
− Fjp

j = Fj−1 (8.93)
2 2 ∆x
(8.94)
8.10.5 Method of Manufactured solution

For the Advection equation we were able to verify our schemes by comparing
with exact solutions, using MES. For the burger equation it is not easy to find an
analytical solution, so in order to verify our schemes we use the MMS approach.
However this requires that our schemes can handle source terms. The new
equation to solve is thus the modified burgers equation
∂u ∂u
+u =Q (8.95)
∂t ∂x
In this chapter we will consider source terms that are a function of x and t
only. The basic approach to adding source terms to our schemes is to simply
add a term Qni to our discrete equations. The schemes mentioned above with
possibility to handle source terms are summarized in Table (8.10.5).
Name of Scheme Scheme order

Upwind un+1
j = unj − ∆t
∆x + ∆tQj
n
1
n n
Fj+1 −Fj−1
Lax-Friedrichs ujn+1 = ∆t2 (un
j+1 + un
j−1 ) − 2∆x + ∆tQnj 1
n+1/2 1
∆t
∆t n
uj+1/2 = 2 uj+1 + uj − 2∆x F (uj+1 ) − F (unj ) +
n n n
2 Qj+1/2
n+1/2
uj−1/2 = 12 unj + unj−1 − 2∆x ∆t ∆t n

Lax-Wendroff F (unj ) − F (unj−1 ) + 2 Qj−1/2 2

n+1/2 n+1/2
ujn+1 = unj − ∆x ∆t
F (uj+1/2 ) − F (uj−1/2 ) + ∆tQnj
n+1/2
upj = unj − ∆x∆t n

macCormack Fj+1 − Fjn + ∆tQj 2
n+1/2
ujn+1 = 12 unj + upj − 12 ∆x ∆t
Fjp − Fj−1
p
+ ∆t

2 Qj
Examples on how to implement these schemes are given below, where we

have used RHS(x, t) instead of Q for the source term:
ftbs or upwind:
def ftbs(u, t):
"""method that solves u(n+1), for the scalar conservation equation with source term:
du/dt + dF/dx = RHS,
where F = 0.5u^2 for the burger equation
with use of the forward in time backward in space (upwind) scheme
Args:
u(array): an array containg the previous solution of u, u(n). (RHS)

t(float): an array
Returns:
u[1:-1](array): the solution of the interior nodes for the next timestep, u(n+1).
"""
u[1:-1] = u[1:-1] - (dt/dx)*(F(u[1:-1])-F(u[:-2])) + dt*RHS(t-0.5*dt, x[1:-1])
return u[1:-1]
Lax-Friedrichs:
def lax_friedrich_Flux(u, t):
with use of the lax-friedrich scheme
Args:
t(float): an array
Returns:
"""
u[1:-1] = (u[:-2] +u[2:])/2.0 - dt*(F(u[2:])-F(u[:-2]))/(2.0*dx) + dt*(RHS(t, x[:-2]) + RHS(t, x
return u[1:-1]
Lax-Wendroff-Two-step:
def Lax_W_Two_Step(u, t):
with use of the Two-step Lax-Wendroff scheme
Args:
u(array): an array containg the previous solution of u, u(n).
t(float): time at t(n+1)
Returns:
"""
ujm = u[:-2].copy() #u(j-1)
uj = u[1:-1].copy() #u(j)
ujp = u[2:].copy() #u(j+1)
up_m = 0.5*(ujm + uj) - 0.5*(dt/dx)*(F(uj)-F(ujm)) + 0.5*dt*RHS(t-0.5*dt, x[1:-1] - 0.5*dx) #u(n+
up_p = 0.5*(uj + ujp) - 0.5*(dt/dx)*(F(ujp)-F(uj)) + 0.5*dt*RHS(t-0.5*dt, x[1:-1] + 0.5*dx)#u(n+0
u[1:-1] = uj -(dt/dx)*(F(up_p) - F(up_m)) + dt*RHS(t-0.5*dt, x[1:-1])
return u[1:-1]
macCormack:
def macCormack(u, t):
with use of the MacCormack scheme
Args:
t(float): an array
Returns:
"""
up = u.copy()
up[:-1] = u[:-1] - (dt/dx)*(F(u[1:]) - F(u[:-1])) + dt*RHS(t-0.5*dt, x[:-1])
u[1:] = .5*(u[1:] + up[1:] - (dt/dx)*(F(up[1:]) - F(up[:-1])) + dt*RHS(t-0.5*dt, x[1:]))
return u[1:-1]
Exercise 11: Stability analysis of the Lax-Friedrich scheme

In this exercise we want to assess the stability of the Lax-Friedrich scheme for
the advection equation as formulated in (8.40).
a) Use the PC-criterion to find a sufficient condition for stability for (8.40).
b) Use the von Neumann method to find a sufficient and necessary condition
for stability for (8.40) and compare with the PC-condition
Chapter 9
Python Style Guide
PEP 8, and Google have python style guides which we generally try to follow,
though we have made some different choices in some respect.
• https://www.python.org/dev/peps/pep-0008/
• https://google-styleguide.googlecode.com/svn/trunk/pyguide.html
9.1 The conventions we use

First off we try to be compliant to the https://www.python.org/dev/peps/pep-
0008/ the summary is as follows:
• Use 4 spaces, not tabs (you can get your editor to do this automatically,
and it is recommended for coding in general).
• If your function has many inputs, list them vertically by having an extra
indent or more
• We use CapitalizedWords and MixedCase mostly in our code. We capitalize
the first letter to signal that this is a class, and we keep it in lowercase if
it is an instance of a class, or a function. This allows us to easily see what
we´re working with.
• We disobey the convention on function names, where PEP 8 wants them to
be lowercase and separated by underscores, we use mixedCase for functions
as well. This is due to underscore being a hassle to use.
• If you have a list of things, add list to the name. Do not use plurals. good:
vesselList, bad: vessels
• Further, as we have a project with a lot of functional files, we often use
"import .... as" method, where we have a list of abbreviations that are
convenient to follow
347
CHAPTER 9. PYTHON STYLE GUIDE 348
9.2 Summary
• Have clear names and use proper namespacing in your code
• Document your code with docstrings adhering to the Goodle docstring

standard. Clearly indicate what is input and what is outout. Especially
side-effects!
• Structure your code so that it is as self-explanatory as possible, and use
comments where additional clarification is useful.
• Remember that all code you write will end up being treated as a "black
box" sooner or later. So make sure that it actually works like one, with
clean input-output dichotomy.
• Exceptions should crash your program, unless you have very specific reasons
why it should not.
• If you absolutely want to stop exceptions, you should not use "except",
as this will catch the system exit and keyboard interrupts. If you want
catch-all use "except Exception as e"
• You should catch, append and re-raise the exception at each level of the
code, so that an informative traceback will appear when the program dies.
• Unit Testing is desirable, and you should test the crucial behavior of each
feature you wish to push to the main project. Also, test that the code
fails the way it should. Few things are as hard to track down as code that
passes wrong data onto other parts in the program
• Keep your Unit tests. Write them well. Systematize them. Make sure
they´re thorough, independent of each other, and fast.
9.3 The code is the documentation... i.e. the

docs always lie!
Write the code as clearly as possible to what it is doing. Clear, succinct variable
names and conventions, and ideally code review.
9.4 Docstring Guide

The google docstring guide is located at Google Style Python Docstrings
Chapter 10
Sympolic computation with

SymPy
In this chapter we provide a very short introduction to SymPy customized for

the applications and examples in the current setting. For a more thorough
presentation see e.g. [9].
10.1 Introduction
SymPy is a Python library for symbolic mathematics, with the ambition to
offer a full-featured computer algebra system (CAS). The library design makes
SymPy ideally suited to make symbolic mathematical computations integrated
in numerical Python applications.
10.2 Basic features

The SymPy library is implemented in the Python module sympy. To avoid
namespace conflicts (e.g. with other modules like NumPy/SciPy) we propose to
import the SymPy as:
import sympy
Symbols. SymPy introduces the class symbols (or Symbol) to represent math-
ematical symbols as Python objects. An instance of type symbols has a set of
attributes to hold the its properties and methods to operate on those properties.
Such symbols may be used to represent and manipulate algebraic expressions.
Unlike many symbolic manipulation systems, variables in SymPy must be defined
before they are used (for justification see sympy.org)
As an example, let us define a symbolic expression, representing the mathe-
matical expression x2 + xy − y
349
CHAPTER 10. SYMPOLIC COMPUTATION WITH SYMPY 350
import sympy
x, y = sympy.symbols(’x y’)
expr = x**2 + x*y -y
expr
Note that we wrote the expression as if "x" and "y" were ordinary Python
variables, but instead of being evaluated the expression remains unaltered.
To make the output look nicer we may invoke the pretty print feature of
SymPy by:
sympy.init_printing(use_latex=True)
expr
The expression is now ready for algebraic manipulation:

expr + 2
x**2 + x*y -y + 2
and
expr + y
x**2 + x*y
Note that the result of the above is not x2 + xy − y + y but rather x2 + xy,
i.e. the −y and the +y are added and found to cancel automatically by SymPy
and a simplified expression is outputted accordingly. Appart from rather obvious
simplifications
√ like discarding subexpression that add up to zero (e.g. y − y or
9 = 3), most simplifications are not performed automatically by SymPy.
expr2 = x**2 + 2*x + 1
expr3 = sympy.factor(expr2)
expr3
Matrices. Matrices in SymPy are implemented with the Matrix class and are
constructed by providing a list of row the vectors in the following manner:
sympy.init_printing(use_latex=’mathjax’)
M = sympy.Matrix([[1, -1], [2, 1], [4, 4]])
M
A matrix with symbolic elements may be constructed by:

a, b, c, d = sympy.symbols(’a b c d’)
M = sympy.Matrix([[a, b],[c, d]])
M
The matrices may naturally be manipulated like any other object in SymPy
or Python. To illustrate this we introduce another 2x2-matrix
n1, n2, n3, n4 =sympy.symbols(’n1 n2 n3 n4’)
N=sympy.Matrix([[n1, n2],[n3, n4]])
N
The two matrices may then be added, subtracted, multiplied, and inverted
by the following simple statements
CHAPTER 10. SYMPOLIC COMPUTATION WITH SYMPY 351
M+N, M-N, M*N, M.inv()
M=sympy.Matrix([[0, a], [a, 0]])
Diagonaliztion:
L, D = M.diagonalize()
L, D
Differentiating and integrating. Consider also the parabolic function which

may describe the velocity profile for fully developed flow in a cylinder.
from sympy import integrate, diff, symbols, pi
v0, r = symbols(’v0 r’)
v = v0*(1 - r**2)
Q = integrate(2*pi*v*r, r)
Q
Q = sympy.factor(Q)
Q
newV = diff(Q, r)/(r*2*pi)

newV
sympy.simplify(newV)
R
compute cos(x)
from sympy import cos
integrate(cos(x), x)
R∞
compute −∞
sin(x2 )
from sympy import sin, oo

integrate(sin(x**2), (x, -oo, oo))
sin(x)
limits. perform limx→0 x
from sympy import limit
limit(sin(x)/x, x, 0)
solving equations. solve the algebraic equation x2 − 4 = 0

from sympy import solve
solve(x**2 - 4*x, x)
solve the differential equation y 00 − y 0 = et

from sympy import dsolve, Function, Eq, exp
y = Function(’y’)
diffeq = Eq(y(t).diff(t, t) - y(t), exp(t))
dsolve(diffeq, y(t))
Bibliography
[1] P.W. Bearman and J.K. Harvey. Golf ball aerodynamics. Aeronaut Q, 27(pt
2):112–122, 1976. cited By 119.
[2] E. Cheney and David Kincaid. Numerical Mathematics and Computing.

Cengage Learning, 4th edition, 1999.
[3] E. Cheney and David Kincaid. Numerical Mathematics and Computing 7th.
Cengage Learning, 7th edition, 2012.
[4] J. Evett and C. Liu. 2,500 Solved Problems in Fluid Mechanics and Hy-
draulics. Schaum’s Solved Problems Series. McGraw-Hill Education, 1989.
[5] C. A. J. Fletcher. Computational Techniques for Fluid Dynamics. 1. ,
Fundamental and General Techniques. Springer series in computational
physics. Springer-Verlag, Berlin, Paris, 1991.
[6] Ernst Hairer, Syvert Paul Norsett, and Gerhard Wanner. Solving Ordinary
Differential Equations I: Nonstiff Problems, volume 1. Springer Science &
Business, 2008.
[7] C. Hirsch. Numerical Computation of Internal and External Flows. Elsevier,
2007.
[8] George A. Hool and Nathan Clarke Johnson. Elements of Structural Theory-
Definitions. Handbook of Building Construction. New York. McGraw-Hill.
Google Books, 1920. p.2.
[9] Robert Johansson. Numerical Python. a Practical Techniques Approach for
Industry. Springer, 2015.
[10] Hans Petter Langtangen. A Primer on Scientific Programming With Python.

Springer, Berlin; Heidelberg; New York, fourth edition, 2011.
[11] P. D. Lax. Hyperbolic Systems of Conservation Laws and the Mathematical
Theory of Shock Waves. Society for Industrial and Applied Mathematics,
1973. Regional conference series in applied mathematics.
352
BIBLIOGRAPHY 353
[12] Randall J. LeVeque. Finite Difference Methods for Ordinary and Partial
Differential Equations: Steady-State and Time-Dependent Problems. Society
for Industrial and Applied Mathematics, Philadelphia, PA, 2007. OCLC:
ocm86110147.
[13] R. W. MacCormack. The effect of viscosity in hypervelocity impact cratering.
Astronautics, AIAA, 69:354, 1969.
[14] John. C. Tannehil, Dale A. Anderson, and Richard H. Pletcher. Computa-
tional Fluid Mechanics and Heat Transfer. Taylor & Francis, 1997.
[15] F.M. White. Viscous Fluid Flow. WCB/McGraw-Hill, 1991.
[16] F.M. White. Fluid Mechanics. McGraw-Hill series in mechanical engineering.

WCB/McGraw-Hill, 1999.

Main

Uploaded by

Copyright:

Available Formats

Main

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Main

Uploaded by

Copyright:

Available Formats

What topics are covered in the document?

What topics are covered in the document?

What numerical methods are discussed?

What numerical methods are discussed?

Numerical Methods for Engineers

Leif Rune Hellevik

Department of Structural Engineering, NTNU

Jan 26, 2018

2 Initial value problems for ODEs 11

2.11 Runge-Kutta of 4th order . . . . . . . . . . . . . . . . . . . . . . 60

3 Shooting Methods 108

4 Finite differences for ODEs 163

4.4.1 Example: Heat exchanger with constant cross-section . . 172

5 Mathematical properties of PDEs 217

6 Elliptic PDEs 229

7.5.3 Crank-Nicolson scheme. θ-scheme . . . . . . . . . . . . . 282

8 hyperbolic PDEs 307

9 Python Style Guide 347

10 Sympolic computation with SymPy 349

1.1 Acknowledgements and dedications

1.2 Check Python and LiClipse plugin

working. Download and run the code systemCheck.py in your Eclipse/LiClipse

print "check for matplotlib ",

print "check for numpy ",

print "check for scipy ",

print "check for sympy ",

1.3 Scientific computing with Python

Initial value problems for

y 00 (x) = f (x, y, y 0 ) (2.3)

2.1.1 Example: A mathematical pendulum

Figure 2.1: An illustration of the mathematical pendulum.

To present the governing equation (2.6) on

θ̈(t) + sin(θ(t)) = 0 (2.7)

θ̈(t) + θ(t) = 0 (2.9)

2.1.2 n-th order linear ordinary differential equations

2.2 Existence and uniqueness of solutions for ini-

The criteria for existence and uniqueness

S = max |f (y, x)|.

We say that f (y, x) is Lipschitz continuous in y over some domain D if there

2.3 Taylor’s method

θ̈(t) + θ(t) = 0 (2.14)

θ(4) (t) = −θ̈(t) → θ(4) (0) = −θ̈(0) = θ0

Taylor’s Method for solution of initial value ODE..

3. Differentiate the ODE as many times needed to obtain the wanted

2.3.1 Example: Taylor’s method for the non-linear math-

θ̈(t) + sin (θ(t)) = 0

2.3.2 Example: Newton’s first differential equation

y 0 (x) = 1 − 3x + y + x2 + xy, y(0) = 0

Series expansion around x = 0 gives

2.4 Reduction of Higher order Equations

y 00 (x) + y(x) = 0, y(0) = a0 , y 0 (0) = b0

y 000 (x) + 2y 00 (x) − (y 0 (x))2 + 2y(x) = x2 (2.21)

y00 (x) =y1 (x)

General procedure to reduce a higher order ODE to a system of

y (m) = f (x, y, y 0 , y 00 , . . . , y (m−1) ) (2.22)

with y = y0 , we get the following system of ODEs:

with the following boundary conditions:

y0 (x0 ) = a0 , y1 (x0 ) = a1 , . . . , ym−1 (x0 ) = am−1

2.4.1 Example: Reduction of higher order systems