Solving Differential Equations On Manifolds: Ernst Hairer
Solving Differential Equations On Manifolds: Ernst Hairer
Solving Differential Equations On Manifolds: Ernst Hairer
Differential Equations
on Manifolds
Ernst Hairer
Universite de Gen`eve
Section de mathematiques
2-4 rue du Li`evre, CP 64
CH-1211 Gen`eve 4
June 2011
Acknowledgement. These notes have been distributed during the lecture Equations
differentielles sur des sous-varietes (2 hours per week) given in the spring term 2011. At
several places, the text and figures are taken from one of the monographs by the author.
Contents
I
II
Introduction by Examples . . . . . . . . . . . . . . . . . . . . . . . . . . .
I.1
I.2
I.3
I.4
1
3
4
6
the
. .
. .
. .
rigid
. . .
. . .
. . .
body .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Submanifolds of Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
II.1
II.2
II.3
II.4
II.5
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Projection methods . . . . . . . . . . . . . . .
Numerical methods based on local coordinates
Derivative of the exponential and its inverse . .
Methods based on the Magnus series expansion
Convergence of methods on submanifolds . . .
Exercises . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
IV Differential-Algebraic Equations . . . . . . . . . . . . . . . . . . . . . . . .
IV.1
IV.2
IV.3
IV.4
IV.5
V
coefficients
. . . . . . .
. . . . . . .
. . . . . . .
. . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7
7
9
11
14
16
19
19
21
23
24
26
27
29
29
30
32
34
36
39
V.1
V.2
V.3
V.4
V.5
V.6
39
42
45
47
50
51
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Recommended Literature
There are many monographs treating manifolds and submanifolds. Many of them can be
found under the numbers 53 and 57 in the mathematics library. Books specially devoted to
the numerical treatment of differential equations on manifolds (differential-algebraic equations) are listed under number 65 in the library.
We give here an incomplete list for further reading: the numbers in brackets (e.g. [MA
65/403]) allow one to find the book without computer search.
R. Abraham, J.E. Marsden and T. Ratiu, Manifolds, Tensor Analysis, and Applications, 2nd edition,
Applied Mathematical Sciences 75, Springer-Verlag, 1988. [MA 57/266]
V. Arnold, Equations Differentielles Ordinaires, Editions Mir (traduction francaise), Moscou, 1974.
[MA 34/102]
U.M. Ascher and L.R. Petzold, Computer Methods for Ordinary Differential Equations and DifferentialAlgebraic Equations, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1998.
K.E. Brenan, S.L. Campbell and L.R. Petzold, Numerical Solution of Initial-Value Problems in
Differential-Algebraic Equations. Revised and corrected reprint of the 1989 original,
Classics in Applied Mathematics 14, Society for Industrial and Applied Mathematics
(SIAM), Philadelphia, PA, 1996. [MA 65/294]
E. Eich-Soellner and C. F
uhrer, Numerical methods in multibody dynamics, European Consortium
for Mathematics in Industry. B.G. Teubner, Stuttgart, 1998.
E. Griepentrog and R. M
arz, Differential-Algebraic Equations and Their Numerical Treatment,
Teubner-Texte zur Mathematik 88, Teubner Verlagsgesellschaft, Leipzig, 1986. [MA
65/256]
E. Hairer, C. Lubich and M. Roche, The Numerical Solution of Differential-Algebraic Systems by
Runge-Kutta Methods, Lecture Notes in Mathematics 1409, Springer Berlin, 1989. [MA
00.04/3 1409]
E. Hairer, C. Lubich and G. Wanner, Geometric Numerical Integration. Structure-Preserving Algorithms for Ordinary Differential Equations, 2nd edition, Springer Series in Computational Mathematics 31, Springer Berlin, 2006. [MA 65/448]
E. Hairer, S.P. Nrsett and G. Wanner, Solving Ordinary Differential Equations I. Nonstiff Problems, 2nd edition, Springer Series in Computational Mathematics 8, Springer Berlin,
1993. [MA 65/245]
E. Hairer and G. Wanner, Solving Ordinary Differential Equations II. Stiff and Differential-Agebraic
Problems, 2nd edition, Springer Series in Computational Mathematics 14, Springer
Berlin, 1996. [MA 65/245]
E. Hairer and G. Wanner, Analysis by Its History, Undergraduate Texts in Mathematics, Springer
New York, 1995. [MA 27/256]
P. Kunkel and V. Mehrmann, Differential-Algebraic Equations. Analysis and Numerical Solution,
EMS Textbooks in Mathematics. European Mathematical Society (EMS), Z
urich, 2006.
[MA 34/325]
S. Lang, Introduction to Differentiable Manifolds, 2nd edition, Universitext, Springer New York,
2002. [MA 57/15]
J.M. Lee, Introduction to Smooth Manifolds, Graduate Texts in Mathematics, Springer New York,
2003. [MA 53/302]
Chapter I
Introduction by Examples
Systems of ordinary differential equations in the Euclidean space Rn are given by
y = f (y),
(0.1)
I.1
Let I1 , I2 , I3 be the principal moments of inertia of a rigid body. The angular momentum
vector y = (y1 , y2 , y3 )T then satisfies Eulers equations of motion
y 1 = (I31 I21 ) y3 y2
y 2 = (I11 I31 ) y1 y3
y 3 = (I21 I11 ) y2 y1
or
y 1
0 y3 y2
y1 /I1
y 2 = y3
0 y1 y2 /I2
.
y 3
y2 y1 0
y3 /I3
(1.1)
Fig. I.1: Eulers equation of motion for I1 = 1.6, I2 = 1, I3 = 2/3; left picture: vector field
on the sphere; right picture: some solution curves.
Introduction by Examples
C(y) =
(1.2)
is exactly
preserved along solutions, a property that can be checked by differentiation:
d
C y(t) = . . . = 0. As a consequence, the solution remains forever on the sphere with
dt
radius that is determined by the initial values. The left picture of Figure I.1 shows the vector
f (y) attached to selected points y of the unit sphere.
To study further properties of the solution we write the differential equation as
y = B(y)H(y)
with
0 y3 y2
B(y) = y3 0 y1
,
y2 y1 0
The function H(y) is called Hamiltonian of the system, whereas C(y) of (1.2) is called
d
H(y(t)) =
Casimir function. Exploiting the skew-symmetry of the matrix B(y), we obtain dt
T
H(y(t)) B(y)H(y(t)) = 0, which implies the preservation of the Hamiltonian H(y) along
solutions of (1.1). Consequently, solutions lie on the intersection of a sphere C(y) = Const
with an ellipsoid H(y) = Const, and give rise to the closed curves of the right picture in
Figure I.1. Solutions are therefore typically periodic.
Numerical solutions are displayed in Figure I.2. The top picture shows the numerical
result, when the explicit Euler method yn+1 = yn + hf (yn ) is applied with step size h =
0.025 and with the initial value y0 = (cos(0.9), 0, sin(0.9)). The numerical solution drifts
away from the manifold. The bottom left picture shows the result of the trapezoidal rule
yn+1 = yn + h2 (f (yn+1 ) + f (yn )) with h = 1, where the numerical solution is orthogonally
Fig. I.2: Top picture: integration with explicit Euler; bottom left picture: trapezoidal rule
with projection onto the sphere; bottom right picture: implicit midpoint rule.
Introduction by Examples
projected onto the sphere after every step. The bottom right picture considers the implicit
mid-point rule yn+1 = yn + hf ( 12 (yn+1 + yn )) with h = 1. Even without any projection, the
solution agrees extremely well with the exact solution. All these behaviours will be explained
in later chapters.
I.2
(2.1)
where u(t) is a control function that permits to steer the motion y(t) of a mechanical system.
Differentiating the algebraic equation g(y(t)) = 0 with respect to time yields g 0 (y)f (y, u) = 0.
Under suitable regularity assumptions this relation permits us to express u as a function of y
(using the implicit function theorem). Inserting u = G(y) into (2.1) gives a differential
equation for y on the manifold M = {y ; g(y) = 0}.
1
4
3
(2.2)
The problem consists in finding the motion (t) of the articulated robot arm such that the
endpoint of the last segment follows a given parametrized curve (t) in the plane and
k(t)k
min
subject to
g((t)) = (t).
Differentiating the algebraic relation with respect to time yields the underdetermined linear
= (t)
(two linear equations for n > 2 unknowns). Among all
equation g 0 ((t))(t)
for (t)
is minimized when this vector is
solutions of this linear system, the Euclidean norm of (t)
0
0
perpendicular to ker g ((t)). Because of (ker g ()) = Im g 0 ()T , this leads to the problem
= g 0 ()T u,
g() = (t),
(2.3)
which is of the form (2.1), if we add the trivial equation t = 1 to the system, and interpret
y = (, t). This is a differential equation on the manifold M = {(, t) ; g() (t) = 0}.
The differentiated constraint yields g 0 ()g 0 ()T u = (t),
1
(t),
I.3
Introduction by Examples
A rich treasure trove of differential equations on manifolds are constrained mechanical systems (or multi-body systems). Let q = (q1 , . . . , qn )T be generalized coordinates of a conservative mechanical system with kinetic energy T (q)
= 21 qT M q (symmetric positive definite mass
matrix M ) and potential energy U (q), which is subject to holonomic constraints g(q) = 0
(here, g : Rn Rm with m < n). The equations of motion are then given by
q = v
M v = U (q) g 0 (q)T
0 = g(q).
(3.1)
v 1 = q1 ,
v 2 = 1 q2 ,
0 = q12 + q22 1,
(3.2)
+ sin = 0.
Remark. In general it is not possible to determine minimal coordinates (where the number
of coordinates equals the number of degrees of freedom of the mechanical system). Even if it
is possible, they are usually only locally defined and the differential equations become much
more complicated as the formulation (3.1). Our next example illustrates such a situation
and shows the importance of considering differential equations on manifolds.
Introduction by Examples
k (t) =
qij (t)
0
for i = 0, 1, 2, j = 1, 2, 3,
for j = 2, 3, 4, 5,
for j = 2, 3, 4, 5, 6,
(3.3)
for k = 3(i 1) + j, i = 1, 2, 3, j = 1, 2, 3,
else.
The constraint condition g(q) = (t) represents 18 (linear and quadratic) equations for 18
unknowns q = (q11 , q12 , q13 , q21 , q22 , q23 , . . . , q63 )T . For a consistent vector (t), this nonlinear
equation possesses as solution a discrete point and a one-dimensional closed curve in R18
(without proof, see also Exercise 7). To get a nontrivial dynamics we assume that the initial
value lies on the one-dimensional curve.
To complete the description of the problem, we assume that the mass of the pieces is
unity and concentrated in their corner, and the motion is without friction. The kinetic and
potential energies are then given by
6
1X
qiT qi ,
T (q)
=
2 i=1
U (q) =
6
X
qi3 ,
i=1
where the potential only takes gravity into account. The equations of motion are obtained
by (3.1) with the constraint replaced by g(q) (t) = 0.
Introduction by Examples
Remark. The fact that the equation g(q) = (t) admits a one-dimensional submanifold as
solution shows that the 18 equations are not independent. For a numerical treatment we can
remove one (carefully chosen) constraint and work with the remaining 17 constraints. This
can be done with the help of a QR decomposition of g 0 (q) which is anyway required during
the integration.
I.4
Exercises
1. Compute all stationary solutions of the system (1.1) and identify them in Figure I.1. Explain
the behaviour of the solutions close to these points.
2. If the principal moments of inertia satisfy I1 = I2 6= I3 , the rigid body is called a symmetrical
top. In this situation, solve analytically Eulers equations of motion (1.1).
3. For a vector = (1 , . . . , n ) of angles consider the function g() of (2.2). Prove that g 0 () is
of full rank 2 if and only if there exists a pair of subscripts i, j such that
i j 6= 0
mod .
4. Consider the problem (2.3) and assume that initial values satisfy i (t0 ) = j (t0 ). Prove that
the solution then satisfies i (t) = j (t) wherever it exists.
5. Find a differential equation (on a submanifold) that describes the solution of the problem
12 + (2 1 )2 + . . . + (n n1 )2 min
subject to the constraint g() (t) = 0, where g() is as in (2.2).
6. Consider a n n matrix M and a m n matrix G (with m < n). Under the assumptions that
M is a symmetric positive definite matrix and G is of full rank m, prove that the matrices
M
G
GT
0
and
G M 1 GT
are invertible.
7. Consider the function g : R18 R18 defined in (3.3), and compute the Jacobian matrix g 0 (q)
for the two (admissible) points
a = (1, 0, 0; 0, 0, 0; 0, 1, 0; 0, 1, 1; 0, 0, 1; 1, 0, 1)
b = (1, 0, 0; 0, 0, 0; 0, 1, 0; 0, 1, 1; 1, 1, 1; 1, 0, 1).
Prove that g 0 (a) is invertible, but g 0 (b) is singular and of rank 17.
Chapter II
Submanifolds of Rn
The Euclidean space Rn is a differentiable manifold. In this chapter we give a short introduction to submanifolds of Rn . Our emphasis is on characterizations that are suitable for
numerical computations. We further discuss the tangent space, differentiable mappings, and
differential equations on submanifolds.
II.1
Submanifolds of Rn are nonlinear analogues of linear subspaces. They extend the notion of
curves and surfaces. In the following a diffeomorphism : U V between open sets is a
continuously differentiable mapping having a continuously differentiable inverse.
Definition 1.1 (submanifold). A set M Rn is a submanifold of Rn if for every a M
there exist open sets U, V Rn with a U and a diffeomorphism : U V such that
(U M) = (U ) (Rk {0}).
The number k is called dimension of M and n k is its codimension. A pair (U, ) is called
chart on M, and the union of all charts is called (maximal) atlas.
x2
x1
Fig. II.1: Definition of a submanifold of Rn .
Figure II.1 illustrates the circle {(x1 , x2 ) ; x21 + x22 = 1} as a submanifold of R2 . A
possible choice for the diffeomorphism (x1 , x2 ) = (, r) is the mapping defined by polar
coordinates x1 = (1 + r) cos , x2 = (1 + r) sin .
Submanifolds of dimension k = 0 are discrete points in Rn . Submanifolds of maximal
dimension k = n are open sets in Rn . Every linear or affine subspace of Rn is a submanifold.
However, the set {(x, y) ; xy = 0} is not a submanifold of R2 because, close to the origin, it
is not diffeomorph to a straight line.
Submanifolds of Rn
T
where yb denotes the vector consisting of the first k components of y. The Jacobian matrix
0 (0) is invertible, so that is a local diffeomorphism close to (0) = a, i.e., there exist
open neighborhoods U1 U of a and V W Rnk of 0, such that : V U1 is a
diffeomorphism. We now put = 1 : U1 V .
The property (U1 M) (U1 ) (Rk {0}) follows immediately from the fact that,
for y (U1 ) with yk+1 = . . . = yn = 0, we have (y) = (yb) U1 (W ) = U1 M. To
prove the inverse inclusion, we take y (U1 M) = (U1 (W )) so that y = ((z)) for
some z W and hence also (y) = (z). If U1 is chosen as a sufficiently small neighborhood
of a, the vectors z and yb = (y1 , . . . , yk )T are both close to 0 (this follows from the fact that
: W U M is a homeomorphism). If we denote by b the first k components of the
function , it follows from (y) = (z) that b(yb) = b(z). However, since b0 (0) is nonsingular,
b is a local diffeomorphism close to 0, and we obtain yb = z. The relation (y) = (z) = (yb)
thus implies yk+1 = . . . = yn = 0, which completes the proof.
Submanifolds of Rn
(d + cos ) cos
(, ) = (d + cos ) sin
sin
of a torus. One can check that 0 (, ) is of maximal rank 2
and that is locally a homeomorphism.
Example 1.5 (Mobius strip). Consider a segment of lenght
2 (parametrized by 1 < t < 1), rotate it around its centre
and, at the same time, move this centre twice as fast along
a circle of radius d. This gives the parametrization
(d + t cos ) cos 2
II.2
Tangent space
x2
.5
Ta M = {tv | t R}.
In the original variables, the tangent is the affine space a + Ta M Rn .
a
.5
x1
Submanifolds of Rn
10
Surfaces in R3 . As an example, consider the ellipsoid
M = (x, y, z)
x
y
z
+ 2 + 2 =1
2
a
b
c
a cos sin
To determine the tangent plane at a = (x0 , y0 , z0 ) = (0 , 0 ) M, we consider the parametric curves (t) = (t, 0 ) and (t) = (0 , t); see Figure II.3. The left picture shows also
0 ).
the tangents (in grey) (t) = a + tv1 with v1 = (
0 ) and (t) = a + tv2 with v2 = (
The vectors v1 and v2 span the tangent space. It is given by a + Ta M, where
Ta M = {t1 v1 + t2 v2 | t1 , t2 R}
with
v1 =
(0 , 0 ),
v2 =
(0 , 0 ).
The tangent of other curves lying in M and passing through a is also in a + Ta M (see the
right picture of Figure II.3).
v1
v2
Fig. II.3: Illustration of the definition of the tangent space.
Definition 2.1 (tangent space). Let M Rn be a submanifold of Rn and let a M. The
tangent space of M at a is the linear space given by
Ta M =
n
R
=v
This definition gives a nice geometric interpretation of the tangent space. Algebraic
characterizations with explicit formulas are given in the following theorem.
Theorem 2.2. Consider a submanifold M Rn of dimension k and let a M.
If, close to a, M is given by a local parametrization : W Rn , i.e., we have
U M = {(z) | z W }, where (z0 ) = a with z0 W Rk , then
Ta M = Im 0 (z0 ) = { 0 (z0 ) t | t Rk }.
(2.1)
(2.2)
= 0 (z0 ) (0)
= 0 (z0 ) t. This implies Im 0 (z0 ) Ta M.
If (t) is a curve lying in M and satisfying (0) = a and (0)
= v, then we have
0
0
g((t)) = 0 and hence also g (a)(0)
Submanifolds of Rn
11
= v,
so that v Ta M. The curve (t) := f ((t)) then satisfies (t) N , (0) = f (a), and it is
continuously differentiable, because it can be written as = (f 1 ) ( ) = f ( ).
by
(Ta f )(v) = w,
Ta M we have w = (0)
Tf (a) N with (t) = f ((t)).
The linear mapping Ta f is called tangent map (or derivative) of f at a.
It is straight-forward to define mappings f : M N of class C k (k times continuously
differentiable mappings). In this case one has to require that the diffeomorphisms of the
charts (U, ) of the manifold are also mappings of class C k .
II.3
satisfies y(t)
Ty(t) M for all t. This motivates the
following definition.
Submanifolds of Rn
12
= 0 (z(t))z(t),
z(t1 ) = z1
with z1 given by y1 = (z1 ). Premultiplication with 0 (z)T yields the following differential
equation for z:
1
z = fe(z),
fe(z) = 0 (z)T 0 (z) 0 (z)T f ((z)).
(3.2)
The matrix 0 (z)T 0 (z) is invertible in z1 (and hence also in a neighborhood), because 0 (z1 )
is known to be of maximal rank. For a sufficiently smooth manifold M the function fe is
of class C 1 . Since (3.2) is a differential equation in an Euclidean space, we can apply the
classical theory which yields the local existence and uniqueness of a solution z(t). Because of
f ((z)) T(z) M = Im 0 (z), the function y(t) = (z(t)) is seen to be a solution of y = f (y).
Global uniqueness. Let I, J be open intervals, and let y : I M and yb : J M be two
solutions of y = f (y) satisfying y(0) = yb(0) = y0 . To prove that both functions coincide on
the interval I J, we consider the set
K = {t I J ; y(t) = yb(t)}.
This set in nonempty (0 K) and closed in I J (y(t) and yb(t) are continuous). Since
I J is a connected set (an interval), it is sufficient to prove that K is also open. In fact, for
t1 K, we can choose a chart of M containing y(t1 ) = yb(t1 ). The above local existence and
uniqueness result shows that we have y(t) = yb(t) for t in a neighborhood of t1 . This proves
that K is open and, consequently, K = I J.
Submanifolds of Rn
13
Maximality of the interval I = I(y0 ). We consider all open intervals J such that the
problem y = f (y), y(0) = y0 admits a solution on J. We then let I(y0 ) be the union of all
these intervals. For t I(y0 ) there exists J with t J, and we can define y(t) as the value
of the function y : J M. By the uniqueness result, this is well defined and provides a
solution on the maximal interval I(y0 ).
The solution of a differential equation depends on the initial data. We adopt the notation
t (y0 ) = y(t) for the solution of y = f (y) at time t corresponding to the initial condition
y(0) = y0 . It is called the flow (exact flow in contrast to a discrete flow) of the differential
equation. We also consider
D = {(t, y0 ) ; y0 M, t I(y0 )}
and
: D M,
(t, y0 ) := t (y0 ).
(3.3)
(3.4)
By the local argument above each of the mappings ti+1 ti (for i {0, . . . , N 2}) is of class
C 1 in a neighborhood of ti (y0 ), and the mapping (t, y) 7 ttN 1 (y) is defined and of class
C 1 for (t, y) in a neighborhood of (tN , tN 1 (y0 )). This proves that D is open and that the
composition (3.4) is of class C 1 .
The existence of such a partitioning follows from a compactness argument. For a fixed
[0, tb ] there exists an open interval I (with I ) and a chart (U , ), such that
U1
U0
y(t0 )
y(t1 )
U2
y(t2 )
y(t3 )
U3
y(t4 )
Submanifolds of Rn
14
y(s) U for all s I . The family {I } [0,t] is an open covering of the compact interval
[0, tb ]. By the HeineBorel theorem we know that already finitely many intervals I cover
the whole interval. This completes the proof of the theorem.
The following result on the propagation of perturbations in initial values will be an
essential ingredient of the convergence analysis of numerical integrators on submanifolds.
Corollary 3.4 (propagation of perturbations). Consider a differential equation y = f (y)
on a submanifold M Rn with a C 1 vector field f : M Rn . Suppose that the solution
b Then there exist > 0 and a constant C, such that
y(t) = t (y0 ) exists for 0 t t.
kt (y1 ) t (y2 )k C ky1 y2 k
0 t tb
for
for all y1 , y2 K (), where the compact neighborhood of the solution is given by
n
K () = y M ; ky (y0 )k .
(3.5)
Proof. As in the proof of Theorem 3.3 we cover the solution t (y0 ) for 0 t tb by
finitely many charts (Ui , i ). Since the sets Ui are open, there exists 0 > 0, such that
K (0 ) Ui for all [ti , ti+1 ] and all i {0, . . . , N 1}. By the smoothness of the flow
mapping (Theorem 3.3) and a compactness argument there exists 0 < 0 such that for
b the solution t (y) remains in Kt (0 ).
[ti , ti+1 ], for y K (), and for t t,
We now consider [ti , ti+1 ] and y1 , y2 K () and we let local coordinates z1 , z2 be
given by i (y1 ) = (z1 , 0) and i (y2 ) = (z2 , 0). The mean value theorem, applied to the C 1
mapping (z) = (t 1
i )(z, 0), yields the existence of a constant Ci such that
1
kt (y1 ) t (y2 )k = k(t 1
i )(z1 , 0) (t i )(z2 , 0)k Ci kz1 z2 k
for all y1 , y2 K (). A compactness argument implies that the constant Ci can be chosen
independent of [ti , ti+1 ] and of t [, tb ]. A further application of the mean value theorem
yields
kz1 z2 k = ki (y1 ) i (y2 )k Di ky1 y2 k,
which proves the statement of the corollary with C = maxi=0,...,N 1 Ci Di .
II.4
A Lie group is a group G which is a differentiable manifold, and for which the product is
a differentiable mapping G G G. We restrict our considerations to matrix Lie groups,
that is, Lie groups which are subgroups of GL(n), the group of invertible n n matrices
with the usual matrix product as the group operation.1
An important example is the set
n
O(n) = X GL(n) ; X T X = I
of all orthogonal matrices, which is a submanifold of dimension n(n1)/2 (see Example 1.6).
With the usual product of matrices the set O(n) is a group with unit element I (the identity).
Since the matrix multiplication is a differentiable mapping, O(n) is a Lie group.
1
Section II.4 is nearly identical to Section IV.6 of the monograph Geometric Numerical Integration by
Hairer, Lubich, and Wanner. For further reading on Lie groups we refer to the monographs Applications of Lie
Groups to Differential Equations by Olver (1986) and to Lie Groups, Lie Algebras and Their Representations
by Varadarajan (1974).
Submanifolds of Rn
15
Tab. II.1: Some matrix Lie groups and their corresponding Lie algebras.
Lie group
Lie algebra
GL(n) = {X ; det X 6= 0}
general linear group
SL(n) = {X ; det X = 1}
special linear group
sl(n) = {Z ; trace(Z) = 0}
special linear Lie algebra
O(n) = {X ; X T X = I}
orthogonal group
so(n) = {Z ; Z T + Z = 0}
skew-symmetric matrices
so(n) = {Z ; Z T + Z = 0}
skew-symmetric matrices
Sp(n) = {X ; X T JX = J}
symplectic group
sp(n) = {Z ; JZ + Z T J = 0}
Table II.1 lists further prominent examples. The symplectic group is only defined for
even n, and the matrix J given by
!
0 I
J=
I 0
determines the symplectic structure on R2n .
As the following lemma shows, the tangent space g = TI G at the identity I of a matrix
Lie group G is closed under forming commutators of its elements. This makes g an algebra,
the Lie algebra of the Lie group G.
Lemma 4.1 (Lie Bracket and Lie Algebra). Let G be a matrix Lie group and let g = TI G
be the tangent space at the identity. The Lie bracket (or commutator)
[A, B] = AB BA
(4.1)
(4.2)
Proof. By definition of the tangent space, for A, B g, there exist differentiable paths
(t), (t) (|t| < ) in G such that (t) = I + tA + o(t) and (t) = I + tB + o(t). Consider
now the path (t) in G defined by
for
t 0.
(t) = ( t)( t)( t)1 ( t)1
An elementary computation then yields
(t) = I + t[A, B] + o(t).
With the extension (t) = (t)1 for negative t, this is a differentiable path in G satisfying
(0) = I and (0)
Submanifolds of Rn
16
Example 4.2. Consider again the orthogonal group O(n), see Example 1.6. Since the
derivative of g(X) = X T X I at the identity is g 0 (I)H = I T H + H T I = H + H T , it follows
from the second part of Theorem 2.2 that the Lie algebra corresponding to O(n) consists of
all skew-symmetric matrices. The right column of Table II.1 gives the Lie algebras of the
other Lie groups listed there.
The following basic lemma shows that the exponential map
exp(A) =
X
k0
1 k
A
k!
yields a local parametrization of the Lie group near the identity, with the Lie algebra (a
linear space) as the parameter space. We recall that the mapping Y (t) = exp(tA)Y0 is the
solution of the matrix differential equation Y = AY , Y (0) = Y0 .
Lemma 4.3 (Exponential map). Consider a matrix Lie group G and its Lie algebra g. The
matrix exponential maps the Lie algebra into the Lie group,
exp : g G,
i.e., for A g we have exp(A) G. Moreover, exp is a local diffeomorphism in a neighbourhood of A = 0.
Proof. For A g, it follows from the definition of the tangent space g = TI G that there
exists a differentiable path (t) in G satisfying (0) = I and (0)
= A. For a fixed Y G,
the path (t) := (t)Y is in G and satisfies (0) = Y and (0)
= AY . Consequently,
II.5
Exercises
1. Consider the 2-dimensional torus of Example 1.4. Find a function g : R3 R such that the
manifold is given by M = {x ; g(x) = 0}. Prove that g 0 (x) 6= 0 for all x M.
2
Result. g(x) = x21 + x22 + x23 + d2 2 4 d2 (x21 + x22 ).
Submanifolds of Rn
17
{(t, t2 ) R2 ; t 0}
{(t2 , t3 ) R2 ; t 6= 0}
{(x, y, z) R3 ; x = y = z = 0}
{(x, y, z) R3 ; x2 + y 2 z 2 = 0}
(5.1)
is a submanifold
of R3 for = 2/13 (see the figure).
T
1
x1 , . . . , x n .
1 xn+1
a) For any x Sn \{N }, prove that (x) is the point where the line trough N and x intersects
the hyperplane xn+1 = 0 (which is identified with Rn ).
z = (x)
Rn
Submanifolds of Rn
18
b) Prove that is bijective, and that its inverse = 1 is given by
(z1 , . . . , zn ) =
1
kzk2 + 1
T
c) For any z Rn , prove that the matrix 0 (z) is of full rank n. Determine for which z Rn
the first n lines of 0 (z) are not linearly independent.
d) For a fixed x Sn \ {N } with xn+1 6= 0, find a chart (U, ) with x U by following the
proof of Lemma 1.3.
8. Let M, N , P be submanifolds, and let g : M N , f : N P be C 1 -mappings. Prove that
the composition f g is a C 1 -mapping, and that its tangent map satisfies
Ta (f g) = Tg(a) f Ta g .
9. Consider a compact submanifold M (e.g., the sphere or the torus) and a C 1 vector field
f (y) on M. Prove that for every y0 M the solution y(t) of the initial value problem
y = f (y), y(0) = y0 exists for all t (, +).
10. Prove that SL(n) is a Lie group of dimension n2 1, and that sl(n) is its Lie algebra (see
Table II.1 for the definitions of SL(n) and sl(n)).
11. Let G be a matrix Lie group and g its Lie algebra. Prove that for X G and A g we have
XAX 1 g.
Hint. Consider the path (t) = X(t)X 1 .
Chapter III
Integrators on Manifolds
We consider ordinary differential equations
y = f (y),
y(0) = y0
(0.1)
on a submanifold M, i.e., we assume that f (y) Ty M for all y M. This chapter is devoted
to the numerical solution of such problems. We discuss projection methods, integrators
based on local coordinates, and Magnus series methods for linear differential equations on
Lie groups. We also show how the global error can be estimated (global convergence).
III.1
Projection methods
We start by assuming that the vector field f (y) is well defined in an open neighborhood of
the manifold M. In principle it is then possible to apply any numerical integrator (Runge
Kutta, multistep, etc.) to the differential equation (0.1) without taking care of the manifold.
However, as we have seen in Chapter I (for example in Figure I.2), the numerical solution will
usually drift away from the manifold and often looses a physical interpretation. A natural
approach for avoiding such unphysical approximations is by projection1 .
Algorithm 1.1 (Standard projection method). Assume that yn M. One step yn 7 yn+1
is defined as follows (see Fig. III.1):
Compute yen+1 = h (yn ), where h is an arbitrary one-step method applied to y = f (y);
project the value yen+1 onto the manifold M to obtain yn+1 M.
y1
y0
ye1
y2
y3
For more details consult the following monographs: Sections IV.4 and V.4.1 of Geometric Numerical
Integration by Hairer, Lubich and Wanner (2006), Section VII.2 of Solving Ordinary Differential Equations II
by Hairer and Wanner (1996), and Section 5.3.3 of Numerical Methods in Multibody Dynamics by EichSoellner and F
uhrer (1998).
20
Integrators on Manifolds
In some situations the projection step is straight-forward. If M is the unit sphere (e.g., for
Eulers equation of motion for a rigid body, Section I.1), we simply divide the approximation
yen+1 by its Euclidean norm to get a vector of length one.
If the manifold is given by a local parametrization y = (z), we compute zn+1 by minimizing k(zn+1 ) yen+1 k in a suitable norm, and then we put yn+1 = (zn+1 ). But this situation
is not important in practice, because we can treat directly the differential equation (II.3.2)
for z, if explicit formulas for the parametrization are known. This yields approximations zn
and yn := (zn ), which lie on the manifold by definition.
Projection step, if the manifold is given as a level set. For all examples of Chapter I
the manifold M is given as the level set of a smooth function g(y) = (g1 (y), . . . , gm (y))T .
This is by far the most important situation. For the computation of yn+1 (projection step)
we have to solve the constrained minimization problem
kyn+1 yen+1 k min
subject to
g(yn+1 ) = 0.
(1.1)
In the case of the Euclidean norm, a standard approach is to introduce Lagrange multipliers
= (1 , . . . , m )T , and to consider the Lagrange function
L(yn+1 , ) =
1
2
(1.2)
0 = g(yn+1 ).
We have replaced yn+1 with yen+1 in the argument of g 0 (y) in order to save some evaluations
of g 0 (y). Inserting the first relation of (1.2) into the second gives a nonlinear equation for ,
which can be efficiently solved by simplified Newton iterations:
i = g 0 (yen+1 )g 0 (yen+1 )T
1
g yen+1 + g 0 (yen+1 )T i ,
i+1 = i + i .
For the choice 0 = 0 the first increment 0 is of size O(hp+1 ), so that the convergence
is usually extremely fast. Often, one simplified Newton iteration is sufficient to achieve the
desired precision.
Internal projection. We assume here that the vector field f (y) is only defined on the manifold M, and not on a whole neighborhood. It may also happen that the differential equation
has a different (stability) behavior outside the manifold. In this case we are interested in
numerical methods that evaluate the vector field only on the manifold.
The idea is the following. We denote by (y) a smooth projection of a vector y onto the
manifold. Since (y) = y for y M, the solution of the differential equation
y = f ((y)),
y(0) = y0 M
(1.3)
is identical to that of (0.1). We then apply our integrator to (1.3) instead of (0.1). For a
RungeKutta method, e.g.,
k1 = f ((yn ))
k2 = f ((yn + a21 hk1 ))
yn+1 = yn + h(b1 k1 + b2 k2 ),
this means that we do not only project yn+1 onto the manifold, but also the vector yn +a21 hk1
before computing k2 .
Integrators on Manifolds
21
(1.4)
where the scalar is given by = det Yen+1 . This leads to the scalar nonlinear equation
T
det(Yen+1 + Yen+1
) = det Yn , for which simplified Newton iterations become
T
det Yen+1 + i Yen+1
T e
1 + (i+1 i ) trace (Yen+1
Yn+1 )1
= det Yn .
If the QR-decomposition of Yen+1 is available from the computation of det Yen+1 , the value of
T e
Yn+1 )1 ) can be computed efficiently with O(n3 /3) flops.
trace((Yen+1
The above projection is preferable to Yn+1 = c Yen+1 , where c R is chosen such that
det Yn+1 = det Yn . This latter projection is already ill-conditioned for diagonal matrices with
entries that differ by several magnitudes.
Example 1.3 (orthogonal matrices). As a second example let us consider Y = F (Y ), where
the solution Y (t) is known to be an orthogonal matrix or, more generally, an n k matrix
satisfying Y T Y = I (Stiefel manifold). The projection step (1.1) requires the solution of the
problem
kY Ye kF min
subject to
Y T Y = I,
(1.5)
where Ye is a given matrix. This projection can be computed as follows: compute the
singular value decomposition Ye = U T V , where U T and V are n k and k k matrices
with orthonormal columns, = diag(1 , . . . , k ), and the singular values 1 . . . k are
all close to 1. Then the solution of (1.5) is given by the product Y = U T V (see Exercise 3
for some hints).
This procedure has a different interpretation: the orthogonal projection is the first factor
of the polar decomposition Ye = Y R (where Y has orthonormal columns and R is symmetric
positive definite). The equivalence is seen from the polar decomposition Ye = (U T V )(V T V ).
III.2
(2.1)
where A+ = (AT A)1 AT denotes the pseudo-inverse of a matrix with full column rank. The
solutions of (0.1) and (2.1) are related via y(t) = (z(t)), so that any approximation zn of
z(tn ) also provides an approximation yn = (zn ) y(tn ). The idea is to apply the numerical
integrator in the parameter space rather than in the space where M is embedded. In contrast
to projection methods (Section III.1), the numerical integrators of this section evaluate f (y)
only on the manifold M.
22
Integrators on Manifolds
Algorithm 2.1 (Local Coordinates Approach). Assume that yn M and that y = (z) is
a local parametrization of M. One step yn 7 yn+1 is defined as follows (see Fig. III.2):
determine zn in the parameter space, such that (zn ) = yn ;
compute zen+1 = h (zn ), the result of the numerical method h applied to (2.1);
define the numerical solution by yn+1 = ( zen+1 ).
It is important to remark that the parametrization y = (z) can be changed at every step.
ze1
z0
z1
y1
ze2
y2
y4
y3
y0
Fig. III.2: The numerical solution of differential equations on manifolds via local coordinates.
There are many possible choices
of local coordinates. For the mathematical
n
o pendulum
of Example I.3.1, where M = (q1 , q2 , v1 , v2 ) | q12 + q22 = 1, q1 v1 + q2 v2 = 0 , a standard
parametrization is q1 = sin , q2 = cos , v1 = cos , and v2 = sin . In the new
coordinates (, ) the problem becomes simply = , = sin . Another typical choice
is the exponential map (Z) = exp(Z) for differential equations on Lie groups. In this
section we are mainly interested in the situation where the manifold is given as the level set
of a smooth function g(y), and we discuss two commonly used choices which do not use any
special structure of the manifold.
Generalized Coordinate Partitioning. We assume that the manifold is given by M =
{y Rn ; g(y) = 0}, where g : Rn Rm has a Jacobian with full rank m < n at y = a. We
can then find a partitioning y = (y1 , y2 ), such that g/y2 (a) is invertible. In this case we
can choose the components of y1 as local coordinates. The function y = (z) is then given by
y1 = z and y2 = 2 (z), where 2 (z) is implicitly defined by g(z, 2 (z)) = 0, and (2.1) reduces
to z = f1 ((z)), where f1 (y) denotes the first n m components of f (y). This approach has
been promoted by Wehage and Haug2 in the context of constrained mechanical systems, and
the partitioning is found by Gaussian elimination with full pivoting applied to the matrix
g 0 (a). Another way of finding the partitioning is by the use of the QR decomposition with
column change.
Tangent Space Parametrization. Let the manifold M be
given as the level set of a smooth function g : Rn Rm . We
compute an orthonormal basis of the tangent space Ta M =
ker g 0 (a) at a = yn , and we collect the basis vectors as columns
of the matrix Q, which is of dimension n (n m). This
matrix satisfies QT Q = I and g 0 (a)Q = 0. We then consider
the parametrization
a (z) = a + Qz + g 0 (a)T u(z),
Qz
g 0 (a)T u
a (z)
(2.2)
where u(z) is defined by g(a (z)) = 0. The existence and local uniqueness of u(z) with
u(0) = 0 follows for small z from the implicit function theorem. In fact, the function
F (z, u) := g(a + Qz + g 0 (a)T u) satisfies F (0, 0) = 0 and its derivative with respect to u is
2
Generalized coordinate partitioning for dimension reduction in analysis of constrained dynamic systems,
Mechanical Design 104 (1982) 247255.
Integrators on Manifolds
23
for (z, u) = (0, 0) the matrix g 0 (a)g 0 (a)T , which is invertible because g 0 (a) is assumed to be
of full rank. Differentiating y(t) = a (z(t)) with respect to time yields
(2.3)
which corresponds to (2.1). If we apply a numerical method to (2.3), every function evaluation requires the projection of an element of the tangent space onto the manifold. This
procedure is illustrated in Fig. III.2, and was originally proposed by Potra and Rheinboldt3
for the solution of the EulerLagrange equations of constrained multibody systems.
III.3
The exponential function exp plays an important role as local parametrization of Lie groups
(Section II.4). In view of the differential equation (2.1) we need the derivative of exp and its
inverse. Elegant formulas are obtained by the use of matrix commutators [, A] = A A.
If we suppose fixed, this expression defines a linear operator A 7 [, A]
ad (A) = [, A],
(3.1)
which is called the adjoint operator. Let us start by computing the derivatives of k . The
product rule for differentiation yields
d k
H = Hk1 + Hk2 + . . . + k1 H,
d
(3.2)
and this equals kHk1 if and H commute. Therefore, it is natural to write (3.2) as
kHk1 to which are added correction terms involving commutators and iterated commutators. In the cases k = 2 and k = 3 we have
H + H = 2H + ad (H)
k1
X
d k
k
H=
i+1
d
i=0
!
ad i (H) ki1 .
(3.3)
This is seen by applying Leibniz rule to k+1 = k and by using the identity (ad i (H)) =
(ad i (H)) + ad i+1
(H).
Lemma 3.1. The derivative of exp =
where
1
k0 k!
k is given by
d
exp H = d exp (H) exp ,
d
X
1
d exp (H) =
ad k (H).
k0 (k + 1)!
(3.4)
On the numerical solution of EulerLagrange equations, Mech. Struct. & Mech. 19 (1991) 118; see also
page 476 of the monograph Solving Ordinary Differential Equations II by Hairer and Wanner (1996).
24
Integrators on Manifolds
Proof. Multiplying (3.3) by (k!)1 and summing, then exchanging the sums and putting
j = k i 1 yields
!
X 1 k1
X
X X
d
1
k
exp H =
ad i (H) ki1 =
ad i (H) j .
i+1
d
i0 j0 (i + 1)! j!
k0 k! i=0
The convergence of the series follows from the boundedness of the linear operator ad (we
have kad k 2kk).
Lemma 3.2. If the eigenvalues of the linear operator ad are different from 2`i with
` {1, 2, . . .}, then d exp is invertible. Furthermore, we have for kk < that
X Bk
(3.5)
ad k (H),
d exp1
(H)
=
k!
k0
where Bk are the Bernoulli numbers, defined by
k0 (Bk /k!)x
k
= x/(ex 1).
III.4
Our next aim is the numerical solution of differential equations (II.4.4) on Lie groups. For
this purpose we consider linear matrix differential equations of the form
Y = A(t) Y.
(4.1)
No assumption on the matrix A(t) is made for the moment (apart from continuous dependence on t). For the scalar case, the solution of (4.1) with Y (0) = Y0 is given by
Z t
Y (t) = exp
0
A( ) d Y0 .
(4.2)
Also in the case where the matrices A(t) and 0t A( ) d commute, (4.2) is the solution of
(4.1). In the general non-commutative case we search for a matrix function (t) such that
R
Y (t) = exp((t)) Y0
solves (4.1). The main ingredient for the solution will be the inverse of the derivative of the
matrix exponential. It has been studied in Section III.3.
Theorem 4.1 (Magnus 1954). The solution of the differential equation (4.1) can be written
as Y (t) = exp((t))Y0 with (t) given by
= d exp1
(0) = 0.
(4.3)
(A(t)),
As long as k(t)k < , the convergence of the series expansion (3.5) of d exp1
is assured.
Proof. Comparing the derivative of Y (t) = exp((t))Y0 ,
d
Y0 = d exp ((t))
Y (t) =
exp((t)) (t)
exp((t)) Y0 ,
(t)
d
Integrators on Manifolds
25
12
which is nonlinear in . Applying Picard fixed point iteration after integration yields
(t) =
Z t
0
1Z t
A() d, A( ) d
A( ) d
2 0 0
#
"
Z
1Z t Z
+
A() d, A() d, A( ) d
4 0 0 0
Z
(4.4)
"
#
Z
1 Zt Z
+
A() d,
A() d, A( ) d + . . . ,
12 0 0
0
which is the so-called Magnus expansion. For smooth matrices A(t) the remainder in (4.4) is
of size O(t5 ) so that the truncated series inserted into Y (t) = exp((t))Y0 gives an excellent
approximation to the solution of (4.1) for small t.
Numerical Methods Based on the Magnus Expansion. The matrix can be considered as local coordinates for Y = exp()Yn . The differential equation (4.3) corresponds to
equation (2.1) in the general situation. Following the steps in Algorithm 2.1 we let n = 0,
we compute an approximation n+1 of (h) given by (4.4) with A(tn + ) instead of A( ),
and we finally put Yn+1 = exp(n+1 ) Yn . For n+1 it is natural to take a suitable truncation of the Magnus expansion with the integrals approximated by numerical quadrature.4 A
related approach is to replace A(t) locally by an interpolation polynomial
b
A(t)
=
s
X
i=1
b
and to solve Y = A(t)
Y on [tn , tn + h] by the use of the truncated series (4.4).
Theorem 4.2. Consider a quadrature formula (bi , ci )si=1 of order p s, and let Y (t) and
b
Z(t) be solutions of Y = A(t) Y and Z = A(t)
Z, respectively, satisfying Y (tn ) = Z(tn ).
p+1
Then, Z(tn + h) Y (tn + h) = O(h ).
b
Proof. We write the differential equation for Z as Z = A(t)Z + (A(t)
A(t))Z and use the
variation of constants formula to get
Z(tn + h) Y (tn + h) =
Z tn +h
tn
b ) A( ) Z( ) d.
R(tn + h, ) A(
Applying our quadrature formula to this integral gives zero as result, and the remainder is
of size O(hp+1 ). Details of the proof are omitted.
4
Iserles and Nrsett, On the solution of linear differential equations in Lie groups (1999);
Zanna, Collocation and relaxed collocation for the Fer and the Magnus expansions (1999).
26
Integrators on Manifolds
Example 4.3. As a first example, we use the midpoint rule (c1 = 1/2, b1 = 1). In this case
the interpolation polynomial is constant, and the method becomes
(4.5)
which is of order 2.
Example 4.4. The two-stage Gauss quadrature is given by c1,2 = 1/2 3/6, b1,2 =
1/2. The interpolation polynomial is of degree one and we have to apply (4.4) to get an
approximation Yn+1 . Since we are interested in a fourth order approximation, we can neglect
the remainder term (indicated by . . . in (4.4)). Computing analytically the iterated integrals
over products of `i (t) we obtain
2
h
3h
(A1 + A2 ) +
[A2 , A1 ] Yn ,
(4.6)
Yn+1 = exp
2
12
where A1 = A(tn + c1 h) and A2 = A(tn + c2 h). This is a method of order four. The terms
of (4.4) with triple integrals give O(h4 ) expressions, whose leading term vanishes by the
symmetry of the method (Exercise 6). Therefore, they need not be considered.
Remark. All numerical methods of this section are of the form Yn+1 = exp(hn ) Yn , where
n is a linear combination of A(tn + ci h) and of their commutators. If A(t) g for all t,
then also hn lies in the Lie algebra g, so that the numerical solution stays in the Lie group
G if Y0 G (this is a consequence of Lemma II.4.3).
III.5
Typically, h (y) is defined implicitly by algebraic equations, and it is well defined only for sufficiently
small h h0 with h0 depending on y. It may happen that there is no uniform h0 > 0 such that h (y) exists
for all y M and for h h0 . By abuse of notation, we nevertheless write h : M M in this situation.
Integrators on Manifolds
27
Notice that the local error has to be estimated only for y0 in the submanifold M. This
is usually much easier than estimating a suitable extension on an open neighborhood of M.
However, this makes sense only if h : M M, which implies that the numerical solution
stays for ever on the submanifold.
Theorem 5.2 (convergence). Consider a sufficiently smooth differential equation y = f (y)
on a submanifold M Rm , and an initial value y0 M such that the solution y(t) = t (y0 )
b If the numerical integrator yn+1 = h (yn ) is of order p and yields
exists for 0 t t.
b then there exists h0 > 0 such that for
approximations satisfying yn M for nh t,
0 < h h0 the global error can be estimated as
kyn y(nh)k c hp
b
for nh t.
The constant c is independent on h, but depends on the length tb of the considered interval.
Proof. We consider the compact neighborhood
y(tn )
K = {y M ; [0, tb ] with ky y( )k }
y0
exact solutions
E1
e1
of the solution, where > 0 is given by CorolE2
e2 e
y
1
lary II.3.4. As long as yn K, it follows from
n1
E3
y2
Definition 5.1 that kyn+1 h (yn )k C0 hp+1 .
.
.
y3
.
Assume for the moment that yn Knh ()
En = en
b where
and h (yn1 ) Knh () for nh = tn t,
numerical method yn
K () = {y M ; ky y( )k }. Using
t0 t1 t2 t3
tn = t
. . .
ttn (yn ) = ttn+1 (h (yn )), Corollary II.3.4 then
yields
kttn+1 (yn+1 ) ttn (yn )k C kyn+1 h (yn )k C C0 hp+1
b Summing up, we thus obtain for nh = tn tb
for tn+1 t t.
kyn y(tn )k
n1
X
j=0
III.6
Exercises
1. For n-dimensional square matrices Y consider the function g(Y ) = det Y . Prove that
g 0 (Y )(HY ) = traceH det Y.
Hint. Expand det(Y + HY ) in powers of .
2. Elaborate Example 1.2 for the special case where Y is a matrix of dimension 2. In particular,
show that (1.4) is the same as (1.2), and check the formulas for the simplified Newton
iterations.
28
Integrators on Manifolds
3. Show that for given Ye the solution of the problem (1.5) is Y = U T V , where Ye = U T V is
the singular value decomposition of Ye .
Hint. Since kU T SV kF = kSkF holds for all orthogonal matrices U and V , it is sufficient to
consider the case Ye = (, 0)T with = diag(1 , . . . , k ). Prove that
k(, 0)T Y k2F
k
X
(i 1)2
i=1
exp() = I +
sin
+
1 sin(/2) 2
/2
for
0 3 2
0 1
= 3
2 1
0
Prove that the first terms of the t-expansion of (t) are given by
(t) = tA(Y0 ) +
t2 0
t3 0
A (Y0 )A(Y0 )Y0 +
A (Y0 )2 A(Y0 )Y02 + A0 (Y0 )A(Y0 )2 Y0
2
6
1
A(Y0 ), A0 (Y0 )A(Y0 )Y0 + . . .
2
6. For the numerical solution of Y = A(t) Y consider the method Yn 7 Yn+1 defined by Yn+1 =
Z(tn + h), where Z(t) is the solution of
b Z,
Z = A(t)
Z(tn ) = Yn ,
b
and A(t)
is the interpolation polynomial based on symmetric nodes c1 , . . . , cs , i.e., we have
cs+1i + ci = 1 for all i.
a) Prove that this method is symmetric.
b) Show that Yn+1 = exp((h)) Yn holds, where (h) has an expansion in odd powers of h.
This justifies the omission of the terms involving triple integrals in Example 4.4.
7. Consider the projection method of Algorithm 1.1, where h represents an explicit RungeKutta method of order p (e.g., the explicit Euler method) and the numerical approximation
is obtained by orthogonal projection onto the submanifold. Prove that, for sufficiently small
h, the projection method is of order p according to Definition 5.1.
Chapter IV
Differential-Algebraic Equations
The most general form of a differential-algebraic system is that of an implicit differential
equation
F (u,
u) = 0
(0.1)
where F and u have the same dimension. We always assume F to be sufficiently differentiable.
A non-autonomous system is brought to the form (0.1) by appending t to the vector u, and
by adding the equation t = 1. If F/ u is invertible we can locally solve (0.1) for u to obtain
an ordinary differential equation. In this chapter we are interested in problems (0.1) where
F/ u is singular.1
IV.1
The simplest and best understood problems of the form (0.1) are linear differential equations
with constant coefficients
B u + Au = d(t).
(1.1)
In looking for solutions of the form u(t) = et u0 (if d(t) 0) we are led to consider the
matrix pencil A + B. When A + B is singular for all values of , then (1.1) has either no
solution or infinitely many solutions for a given initial value (Exercise 1). We shall therefore
deal only with regular matrix pencils, i.e., with problems where the polynomial det(A + B)
does not vanish identically. The key to the solution of (1.1) is the following simultaneous
transformation of A and B to canonical form.
Theorem 1.1 (Weierstrass 1868, Kronecker 1890). Let A + B be a regular matrix pencil.
Then there exist nonsingular matrices P and Q such that
!
C 0
,
P AQ =
0 I
I 0
P BQ =
0 N
(1.2)
1
0
.. ..
.
.
Ni =
0 1
0
0
of dimension mi ,
(1.3)
The text of this chapter is taken from Section VII.1 of the monograph Solving Ordinary Differential
Equations II by Hairer and Wanner (1996).
30
Differential-Algebraic Equations
Proof. (Gantmacher 1954 (Chap. XII), see also Exercises 3 and 4). We fix some c such that
A + cB is invertible. If we multiply
A + B = A + cB + ( c)B
by the inverse of A + cB and then transform (A + cB)1 B to Jordan canonical form we
obtain
I 0
J 0
+ ( c) 1
.
(1.4)
0 I
0 J2
Here, J1 contains the Jordan blocks with non-zero eigenvalues, J2 those with zero eigenvalues
(the dimension of J1 is just the degree of the polynomial det(A+B)). Consequently, J1 and
I cJ2 are both invertible and multiplying (1.4) from the left by blockdiag (J11 , (I cJ2 )1 )
gives
!
!
J11 (I cJ1 ) 0
I
0
+
.
0
I
0 (I cJ2 )1 J2
The matrices J11 (I cJ1 ) and (I cJ2 )1 J2 can then be brought to Jordan canonical form.
Since all eigenvalues of (I cJ2 )1 J2 are zero, we obtain the desired decomposition (1.2).
Theorem 1.1 allows us to solve the differential-algebraic system (1.1) as follows: we
premultiply (1.1) by P and use the transformation
y
,
z
u=Q
P d(t) =
(t)
.
(t)
N z + z = (t).
(1.5)
The equation for y is just an ordinary differential equation. The relation for z decouples
again into k subsystems, each of the form (with m = mi )
z2 + z1 = 1 (t)
..
.
zm + zm1 = m1 (t)
zm = m (t).
(1.6)
Here zm is determined by the last equation, and the other components are computed recursively by repeated differentiation. Exactly m 1 differentiations are necessary to obtain
(m1)
z1 (t) = 1 (t) 2 (t) + 3 (t) . . . + (1)m1 m
(t).
(1.7)
The integer (max mi ) is called the index of nilpotency of the matrix pencil A + B. It does
not depend on the particular transformation used to get (1.2) (see Exercise 5).
IV.2
Differentiation index
The previous example shows that certain equations of the differential-algebraic system (1.6)
have to be differentiated m 1 times to get an explicit expression of all solution components.
One more differentiation gives ordinary differential equations for all components. This motivates the following index definition for general nonlinear problems (Gear and Petzold 1983,
1984; Gear, Gupta, and Leimkuhler 1985, Gear 1990, Campbell and Gear 1995).
Differential-Algebraic Equations
31
Definition 2.1. Equation (0.1) has differentiation index m, if m is the minimal number of
analytical differentiations
dF (u,
u)
=0,
F (u,
u) = 0,
dt
dm F (u,
u)
... ,
=0
dtm
(2.1)
such that equations (2.1) allow us to extract by algebraic manipulations an explicit ordinary
differential system u = a(u) (which is called the underlying ODE).
Note that for linear equations with constant coefficients the differentiation index and
the index of nilpotency are the same. Let us discuss the (differentiation) index for some
important special cases.
Systems of index 1. Differential-algebraic systems of the form
y = f (y, z)
0 = g(y, z)
(2.2)
have no occurrence of z.
We therefore differentiate the second equation of (2.2) to obtain
z = gz1 (y, z)gy (y, z)f (y, z)
which is possible if gz is invertible in a neighbourhood of the solution. The problem (2.2),
for invertible gz , is thus of differentiation index 1.
In practice, it is not necessary to know the differential equation for z. If initial values
satisfy g(y0 , z0 ) = 0 (we call them consistent) and if the matrix gz (y0 , z0 ) is invertible, then
the implicit function theorem guarantees the existence of a unique function z = (y) (defined
close to (y0 , z0 )) such that g(y, (y)) = 0. The problem then reduces locally to the ordinary
differential equation y = f (y, (y)), which can be solved by any numerical integrator.
Systems of index 2. In the system
y = f (y, z)
0 = g(y)
(2.3)
where the variable z is absent in the algebraic constraint, we obtain by differentiation of the
second relation of (2.3) the hidden constraint
0 = gy (y)f (y, z).
(2.4)
If gy (y)fz (y, z) is invertible in a neighbourhood of the solution, then the first equation of
(2.3) together with (2.4) constitute an index 1 problem. Differentiation of (2.4) yields the
missing differential equation for z, so that the problem (2.3) is of differentiation index 2.
If the initial values satisfy 0 = g(y0 ) and 0 = gy (y0 )f (y0 , z0 ), we call them consistent. If
in addition the matrix gy (y0 )fz (y0 , z0 ) is invertible, the implicit function theorem implies the
local existence of a function z = (y) satisfying gy (y)f (y, (y)) = 0 in a neighborhood of y0 .
We thus obtain the differential equation
y = f (y, (y))
on the manifold
M = {y ; g(y) = 0}.
The property f (y, (y)) Ty M follows from gy (y)f (y, (y)) = 0. All numerical approaches
of Chapter III can be applied to solve such problems.
32
Differential-Algebraic Equations
System (2.3) is a representative of the larger class of problems of type (2.2) with singular
gz . If we assume that gz has constant rank in a neighbourhood of the solution, we can
eliminate certain algebraic variables from 0 = g(y, z) until the system is of the form (2.3).
This can be done as follows: if there exists a pair (i, j) such that gi /zj 6= 0 at the initial
value then, by the implicit function theorem, the relation gi (y, z) = 0 permits us to express
zj in terms of y and the other components of z. We can thus eliminate the variable zj
from the system. Repeating this procedure we arrive at the situation, where gz vanishes at
the initial value. From the constant rank assumption it follows that gz vanishes in a whole
neighborhood of the initial value, so that g is already independent of z.
Systems of index 3. Problems of the form
y = f (y, z)
z = k(y, z, u)
0 = g(y)
(2.5)
is invertible
(2.6)
in a neighborhood of the solution. To see this, we differentiate twice the algebraic relation
of (2.5), which yields
0 = (gy f )(y, z),
(2.7)
A third differentiation permits to express u in terms of (y, z, u) provided that (2.6) is satisfied.
This proves index 3 of the system (2.5).
Consistent inital values (y0 , z0 , u0 ) must satisfy g(y0 ) = 0 and the two conditions (2.7).
Under the condition (2.6) an application of the implicit function theorem permits to express
u in terms of (y, z) from the second relation of (2.7), i.e., u = (y, z). Inserting this relation
into the differential-algebraic system (2.5) yields an ordinary differential equation for (y, z)
on the manifold
M = {(y, z) ; g(y) = 0, gy (y)f (y, z) = 0}.
The assumption (2.6) implies that gy (y) and gy (y)fz (y, z) have full rank, so that M is a
manifold. It follows from (2.7) that the vector field lies in the tangent space T(y,z) M for all
(y, z) M.
IV.3
Control problems
Many problems of control theory lead to ordinary differential equations of the form
y = f (y, u),
where u represents a set of controls. These controls must be applied so that the solution
satisfies some constraints 0 = g(y) (or 0 = g(y, u)). They often lead to a differential-algebraic
system of index 2, as it is the case for the example of Section I.2.
Optimal control problems are differential equations y = f (y, u) formulated in such a way
that the control u(t) has to minimize some cost functional. The EulerLagrange equation
then often becomes a differential-algebraic system (Pontryagin, Boltyanskij, Gamkrelidze
Differential-Algebraic Equations
33
& Mishchenko 1961, Athans & Falb 1966, Campbell 1982). We demonstrate this on the
problem
y = f (y, u),
y(0) = y0
(3.1)
with cost functional
J(u) =
Z 1
(3.2)
For a given function u(t) the solution y(t) is determined by (3.1). In order to find conditions
for u(t) that minimize J(u) of (3.2), we consider the perturbed control u(t) + u(t) where
u(t) is an arbitrary function and a small parameter. To this control there corresponds a
solution y(t) + y(t) + O(2 ) of (3.1); we have (by comparing powers of )
y(t)
= fy (t)y(t) + fu (t)u(t),
y(0) = 0,
where, as usual, fy (t) = fy (y(t), u(t)), etc. Linearization of (3.2) shows that
J(u + u) J(u) =
Z 1
0
so that
Z 1
0
y (t)y(t) + u (t)u(t) dt = 0
(3.3)
is a necessary condition for u(t) to be an optimal solution of our problem. In order to express
y in terms of u in (3.3), we introduce the adjoint differential equation
v = fy (t)T v y (t)T ,
v(1) = 0
y (t)y(t) dt =
Z 1
0
Since this relation has to be satisfied for all u we obtain the necessary relation
v T (t)fu (t) + u (t) = 0
by the so-called fundamental lemma of variational calculus.
In summary, we have proved that a solution of the above optimal control problem has to
satisfy the system
y = f (y, u),
y(0) = y0
T
T
v = fy (y, u) v y (y, u) ,
v(1) = 0
(3.4)
T
0 = v fu (y, u) + u (y, u).
This is a boundary value differential-algebraic problem. It can also be obtained directly from
the Pontryagin minimum principle (see Pontryagin et al. 1961, Athans and Falb 1966).
Differentiation of the algebraic relation in (3.4) shows that the system (3.4) has index 1
if the matrix
n
X
2 fi
2
vi
(y,
u)
+
(y, u)
u2
u2
i=1
is invertible along the solution. A situation where the system (3.4) has index 3 is presented
in Exercise 7.
34
Differential-Algebraic Equations
IV.4
Mechanical systems
An interesting class of differential-algebraic systems appears in mechanical modeling of constrained systems. A choice method for deriving the equations of motion of mechanical systems is the Lagrange-Hamilton principle, whose long history goes back to merely theological
ideas of Leibniz and Maupertuis.
Mechanical systems in minimal coordinates. Let q = (q1 , . . . , qn )T be minimal2 generalized coordinates of a system and vi = qi the velocities. Suppose a function L(q, q)
is given;
then the Euler equations of the variational problem
Z t2
L(q, q)
dt = min !
t1
are given by
d
dt
L
qk
L
= 0,
qk
k = 1, . . . , n,
(4.1)
which represent a second order differential equations for the coordinates qk . The great
discovery of Lagrange (1788) is that for L = T U , where T (q, q)
= 12 qT M (q) q (with
a symmetric positive matrix M (q)) is the kinetic energy and U (q) the potential energy,
the differential equation (4.1) describes the movement of the corresponding conservative
system. Written as a first order differential equation, it is given by
q = v
M (q) v = f (q, v),
(4.2)
Example 4.1. The mathematical pendulum of length ` has one degree of freedom. We
choose as generalized coordinate the angle = q1 such that T = m `2 2 /2 and U =
`mg cos . Then (4.1) becomes `
= g sin , the well-known pendulum equation.
Constrained mechanical systems. Suppose now that the generalized coordinates q =
(q1 , . . . , qn )T are constrained by the relations g1 (q) = 0, . . . , gm (q) = 0 (or shortly g(q) = 0)
on their movement. If these relations are independent (we assume that g 0 (q) has full rank
m) the number of degrees of freedom is n m. An example is the mathematical pendulum
considered in Cartesian coordinates. We again assume that the kinetic energy is given by
T (q, q)
= 12 qT M (q) q with a symmetric positive matrix M (q), and the potential energy is
U (q). To obtain the equations of motion we proceed in three steps:
we introduce minimal coordinates of the system, i.e., a parametrization q = (z) of
the submanifold N = {q ; g(q) = 0},
we write down the equations of motion in minimal coordinates z, and
we rewrite these equations in the original variables q.
Using our parametrization q = (z) and its time derivative q = 0 (z)z,
the kinetic and
potential energies become
Tb (z, z)
= T ((z), 0 (z)z)
=
2
1 Tc
z M (z) z
2
with
Minimal means that the dimension of q equals the number of degrees of freedom in the system.
Differential-Algebraic Equations
35
b
and Ub (z) = U ((z)). With the Lagrangian L(z,
z)
= L((z), 0 (z)z)
= Tb (z, z)
Ub (z) the
equations of motion, written in minimal coordinates z, are therefore
b
b
d L
L
(z, z)
(z, z)
= 0.
dt z
z
(4.3)
We have to rewrite these equations in the original variables q. Using the relations
b
L
L
(z, z)
=
(q, q)
0 (z)
z
q
b
L
L
L
(z, z)
=
(q, q)
0 (z) +
(q, q)
00 (z)(z,
)
z
q
q
b
d L
d L
L
(z, z)
=
(q, q)
0 (z) +
(q, q)
00 (z)(z,
)
dt z
dt q
q
d L
L
(q, q)
(q, q)
0 (z) = 0.
dt q
q
(4.4)
Any vector w satisfying wT 0 (z) = 0 is orthogonal to the image Im 0 (z). However, from
the characterization of the tangent space (Theorem II.2.2) we know that Im 0 (z) = Tq N =
ker g 0 (q). Using the identity (ker g 0 (q)) = Im g 0 (q)T , we obtain that the equation (4.4) is
equivalent to
T
L
d L
= g 0 (q)T
(q, q)
(q, q)
dt q
q
which can also be written as
q = v
M (q) v = f (q, v) G(q)T
0 = g(q),
(4.5)
where we denote G(q) = g 0 (q), and f (q, v) is as in (4.2). For the mathematical pendulum,
written in Cartesian coordinates, these equations have been considered in Example I.3.1.
Various formulations are possible for such a problem, each of which leads to a different
numerical approach.
Index 3 Formulation (position level, descriptor form). If we formally multiply the second
equation of (4.5) by M (q)1 , the system (4.5) becomes of the form (2.5) with (q, v, ) in the
roles of (y, z, u). The condition (2.6), written out for (4.5), is
G(q)M (q)1 G(q)T
is invertible .
(4.6)
This is satisfied, if the rows of the matrix G(q) are linearly independent, i.e., the constraints
g(q) = 0 are independent. Under this assumption, the system (4.5) is an index 3 problem.
Index 2 Formulation (velocity level). Differentiation of the algebraic relation in (4.5) gives
0 = G(q)v.
(4.7)
If we replace the algebraic relation in (4.5) by (4.7), we obtain a system of the form (2.3)
with (q, u) in the role of y and in that of z. One verifies that because of (4.6) the first two
equations of (4.5) together with (4.7) represent a problem of index 2.
36
Differential-Algebraic Equations
M (q) GT (q)
G(q)
0
v
f (q, v)
=
.
gqq (q)(v, v)
(4.8)
This allows us to express v and as functions of q, v, provided that the matrix in (4.8) is
invertible (see Exercise I.6). Hence, the first equation of (4.5) together with (4.8) consitute
an index 1 problem.
All these formulations are mathematically equivalent, if the initial values are consistent,
i.e., if (q0 , v0 ) satisfy g(q0 ) = 0 and g 0 (q0 )v0 = 0, and if 0 = (q0 , v0 ) where the function
(q, v) is defined by (4.8). However, if for example the index 1 or the index 2 system is integrated numerically, the constraints of the original problem will no longer be exactly satisfied.
It is recommended to consider the problem as a differential equation on the manifold, and
to force the solution to remain on the manifold.
Constrained mechanical system as differential equation on a manifold. Inserting
the function (q, v) obtained from (4.8) into the system (4.5), the first two equations of (4.5)
represent an ordinary differential equation on the submanifold
M = {(q, v) ; g(q) = 0, g 0 (q)v = 0}.
This is equivalent to the index 1 formulation. Applying the numerical techniques of Chapter III (projection methods and local state space form approaches) to the problem, one has
to be careful that the numerical solution not only satisfies the given constraint g(q) = 0, but
also the hidden constraint g 0 (q)v = 0.
IV.5
Exercises
1 2
2 4
u 1
u 2
1 0
1 1
u1
u2
2
t
3t
Is there a solution for every pair of initial values (u1 (0), u2 (0)) ?
2. Prove that the initial value problem
B u + Au = 0,
u(0) = 0
QAZ =
A11 A12
,
0 A22
QBZ =
B11 B12
0 B22
(5.1)
are both triangular. Furthermore, the submatrices A22 and B11 are invertible, and the
diagonal elements of B22 are all 0.
Differential-Algebraic Equations
37
Hint (compare with the Schur decomposition of a matrix). Let 1 be a zero of det(A + B)
and v1 6= 0 be such that (A + 1 B)v1 = 0. Verify that Bv1 6= 0 and that
AZ1 = Q1
1
,
0
Ae
BZ1 = Q1
e
0 B
where = kBv1 k/kv1 k, and Q1 , Z1 are unitary matrices (orthogonal if 1 is real) whose first
e is again
columns are scalar multiples of Bv1 and v1 , respectively. The matrix pencil Ae + B
e = Const which implies that
regular and this procedure can be continued until det(Ae + B)
e
e
e with
det B = 0. In this case we take a vector v2 6= 0 such that Bv2 = 0 and transform Ae + B
e 2 and v2 , respectively. For a practical
unitary matrices Q2 , Z2 , whose first columns are Av
computation of the decomposition (5.1) see the monograph of Golub and Van Loan (1989),
Section 7.7.
4. Under the assumptions of Exercise 3 show that there exist matrices S and T such that
I S
0 I
I S
0 I
A11 A12
0 A22
B11 B12
0 B22
I T
0 I
I T
0 I
A11 0
,
0 A22
B11 0
.
0 B22
=
=
(5.2)
(5.3)
and can be computed as follows: the first column of T is obtained from (5.3) because B11 is
invertible and the first column of SB22 vanishes; then the first column of S is given by (5.2)
because A22 is invertible; the second column of SB22 is then known and we can compute the
second column of T from (5.3), etc.
5. Prove that the index of nilpotency of a regular matrix pencil A + B does not depend on the
choice of P and Q in (1.2).
Hint. Consider two different decompositions of the form (1.2) and denote the matrices which
appear by C1 , N1 and C2 , N2 , respectively. Show the existence of a regular matrix T such
that N2 = T 1 N1 T .
6. For the linear initial value problem
y = A(t)y + f (t),
y(0) = 0
Z 1
g(t)T y(t) dt =
Z 1
v(1) = 0.
Z
1 1
y(0) = y0
38
Differential-Algebraic Equations
a) Prove that J(u) is minimal if and only if
y = Ay + Bu + f (t),
v =
AT v
0 =
BTv
Cy,
y(0) = y0
v(1) = 0
(5.4)
+ Du.
`1
m1
`2
m2
a) Determine the constraints and give the descriptor form (differential-algebraic equation of
index 3) of the equations of motion for the mechanical system in Cartesian coordinates.
b) Let and be the generalized coordinates of the double pendulum. Write the equations
of motion in terms of these minimal coordinates.
Chapter V
Numerical Methods for DAEs
We have seen in Chapter IV how differential-algebraic equations (DAEs) can be interpreted as
differential equations on manifolds. Therefore, all numerical approaches (projection methods
and integrators based on local coordinates) discussed in Chapter III can be applied to solve
these problems. Here, we consider direct numerical methods for problems of the form
M u = F (u),
u(0) = u0 ,
(0.1)
where M is a constant, but possibly singular matrix, and the initial value is such that the
problem possesses a unique solution. For this it is necessary that F (u0 ) lies in the range of
the matrix M . All problems of Chapter IV can be written in this form. For the problems of
the form (IV.2.2) or (IV.2.3) the matrix M is diagonal, with entries 1 in the first part, and
entries 0 in the rest. For an implicit differential equation F0 (v,
v) = 0, we can introduce a
new variable for the derivative and thus obtain the system v = w, F0 (w, v) = 0, which is of
the form (0.1) for the vector u = (v, w).
In the first sections of the present chapter, we consider a numerical approach which
requires only the knowledge of the data M and F (u) of the problem, and not that of the
underlying manifold of the DAE. It can be sketched as follows:
apply formally any numerical method to the differential equation u = M 1 F (u),
rewrite the formulas in such a way that the inverse of M is no longer present,
investigate whether the resulting numerical scheme makes sense for singular M .
Whereas the definition of the numerical schemes, following this approach, is extremely simple,
their analysis (local accuracy, stability, and convergence) needs more effort.
V.1
Let us start with applying the above approach to the explicit and implicit Euler methods.
For the explicit Euler method we obtain
un+1 = un + h M 1 F (un )
or
M (un+1 un ) = h F (un ).
If the matrix M is singular, this relation does not permit us to compute un+1 for a given
un , and the above approach does not lead to a numerical approximation. The implicit Euler
method yields
M (un+1 un ) = h F (un+1 ),
(1.1)
40
which represents a nonlinear system for un+1 . Application of simplified Newton iterations
requires the solution of linear equations with the matrix
M h F 0 (un ).
(1.2)
If the matrix pencil, formed by the matrices M and F 0 (un ) is regular, then the matrix (1.2) is
invertible for sufficiently small step size h, and simplified Newton iterations are feasible. We
shall study in the next sections, when the solution un+1 of (1.1) exists, so that the implicit
Euler method is well defined for small h.
Linear multistep methods. Applying a multistep formula to the system u = M 1 F (u)
and multiplying the relation with M yields (notice that k 6= 0)
M
k
X
j un+j = h
j=0
k
X
j F (un+j ).
(1.3)
j=0
If the method is explicit, i.e., k = 0, this relation does not permit the computation of un+k
when M is singular. Therefore, only implicit methods make sense in this context. As for the
implicit Euler method, an application of simplified Newton iterations leads to linear systems
with the matrix
k M h k F 0 (un+k1 ).
This again requires the matrix pencil formed by M and F 0 (un+k1 ) to be regular.
RungeKutta methods. Using this approach with RungeKutta methods as numerical
integrator leads to the system
M (Uni un ) = h
M (un+1 un ) = h
s
X
j=1
s
X
aij F (Unj ),
i = 1, . . . , s
(1.4)
bi F (Uni ).
i=1
Consider first the upper relation of (1.4), which is supposed to define the internal stages
Uni for i = 1, . . . , s. Applying simplified Newton iterations yields linear systems with the
matrix1
I M h A F 0 (un ).
(1.5)
Suppose that the invertible matrix T is such that T 1 AT is upper triangular with the
eigenvalues i of A on the diagonal. The matrix T I then transforms (1.5) to block upper
triangular form with diagonal blocks of the form M hi F 0 (un ). If the matrix pencil formed
by M and F 0 (un ) is regular, and if i 6= 0 for all i (which means that A is non-singular)
then the matrix (1.5) is invertible for sufficiently small h, and simplified Newton iterations
can be performed.
Assume for the moment that the system (1.4) has a (locally) unique solution Un1 , . . . , Uns .
The right-hand side of the lower relation of (1.4) is then determined, and it seems hopeless to
get a unique approximation un+1 when M is singular. However, if the RungeKutta matrix
A = (aij )si,j=1 is invertible, we can compute the vector (F (Un1 ), . . . , F (Uns )) from the upper
part of (1.4) and insert it into the lower part. This gives
M (un+1 un ) =
s
X
i=1
bi
s
X
wij M (Unj un ),
j=1
For two matrices A and B, the tensor product is defined as A B = (aij B)si,j=1 .
41
where wij are the entries of the inverse A1 of the RungeKutta matrix. As long as M is
invertible, we can simplify this relation by M and thus obtain
un+1 un =
s X
s
X
j=1
bi wij (Unj un ).
(1.6)
i=1
For invertible M , the complete system (1.4) is therefore equivalent to the system, where the
lower relation of (1.4) is replaced with (1.6). This formulation is perfectly adapted to the
solution of problems (0.1) with singular M .2
Invariance with respect to linear transformations. In many situations (either for
theoretical investigations or for practical issues like step size selection) it is convenient to
have a very simple form of the matrix M in (0.1). We can always decompose the matrix M
(e.g., by Gaussian elimination with total pivoting) as
M =S
I 0
0 0
(1.7)
T,
where S and T are invertible matrices and the dimension of I represents the rank of M .
Inserting this into (0.1), multiplying by S 1 , and using the transformed quantities
Tu =
y
,
z
S 1 F (u) = S 1 F T 1
y
z
f (y, z)
g(y, z)
(1.8)
gives
y = f (y, z)
0 = g(y, z),
(1.9)
a problem that has been extensively studied in Chapter IV. At the moment we do not make
any assumption on the index of the problem.
It is interesting to note that all numerical methods considered in this section are invariant
with respect to this transformation. If we consider transformed variables
T un =
yn
,
zn
T Uni =
Yni
Zni
(1.10)
also for the numerical solution, this means that the diagram
problem (0.1)
transf. (1.8) -
numer. method
?
{un }
problem (1.9)
numer. method
transf. (1.10)-
{yn , zn }
By the way, the use of (1.6) is recommended for an implementation of implicit RungeKutta methods.
42
V.2
Index 1 problems
y(0) = y0
z(0) = z0
(2.1)
with initial values satisfying g(y0 , z0 ) = 0. In this section we assume that gz is invertible along
the solution, so that the problem is of differentiation index 1. As discussed in Section IV.2,
the algebraic equation of (2.1) can then be solved for z and yields and equivalent relation
z = (y). In this section we study the accuracy and convergence of multistep methods as
well as RungeKutta methods.
Linear multistep methods. For the problem (2.1), a linear multistep method applied in
the form (1.3) reads
k
X
j=0
j yn+j = h
k
X
j f (yn+j , zn+j ),
0=
j=0
k
X
j g(yn+j , zn+j ).
(2.2)
j=0
s
X
i = 1, . . . , s
j=1
0 = g(Yni , Zni ),
yn+1 yn = h
s
X
i = 1, . . . , s
bi f (Yni , Zni )
(2.3)
i=1
zn+1 zn =
s X
s
X
j=1
bi wij (Znj zn ).
i=1
The second relation shows that the internal stages satisfy Zni = (Yni ) for i = 1, . . . , s.
Consequently, the y component is precisely the same as if we apply the RungeKutta method
to the ordinary differential equation y = f (y, (y)). Classical convergence results therefore
yield yn y(nh) = O(hp ) on compact intervals 0 nh T , where p denotes the order of
the method.
If the method is stiffly accurate, i.e., the RungeKutta coefficients satisfy asj = bj for all j,
P
then we have yn+1 = Yns . Moreover, the RungeKutta coefficients satisfy si=1 bi wij = 0 for
43
j = 1, . . . , s 1, and si=1 bi wis = 1. Consequently, we have zn+1 = Zns and thus also zn+1 =
(yn+1 ). The convergence estimate for the y component therefore implies zn z(nh) = O(hp )
on compact intervals 0 nh T .
For methods that are not stiffly accurate, the so-called stage order plays an important
role. One says that a RungeKutta method has stage order q, if the coefficients satisfy the
simplifying condition
P
C(q) :
s
X
aij ck1
=
j
j=1
cki
,
k
i = 1, . . . , s,
k = 1, . . . q.
(2.4)
This is equivalent to sj=1 aij p(cj ) = 0ci p( ) d for polynomials p( ) of degree q 1, and
means that the quadrature rules for the internal stages have an order at least q. For the z
component we have the following convergence result.
P
Theorem 2.1 (order reduction3 ). Consider the system (2.1) with initial values satisfying g(y0 , z0 ) = 0, and assume that gz is invertible in a neighborhood of the exact solution
(y(t), z(t)). Let the Runge-Kutta method be of order p, of stage order q, with invertible maP
P
trix A, and denote4 = 1 sj=1 si=1 bi wij . Then the numerical solution of (2.3) has global
error satisfying
zn z(nh) = O(hr ) for tn = nh T,
where
a) r = p for stiffly accurate methods,
b) r = min(p, q + 1) if the stability function satisfies 1 < 1,
c) r = min(p 1, q) if = +1.
d) If || > 1, the numerical solution diverges.
Proof. Part (a) has already been discussed. For the remaining cases we proceed as follows:
we first observe that condition C(q) and order p imply
z(tn + ci h) = z(tn ) + h
z(tn+1 ) = z(tn ) + h
s
X
j=1
s
X
aij z(t
n + cj h) + O(hq+1 )
(2.5)
bi z(t
n + ci h) + O(hp+1 ).
(2.6)
i=1
(2.8)
Our next aim is to estimate Zn . For this we have to consider the y component of the
system. By definition of the method, the values yn , Yni are those of the Runge-Kutta method
3
This order reduction in the z component was first studied in a more general context by L.R. Petzold,
Order results for implicit RungeKutta methods applied to differential/algebraic systems. SIAM J. Numer.
Anal. 23 (1986) 837852.
4
The expression equals the value at infinity of the stability function.
44
applied to y = f (y, (y)). It thus follows from the classical convergence theory for ordinary
differential equations that yn y(tn ) = ep (tn )hp + O(hp+1 ). Since equation (2.5) also holds
with z(t) replaced by y(t), we can subtract this formula from the first relation of (2.3) and
so obtain
Yni y(tn + ci h) = yn y(tn )
+ h
s
X
+ O(hq+1 ).
j=1
with
= min(p, q + 1).
By the second relation of (2.3) we have Zni z(tn + ci h) = (Yni ) (y(tn + ci h)) = O(h ),
and equation (2.8) becomes
zn+1 = zn + n+1 ,
n+1 = O(h ).
where
n
X
ni i ,
i=1
because z0 = 0. This proves the statement for 6= 1. For the case = 1 the error zn
is a sum of differences j+1 j . Since n+1 is actually of the form n+1 = d(tn )h + O(h+1 )
we have j+1 j = O(h+1 ) and the statement also follows in this situation.
Example 2.2 (Radau IIA methods). One of the most important integrators for the numerical solution of differential-algebraic equations are the so-called Radau IIA methods. The
nodes c1 , . . . , cs are the zeros of
ds1
xs1 (x 1)s ,
dxs1
and the weights b1 , . . . , bs are chosen such that the quadrature formula is interpolatory,
which implies that it is of order p = 2s 1. Ehle (1969) and Axelsson (1969) independently
proposed to consider coefficients aij by imposing condition C(s) of (2.4). The special case for
s = 1 is nothing other than the implicit Euler method. The coefficients (matrix aij together
with the ci in the left column and the bj in the bottom row) are given in Table V.1 for the
cases s = 2 and s = 3.
The methods have classical order p = 2s1, stage order q = s, the RungeKutta matrix is
invertible, and the weights satisfy bj = asj for all j. For more details we refer to Section IV.5
of the monograph Solving Ordinary Differential Equations II by Hairer and Wanner.
5
12
3
4
3
4
1
12
1
4
1
4
4 6
10
4+ 6
10
88 7 6
360
296 + 169 6
1800
16 6
36
16 6
36
296 169 6
1800
88 + 7 6
360
16 + 6
36
16 + 6
36
2 + 3 6
225
2 3 6
225
1
9
1
9
V.3
45
Index 2 problems
y(0) = y0
z(0) = z0
(3.1)
where the initial values satisfy g(y0 ) = 0 and gy (y0 )f (y0 , z0 ) = 0. We assume that f and g
are sufficiently differentiable and that
gy (y)fz (y, z)
is invertible
(3.2)
in a neighbourhood of the solution, so that the problem has index 2. Recall that this problem
can be considered as a differential equation on the manifold M = {y ; g(y) = 0}.
In this section we restrict our considerations to implicit RungeKutta methods with
invertible matrix (aij ), and coefficients satisfying bj = asj for all j (stiffly accurate methods).
For the problem (3.1) they are defined by
Yni yn = h
s
X
0 = g(Yni ),
i = 1, . . . , s
(3.3)
j=1
with a numerical approximation after one step given by yn+1 = Yns , zn+1 = Zns . Notice that
the internal stages and the numerical solution do not depend on zn . The value of zn only
specifies the solution branch of gy (y)f (y, z) = 0 to which the expressions Znj remain close.
Moreover, the numerical solution yn stays on the manifold M for all n.
The convergence results of this section are also valid for index 2 systems of the form
y = f (y, z), 0 = g(y, z), if they can be transformed to (3.1) without any differentiation
(see the discussion of index 2 systems in Section IV.2). This is because the method (3.3) is
invariant with respect to these transformations.
Theorem 3.1 (existence and uniqueness of numerical solution). Consider yn M, let
be a value satisfying gy (yn )f (yn , ) = 0, and assume that (3.2) holds in a neighborhood of
(yn , ). If the RungeKutta matrix (aij ) is invertible, then there exists h0 > 0 such that the
nonlinear system (3.3) possesses for |h| h0 a locally unique solution which satisfies
Yni yn = O(h),
Zni = O(h).
(3.4)
Proof. We shall prove that the solution (Yni , Zni ) of (3.3) can be expressed as a smooth
function of h (for sufficiently small h). A direct application of the implicit function theorem
is not possible due to the presence of the factor h in front of the Znj dependence.
The idea is to use the fundamental theorem of calculus
g(Yni ) g(yn ) =
Z 1
0
gy yn + (Yni yn ) (Yni yn ) d,
gy yn + (Yni yn ) d
s
X
i = 1, . . . , s,
(3.5)
j=1
which is the discrete analogue of the hidden constraint gy (y)f (y, z) = 0. We now apply the
implicit function theorem to the system formed by (3.5) and the first relation of (3.3). For
46
h = 0, our assumptions imply that the values Yni = yn and Zni = satisfy the system.
Furthermore, the derivative with repect to (Yni , Zni ) at h = 0 and (Yni , Zni ) = (yn , ) is of
the form
!
I I
0
,
O(1) A (gy fz )(yn , )
which is invertible because of (3.2). Therefore the implicit function theorem yields the
existence of a locally unique solution of (3.3).
The method (3.3) represents a numerical one-step method on the manifold M. In view
of an application of the convergence theorem of Section III.5 we have to study the local
error. Recall that the local error is the difference (yn+1 yn (tn+1 ), zn+1 zn (tn+1 )), where
(yn (t), zn (t)) is the solution of (3.1) with consistent initial values yn (tn ) = yn , zn (tn ) = zn .
Theorem 3.2 (local error estimate). Consider a differential-algebraic equation (3.1) satisfying (3.2), and apply an implicit RungeKutta method (3.3) with invertible matrix (aij ) and
coefficients satisfying bj = asj for all j. If the quadrature formula formed by (bi , ci )si=1 is of
order p, and the method has stage order q, then we have the estimate
yn+1 yn (tn + h) = O(hmin(p+1,q+2) ),
Proof. Inspired by the proof of Theorem 3.1, we consider the nonlinear system for (Yi , Zi ),
i = 1, . . . , s,
Yi y n = h
s
X
aij f (Yj , Zj ) + hi
j=1
Z 1
0
gy yn + (Yi yn ) d
X
s
(3.6)
aij f (Yj , Zj ) + i = 0,
j=1
where the second equation is known to be equivalent to g(Yi ) = 0. For i = 0 we obtain the
numerical solution (Yi , Zi ) = (Yni , Zni ) of (3.3). The exact solution at the quadrature points
(Yi , Zi ) = (yn (tn + ci h), zn (tn + ci h)) satisfies (3.6) with i = O(hq ) for i = 1, . . . , s 1, and
s = O(hp ). We are interested in the dependence of the solution (Yi , Zi ) on the parameters
i , when the step size h 6= 0 is fixed. We see that the derivative of the system (3.6) with
respect to (Yi , Zi ) at the numerical approximation (Yni , Zni ) is of the form
!
I I + O(h)
O(h)
,
O(1)
A (gy fz )(yn , zn ) + O(h)
which is invertible for sufficiently small h. The implicit function theorem therefore implies
that
Yni yn (tn + ci h) = O(),
Zni zn (tn + ci h) = O(),
where = maxi=1,...,s i = O(hq ). This proves the estimate for the local error of the z
component. Some further considerations are necessary for the local error of the y component.
First, we notice that due to the factor h in the right-hand side of the upper equation of
(3.6), we have the improved estimate Yni yn (tn + ci h) = O(hq+1 ) for all i. For the local
error yn+1 = yn+1 yn (tn + h) of the y component we thus obtain
yn+1 = h fz (yn , zn )
s
X
j=1
(3.7)
47
s
X
j=1
Since gy (yn )fz (yn , zn ) is invertible by (3.2), the expression h sj=1 bj (Znj zn (tn + cj h)) is
of size O(hq+2 ) + O(hp+1 ). Inserted into (3.7) we finally get the stated estimate for the y
component.
P
Remark. Whereas the estimate for the local error of the z component is in general optimal,
that for the y component can be improved in some interesting situations. For example, for the
Radau IIA methods of Example 2.2, we have for the y component yn+1 yn (tn +h) = O(hp+1 ).
This property is known as superconvergence.
Convergence for the y component. The numerical method (3.3) can be considered as
a mapping yn 7 yn+1 on the submanifold M. The approximations zn only influence the
choice of the solution, when the equation 0 = gy (y)f (y, z) has more than one solutions z for
a given y. Theorem III.5.2 can therefore be applied and yields the estimate for the global
error
yn y(tn ) = O(hmin(p,q+1) )
for tn = nh T.
Convergence for the z component. The numerical solution zn is defined locally and
there is no propagation of errors. The error is therefore a superposition of the local error for
the z component and the global error of the y component. Since we have p q for stiffly
accurate methods, this implies
zn z(tn ) = O(hq )
V.4
for tn = nh T.
(4.1)
where G(q) = g 0 (q). This system has many remarkable properties: it exactly preserves the
total energy
1
H(p, q) = pT M 1 p + U (q),
(4.2)
2
48
and the flow is a symplectic and volume preserving transformation. It is not the aim of
this lecture to discuss these topics5 , we concentrate on the fact that the system (4.1) is a
differential-algebraic equation of index 3 and can be considered as a differential equation on
the manifold
n
o
M = (p, q) ; g(q) = 0, G(q)M 1 p = 0 .
(4.3)
Symplectic Euler method for constrained mechanical systems. We integrate the
p and variables by the implicit Euler and the q variable by the explicit Euler method. This
leads to the discretization
pbn+1 = pn h (U (qn ) + G(qn )T n+1 )
qn+1 = qn + h M 1 pbn+1
0 = g(qn+1 ).
(4.4)
The numerical approximation (pbn+1 , qn+1 ) satisfies the constraint g(q) = 0, but not the
hidden constraint G(q)M 1 p = 0. To get an approximation (pn+1 , qn+1 ) M, we append
the projection
pn+1 = pbn+1 h G(qn+1 )T n+1
(4.5)
0 = G(qn+1 ) M 1 pn+1 .
Let us discuss some basic properties of this method.
Existence and Uniqueness of the Numerical Solution. Inserting the definition of qn+1
from the second line of (4.4) into 0 = g(qn+1 ) gives a nonlinear system for pbn+1 and hn+1 .
Due to the factor h in front of M 1 pbn+1 , the implicit function theorem cannot be directly
applied to prove existence and uniqueness of the numerical solution. We therefore write this
equation as
Z 1
0 = g(qn+1 ) = g(qn ) +
G qn + (qn+1 qn ) (qn+1 qn ) d.
We now use g(qn ) = 0, insert the definition of qn+1 from the second
and divide
line of (4.4)
b
by h. Together with the first line of (4.4) this yields the system F pn+1 , hn+1 , h = 0 with
p pn + hU (qn ) + G(qn )T
F p, , h = Z
G qn + hM
p M
p d
F
I
G(qn )T
pn , 0, 0 =
,
1
G(qn )M
0
(p, )
and this matrix is invertible, because we always assume the matrix G(q) to be of full rank.
Consequently, an application of the implicit function theorem proves that the numerical solution (pbn+1 , hn+1 ) (and hence also qn+1 ) exists and is locally unique for sufficiently small h.
The projection step (4.5) represents a linear system for pn+1 and hn+1 with invertible matrix.
5
They are treated in the monograph Geometric Numerical Integration by Hairer, Lubich, and Wanner.
This and the next section are taken from this monograph.
49
Convergence of Order 1. The above use of the implicit function theorem yields the rough
estimates
pbn+1 = pn + O(h), hn+1 = O(h), hn+1 = O(h),
which, together with the equations (4.4) and (4.5), give
qn+1 = qn (tn+1 ) + O(h2 ),
where (pn (t), qn (t)) is the solution of (4.1) passing through (pn , qn ) M at t = tn . Inserting
these relations into the second equation of (4.5) we get
0 = G(qn (t))M 1 pn (t) G(qn (t))M 1 G(qn (t))T + O(h2 )
at t = tn+1 . Since G(qn (t))M 1 pn (t) = 0 , and G(qn (t))M 1 G(qn (t))T is invertible, we have
= O(h2 ). The local error is therefore of size O(h2 ) in both components.
The convergence proof is now a direct application of Theorem III.5.2, because the method
is a mapping h : M M on the solution manifold. This proves that the global error
satisfies pn p(tn ) = O(h) and qn q(tn ) = O(h) as long as tn = nh Const.
Numerical Experiment (spherical pendulum). We denote by q1 , q2 , q3 the Cartesian
coordinates of a point with mass m = 1 that is connected with a massless rod of length
` = 1 to the origin. The kinetic and potential energies are T = 21 (q12 + q22 + q32 ) and U = q3 ,
respectively, and the constraint is the fixed length of the rod. We thus get the system
p1 = q1
p2 = q2
p3 = 1 q3
q1 = p1
q2 = p2
q3 = p3
0 =
1 2
(q
2 1
(4.6)
Figure V.1 (upper picture) shows the numerical solution (vertical coordinate q3 ) over many
periods obtained by method (4.4)-(4.5). We observe a regular qualitatively correct behavior.
For the implicit Euler method (i.e., the argument qn is replaced with qn+1 in (4.4)) the
numerical solution, obtained with the same step size and the same initial values, is less
satisfactory. Already after one period the solution deteriorates and the system loses energy.
1.0
.5
component q3
symplectic Euler
.0
.5
50
100
1.0
.5
component q3
implicit Euler
.0
.5
50
100
Fig. V.1: Spherical pendulum problem solved with the symplectic Euler method (4.4)(4.5) and with the implicit Euler method; initial value q0 = (sin(1.3), 0, cos(1.3)), p0 =
(3 cos(1.3), 6.5, 3 sin(1.3)), step size h = 0.01.
50
V.5
The numerical method (4.4)-(4.5) is only of order 1 and it is not symmetric. Here we present
an algorithm that is of order 2, symmetric and symplectic. The original derivation is based
on the fact that the system (4.1) is equivalent to the second order differential equation
M q = U (q) G(q)T with constraint g(q) = 0.
SHAKE. Ryckaert, Ciccotti, and Berendsen (1977) propose the method
(5.1)
for computations in molecular dynamics. The p component, not used in the recursion, is
approximated by the symmetric finite difference pn = M (qn+1 qn1 )/2h.
RATTLE. The three-term recursion (5.1) may lead to an accumulation of round-off errors,
and a reformulation as a one-step method is desirable. Introducing a new variable via
qn+1 qn = hM 1 pn+1/2 , the method (5.1) becomes pn+1/2 pn1/2 = h(U (qn )+G(qn )T n )
and the momentum approximation leads to pn+1/2 + pn1/2 = 2pn . Elimination of either
pn+1/2 or pn1/2 leads to the formulae
pn+1/2 = pn
h
U (qn ) + G(qn )T n
2
qn+1 = qn + hM 1 pn+1/2 ,
pn+1 = pn+1/2
(5.2)
0 = g(qn+1 )
h
U (qn+1 ) + G(qn+1 )T n+1 .
2
The difficulty with this formulation is that n+1 is not yet available at this step (it is computed together with qn+2 ). As a remedy, Andersen (1983) suggests replacing the last line in
(5.2) with a projection step similar to (4.5)
pn+1 = pn+1/2
h
U (qn+1 ) + G(qn+1 )T n
2
(5.3)
0 = G(qn+1 )M 1 pn+1 .
This modification, called RATTLE, has the further advantage that the numerical approximation (pn+1 , qn+1 ) lies on the solution manifold M.
Theorem 5.1. The RATTLE method is symmetric, symplectic, and convergent of order 2.
Proof. If we add the consistency conditions g(qn ) = 0, G(qn )M 1 pn = 0 of the initial values
to the RATTLE algorithm, the symmetry of the method follows at once by exchanging
h h, pn+1 pn , qn+1 qn , and n n . We do not discuss the symplecticity in this
lecture, and refer to the monograph Geometric Numerical Integration.
The implicit function theorem applied to the two systems (5.2) and (5.3) shows that
pn+1/2 = pn + O(h),
hn = O(h),
hn = O(h),
Convergence of order one follows therefore in the same way as for method (4.4)-(4.5) by
applying the convergence Theorem III.5.2. Since the order of a symmetric method is always
even, this implies convergence of order two.
V.6
51
Exercises
1. (Gear, Hsu, and Petzold 1981, Gear and Petzold 1984). Consider the problem
!
0 0
1 t
y
1
t
+
z
0 1+
y
z
f (t)
.
g(t)
(6.1)
Prove that the system (6.1) has differentiation index 2 for all values of , and that the
d
z-component of the exact solution is given by z(t) = g(t) dt
f (t).
2. A straight-forward application of the implicit Euler method to the differential-algebraic equation (6.1) would be
0
0
1 tn+1
1 tn+1
yn+1 yn
+h
0 1+
zn+1 zn
yn+1
zn+1
f (tn+1 )
=h
.
g(tn+1 )
(6.2)
f (tn+1 ) f (tn )
1
g(tn+1 )
.
zn +
1+
1+
h
zn+1 =
Hence, the method is convergent for > 1/2, but unstable for < 1/2. For = 1 the
numerical solution does not exist.
3. Introducing the new variable u = z,
the system (6.1) becomes equivalent to
g(t)
y
0 1+ t
y
1 0 0
0
1 z = 0 ,
0 1 0 z + 0
f (t)
u
1
t
0
u
0 0 0
(6.3)
which is of the form (0.1). Prove that this system has differentiation index 3.
4. Using the approach of the present chapter, apply the implicit Euler method to the system
(6.3). Is the resulting discretization equivalent to (6.2)?
5. Consider the differential-algebraic equation
y = (ez1 + 1)/2,
0=yt
with consistent initial values y(0) = 0 and z(0) = 1. Prove that we are concerned with a
problem of index 2, and the corresponding manifold is M = {(t, y) ; y t = 0}.
Prove that the implicit Euler method, applied to this problem with starting approximation
y0 = h and z0 = 1 does not have a solution.
Remark. This exercise shows that a numerical method for index 2 problems may fail if the
initial value is not O(h2 ) close to the manifold.