HAAR WAVELET APPROACH TO

ORDINARY DIFFERENTIAL EQUATIONS

A Thesis

Presented to the

Faculty of

California State Polytechnic University, Pomona

In Partial Fulfillment

Of the Requirements for the Degree

Master of Science

In

Mathematics

By

Robert Chen

2016

 SIGNATURE PAGE

THESIS: HAAR WAVELET APPROACH TO

ORDINARY DIFFERENTIAL EQUATIONS

AUTHOR: Robert Chen

DATE SUBMITTED: Spring 2016

Department of Mathematics and Statistics

Dr. Ryan Szypowski

Thesis Committee Chair
Mathematics & Statistics

Dr. Hubertus von Bremen

Mathematics & Statistics

Dr. John Rock

Mathematics & Statistics

ii

 ACKNOWLEDGMENTS

Thank you Ryan and all my colleagues from Cal Poly Pomona for your support!

This thesis is dedicated to my dearest grandmas.

Hallelujah

感謝Ryan和加州理工大學波莫那分校各位同事的支持!

此論文獻給我最親愛的兩位奶奶。

哈利路亞

iii

 ABSTRACT

The Haar wavelet was one of the first wavelet bases that was developed by Alfred Haar

in 1909. Haar used these functions to give an example of an orthonormal system for

the space of square-integrable functions on the unit interval [0, 1]. We will further study

his method by applying Haar wavelets as a numerical method to solve different types of

ordinary differential equation, such as a higher order ODE and stiff ODE. Moreover, by

comparing Haar’s method to other numerical method such as Heun’s method, RK-45,

and Implicit Euler’s method, we can determine how well the Haar’s method works in

solving ODE.

iv

Contents

1 Overview 1

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Popular Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Wavelets 3

2.1 Introduction to Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.2 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2.3 What are Wavelets? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2.4 Haar Wavelet Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

3 Haar Wavelet and Integration 10

3.1 Discrete Haar Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . 10

3.2 Integration of Haar Wavelet . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Haar Wavelet Method for Solving Differential Equations 17

4.1 Applying Haar wavelets to Differential Equations . . . . . . . . . . . . 17

4.2 Numerical Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

5 Conclusion 37

Bibiliography 38

vi

List of Figures

2.1 Time vs Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 h00 (x) → h33 (x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1 H00 (t) → H33 (t) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.1 Haar VS Analytic solution (Example 1: Second order ODE) . . . . . . 20

4.2 Haar VS Analytic: Error (Example 1: Second order ODE) . . . . . . . 21

4.3 Haar VS Analytic VS Heun: Solution (Example 1: Second order ODE) 28

4.4 Haar VS Heun: Time, Error (Example 1: Second order ODE) . . . . . . 29

4.5 time vs error: Haar and Heun (Example 1: Second order ODE) . . . . . 30

4.6 Haar VS ode45 (Example 2: Forth order ODE) . . . . . . . . . . . . . 31

4.7 Error between Haar and ode45 (Example 2: Forth order ODE) . . . . . 32

4.8 m vs error, time vs error: Heun and Haar (Example 3: Second order stiff

ODE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.9 Haar vs Implicit Euler vs Analytic (Example 3: Second order stiff ODE) 34

4.10 m vs error, m vs time, Haar and IEM (Example 3: Second order stiff ODE) 35

4.11 Time vs Error: Stiff (Example 3: Second order stiff ODE) . . . . . . . . 36

vii

List of Tables

4.1 Haar VS Analytic: Error (Example 1: Second order ODE) . . . . . . . 19

4.2 Error between Haar and ode45 (Example 2: Forth order ODE) . . . . . 24

4.3 Haar vs Analytic, Haar vs Implicit Euler: Error (Example 3: Second

order stiff ODE) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

viii

Chapter 1

Overview

1.1 Introduction

Numerical methods for solving Ordinary Differential Equations (ODEs) have been well

studied throughout the century, yet people are using different methods for different types

of questions. The main factor that determines how good a method is comes down to

the calculation time and accuracy. If a method solves ODEs within hundredth, or even

thousandth, of a second, but gives tremendous error, one might consider other ways. On

the other hand, if a method gives accuracy with error less than 10−6 but takes a day to

calculate, one might consider this method inefficient as well. In the following chapters,

we will introduce the Haar Wavelets and apply it as a numerical method to solve different

types of ODEs.

1.2 Popular Applications

A wavelet is a mathematical function useful in signal processing and image compression.

Wavelets, or the wavelet transformation, convert an image into a series of mathematical

expressions, usually matrices, which requires less space to store. To view the compressed

image, one must decode the file and apply the inverse transform. By applying a discrete

Haar wavelet transform, we can use the output to identify the edges in the image. In

term of signal processing, wavelets are often used to denoise signals. Though image

compression and signal processing are two main application of wavelets, we will discuss

wavelets in numerical methods for solving differential equations.

Chapter 2

Wavelets

2.1 Introduction to Wavelets

The Fourier transform is a useful tool when we want to analyze a signal in the frequency

domain. However, if we take the Fourier transform over the whole interval, we can not

identify specifically at what instant the frequency rises. The short-time Fourier transform

uses a sliding window to find spectrogram, which gives information on both time and

frequency, but the length of window limits the resolution of frequency. With the wavelet

transform, we can change the scale to have it analyze the signal at different scale.

2.2 Orthogonality

Why is orthogonality an important property when constructing a basis? For example, if

{v1 , v2 , v3 } is a basis for R3 , we can write any v ∈ R3 as a linear combination of v1 , v2 ,

and v3 in a unique way; that is v = a1 v1 + a2 v2 + a3 v3 where a1 , a2 , a3 ∈ R. While we

know that a1 , a2 , a3 are unique, we do not have a way of finding them without doing

some explicit calculations. On the other hand, if {w1 , w2 , w3 } is an orthonormal basis for

R3 , we can write any v ∈ R3 as

v = (v · w1 )w1 + (v · w2 )w2 + (v · w3 )w3 .

In this case, we have an explicit formula for the unique coefficients in the linear combi­

nation.

The Fourier basis,

b = {1, cos(x), sin(x), cos(2x), sin(2x), cos(3x), sin(3x), ...} (2.1)

is a orthogonal basis of L2 (−π, π); that is,

π
f (x)g(x)dx = 0, for any f (x), g(x) ∈ b, and f (x) = g(x). (2.2)
−π

Thus, we have a simple way to calculate the Fourier coefficients an and bn ,

1 π nπx
an = f (x) cos dx, n ≥ 0,
π −π p
(2.3)
1 π nπx
bn = f (x) sin dx, n ≥ 1.
π −π p
A Fourier series of a function is
∞
a0
s(x) =
+ ∑ [an cos(nx) + bn sin(nx)] (2.4)
2 n=1

In the later part of this chapter, we will construct the Haar basis and see that it is also an

orthogonal basis.

2.3 What are Wavelets?

Wavelets are defined in terms basis functions, with two characteristics, location and scale.

In general, a wavelet basis function is represented by the function

1 t −b
φa,b (t) = √ φ , (2.5)
a a

where a is the scale parameter that determines the width of the function, and b is the

location parameter that determines where we are localized, or where the support is. The

function φ is called mother wavelet. To understand more about how wavelets works,

we will demonstrate with a time-signal cartoon (Fig. 2.1). For a wavelet function with

small a, it has a narrower window that gives nice time localization and can capture high

frequency. Then we slide that window across time by varying b to capture all the high

frequencies first. After we have done that, we will remove the high frequencies we have

captured and make a larger. When a is large, it has a wider window and can capture low

frequencies, but we lose some of the time localization. We keep making a larger every

time until it captures the whole graph.

The graph on the right of fig. 2.1 gives another elaboration to this. In order to capture

high frequency, we will need large time steps. After that, we decrease the number of

time steps which captures lower frequencies but less time localization. Then repeat the

process until the time step covers the whole signal. We will start by building the Haar

wavelets.

2.4 Haar Wavelet Basis

The first orthogonal basis of wavelets was developed by a Hungarian mathematician Al­

fred Haar. [3] Up until this day, Haar wavelets still remain one of the most used wavelets

in the field of signal and image processing. The construction of these functions are rela­

tively simple. In this construction, we assume the interval of interest is [0, 1]. Equation

(2.6) is the general form of a Haar wavelet, i denotes the generation of the function, also

known as the scaling parameter. Every generation cuts the unit interval into 2i−1 steps. j

denotes the location of the function, where i ∈ N and j = 0, 1, . . . , 2i−1 − 1.

Figure 2.1: Time vs Frequency

 hi j (x) = 2(i−1)/2 h10 (2i−1 x − j), (2.6)

with

h00 (x) = 1, x ∈ [0, 1],

and
⎧
⎪
⎪
⎪
⎪ 1 for 0 < x < 12
⎨
h10 (x) = −1 for 12 < x < 1 .
(2.7)

⎪
⎪
⎪
⎩
0 otherwise

We can see that h00 and h10 are orthogonal by multiplying them together and integrat­
1 1
ing from 0 to 1. Integrating from 0 to 2 we get 1, and from 2 to 1 we get -1 (Fig. 2.2). To

continue building the next wavelet functions, we will compress h10 (x) horizontally and

stretch the function vertically,

√
⎧
1
⎪ 2 for 0 < x < 4
⎪
⎪
⎪
√
⎨ √
h20 (x) = 2h10 (2x) −
2 for 14 < x < 12 (2.8)

⎪
⎪
⎪
⎩
0
⎪
otherwise

√
⎧
⎪
⎪
⎪ 2 for 12 < x < 34
√

⎪
⎨ √
h21 (x) = 2h10 (2x − 1) −
2 for 34 < x < 1 (2.9)

⎪
⎪
⎪
⎩
0
⎪
otherwise

We continue to do the same to get wavelet functions with higher resolution (Equation

(2.10)).
h30 (x) = 2h10 (4x)

h31 (x) = 2h10 (4x − 1)

(2.10)
h32 (x) = 2h10 (4x − 2)

h33 (x) = 2h10 (4x − 3)

 As we keep constructing hi j as i grows big, the interval on which hi j is nonzero,

called the support, gets narrower and narrower. At some point, we can throw them away

at a minimal loss of information. For example, h6k in equation (2.11) has a step size of
1 1 1
32 . If we keep repeating the procedure, h15k would have a step size of 215
= 32768 at

which we may ignore without losing resolution, depending on the application.

√
h4k = 2 2h10 (8x − k) k = 0, 1, . . . 7

h5k = 4h10 (16x − k) k = 0, 1, 2, . . . 15 (2.11)

√
h6k = 4 2h10 (32x − k) k = 0, 1, 2, . . . 31
By the construction of Haar wavelets, these functions are orthogonal to each other. That

is, ⎧
⎨ 0, α = β
⎪
< hα (x), hβ (x) >= hα (x)hβ (x)dx = δαβ = (2.12)
⎩
1, α = β
⎪

Figure 2.2: h00 (x) → h33 (x)

Chapter 3

Haar Wavelet and Integration

3.1 Discrete Haar Wavelets

The use of orthogonal functions to construct operational matrices for solving identifi­

cation and optimization problems of dynamic systems was initially established in 1975,

when the Walsh-type operational matrix was constructed by Chen and Hsiao [2]. The

main idea of this technique is to convert a differential equation into an algebraic equa­

tion, and hence the solution process will be much simplified.

In section 2.4, we have been using x as the independent variable, but since time is

the more used variable in terms of dynamic systems, we will use t as our independent

variable. The fundamental idea starts from integrating a matrix φ (t), and we will approx­

imate it using the following,

t
φ (τ)dτ ∼
= Pφ (t) (3.1)
0

10

where
⎡ ⎤
φ (t)
⎢ 0 ⎥
⎢ ⎥
⎢ φ1 (t) ⎥
φ (t) =
⎢ ⎥ .

⎢ ⎥
..
.
⎢ ⎥
⎢ ⎥
⎣
⎦

φm−1 (t)
φi (t)’s are the basic orthogonal functions on a certain interval [a, b], which in the case of

Haar wavelets, the interval is [0, 1]. And matrix P is uniquely determined based on the

orthogonal functions φi (t).

In order to solve differential equation models of dynamic systems, usually one needs

to perform integration to solve the problem. Let us consider the Haar wavelet function

from section 2.4, and the integration of these functions (Fig. 3.1). Define Hi j (t) by
t
Hi j (t) = hi j (τ)dτ (3.2)
0

We also know, from earlier, by the construction of Haar wavelets, all of them are or­

thogonal to each other (Equation (2.12)). Functions with this property are well suited as

a transform basis. Given any function y(t) ∈ L2 (0, 1), or square integrable function y(t)

over the interval [0, 1], y(t) can be expanded as a series of Haar wavelets [2]:

y(t) = co h00 (t) + c1 h10 (t) + c2 h20 (t) + c3 h21 (t) + . . . (3.3)
1
ci = 2i y(t)hi (t)dt (3.4)
0

Similar to Fourier series, the more terms we have equation (3.3), the better our approxi­

mation is going to be. The first four Haar wavelets can be expressed as the following:

h00 (t) = [1 1 1 1]

h10 (t) = [1 1 −1 −1]

(3.5)
h20 (t) = [1 −1 0 0]

h21 (t) = [0 0 1 −1]

11

Figure 3.1: H00 (t) → H33 (t)

12

The number of entries in each function from (3.5) signifies the interval [0, 1] is cut into 4

equal pieces, for instance, h20 (t) = [1 −1 0 0] represent the wavelet function that is 1

from t = 0 to t = 41 , -1 from t = 1
4 to t = 21 , 0 from t = 1
2 to t = 43 , and 0 from t = 3
4 to

t = 1. We obtain the Haar matrix of 4th order when we combine everything in equation

(3.5) into a matrix:

⎡ ⎤
1 1 1 1

⎢ ⎥
⎢ ⎥
⎢1 1 −1 −1
⎥
⎥
∼
H4 (t) = ⎢
⎢
⎥ (3.6)

⎢1 −1 0 0
⎥
⎢ ⎥
⎣
⎦

0 0 1 −1

Example
We want to transform a piecewise constant function y(t) = [9 1 2 0]. The Haar coefficient

can be found using equation (3.4), but here we want to propose an alternative method.

We want to transform y(t) such that

y(t) = cT H4 (t) (3.7)

where c is the Haar coefficient vector. It is very convenient to find c using matrix inver­

sion:
⎡ ⎤
1 1 2 0

⎢ ⎥
⎢ ⎥
1⎢⎢1 1 −2 0
⎥
⎥
H4−1 (t) = ⎢ (3.8)

4
⎢ 1
⎥
−1 0 2
⎥
⎥
⎢
⎣
⎦

1 −1 0 −2

cT = y(t)H4−1 = [3 2 4 1]. (3.9)

Therefore, we have the transformation

y(t) = 3h00 (t) + 2h10 (t) + 4h20 (t) + h21 (t) (3.10)

13

3.2 Integration of Haar Wavelet

One tool that is often used to solve differential equation models of dynamic systems is

integration. The technique we are introducing is using a similar approach. We will start

by looking at the integration of the first four Haar wavelets:

t 1
h00 (t)dt = t, 0 ≤ t < 1 [1 3 5 7] (3.11)
0 8
⎧ ⎫
t ⎨
t, 0 ≤ t < ⎬
1

⎪ 1 ⎪
2
h10 (t )dt = [1 3 3 1] (3.12)
0 ⎪ ⎭
8
⎩
1 − t ,
1 ≤ t < 1 ⎪
2
⎧ ⎫
1 ⎪

⎨
t, 0≤t <
t
⎪
4
⎬ 1

h20 (t )dt = [1 1 0 0] (3.13)

0 ⎩
1 − t,
⎪ 1 1
≤ t < ⎭

⎪ 8
2 4 2
⎧ ⎫
⎨
t − 1 , 1
≤ t < 4 ⎬

3⎪
t
⎪
2 2 1

h21 (t)dt = [0 0 1 1] (3.14)

0 ⎩
1 − t,
3 ≤ t < 1 ⎭

⎪ ⎪ 8
4

By writing equation (3.11) to (3.14) together in a matrix, we obtain

⎡ ⎤
1 3 5 7

⎢ ⎥
⎢ ⎥
t 1 ⎢ 1 3 3 1
⎥
⎢
H4 (t)dt (3.15)

⎥
8
⎢ 1 1 0 0
⎥
⎢ ⎥
0
⎢ ⎥
⎣
⎦

0 0 1 1

Let the integrals be expanded into Haar series, such that,

t
H4 (t)dt = P4 H4 (t) (3.16)
0

gives gives
t
P4 = H4 (t)dtH4−1 (t) (3.17)
0

14

and
⎡ ⎤
4 −2 −1 −1

⎢ ⎥
⎢ ⎥
1 ⎢ ⎢ 2 0 −1 1
⎥
P4 = (3.18)

⎥
2 × 4
⎢ 1/2 1/2 0
⎢ ⎥
0
⎥
⎥
⎢
⎣
⎦

1/2 −1/2 0 0

By following similar procedure, we can build another integration matrix, P8 , with higher

resolution:
⎡ ⎤
8 −4H1
⎢ ⎥ ⎡ ⎤
⎢ −1 ⎥
⎢ 4H1
1
⎢ 0 −2H2 −H4 ⎥ 1
⎢ 16P4 −H4 ⎥
P8 =
⎢ ⎥ =
⎣
(3.19)

⎥
16
⎢ ⎥ 16

⎢ 4H2−1 0
⎥ H4−1 0

⎣
⎦

−1
H4 0

In general for an mth-order system with m = 2 j , j ∈ N, the Pm is given as

⎡ ⎤
1 ⎢ 2mPm/2 −Hm/2 ⎥
Pm = ⎣
(3.20)

2m
H −1
⎦

m/2 0

A general proof of the construction of Pm can be found in [2]. Here, we will verify Pm

for m = 1, 2, 4, 8, .... The Haar matrix Hm is defined by m Haar functions in row vector

form as in equation (3.6). Specifically,

⎡ ⎤
h (t)
⎢ 00 ⎥
⎢ ⎥
⎢ h10 (t) ⎥
Hm (t) = ⎢ (3.21)

⎢ ⎥
.. ⎥
.
⎢ ⎥
⎢ ⎥
⎣
⎦

h j, j−1 (t)

H1 = [1], H1−1 = [1] (3.22)

⎡ ⎤ ⎡ ⎤
⎢ 1 1
⎥ 1 ⎢ 1 1
⎥
H2 =
⎣
⎦
,
H2−1 = (3.23)

2
1 −1

⎣
⎦

1 −1

15

The operation matrix Pm is the Haar transform coefficient matrix of these integrals de­

fined in equation (3.16).

⎡ ⎤
t
Pm =
⎣
Hm (τ)dτ ⎦ Hm−1 (3.24)

⎡
⎤

1 1 ⎢ 2 −1
⎥
P1 = [ ], P2 = ⎣

2 4 1 0

⎡ ⎤
8 −4 −2 −2

⎢ ⎥
⎢ ⎥
1 ⎢ 4 0 − 2 2
⎥
P4 =
⎢ ⎥
16 ⎢ 1 1 0 0
⎥
⎢ ⎥
⎢ ⎥
⎣
⎦

1 −1 0 0

⎡ ⎤
⎢ 32 −1 −8 −8 −4 −4 −4 −4
⎥ (3.25)

⎢ ⎥
⎢ 16 0 −8 8 −4 −4 4 4 ⎥
⎢ ⎥
⎢ ⎥
⎢ 4 4 0 0 −4 4 0 0 ⎥
⎢ ⎥
⎢ ⎥
⎢ ⎥
1 ⎢ 4 −4 0 0 0 0 −4 4 ⎥
P8 =
⎢ ⎥
64 1 1 2 0 0 0 0 0
⎢ ⎥
⎢ ⎥
⎢ ⎥
⎢ ⎥
⎢
⎢ 1 1 −2 0 0 0 0 0 ⎥
⎥
⎢ ⎥
⎢ ⎥
⎢ 1 −1 0 2 0 0 0 0 ⎥
⎢ ⎥
⎣
⎦

1 −1 0 −2 0 0 0 0

16

Chapter 4

Haar Wavelet Method for Solving

Differential Equations

4.1 Applying Haar wavelets to Differential Equations

Let us consider a linear system

ẋ(t) = Ax(t) + Bu(t), x(0) = x0 (4.1)

with n states x(t) and p inputs u(t) that can be described in the state equation with prop­

erly dimensioned matrices A and B. Assume that u(t) is square integrable in the interval

0 ≤ t < 1. It can be expressed as a Haar series expansion

u(t) = GH(t) (4.2)

where G is p × m, m being the size of matrix H which determines the resolution of the

Haar matrix (3.6). G can be obtained from equation (3.4) for different i as a vector of

coefficients of the Haar wavelets.

17

Furthermore, we also want to expand ẋ(t) into a Haar series:

ẋ(t) = FH(t) (4.3)

After integration, we have

t t
x(t) = ẋ(τ)dτ + x0 = F H(τ)dτ + x0 = FPH(t) + x0 . (4.4)
0 0

Since P, H, and x0 are known, if we can solve for F, then we have x(t). By substituting

equation (4.2), (4.3), and (4.4) into equation (4.1), we obtain

FH(t) = AFPH(t) + Ax0 + BGH(t)

⇒ [F − AFP]H(t) = {[Ax0 , 0, 0, . . . , 0] + BG}H(t) (4.5)

⇒ F − AFP = [Ax0 , 0, 0, . . . , 0] + BG ≡ G1

The zero entries in H and P will greatly simplify the solution processes. Equation (4.5)

can also be expressed as

A−1 F − FP = A−1 G1 . (4.6)

For nonsingular matrix A, this is a Lyapunov matrix equation [2]. By using lyap(A−1 , −P, A−1 G1 )

from MATLAB, we can obtain F directly with A and G1 being given.

4.2 Numerical Analysis

MATLAB has a sparse function that converts a full matrix to sparse form by squeezing

out any zero elements. This function will save up a lot of computation time due to the

large amount of zero entries in matrix P and H.

Example
Consider the differential equation

y'' + 4y' + 3y = 1 (4.7)

18

with initial condition

y(0) = 0, y' (0) = 1.

In order to apply the Haar’s method, we want to represent equation (4.7) as an equation

of matrices.
⎡ ⎤

⎢y⎥
Let x =
⎣
⎦,
we can rewrite equation (4.7) as

y'
⎡ ⎤ ⎡ ⎤
⎢ 0 1
⎥ ⎢0⎥
ẋ =
⎣
⎦
x +
⎣
⎦
u(t) (4.8)

−3 −4 1

with u(t) = GH. (Equation (4.2))

The analytic solution is

1 e−3t
y(t) = −
3 3
First, we want to know how well Haar’s method works in terms of error. Though there

are many ways to calculate error using different norms, we will be using the infinity norm

to calculate the error, e.

||e||∞ = max{e}

Graph 4.1 gives some idea of how fit the Haar’s method is as we increase m by a power

of 2. Table 4.1 shows the error between the Haar’s method and the analytic solution at

different value of m. As we increase m from 2 p to 2 p+1 , the error tends to drop by half,

this means the Haar’s method for solving differential equation is first order.

m 4 8 16 32 64 128

Error 0.0909 0.0526 0.0286 0.0149 0.0076 0.0039

Table 4.1: Haar VS Analytic: Error (Example 1: Second order ODE)

19

Figure 4.1: Haar VS Analytic solution (Example 1: Second order ODE)

20

 Figure 4.2: Haar VS Analytic: Error (Example 1: Second order ODE)

However, Haar’s method is still a first-order. There are other methods that has higher

order than the Haar’s method, for example, Heun’s method is a second-order RK method.

Let us compare Haar’s method and Heun’s method. Figure 4.3 compares the Haar’s

method and Heun’s method to the analytic solution. Graphically, Heun’s method is a little

better than Haar at m = 8. Let us compare the error and time between these two methods.

Figure 4.4 shows the time and error difference between Haar and Heun’s method. In

terms of time as m increases, Haar remains fairly fast because as m increases, the matrices

P and H are much ”sparser”, or has more zero entries. Sparse matrix multiplication in

Matlab is still fast even when the size of the matrices increases. Whereas the time taken

for Heun’s method increases in a somewhat linear fashion.

21

 The graph on the right of figure 4.4 shows the error for both methods as m increases.

When m = 8, the error is very close for both methods, but as we increase m, we can see

that Heun’s method has an advantage. Heun’s error decreases much faster than Haar’s as

m increases. This is due to the order of these two methods.

It is hard to compare the efficiency of these two methods. Haar is better in computa­

tion time, but worse in error reducing. In order to better analyze the two methods, let’s

look at time and error together.

In figure 4.5, the Haar’s method is better if the user is looking for an accuracy of

at most 10−4 . From the figure, we can see that Haar and Heun are going to cross at

somewhere below Error= 10−4 . Therefore, if we want a better accuracy than 10−4 ,

Heun’s method is more efficient than Haar’s method.

22

Example
Now let’s look at a higher order differential equation:

y(IV ) + 2y''' − 3y'' + 3y' − 5y = k sint (4.9)

with initial condition

y(0) = 0, y' (0) = 1, y'' (0) = 0, y''' (0) = 0.

Similarly,
⎡
we ⎤
want to rewrite equation (4.9) as an equation of matrices.
y
⎢ ⎥
⎢ ⎥
⎢ ' ⎥
⎢y ⎥
Let x =
⎢ ⎥, we can rewrite equation (4.9) as

⎢ '' ⎥
⎢y ⎥
⎣
⎦

y'''
⎡ ⎤ ⎡ ⎤
0 1 0 0 0

⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎢0 0 1 0 ⎥ ⎢ 0
⎥
ẋ =
⎢ x +
⎢ ⎥ u(t) (4.10)

⎢ ⎥ ⎢ ⎥
⎥
⎢0 0 0 1 ⎥ ⎢ 0
⎥
⎢ ⎥ ⎢ ⎥
⎣
⎦
⎣
⎦

5 −3 3 −2 k

with u(t) = GH. Since now we have k sint on the right hand side of equation (4.9), matrix

G will be the Haar coefficients of the function sint, the constant k is captured in equation

(4.10). To do this, we pick the mesh, m, first, and then take the value value of sint and

the endpoint of every mesh to get the function value in a vector, W , and find G using

G = W Hm−1 (4.11)

The analytic solution takes a long time to obtain, instead, we will compare Haar’s

method to MATLAB’s built-in solver ode45. Since ode45 is of medium order, we should

be able to use ode45 as an analytic solution to this question.

23

 m 4 8 16 32 64 128

Error(Haar - ode45) 0.1245 0.0625 0.0312 0.0156 0.0078 0.0039

Table 4.2: Error between Haar and ode45 (Example 2: Forth order ODE)

We can see from figure 4.6 again that Haar’s method works nicely when m is large.

It is very close to what ode45 does as m increases. Table 4.2 the maximum different

between Haar’s method and ode45 at different values of m. The difference is halved

every time we double m, therefore, again, concluding that the Haar’s method is of order

1.

24

Example
A stiff ordinary differential equation often poses troubles for explicit method such as

Heun’s method. The solution can be unstable unless if the step size is taken to be very

small. The idea is that some terms in the equation lead to a rapid variation in the solution,

such as vanishing at a fast rate. For example,

y'' + 1001y' + 1000y = 0 (4.12)

with initial condition y(0) = 1, y' (0) = 0 is a stiff ODE problem with analytic solution
1
y= (1000e−t − e−1000t ). (4.13)
999
The term e−1000t decreases to 0 much faster than e−t , and that will be a trouble when we

try to solve this using Heun’s method.

From left of figure 4.8, we can see that when error of Heun’s method increases rapidly

when m increases until m ≈ 103 . Whereas the error for Haar’s method is consistently

small when compared to Heun’s method. Right of figure 4.8 shows that Haar’s method

is definitely a better choice than Heun’s method in terms of run time and accuracy. How­

ever, since Heun’s method is explicit, we know Heun’s method is not our first choice

when it comes to solving stiff ODE, it would not be fair to compare Haar’s method to

Heun’s.

Implicit Euler’s method (IEM), or backwards Euler’s method, is similar to the stan­

dard Euler’s method, but differs in that it is an implicit method. Implicit methods find

a solution by solving an equation involving both the current state of the system and the

later one. For instance, the forward, or standard, Euler’s method

1
(yi+1 − yi ) = Ayi
h
yields

yi+1 = (I + hA)yi

25

for i = 0, 1, 2, ...

Whereas, the backwards, or implicit, Euler method

1
(yi+1 − yi ) = Ayi+1
h

yields

yi+1 = (I − hA)−1 yi

for i = 0, 1, 2, ... Implicit Euler’s method has order one and is L-stable, which makes it a

good choice when we want to solve a stiff equation.

Based on figure 4.9, Haar’s method and implicit Euler’s method seem to work fine for

this stiff equation even when m is small. Unlike Heun’s method where the error is huge

if m is not big enough. Now, it would be more reasonable to compare Haar’s method and

IEM in details.

m 4 8 16 32 64 128

Haar Error 0.1102 0.0579 0.0294 0.0145 0.0073 0.0038

Implicit Euler Error 0.0418 0.0219 0.0112 0.0057 0.0029 0.0014

Haar Error/IEM Error 2.6394 2.6453 2.6177 2.5456 2.5565 2.6398

Table 4.3: Haar vs Analytic, Haar vs Implicit Euler: Error (Example 3: Second order

stiff ODE)

Table 4.3 tells us that IEM is definitely a better choice in terms of minimizing error at

different step sizes. Haar Error is about 2.6 times Implicit Euler Error, which is somewhat

significant. Now let’s look at the graph 4.10, Haar’s time curve is below Euler’s curve

for all possible values of m. Here we can really see the power of MATLAB’s sparse

mode on saving computational time. IEM is better in accuracy whereas Haar’s method

26

is fast in time. In order to further compare these two methods, figure 4.11 shows both

methods in terms of their time and error, although IEM is good on accuracy, but when

we want efficiency, Haar’s method is a better one. Haar’s method can be more accurate

without significant additional computation time. On the other hand, if we can one more

digit of accuracy with IEM, it takes 10 times longer to go from a 10−2 to 10−3 . Since

both methods are first order, Haar’s method has the upper hand of applying its method in

MATLAB with sparse mode to reduce a lot of the computation time needed.

27

Figure 4.3: Haar VS Analytic VS Heun: Solution (Example 1: Second order ODE)

28

Figure 4.4: Haar VS Heun: Time, Error (Example 1: Second order ODE)

29

Figure 4.5: time vs error: Haar and Heun (Example 1: Second order ODE)

30

Figure 4.6: Haar VS ode45 (Example 2: Forth order ODE)

31

Figure 4.7: Error between Haar and ode45 (Example 2: Forth order ODE)

32

Figure 4.8: m vs error, time vs error: Heun and Haar (Example 3: Second order stiff

ODE)
33
Figure 4.9: Haar vs Implicit Euler vs Analytic (Example 3: Second order stiff ODE)

34

Figure 4.10: m vs error, m vs time, Haar and IEM (Example 3: Second order stiff ODE)

35

Figure 4.11: Time vs Error: Stiff (Example 3: Second order stiff ODE)

36

Chapter 5

Conclusion

The Haar wavelet orthogonal functions and their integration matrices have been intro­

duced to solve higher order ODE and stiff ODE. The sparsity of matrices P and H brings

down the computational time dramatically thanks to the sparse function in MATLAB.

The method is of first order; however, due to the fast computational time, we can make

step sizes smaller to get better accuracy without adding any significant amount of com­

putational time as seen in several numerical examples.

In a theoretical point of view, Haar’s method is simply derived and proved. Out of

many well-known wavelets, Haar’s is considered the simplest one. This method has two

main advtages: (i) it is suitable for non-stiff and stiff ODE. Regardless of the types of

problem, Haar’s method is stable in terms of error reducing versus step sizes. (ii) Fast

computational time as mentioned earlier. This approach of Haar’s can also be extended

to other wavelets, such as Daubechies’ wavelets. Daubechies wavelets are much more

complicated in terms of construction compared to the Haar wavelets. Also an orthogonal

basis, Daubechies wavelets can be applied in a similar way.

37

Bibliography

[1] Chun-Lin, Liu, A Tutorial of the Wavelet Transform, 2010

[2] C. F. Chen, C. H. Hsiao, Haar wavelet method for solving lumped and distributed-

parameter systems, IEEE, 1997

[3] P. van Fleet, Discrete Wavelet Transformations: An Elementary Approach with Ap­

plications, 1st edition, Wiley 2008.

[4] J.P. Gabardo Orthogonal Functions and Fourier series, McMaster University, 2009.

[5] L. Sadun, Haar Wavelet, 2014.

[6] David M. Bressoud, A Radical Approach to Lebesgue’s Theory of Integration,

MAA, 2008.

38