Hans-Georg Stark Wavelets and Signal Processing

Hans-Georg Stark
Wavelets and Signal Processing
An Application-Based Introduction
With 67 Figures and 7 Tables
123
Professor Dr. Hans-Georg Stark FH Aschaffenburg - University of Applied Sciences
FB Ingenieurwissenschaften Würzburger Str. 45 63743 Aschaffenburg Germany hans-ge
[email protected]
Library of Congress Control Number: 2005921923
ISBN 3-540-23433-0 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or
part of the material is concerned, speci cally the rights of translation, reprint
ing, reuse of illustrations, recitation, broadcasting, reproduction on micro lm or
in any other way, and storage in data banks. Duplication of this publication or
parts thereof is permitted only under the provisions of the German Copyright La
w of September 9, 1965, in its current version, and permission for use must alwa
ys be obtained from Springer. Violations are liable to prosecution under the Ger
man Copyright Law. Springer is a part of Springer Science+Business Media springe
ronline.com © Springer-Verlag Berlin Heidelberg 2005 Printed in The Netherlands Th
e use of general descriptive names, registered names, trademarks, etc. in this p
ublication does not imply, even in the absence of a speci c statement, that such n
ames are exempt from the relevant protective laws and regulations and therefore
free for general use. Typesetting: By the author Production: LE-TEX Jelonek, Sch
midt & Vöckler GbR, Leipzig Cover design: medionet AG, Berlin Printed on acid-free
paper 7/3142/YL - 5 4 3 2 1 0
To Fabi and Paula
Preface
“The idea of this book arose in a conversation with H. A. Bethe, who remarked that
a little book about modern eld theory which contained only memorable results wou
ld be a good thing”1 . As a graduate student working in theoretical physics, in pa
rticular desperately needing a readable and compact treatment of quantum eld theo
ry, I came across a beautifully written book: “PCT, Spin and Statistics and all th
at” [35]. Here we are dealing with signal processing instead of quantum eld theory
and the present book does not claim to reach a level of clarity and deepness com
parable to the respective achievement in [35]. Nevertheless, the basic intention
of this book perfectly is described by the above quotation when replacing “ eld the
ory” with “wavelet analyis”: It should provide a quick and readable introduction to th
e wavelet topic for anyone interested in learning about wavelets and/or working
with them and hopefully will contain only memorable results. What does the pro le
of the “ideal reader” of this little book look like? The largest portion of the cove
red material should be readable for anyone with an elementary technical backgrou
nd and familiar with basic notions of calculus like integrals and complex number
s. Thus these parts should be accessible for readers with an engineering or scie
nce education on a bachelor level who are interested in a short and compact intr
oduction into wavelet applications. The target group includes academia and (in p
articular) practitioners in industry. Some parts of the book are of interest mai
nly for readers with a specialization on, e.g., electrical engineering or commun
ications engineering. For these parts basic familiarity with notions from signal
analysis like Fourier transforms and digital ltering will be helpful. Elementary
information about these topics is collected in the appendix. There are some exc
ellent introductory books on wavelets and wavelet applications mainly focussing
on the discrete wavelet transform (cf., e.g. [38]).
1
Quotation from the preface of [35]
VIII
Preface
This book will contain a fairly large part dealing with the continuous transform
, since in my opinion the continuous transform provides a very intuitive insight
into the essence of wavelet techniques and since the continuous version is clos
e to the Fourier transform, a standard tool in engineering. As with all books, s
election and presentation of the material re ect the author’s view of the topic. My
hope is that after reading the book the reader will have gained an intuitive ins
ight into what wavelets are. This insight should then provide a basis on which t
o decide, whether wavelets are interesting for the particular needs of the reade
r and, if yes, how he or she wants to apply them. Therefore this book is intende
d to serve as an introductory guide to the topic. Moreover, the wavelet story is
a story about a fascinating and exciting scienti c development. If this fascinati
on can be felt during reading, this is among the best I can hope for. It is a pl
easure to thank Eva Hestermann-Beyerle and Monika Lempe from Springer-Verlag for
their patience and professional support. Their comments had substantial in uence
on sharpening the pro le of this book. The work reported on in section 4.2 would n
ot have been possible without the collaboration and support of Gernod Laufk¨tter,
Andreas Divivier and Thomas o Goseberg, my friends and colleagues at tecmath AG.
I am very grateful for the opportunity to work with them. Dr. Hans-J¨ rgen Stahl
checked the English of chap. 1, a copyeditor from u Springer-Verlag did so with
the complete manuscript. I am indebted to both for valuable hints which helped m
e to improve spelling and style.
Ascha enburg, spring 2005
Hans-Georg Stark
Contents
1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 1 1.1 Signals and Signal Processing . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 1 1.2 Local Analysis . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Transf
orms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 2 1.2.2 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 3 1.2.3 Short Time Fourier Transform (STFT) . . . . . . . . . . .
. . . . 4 1.2.4 Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 4 1.2.5 Visualization . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 7 1.2.6 Fourier vs. Wavelet Transform - A Com
parison Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 9 1.3 A Roadmap for the Book . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 11 Continuous Analysis . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 The Short Time Fourier
Transform (STFT) . . . . . . . . . . . . . . . . . 2.1.1 De nition, Computation an
d Reconstruction . . . . . . . . . . 2.1.2 Phase Space and Localization Paramete
rs . . . . . . . . . . . . 2.1.3 Implementation with MATLAB and Visualization .
. . . . 2.2 The Continuous Wavelet Transform (CWT) . . . . . . . . . . . . . . .
. . 2.2.1 De nition, Computation and Reconstruction . . . . . . . . . . 2.2.2 Wav
elet Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.3 Implementation with MATLAB and Visualization . . . . . 2.2.4 Application:
Detection of Signal Changes . . . . . . . . . . . . . 2.3 Case Studies . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
.3.1 Analysis of Sensor Signals . . . . . . . . . . . . . . . . . . . . . . . .
. . 2.3.2 Analysis and Classi cation of Audio Signals . . . . . . . . . . 2.4 Note
s and Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . The Discrete Wavelet Transform . . . . . . . . . . . . . . . . . . . .
. . . . . . . 3.1 Redundancy of the CWT and the STFT . . . . . . . . . . . . .
. . . . . . 3.2 The Haar-System . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 3.2.1 Continuous-Time Functions . . . . . . . .
. . . . . . . . . . . . . . . . . 13 14 14 18 19 21 21 26 29 32 33 33 36 40 43 4
3 45 46
2
3
X
Contents
3.2.2 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 3.3 Generalization to Daubechies-Wavelets . . . . . . . . . . .
. . . . . . . . . . 3.3.1 From Filters to Functions . . . . . . . . . . . . . .
. . . . . . . . . . . . 3.3.2 Transfer Properties . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 3.4 Multiscale Analysis . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 One-Dimensional Sign
als . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Two-Dimensional
Signals (Images) . . . . . . . . . . . . . . . . . . . 3.4.3 Implementations wit
h the MATLAB Wavelet Toolbox . . 3.4.4 Generalization: Biorthogonal Filters . .
. . . . . . . . . . . . . . . 3.5 A Unifying Viewpoint: Basis Systems . . . . .
. . . . . . . . . . . . . . . . . 3.5.1 One-Dimensional Signals . . . . . . . .
. . . . . . . . . . . . . . . . . . . 3.5.2 Two-Dimensional Signals . . . . . .
. . . . . . . . . . . . . . . . . . . . . 3.5.3 Computation and Visualization wi
th MATLAB . . . . . . . 3.6 Case Studies . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Energy Compaction and Com
pression . . . . . . . . . . . . . . . . 3.6.2 Denoising a Sensor Signal / Real-
Time Properties of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 3.7 Notes and Exercises . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 4
49 53 56 59 60 61 65 69 73 75 76 79 81 81 81 88 91
More Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 95 4.1 The Transform Compression Scheme . . . . . . . . . .
. . . . . . . . . . . . . 95 4.1.1 The General Procedure . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 97 4.1.2 Entropy Coders . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 99 4.1.3 Optimal Quantization and
Examples . . . . . . . . . . . . . . . . . 108 4.1.4 MATLAB Implementation . .
. . . . . . . . . . . . . . . . . . . . . . . . 113 4.2 Wavelet-Based Similarity
Retrieval in Image Archives . . . . . . . . 116 4.3 Notes and Exercises . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 125 5.1 Fourier Transform and Uncertainty Relation . . . . .
. . . . . . . . . . . 125 5.2 Discrete Fourier Transform (DFT) . . . . . . . .
. . . . . . . . . . . . . . . . . 128 5.3 Digital Filters . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 5.4 Solution
s to Selected Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
34 5.4.1 Problems from Sect. 2.4 . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 134 5.4.2 Problems from Sect. 3.7 . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 138 5.4.3 Problems from Sect. 4.3 . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 142 5.5 Notations and Symbols . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 146
5
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 147 Index . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 149
1 Introduction
1.1 Signals and Signal Processing
Wavelet analysis had its origins in the mid-eighties. From the very beginning it
was driven by application needs: The desire to analyze seismic signals more sen
sitively than with Fourier techniques led to the rst appearance of the continuous
wavelet transform formula. In parallel it turned out that the new technique cou
ld be applied successfully to certain problems in theoretical physics as well as
in pure mathematics. For one of the earliest collections of research and survey
papers documenting the state of the art the reader is referred to [6]. It soon
turned out that wavelet analysis successfully could be applied to many types of
signal processing problems: In signal analysis the detection of discontinuities
or irregularities was tackled with wavelets. The analysis of medical signals lik
e electrocardiograms of the heart is one of the rst reported examples of disconti
nuity detection (see [6]). For more applications, like the analysis of sensor si
gnals in robotics, cf. sect. 2.3.1. Signal compression is another impressive exa
mple of wavelet applications. JPEG 2000, the present version of the internationa
l standard on still image compression is based on wavelet techniques (see, e.g.,
[36]). Wavelet applications both in signal analysis and signal compression shal
l be described in more detail in later sections. This chapter serves as a brief
introduction to the main features of the wavelet transform by comparing wavelet
transform with Fourier transform, the standard tool of signal analysis. For that
purpose we shall work out the common aspects of wavelet and Fourier transforms
and point out the main di erences. For understanding the following section, the kn
owledge of the Fourier transform is not a necessary prerequisite. On the other h
and, of course, it would be useful, if the reader already had some experience wi
th applications of the Fourier transform. Basic facts about the Fourier transfor
m are collected in the appendix, sections 5.2 and 5.3.
2
1 Introduction
Mathematical symbols, used throughout this book, are explained upon their rst app
earance. They are collected in sect. 5.1 of the appendix.
1.2 Local Analysis
In this section we will deal with signals which may be represented by a function
f (t) depending on time t. We shall assume that t is a continuously varying par
ameter; thus f (t) is called a “continuous-time signal”. We shall try to transform f
(t) into a representation, which encorporates the desired information about the
signal as compactly as possible. The Fourier transform (cf. sections 1.2.2 and
1.2.3) supplies information about the contribution of certain frequencies to the
signal, the wavelet transform (cf. sect. 1.2.4) indicates whether details of a
certain size are present in a signal and quanti es their respective contribution.
Both transforms are called “local” if they not only globally measure frequencies and
detail sizes, respectively, but also indicate where they are located in the sig
nal f (t). There are many applications for the kind of signal information descri
bed above – we explicitly mention signal classi cation and data compression. These a
pplications are described in more detail later, in the subsections below we indi
cate how frequencies and detail sizes may be measured. Furthermore, we will work
out the aspects which are common to both transforms and illuminate the respecti
ve di erences. The transform results shall be visualized and we will give an examp
le which serves as an illustration for the above-mentioned compactness of the re
spective signal representations. The purpose of this chapter is to introduce the
ideas underlying Fourier and wavelet transforms, respectively. For more - in pa
rticular for mathematical - details the reader is referred to the following chap
ters. 1.2.1 Transforms All transforms of the signal f (t) described in this sect
ion share a common computation principle: The signal is multiplied with a certai
n “analysis function” and integrated about the time domain. In a symbolic notation t
he prescription for performing a transform reads
+∞
f (t)
transf orm
−→
f (u)g(u) du
−∞
(1.1)
The “analysis function” g(u) characterizes the chosen transform. In general it may b
e a complex function, the overline denotes the complex conjugate entity. g(u) in
a certain way depends on the parameters, i.e. frequencies or detail sizes, to b
e measured (see below). Thus, by the computation principle given above the trans
formed entity will depend on these parameters. In other
1.2 Local Analysis
3
words: the transformed entity again will be a function. These functions we shall
denote with “transform” or “transformed signal”. Another common aspect of all transform
s discussed in this section is invertibility: From the transformed signal the or
iginal signal f (t) may be reconstructed. This is essential for understanding th
e comparison experiment carried out in sect. 1.2.6. 1.2.2 Fourier Transform The
parameter relevant for the Fourier transform is the circular frequency ω, the anal
ysis function reads gω (u) = ejuω . Thus the transformed signal is a ˆ function depend
ing on ω and it is denoted ith f (ω):
+∞
ˆ f (ω) =
−∞
f (u)gω (u) du
(1.2)
Figure 1.1 illustrates the above computation recipe by plotting both curves ˆ requ
ired for computing f (π). The signal is shown as a solid curve, the real
art of t
he analysis function gπ (u) = ejuπ is dashed. Obviously, it is an harmonic oscillati
on with circular frequency ω = π.
4 f(u) 3
2 Re(g (u)) π 1
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.1. Fourier transform: signal and analysis function
ˆ Why does f (π) measure the a

earance of ω = π in the signal? The qualitative argument
 is as follows: If in some time interval the signal oscillates with circular fre
quency ω = π, the signal and the analysis function have a
4
1 Introduction
constant mutual
hase shift in this interval and therefore
rovide a nonzero ˆ con
tribution to f (π). Yet there is no
ossibility to localize the a

earance of the
circular freˆ quency: If (the absolute value of) f (π) is “large”, we only know that the
signal contains the circular frequency π, but we do not know where it a

ears, si
nce the analysis function extends over the whole real axis. The only label
aram
eterizing the analysis function is circular frequency. 1.2.3 Short Time Fourier
Transform (STFT) This transform sometimes also is called “Windowed Fourier Transfo
rm” (WFT). The STFT looks for the a

earance of the circular frequency ω at a certai
n time t. The corresponding analysis function reads: g(ω,t) (u) = ejuω (u − t). Here
(u) is a “ indo function”, usually centered about the origin (for an example see b
elo ). In the expression (u − t) this indo is shifted to the desired time t. No
the transformed signal depends on ω and t! Since it also ill depend ˆ on the shap
e of the indo function, it is denoted ith f (ω, t):
+∞
ˆ f (ω, t) =
−∞
f (u)g(ω,t) (u) du
(1.3)
For a box indo of idth 2, centered symmetrically about 0, ω = π and t = 8, the
com
utation
rinci
le is illustrated in Fig. 1.2. Again the dashed curve shows t
he real
art of the analysis function g(π,8) (u); it is obviously now localized at
t = 8, since w(u − 8) denotes the box window, shifted by 8 units to the right. In
general, the analysis function will be localized at the res
ective “analysis time”
t . Therefore the transform
rovides not just global information about the a

ea
rance of a certain circular frequency, but in addition the time of this a

earan
ce. The
rocedure described so far has a disadvantage: If in the above exam
le o
ne is interested in small details of the signal around t = 8, the corres
onding
frequency of the analysis function must be increased. As an exam
le Fig. 1.2 is
redrawn for ω = 6π in Fig. 1.3. Obviously the window width is constant and non ada
t
ive: If one is interested in very tiny signal details (high frequencies) in only
a small neighborhood of t = 8, eventually signal
arts, which actually are “not o
f interest”, also will be analyzed. Zooming into small details  analogously to a
microsco
e  is not su

orted. 1.2.4 Wavelet Transform The wavelet transform has
such a zooming
ro
erty. In contrast to the Fourier transform, the wavelet tran
sform does not look for circular frequencies but
1.2 Local Analysis
4 f(u) 3
5
2 Re(g 1
(π,8)
(u))
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.2. STFT: signal and analysis function for ω = π
4 f(u) 3
2 Re(g 1
(6π,8)
(u))
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.3. STFT: signal and analysis function for ω = 6π
rather for detail sizes a at a certain time t. Instead of detail sizes we also w
ill s
eak of “scale factors”, both notions will be used equivalently. As mentioned a
lready, high frequencies corres
ond to small details and vice versa, thus, when
com
aring wavelet with Fourier transforms we have to take into account that freq
uencies and detail sizes are inversely
ro
ortional to each other: There exists
a constant β such that
6
1 Introduction

β . (1.4) ω We shall no rie y indicate, ho the avelet transform is computed. Consi
der a (real or complex) analysis function g, oscillating around the a=
+∞
u-axis (mathematically:
−∞
g(u) du = 0) and decreasing ra
idly for u → ±∞.
Such a function is called a “wavelet”. In eq. 1.4, relating scale factors with frequ
encies, the constant β depends on g. Starting from g consider the follo ing family
of functions: g(a,t) (u) = 1 √ g u−t . The members of this family are generated fro
m g by shifting the a a function to t followed by shrinking (a < 1) or dilating
(a > 1) the width of the function. The wavelet transform now reads:
+∞
Lg f (a, t) =
−∞
f (u)g(a,t) (u) du
(1.5)
For the “Haar wavelet”
⎧ ⎨1 0 ≤ u < 1 2 g(u) = −1 1 ≤ u < 1 2 ⎩ 0 else
1 2
the com
utation of Lg f (a, t) with a =
4 f(u) 3
and t = 8 is illustrated in Fig. 1.4.
2 g((u−t)/a) 1
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.4. Wavelet transform: signal and analysis function for a =
1 2
The reader may note that the Haar wavelet, originally situated in the interval [
0, 1) now has been shifted to the right by 8 units and its width has
1.2 Local Analysis
7
shrunk by the factor 1 , corres
onding to the chosen values of t and a. For 2 a
= 1 and t = 8 we obtain Fig. 1.5. 4
4 f(u) 3
2 g((u−t)/a) 1
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.5. Wavelet transform: signal and analysis function for a =
1 4
Com
are Figs. 1.4 and 1.5 with Figs. 1.2 and 1.3 and note that the wavelet trans
form shows the desired zooming
ro
erty in contrast to the STFT: When searching
for smaller and smaller details (higher and higher frequencies) with the wavelet
transform, the corres
onding analysis function is oscillating faster and is con
tracted. 1.2.5 Visualization ˆ Both the STFT fw (ω, t) andthe avelet transform Lg
f (a, t) are functions depending on t o varia les. A suita le visualization of t
hese functions is of essential importance in signal analysis. A ide-spread grap
hical representation of t o-dimensional functions is the use of contour lines. I
n signal analysis one usually
 prefers the visualization of the a solutevalues o
f the respective transforms y gray values. High values are coded ith right, l
o values
 ith dark gray values. Figure1.6 sho s such a visualization
 for the S
TFT (a ove) and the avelet
 transform ( elo ). As a signal in oth cases the “chir
p” f (t) = sin(t2 ) has een used. The chirp is an harmonic oscillation sin(ωt), ho
se circular frequency
 increases ith t: ω = t. The linear increase of frequency is
clearly visi le ith the STFT (see the upper part of Fig. 1.6). Since (cf. eq.
1.4) detail size a and frequency ω are inversely proportional ith respect to each
other, for the avelet transform one ould expect a
8
1 Introduction
f(t)=sin(t2). Parameters: δ =5,k
x min
=1,δ =1,k
k
max
=50,σ=0.2
0.5 0 −0.5
0
1
2
3
4
5
6
7
8
9
10
0 5 10 15 20 25 30 0 1 2 3 4 5 t 6 7 8 9 10 ω
f(t)= in(t2). Scale parameter : a tart=1,δa=1,a top=150.
0.5 0 −0.5
0
1
2
3
4
5
6
7
8
9
10
0.2 0.4 0.6 a 0.8 1 1.2 1.4 0 1 2 3 4 5 t 6 7 8 9 10
Fig. 1.6. Above: STFT of the chirp ignal f (t) = in(t2 ). Belo : Wavelet tran 
form of the chirp ignal f (t) = in(t2 ).

behavior corre pon ing to a hyperbola (i.e. proportional to  1 ). The lo er  t par
t of Fig. 1.6 ho  exactly thi  behavior. Since  the STFT epen  on t an ω, the
gray value co ing of the STFT ha  been performe
 on the tω plane. Thi  plane i a
l o calle “pha e plane”, the corre pon ing gray value co ing “pha e pace
 repre entat

ion” of the STFT.Analogou ly the t a plane i  calle “ cale plane” an the corre pon
ing gray value co ing of the avelet tran form “ cale pace repre entation” of the
avelet tran form.
1.2 Local Analy i 
9
1.2.6 Fourier v . Wavelet Tran form  A Compari on Experiment In a certain en e
, the zooming property
 of the avelet tran form en ure  that characteri tic feat
ure  of the analyze ignal on a certain cale are ell repre  ente  by the tran 
form value  corre pon ing to thi  cale factor, i.e. not i tribute  among other
cale factor . Moreover, the e tran form value
  ill be localize at the re pec
tive ignal part , here the above mentione feature   are pre ent. The e concent

ration propertie   both ith re pect to cale an time  may be formulate math
ematically in a more rigorou  ay; the purpo e of thi  ection i , to give a pla
u ibility argument for the above tatement  by performing
 a compari on experiment
ith the Fourier tranform. The ignal i playe in  Fig.  a ection from
1.7 i
the
 beginning of an au io ignal.
 Roughly in the mi le, the u en tart of ou
n clearly can be recognize .
0.15
0.1
0.05
0
−0.05
−0.1
−0.15
−0.2
−0.25 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Fig. 1.7. “Attack ignal”

Such ignal  (“attack
 ignal ”) hall locally, i.e. in a mall neighborhoo of the p
oint, here the oun tart , contain high frequencie , equivalently, there ill
be ra tic change  on a mall  cale.
 In uch a ituation the
 zooming property o
f the avelet tran form houl be a vantageou  hen compare ith the Fourier  tr
an form. To con rm thi  conjecture, the follo ing experiment ha  been carrie out:
1. Compute the Fourier tran form of the ignal, keep tho e 4% of the value  of
 
the tran forme ignal, having the large t ab olute value . Put the remaining
 tr
an form value  equal to zero
 an recon truct the ignal from thi  mo
 i e tran for
m (remember that, a  tate in ect. 1.2.1, all tran form  i cu e here are in
vertible).
10

1 Intro uction
 
2. Perform the ame proce ure ith the avelet tran form in tea of the Fourier
tran form. The re ult  of the experiment are ho n in Fig. 1.8. The a he curve 
ho  the re ult of the Fourier
 recon truction, the avelet recon truction i 
i playe by + ymbol , the oli line repre ent  the original
 ignal. When compa
ring ith Fig. 1.7, ob erve that the curve  ho an enlarge ection of the ign
al from Fig. 1.7 in a neighborhoo of the point here the oun tart .
0.1
0.05
0
−0.05
−0.1
−0.15
−0.2
−0.25 3680 3700 3720 3740 3760 3780 3800 3820 3840
Fig. 1.8. “Attack ignal”: recon truction 
Obviou ly, hen u ing the Fourier tran form, the uppre ion of 96% of the tran 
form ignal value   namely
 tho e tran
 form ignal value  having lo er ab olute
value
  than the retaine one  lea  to a global moothing (lo pa  ltering) an
therefore the local peak  uring the attack phae are not repro uce any more. 
 t, applying the
In contra  ame uppre ion proce ure to the avelet  tran form
oe  not i turb the repro uction of the e peak . Thi  i  a clear in ication for
the above mentione concentration
  propertie  of the avelet tran form value .A 
a nal remark e houl in icate that  the original ignal
 a  not repre ente a 
a continuou time ignal, but it a  i cretely ample . For uch ignal  there
exi t variant  of the continuou  formulae
 1.2 an 1.5, re pectively, ith hich 
the above experiment ha  been carrie out. The e “ i crete tran form ” hall be e c
ribe in later ection . Moreover,  a  ith any avelet tran form application, th
e re ult
 of the above experiment epen  on the cho
 en avelet  g. There ult  i
playe in Fig. 1.8 have been obtaine u ing the b4  avelet e cribe in a late
r ection.
 
1.3 A Roa map for the Book
11

1.3 A Roa map for the Book    
The topic  ketche in thi  intro uction ill  be e cribe in more etail in cha
p. 2. Que tion  like “ hat i  an optimal ino function  for the Short Time Fourie 
r Tran form?”  or “ho may vi ualization  a  i playe in Fig. 1.6 be accompli he ?” i
ll be treate  there. Moreover, e ill in icate ho the original ignal may be r
econ tructe from the re pective tran form . Thi  i  of crucial importance for a
pplication  like ignal compre ion: There, tran form  of the original ignal  u
ually are compute in a r t tep. Sub equently the e tran   form  are mo  in uch
i e
a ay that the require torage  pace for the tran forme ignal i  re uce con
i erably.
  Finally, in the “ ecompre ion”  tep the ignal  are recon tructe from t
he mo i e tran form . In chap. 2 al o a brief urvey i  given  of typical ignal a
naly i  application  of the Short Time Fourier  Tran  form an the avelet
 tran fo
rm, re pectively. T o in u trial ca e tu ie  are e cribe in more etail. Sect
ion 2.4 contain  ome exerci e . Partly the e exerci e  ill be of “paper an pen 
type”, but they al o ill con i t in riting  computer  program  . A  a programming
  p
latform e u e the tool MATLAB, hich i  i e prea an a e facto tan ar in t
he engineering community. Of cour e it i  po ible to u e the book ju t a  a ref 
erence gui eto avelet technique  ithout performing  any programming.
  For rea e
r  intere te in u ing the oft are e cribe in thi  book an in eveloping  the
ir o n program
  , ho ever, the u  e of MATLAB ill be a prerequi
 ite. Rea er  havi
ng alrea y ome familiarity ith programming language  an looking for a compact
ly ritten  intro uction  to MATLAB are referre to [12]. Thi  book provi e  a ver
y nice an e cient e cription of MATLAB’  main feature . It i  ritten  in German; i
f thi  turn  out to be an ob tacle,  reference [30]
 i  highly recommen e . The MA
TLAB oft  are hich
  i  i cu e in thi  book an
 ha  been ritten by the author
may be o nloa e from . pringeronline.com/ e/3 540 23433 0 Mo t of it requi
re  only the ba i  ver ion of MATLAB. Some program , ho ever, make u e of the MA
TLAB Wavelet Toolbox,
 a collection of aveletrelate
 ignal proce ing algorithm

 . It  u e i  e cribe in the u er’  gui e [24],  hich i  not ju t a oft are han
book. It i  moreover a beautifully ritten intro uction to avelet technique .
Since in the author’    opinion MATLAB together ith the avelet toolbox ill turn o
ut to be a tan ar oft are platform for avelet  relate ignal proce ing, the
pre ent book al o ill provi e a hort intro uction to the mo t important compo
nent  of the MATLAB Wavelet Toolbox. In chapter  1 an 2, re pectively,  ignal 
are con i ere tobe continuou   time ignal , even though  computerize ver ion  o
f the algorithm  e cribe there nece arily involve a i cretization. A i erent
per pective i  given in chap.  3: Here from the very  beginning  ignal are icre
te equence  of number  an the avelet tran form e cribe there i  e igne fo
r uch equence .
12

1 Intro uction
 
The corre pon ing notion , avelet con truction , ignal tran form an recon tru
ction formulae are given in thi  chapter. Mo t important for practical  applicati
on  i  the exi tence of fat algorithm  both for tran formation an recon tructi
on. They are al o e cribe in thi chapter together ith MATLAB  implementation 
. Thereafter again application  an ca e tu ie  are pre ente . In thi  context
e hall alo comment on real time propertie  of fa t avelet
 algorithm
  . Sectio
n 3.7 provi
 e  exerci e . Chapter 4 i  evote to a more etaile e cription of
ome a itional application  of the avelet tran form. Fir t e hall focu  on
the mo t popular
 application, namely ata compre ion. Sub equently, an applicat 
ion relate to retrieving image  from ataba e management ytem  i  e cribe .
Again at the en of thi  chapter ome exerci e  are pre ente . A  far a  mathema
tic  i  concerne , e a ume ome familiarity of the rea er ith ba ic calculu 
like integral calculu  an complex number . We hall give no rigorou  proof  of
mathematical tatement , rather e ant to provi e ome intuitive in ight intot
he e ence of the e tatement  an their practical meaning. More mathematical  e
tail  relate to backgroun
  an application  of avelet  are collecte in the
 ap
pen ix. It i  inten e to upport the rea  er, if  he or he feel  that ome a it
ional information  oul be helpful to un er tan the main bo y of the text. More
over,
 the
 appen ix contain  olution   to electe  problem  from the exerci e  an
provi e  a li t of frequently u e ymbol  an notation .
2 Continuou  Analy i 
 
In chap. 1 e intro uce  local tranform  of continuou time ignal . The e tran
form  no ill be tu ie in more etail. In particular the  concept of pha e p
ace localization ill lea to the election of a proper in o function for the
STFT. Moreover, e ill ketch fa  t algorithm  an their implementation in MATLA
B for computing both the STFT an the continuou   avelet tran form.  For both tra
nform  recon truction
 formulae ill be
  provi e . Finally t o in u trial ca e t
u ie  are pre ente . The r t ca e tu y eal  ith application  of the avelet tr
an form an the STFT, re pectively, to the analy  i  of ignal  occurring at a (l
ight) arc el ing  proce . The econ ca e tu y e cribe , ho the above tran f
orm may be u e for cla ifying  au io ignal
  occurring at certain  in pection p
roce ure  in the  automotive
 in u try. Rea er  primarily intere
  te in applicatio
n  may r t tu y the e nition part in ection    2.1.1 an 2.2.1, re pectively, the
n kip the re t of the e ection  an procee irectly to ection  2.2.2,2.2.4
an 2.3. Throughout
 thi  ection the continuou time ignal to be analyze hall
be enote ith f (t). For  completene  e al o require f (t) to have nite energ
y (cf. ect. 5.1). A  note there, e entially all practically  relevant ignal 
ful ll thi  requirement.
 When numerical algorithm  an application enter the pict
ure, e ill not eal ith  continuou  time ignal  any more. In tea , e hall c
on i er equence  obtaine by ampling thecontinuou time ignal  f (t) for t =
0, TS , 2TS , . . . , (N− 1)TS . Here TS enote  the “ ampling i tance” an the equ
ence element  are enote ith fk := f (kTS ) (k = 0,  . . . , N −1). Note, that in
the engineering community al o f [k] i  u e in tea of fk . Putting everything
together, e hall a here to the follo ing notation: f = {fk }N −1 = {f (kTS )}N −1
k=0 k=0 (2.1)

 hall u e the ymbol f both to repre ent either a continuou time ignal or a
We
i cretely ample ignal. The actual meaning al ay  hall be
14
2 Continuou  Analy i 
 
clear from the re pective
 context.
 Our notation  concerning i cretely ample 
ignal  are complete ith the e nition  for the ampling rate νS a d the correspod

ig circular frequecy ωS : 1 TS 2π ωS := TS νS :=
(2.2) (2.3)
2.1 The Short Time Fourier Trasform (STFT)
2.1.1 De itio, Computatio ad Recostructio De itio As described i sect. 1.2.
3, the STFT performs a local frequecy aalysis by shiftig a ido fuctio
to time t ad subsequetly computig the Fourier trasform (cf. eq. 5.1) of the
product of the sigal ad the ido :
+∞
ˆ f (ω, t) =
−∞
f (u) (u − t)e−jωu du
(2.4)
As idicated, this formula is also valid he the ido fuctio is complex val
ued. I this case, the overlie deotes complex cojugatio. The ˆ subscript i
the expressio f (ω, t) idicates the depedecy of the trasformed sigal o the
chose ido fuctio . Usig the Fourier trasform correspodece pairs itr
oduced i sect. 5.1, eq. 2.4 also may be ritte as ˆ f (t) (t − τ ) ◦ − • fw (ω, τ ) (2.5)
      
This equa ion is he key  boh for designinga fas algorihm forcompu  ing he  S
TFT and for recons ruc ing he  signal from he
 STFT. For
 he res  of his sec io
n basic knowledge
 of Fourier
 ransforms and he discre
 e Fourier ransform as pr
ovided in heappendix, sec ions  5.1 and 5.2, respec ively, is required.
 Readers

primarily
 in eres
 edin applica ions  may proceed from here o sec . 2.3.1 and
hen con inue wi  h sec . 2.2.1.
 Compu
 a ion Given a sampled signal  {f (kTS )}N −1  (
cf. eq.
  2.1), he ask is o compu e a k=0 ˆ sampled version of he STFT, i.e. o
compu e he sequence {fw (ω, kTS )}N −1 . k=0
2.1 The Short Time Fourier Trasform (STFT)
15
It ill tur out, that a fast algorithm ca be give for suitably sampled ωvalues.
We describe o both the algorithm ad the restrictios uder hich it is valid
. f Assume f ad to be bad limited ad let ωmax ad ωmax deote the maximum fre
quecies of f ad , respectively (cf. sect. 5.2). Aalogously, letωmax de
ote th
e maximum frequecy of f (t) (t − τ ). Then, as implied already by he no a ion, ωmax
ideed does ot deped o τ and, moreover,
f w ωmax ≤ ωmax + ωmax .
(2.6)
Without goig ito details, e metio that this iequality essetially is a co
sequece from the fact that the spectra of f ad are covolved ith each other
. No de e ωS correspodig to eq. 2.3 ad assume that the samplig is such that th
e iequality
f ωS > 2(ωmax + ωmax ).
(2.7)
is ful lled. The eq. 2.6 implies that ωS > 2ωmax . Thus,
 the
   
 Sha o co ditio 5.12
 i
s ful lled
 for f (t) (t − τ ).
 Correspondence
  pair 2.5
  oge her wi h heorem 5.1 hen
leads o he following ˆ Fas algori hm for he compu a ion of fw (ω, t): 1. De e the
frequecy samplig ωk = here mN =
N 2 N −1 2
k ωS (k = 0, . . . , mN ) N
(2.8)
(N eve) (N odd)
(2.9)
  
2. For τ = 0, TS , 2TS , . . . , (N − 1)TS perform he following  s eps: a) De  ne he s
equence {f (kTS
 )w(kTS
 − τ )}N −1 and compu e k=0 i s DFT via he FFT algori hm. Le
he DFT be deno ed wi h {Gk }N −1 . k=0 b) Then
 ˆ fw (ωk , τ ) = TS Gk (k = 0, . . . , mN
). Some commen
 s are in place: 1. Usually he sampled signal – in par icular he
sampling ra e ωS – is give. Thus, if the Shao coditio 5.12 is true for f , e
q. 2.7 implies that ωmax should be as small as possible i order to avoid distorti
os by aliasig e ects (cf. sect. 5.2). (2.10)
16
2 Cotiuous Aalysis
2. The remarks made after theorem 5.1 apply also here. Sice itely sampled siga
ls ever are bad limited i the strict sese, at least the spectra of f ad m
ust be “su cietly small” outside the respective itervals f f [−ωmax , ωmax ] ad [−ωm
ax ] such that eq. 2.10 is approximately true. Reco structio I this subsectio
  
e shall cosider the questio ho to recostruct the origial cotiuous time 

 
ite e ergy sig al f (t) from the tra  
 sformed sig al ˆ f (ω, τ ). Again we s ar from
correspondence
   pair 2.5. Applying he Fourier inversion formula 5.2 o 2.5 we ob
ain 1 f ( )w( − τ ) = 2π
+∞
ˆ fw (ω, τ )e+jωt dω
−∞
  o both sides ith (t − τ ) and inegraing wih respec
Multiplyi g this equatio
 
o τ we ob ain for he lef hand side:
+∞ +∞ 2
  
f ( )|w( − τ )| dτ = f ( )
−∞ −∞

|w( − τ )|2 dτ
+∞

= f ( )
−∞
|w(u)|2 du
        
In proceeding
 from
 he rs o he second line he subs i u ion u = − τ has been mad
e. For he righ hand side we ob ain 1 2π
+∞ +∞

ˆ fw (ω, τ )w( − τ )e+jωt dωdτ
−∞ −∞
     
Equaingbo h sides and resolving for f ( ) we nally ob ain he “con inuous STFT re
cons ruc ion formula”
+∞ +∞

f ( ) =
−∞ −∞

ˆ fw (ω, τ )w( − τ )e+jωt dωdτ
+∞
(2.11) |w(u)|2 du
2π
−∞
Note that the denominator of this formula does only de
end on the chosen window
function w. Thus it may be
recom
uted and stored. But even then
2.1 The Short Time Fourier Transform (STFT)
17
for
ractical a

lications the usefulness of eq. 2.11 is limited. For sam
led si
gnals and the corres
onding STFT the numerical a

roximation of the double integ
ral in the numerator by some quadrature formula leads to an unacce
table com
uta
tional e ort. Instead, assume that the sam
ling {f (kTS )}N −1 (cf. eq. 2.1) of a co
ntinuousk=0 time nite energy signal f (t) is given, which ful lls eq. 2.7. Assume,
moreover,
 that thealgorithm
 described in eqs. 2.8 – 2.10 has been performed, resu
l ing in he compu a ion of 1 mN ˆ ωS ) f (ω, τ ) (τ = 0, TS , . . . , (N − 1)TS ; ω = 0,
. . . , N N ith mN give by eq. 2.9. The e use the correspodece pair eq. 2.
5 agai to recostruct {f (kTS
 )}N −1 as follo s: k=0 Fast STFT recostructio:
k 1. For a give τ de ne wi h ωk = N ωS (k = 0, .. . , mN ) the seˆ quece Gk := T1 f (ωk

, τ )(k = 0, . . . , N − 1), where he coe S cien s Gk (k = mN + 1, . . . , N − 1) resul
from
 
GN −1 = G1
 , GN −2 = G2 , . .. (cf. sec . 5.2). 2. Then apply he inverse DFT (eq. 
5.14) o his sequence, resul ing in he sequence {f (kTS )w(kTS − τ )}N −1 . Schema i
cally: k=0 {Gk }N −1 −→ {f (kTS )w(kTS − τ )}N −1 k=0 k=0
IDF T
!
!
(2.12)
     
3. Perform he s eps above for a sui able subse {τi }I of τ values,  i=1 such ha τ1
= 0, τI  = (N −1)TS and w(kTS − τi ) is nonzero  for any kTS in he in erval [τi , τi+1 ].
hen on hese in
 ervals f (kTS ) may be recons ruc
 ed by dividing he sequence ob
ained above
 hrough w(kTS − τi ). We remark ha  he signal can be recovered
 by per
forming
 s eps
 1 and 2 above for every
  τ value. S ep 3 aims a reducing
 he compu
a ional e or , since he IDFT compu  a ionneeds o be carried ou only
 fora subse
 of τ values. MATLAB implemen
 a ions of he STFT algori hms presen ed in his sec
ion are discussed in sec . 2.1.3.
18

2 Con inuous Analysis
    
2.1.2
 Phase Space
 and
 Localiza ion Parame ers  In his sec ion weadop and deepe
n he viewpoin
 on he STFT described
 in sec . 1.2.3.
 The concep of localiza
 io
n parameers in roduced in sec . 5.1
 will lead us o he choice of a cer ain win
dow func ion w. Thus rs we rewri e he STFT formula 2.4 as
+∞
ˆ fw (ω, t) =
−∞
f (u)g(ω,t) (u) du
(2.13)
here g(ω,t) (u) = ejuω (u − t) (2.14)
(cf. sect. 1.2.3). Assume that the ido fuctio has the follo ig localizatio
 parameters (for the correspodig de itios cf. eqs. 5.6 – 5.9): t = 0, ω = 0, ∆t
, ∆ω . The it is a easy exercise to verify that the localizatio parameters of g(ω,
t) read tg(ω,t) = t, ωg(ω,t) = ω, ∆tg(ω,t) = ∆t , ∆ω g(ω,t) = ∆ω . The correspodig cell
e plae (cf. sect. 5.1) are dra  i Fig. 2.1.
t
∆ω
ω ∆t
?
Fig. 2.1. Cells i phase plae, occupied by (solid) ad g(ω,t) (dashed) , respec
tively.
Thus, the trasitio = g(0,0) → g(ω,t) correspods to a pure shift of the cell i
the phase pla e. Shape ad size remai uchaged! Obviously, a optimal resoluti

o both i time ad frequecy ca be reached if ∆t ad ∆ω both are as small as poss
ible. Havig the fast STFTalgorithm 2.8 – 2.10 i mid, e remark that coditio 2
.6 is aother strog argumet for havig a as small as possible ∆ω value.
2.1 The Short Time Fourier Trasform (STFT)
19
Because of the Heiseberg ucertaity priciple 5.10 ∆t ad ∆ω caot both get arbi
trarily small. Therefore the optimal choice ith respect to time a d frequecy r

esolutio is the Gauss fuctio g give i 5.11. It miimizes the product ∆t ∆ω . T
he STFT ith the correspodig ido
t2 1 (t) = √ e− 2 . 4 π
(2.15)
is also called “Gabor transform”. 2.1.3 Im
lementation with MATLAB and Visualization
For this section we assume the reader to have basic familiarity with MATLAB. In

articular, knowledge of matrix and vector mani
ulations and functions like fft
(f) (com
utation of the DFT of the sequence f) and ifft(F) (com
utation of the I
DFT of the sequence F) is required. We turn now to a short descri
tion of a MATL

   
AB im lementation  ofthe STFT algorithm
 2.8– 2.10. Subsequen ly, he implemen a i
on of he recons ruc ion algori   hm leading
 o 2.12 is discussed. The correspondi
ng MATLAB  m  les have been  es ed wi h MATLAB 6.5,release
 13 and may be downloade
d from he  URL given
 in sec
  . 1.3. Fas
  STFT Compu a ion The algori hm is implem

en ed in he func ion  mys f . A pro o ype call
  of his func
 ion reads [ ,y,ma ri
x]=mys f (signal,T,s
 ep ,ommin,sepom,ommax,  ex ); We rs give  a shor explana io
n of heinpu parame ers. The s ring  ex allowsfor a commen o be included i
n he i le of he graphics genera edby he func ion. The  sampled signal  is rep
resen ed by he vec or signal,  T
 deno es he sampling dis
 ance, ommin, s epom, o
mmax are minimum  index, s epwid h and maximum index in he frequency sampling 2.
8, for which he ˆ desired fw (ω, t) values are computed. Thus, i particular, the u
ser must take care of the restrictio ommax ≤ mN ith mN de ed by 2.9. N deotes th
e sigal legth. I order to reduce the computig e ort, stept as itroduced. For
, e.g., stept=3, steps 2a) ad 2b) of the algorithm are performed oly for every
3rd value of the time vector ˆ t=(0:(N 1))*T. The f (ω, t) values for itermediate
times are obtaied by replicatig the last computed values. ˆ As a result of the
algorithm the computed f (ωk , ti ) values are stored i the matrix matrix, ith
k umberig ro s ad i umberig colums. This
20
2 Cotiuous Aalysis
matrix is visualized as, e.g., i Fig. 1.6 usig the imagesc fuctio. Together
ith the sigal y ad the time vector t it is retured by the fuctio. The basi
c computatio ad replicatio code is listed belo : %Loop for icreasig time fo
r idex=1:stept:N, y=feval( ido ,(t t(idex))/sigma).*sigal; colum=coj((fft(
y))’); colum=colum(ommi:stepom:ommax); ifill=mi(stept,N idex+1); for i=1:ifil
l, matrix(:,idex+i 1)=colum; ed ed The ido fuctio must be provided as a
fuctio le, its ame is stored i the strig ido , sigma deotes a additioa
l user cotrol of the ido idth. Fast STFT Recostructio The correspodig f
uctio reads myistft. A prototype call: [t,reco,matrix]=myistft(matrix,T,stept)
; Here matrix deotes the result of a STFT computatio ith mystft. Sice for t
he recostructio algorithm the complete frequecy samplig 2.8 is required, i
the precedig mystft call oly the sigal, the samplig distace ad the time i
cremet should be speci ed, sice the complete frequecy samplig the is computed
by default. Moreover, for the STFT computatio ad the recostructio obviously
the same ido fuctio ad correspodig idth must be used, mystft ad myist
ft request these etities from the user. stept deotes the legth of the recost
ructio itervals i multiples of the samplig time T (cf. step 3 of the recost
ructio algorithm). It is recommeded to choose it idetical ith the time step
parameter i the precedig mystftcall, provided the shifted ido values do ot
vaish o this iterval (cf. step 3 of the recostructio algorithm). Thus a ty
pical dialogue is as follo s: >> [t,y,matrix]=mystft(sigal,.017,10); Eter id
o (i quotes): ’gauss’ Wido idth: 1 >> [t,recostft,matrix]=myistft(matrix,.017,1
0); Eter ido (i quotes): ’gauss’ Wido idth: 1 >> For more implemetatio det
ails the reader is referred to the source le.
2.2 The Cotiuous Wavelet Trasform (CWT)
21
2.2 The Cotiuous Wavelet Trasform (CWT)
2.2.1 De itio, Computatio ad Recostructio De itio The (cotiuous) avelet t
rasform has bee listed already i sect. 1.2.4, formula 1.5. We shall o give
the precise formulatio usig the symbol ψ for the wavelet in accordance with the
general use (cf., e.g., [7]). Choose a nite energ function ψ(t) ful lling the “admissi
bilit condition”
∞
cψ := 2π
−∞
ˆ |ψ(ω)|2 dω < ∞. |ω|
(2.16)
     
Any ni eenergy
 func ion sa isfying
 2.16 will
 be called
 a “wavele
 ”. Then he “con inuo
us wavele ransform” (CWT) of he signal f ( ) is deno ed wi h Lψ f (a, t) and read
s
∞
1 Lψ f (a, t) = √ cψ
1 |a|
ψ
−∞
u−t a
f (u) du (a = 0), (t ∈ R).
(2.17)
Again, the overline denotes com
lex conjugation if ψ(t) is com
lex valued. Some re
marks might be a

ro
riate here: 1. When com
aring formula 2.17 with formula 1.5
, note that in the latter formula for sim
licit reasons the constant factor √1 ψ ,
related to the c admissibilit condition 2.16, has been omitted. 2. For
ractica
ll relevant wavelets admissibilit condition 2.16 is ful lled, when
+∞
ψ(t) dt = 0.
−∞
(2.18)
Thus, as stated in sect. 1.2.4, ψ will oscillate around the t axis, since the cont
ributions of
ositive and negative function values to the total area, bounded b
the function gra
h and the t axis, must cancel each other. Since, moreover, ψ(t)
is of nite energ, for t → ±∞ the function ψ(t) will decrease ra
idl. Both facts taken t
ogether ex
lain the term “wavelet” for the function ψ(t). The “Haar wavelet” has alread b
een mentioned in sect. 1.2.4. In sect. 2.2.2 it once more will be treated togeth
er with further wavelet exam
les.
22
2 Continuous Analsis
3. As ex
lained in sect. 1.2.4, the CWT 2.17 is a kind of multiresolution anals
is, since Lψ f (a, t)
rovides information about signal details of size ≈ a. Corres

ondingl, a will be called “detail size” or “scale factor”. As noted in 1.2.4, scale fac
tors and frequencies are inversel
ro
ortional to each other. Since the
ro
ort
ionalit constant in eq. 1.4 de
ends on the wavelet, the correct relation reads βψ .
(2.19) ω Readers maily iterested i applicatios may skip the rest of this sect
io ad proceed from here to sect. 2.2.2 ad the to sect. 2.2.4. The rest of th
is sectio ad sect. 2.2.3 require basic ko ledge of Fourier trasforms ad the
discrete Fourier trasform (cf. appedix, sectios 5.1 ad 5.2, respectively).
Equatio 2.17 may be re ritte as a covolutio: With a= ψa (t) := ψ we obtain 1 Lψ f
(a, t) = √ cψ 1 |a|
∞

− a
(2.20)
ψa (t − u)f (u) du.
−∞
Then from the convolution theorem (eq. 5.5) we ma conclude that 1 Lψ f (a, t) ◦ − • √ cψ 1
|a| ˆ ˆ ψa (ω)f (ω). (2.21)
Similarly to eq. 2.5 i the STFT case, eq. 2.21 is the key both for a fast CWT c
omputatio algorithm ad for recostructig the sigal from the CWT. Computatio
Give a sampled sigal {f (kTS )}N −1 (cf. eq. 2.1), e ill describe o a k=0
algorithm to compute a sampled versio of the CWT. As a result, e ill compute
the sequece {Lψ f (a, kTS )}N −1 for a = 0. We describe now both k=0 the algorithm
and the restrictions under which it is valid. f ψ ψa Assume f and ψ to be band limited
and let ωmax , ωmax ad ωmax deote the maximum frequecies of f , ψ and ψa , res
ectivel
 (cf. sect. 5.2). Analogousl, let ωmax deote the maximum frequecy of Lψ f (a, t)
. Then it is not hard to show that
ψa ωmax =
1ψ ω . |a| max
  
2.2 The Con inuous Wavele Transform (CWT)
23
      
Now assume ha he sampling is such ha he Shannon condi  ion 5.12 is valid
 bo  
h for f ( )and ψa (t), i.e. 1ψ ω . (2.22) |a| max Then he Shannon condi ion au oma
ically is sa is ed for Lψ f (a, t):
f ωS > 2 max ωmax ,
ωS > 2ωmax.
(2.23)
This is true, sice by 2.21 the spectrum of Lψ f (a, t) will vanish if the s
ectru
m of ψa (t) or the s
ectrum of f (t) vanishes. Therefore
f ωmax = mi ωmax ,
1ψ ω |a| max
   
and 2.22 hen implies 2.23. Thus, if 2.22 is valid, he spec ra of f ( ), ψa (t) a


nd Lψ f (a, t) can be com uted with the DFT as described in theorem 5.1, eq. 5.16.
This leads to the following Fast algorithm for the com
utation of Lψ f (a, t) (a
= 0): 1. De ne the sequence {ψa (kTS )}N −1 , where ψa is de ned in eq. k=0 2.20. 2. Com
u
te the DFT of this sequence resulting in a sequence {Ak }N −1 . Moreover, com
ute
the DFT {Fk }N −1 of k=0 k=0 {f (kTS )}N −1 . k=0 3. A

l the IDFT to the
roduct s
equence √ 1 {Fk Ak }N −1 to k=0 obtain {Lψ f (a, kTS )}N −1 . Schematicall: k=0
cψ |a|
  
1 IDF T {Fk Ak }N −1 −→ {Lψ f (a, kTS )}N −1 . k=0 k=0 cψ |a| We conclude his sec ion wi h
some remarks:
(2.24)
  
1.
 The algori
 hm can be performed
 for
 any scale fac or a = 0. Prac ically releva 
n are posi ive scale fac ors; for he sampling considered  here, one usually  ak
es a = TS , 2TS , . . . , (N − 1)TS . 2. Again,
 usually  he sampling
  dis ance TS i
s given
 a priori. Thus for small scale fac ors one mus
 be aware ha
 eq.  i
2.22
s no valid any more. Therefore
  he algori hm inevi
 ably will lead o or ion
dis
s for a → 0. 3. In analogy o he STFT algori hm, he remarks made afer heorem  5
.1 apply also here. Since ni ely sampled signals
 never are band limi ed
   in he s 
ric sense,
 again “maximum
 frequencies”
 mus be unders ood such f f ha he respec
ive spec ra are su cien ly small ou side [−ωmax , ωmax ] ψ ψ and [−ωmax , ωmax ], respectiv
24
2 Cotiuous Aalysis
Recostructio I this subsectio e shall cosider the questio ho to recostr
uct the origial cotiuous time ite eergy sigal f (t) from the CWT Lψ f (a, t).
Similarl to the STFT case we start from the corres
ondence
air eq. 2.21. An e
as calculation shows how the Fourier transform of ψa (t) ma be ex
ressed in term
s of the Fourier transform of ψ. Inserting the result in the corres
ondence
air 2
.21 one obtains Lψ f (a, t) ◦ − • |a| ˆ ˆ ψ(ωa)f (ω). cψ (2.25)
Recall (cf. sect. 5.1) that this relation is onl a shorthand notation for
+∞
Lψ f (a, t)e−jωt dt =
−∞
|a| ˆ ˆ ψ(ωa)f (ω). cψ
(2.26)
The detailed calculations in the following
rocedure are left to the reader. We
shall onl write down the basic ste
s and the ideas behind them: 1. Multi
l bot
h sides of eq. 2.26 with ˆ ψ(ωa) . 3√ |a| 2 cψ and integrate with res
ect to a. Recalling
de nition 2.16 of cψ , the
ur
ose ˆ of this
rocedure is to eliminate all terms de
en
ding on ψ on the righthand side. After the multi
lication and integration
rocedur
e, the new right hand side will read 1ˆ f (ω). 2π Equating this with the corres
onding
new left hand side, we obtain a ˆ formula which com
utes f (ω) from the CWT Lψ f (t,
a). 2. A

ling now the Fourier inversion formula 5.2 to both sides we end u
wi
th the desired reconstruction formula. It reads 2π f (t) = √ cψ
∞ ∞
Lψ f (a, u)
−∞ −∞
1 ψ |a|

−u a
duda a2
(2.27)
       
Wi
 h he same reasoning
  as for
 he STFT recons rucion formula
 2.11 we no e ha
he above reconsruc ion in egral is only of limi ed prac ical relevance if sam
pled signals and he corresponding CWT are given.
  
2.2 The Con inuous Wavele Transform (CWT)
25
     
Insead  inanalogy o he STFTprocedure   one may ob
 ain a recons  ruc ion alg
ori hm essen ially
  by reversing he s eps of he compu a ion algori  hm given abo
ve. Assume ha he sampling {f (kTS )}N −1
  (cf. eq. 2.1) of a con inuous  ime k=0
ni e
   energy signal
 f ( ) is given such ha eq. 2.22 is ful lled.
 Assume, moreover
, ha he  algori hm described above has been performed. Thus af er evalua ing e
q. 2.24, he sequence {Lψ f (a, kTS )}N −1 . k=0 is given. Then we use the corres
on
dence
air eq. 2.21 again to reconstruct {f (kTS )}N −1 as follows: k=0 Fast CWT r
econstruction: 1. Com
ute the sequence {ψa (kTS )}N −1 , where ψa is de ned in k=0 eq. 2
.20 and com
ute the corres
onding DFT denoted with {Ak }N −1 . k=0 2. Com
ute the
DFT of the sequence {Lψ f (a, kTS )}N −1 . Let us k=0 denote it with {Bk }N −1 . k=0 3
. Com
ute the sequence {Fk }N −1 where k=0 Fk = cψ |a| Bk . Ak (2.28)
    
4. Then apply he inverse DFT  (eq. 5.14) o his sequence, resul ing in he sequ
ence {f (kTS )}N −1 . Schema ically: k=0 {Fk }N −1 −→ {f (kTS )}N −1 k=0 k=0
IDF T
(2.29)
      
Noe ha eq. 2.28  requires Ak o be nonzero for all k. If his is no he case, 
s eps 1 – 3 mus be repea ed forsui able a values, such ha {Fk }N −1 k=0  comple
ely may be recovered before compu ing 2.29. Never heless,
 if he scale fac or a 
is such ha Ak= 0 (k = 0, . . . , N −1), he algori hm described above leads o
a recovery of he sampled
 signal
 from
 hecorresponding
  CWT from a single
 a val
ue! This is an indica ion for he fac ha he informa ion s ored in he Lψ f (a,
t) coe cients is highl redundant. We will reduce this redundanc later (cf. cha

. 3). MATLAB im
lementations of the CWT algorithms
resented in this section are
discussed in sect. 2.2.3.
26
2 Continuous Analsis
2.2.2 Wavelet Exam
les As mentioned in sect. 2.2.1, most
racticall relevant wa
velets satisf the admissibilit condition 2.16 if relation 2.18 is ful lled. Gene
ralizing this relation we shall construct “wavelets ψ(t) with M vanishing moments” (cf
. eq. 2.31). The
ractical im
lications of condition 2.31 will be illustrated be
low. Construction
rinci
le Let Φ(t) denote a piecewise smooth function being M -t
imes di erentiable and satisfying some additional slight technical restrictions. D
e ne the wavelet ψ(t) b dM Φ (t). dtM Then one may show that ψ(t) has M vanishing momen
ts, i.e. ψ(t) :=
+∞
(2.30)
ψ(t)tm dt = 0 (m = 0, 1, . . . , M − 1).
−∞
(2.31)
The interested reader is encouraged to
rove this in the exercises. M = 1: The H
aar wavelet Consider ⎧ 0≤t< 1 ⎨t 2 ΦH (t) = 1 − t 1 ≤ t < 1 2 ⎩ 0 else
The corres
onding wavelet is called “Haar wavelet” and reads ⎧ ⎨1 0 ≤ t < 1 2 dΦH (t) = −1
< 1 ψH (t) := 2 ⎩ dt 0 else Condition 2.31 for the Haar wavelet reads
+∞
(2.32)
ψH (t) dt = 0.
−∞
2.2 The Continuous Wavelet Transform (CWT)
27
M = 2: The Mexican hat wavelet Here
1 t2 2 ΦMH (t) = − √ π − 4 e− 2 3
The corres
onding wavelet is called “Mexican hat wavelet” and reads ψMH (t) :=
1 t2 d2 ΦMH 2 (t) = √ π − 4 (1 − t2 )e− 2 2 dt 3
(2.33)
Condition 2.31 for the Mexican hat wavelet reads
+∞ +∞
ψMH (t) dt =
−∞ −∞
ψMH (t)t dt = 0.
Plots Both wavelets treated so far are shown in Fig. 2.2. The wavelets are
lott
ed as solid lines, the res
ective Φ-functions are dashed.
1.5 1 0.5 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 2 ψH(t) Φ (t) H
1 ψ 0.5 0 −0.5 −1 −4
MH
(t)
ΦMH(t) −3 −2 −1 0 1 2 3 4
Fig. 2.2. Haar wavelet and Mexican hat wavelet (below).
Practical im
lications of the vanishing moments condition When for a given scale
a and a given time t the CWT
28
2 Continuous Analsis
1 Lψ f (a, t) = √ cψ
1 |a|
∞
ψ
−∞
u−t a
f (u) du
is com
uted, onl those u values will contribute, where ψ u−t is nonzero. a Assume t
hat these values constitute an interval I(a,t) . Now assume that ψ(t) has M vanish
ing moments and the signal f (u) on I(a,t) ma be modeled b a
olnomial of deg
ree k with k < M . Then one ma show that Lψ f (a, t) = 0. Again, the interested r
eader ma work on the
roof in the exercises. This result can be inter
reted as
follows: Assume that ever “smooth”
art of the signal ma be re
resented b a certa
in
olnomial and signal changes are modeled b switching the
olnomial to a ne
w one. In other words: The signal is modelled b
iecewise
olnomial functions.
Then signal changes will be well localized b the CWT
rovided the wavelet ψ(t) h
as enough vanishing moments. The reason is that in the smooth
arts the CWT will
vanish if the number of vanishing moments is large enough and the CWT will be c
oncentrated around the time values, where signal changes (i.e. changes in the
o
lnomials) occur. This is illustrated in Fig. 2.3. Here the signal consists of t
wo di erent straight lines (i.e.
olnomials of degree k = 1) connected together a
t t = 5000. Recall (see above) that the Mexican hat wavelet has 2 vanishing mome
nts, whereas the Haar wavelet onl has 1 vanishing moment. Thus, one ex
ects tha
t res
ective CWT should be better concentrated around the connection time for th
e Mexican hat wavelet than for the Haar wavelet. This is clearl visible in Fig.
2.3. Note also that the Haar wavelet leads to nonzero CWT values in the “smooth” si
gnal
arts in contrast to the Mexican hatwavelet. The Morlet wavelet We close th
is section with the “Morlet wavelet” which was among the rst wavelets to be ex
lored
for signal analsis
ur
oses [11]. It is com
lex valued and reads ψM (t) = π − 4 (e−jαt − e
1 α2 2
)e− 2 (α = π
t2
2 ). ln 2
(2.34)
Since the corres
onding w velet tr nsform LψM f ( , t) will be com
lexv lued fun
ction, it m  be decom
osed into bsolute v lue nd
h se corres
onding to LψM f (
, t) = |LψM f ( , t)|ejΦM ( ,t) . (2.35)

or ossible

lic tions of the
h se function ΦM ( , t) the re der is referred t
o sect. 2.2.4.
2.2 The Continuous W velet Tr nsform (CWT)
4 2 0 Sign l
29
1000
2000
3000
4000 5000 6000 CWT with H r−W velet
7000
8000
50 100 150 200 1000 2000 3000 4000 5000 6000 CWT with Mexic n−h t−W velet 7000 8000
50 100 150 200 1000 2000 3000 4000 5000 6000 7000 8000

ig. 2.3. To
: Sign l. Middle: CWT with the H r w velet. Bottom: CWT with the M
exic n h t w velet.
2.2.3 Im
lement tion with MATLAB nd Visu liz tion At the beginning of sect. 2.1
.3 some
relimin r st tements concerning MATLAB h ve been m de. The

l lso
to this section nd will not be re
e ted here. In this section we sh ll describ
e MATLAB im
lement tions both of the CWT  lgorithm le ding to 2.24 nd of the co
rres
onding reconstruction lgorithm le ding to 2.29. Ag in, the res
ective MATL
AB m  les h ve been tested with

MATLAB 6.5, rele se 13 nd m  be downlo ded from
the URL given in sect. 1.3. st CWT com
ut tion The corres
onding function le re
ds mcwt. A s m
le function c ll is shown in the next line: [t,,m trix]=mcwt(
sign l,T,st rt,ste
,sto
,comment); An logousl to mstft the vector sign l denot
es the s m
led sign l, T the s m
ling dist nce, the string comment is included i
n the title of the gr
hics gener ted b the function. The
r meters st rt,ste

nd sto
denote minimum index, increment nd m ximum index of the sc le f ctors
for which the CWT is com
uted, me sured in multi
les of T. Thus in MATLAB not t
ion the resulting sc le v lues re given b the vector (st rt:ste
:sto
)*T. As
result of the lgorithm the com
uted Lψ f ( i , kTS ) v lues re stored in the m
trix m trix, with k numbering columns nd i numbering rows. The i v lues re t
he entries of the bove vector cont ining the sc le v lues. This
30
2 Continuous An lsis

m trix g in is visu lized s, e.g., in ig. 1.6 using the im gesc function. Sin
ce for com lex v lued w velets the h se
lot of this m trix might be of interes



t (cf. sect. 2.2.4), the function llows both for visu liz tion of the bsolut
e v lue of m trix nd of its
h se. The


b sic com ut tion code is listed below %
Initi liz tions of sc le f ctors nd ourier tr nsform =(st rt:ste
:sto
)*T; ro
ws=length( ); h t=fft(); %M trix Initi liz tion m trix=zeros(ze,n); %Loo
for
incre sing sc le f ctors for i=1:rows,
si_sc le=conj(fev l(w v, t./ (i)));
si_
sc le_h t=fft(
si_sc le); %Time

tr nsl tion such th t minim l time=0 tr ns=ex
((
j*t(1)*(0:(N 1))*oms/N)); % ourier tr nsform of w velet tr nsform; conv_h t=((
h t.*
si_sc le_h t).*tr ns)/sqrt( (i)); m trix(i,:)=ifft(conv_h t); end The w ve
let must be
rovided s function le, whose n me is stored in the string w v. Th
e code should be f irl self ex
l n tor for re der used to MATLABnot tion nd
h ving worked through sect. 2.2.1. A comment on the time tr nsl tion vector tr
ns should be dded, however. As mentioned in sect. 2.2.1 the CWT will be com
ute
d for the s m
led time v lues 0, TS , 2TS , . . . , (N − 1)TS . Thus, in
rticul
r, ll time v lues will be ≥ 0. On the other h nd, the com
ut tion of ψ (t) := ψ −t
is required. This im
lies th t for w velets, which re zero for t < 0 ψ (t) will
v nish for t > 0, thus in
rticul r for the bove time s m
ling. The H r w vel
et 2.32 is sim
le ex m
le for such w velet. Therefore in mcwt the time vect
or t is designed such th t t = 0 corres
onds to the middle of the sign l. Hence,
the resulting CWT must be shifted with res
ect to the time dom
in such th t ft
er te shift the sm llest t v lue corres
onds to t = 0. In the ourier dom in thi
s time shift m  be re lized b multi
ic tion with
2.2 The Continuous W velet Tr nsform (CWT)
31

h se f ctors, which re collected in the vector tr ns. In the de nition of this v

ector oms denotes the s m
ling circul r frequenc. st CWT reconstruction The r
econstruction lgorithm is im lemented in the function le micwt. A
rotot
e c l


l to this function re ds [t,reco,m trix]=micwt(m trix,T,st rt,ste
,sto
); m tri
x is the result from
receding c ll of mcwt, T g in denotes the s m
ling dis
t nce. As with the mcwt function from the res
ective in
ut
r meters the sc le
vector =(st rt:ste
:sto
)*T is constructed. The entries of these vectors re s
c nned in order to select
com
onent i such th t the sequence {Ak }N −1 resultin



g from l ing the D T to the sequence k=0 {ψ i (kTS )}N −1 is nonzero for ll k (c
f. com
ut tion ste
2.28 in the reconk=0 struction lgorithm). If such n i doe
s not exist, the
rogr m will not work
ro
erl! It is recommended to choose the
sc le rel ted in
ut
r meters st rt, ste
nd sto
identic l to the res
ective

r meters in the
receding mcwtc ll. Thus, t
ic l di logue re ds: >> [t,,
m trix]=mcwt(,.01,1,1,200,’’); Enter w velet (in quotes): ’mex_h t’ >> [t,reco,m trix]
=micwt(m trix,.01,1,1,200); Enter w velet (in quotes): ’mex_h t’ Note th t obviousl
 both in the mcwt  nd the micwt c ll the s me w velet must be selected! In b
oth functions the n me of the w velet function le is requested from the user. The
reconstructed sign l is stored in the vector reco. A note on the MATLAB W velet
Toolbox The MATLAB W velet Toolbox [24]
rovides the cwt function in order to

erform CWT. Below t
ic l function c ll is shown: m trix=cwt(,1:1:200,’mexh’);
The me ning of the res
ective
r meters should be cle r from the discussion bo
ve. Note th t the st rt , ste
 nd sto

r meters from bove must be merged to
n in
ut vector of the kind st rt:ste
:sto
. Note lso th t the chosen w velet
must be s
eci ed b corres
onding in
ut string, in the bove ex m
le ’mexh’ selects
the Mexic n h t w velet 2.33. As with ll other fe tures of the MATLAB W velet T
oolbox the CWT lso m  be invoked from gr
hic l user interf ce which is l un
ched
b the w vemenu
comm nd. The other loc l

tr nsforms tre ted in this ch
ter

(ST T, inverse ST T, inverse CWT) re not su orted b the MATLAB W velet Toolb
ox.
32
2 Continuous An lsis
2.2.4 A

lic tion: Detection of Sign l Ch nges In sect. 2.2.2 we modeled sign ls
b
iecewise
olnomi l functions. “Smooth”
rts of the sign l corres
onded to tim
e interv ls, where the sign l w s re
resented b xed
olnomi l. The time v lue
s where two di erent
olnomi ls re connected corres
onded to “sign l ch nges”. In th
is section we will consider more subtle ex m
le nd discuss CWT ex
eriments wi
th v ring w velets. The model sign l re ds f (t) = cos(t − π) t < π cos(2t − 2π) t ≥ π
Obviousl, the rst deriv tive df (t) is continuous, where s the second dt deriv t
ive is discontinuous t t = π. Thus, we h ve kind of “hidden” sign l ch nge occurrin
g not in the sign l itself but in some higher deriv tive. In the rst ex

eriment w
e com
uted the CWT of this sign l with the com
lex Morlet w velet 2.34. igure 2
.4 visu lizes the
h se ΦM ( , t) of LψM f ( , t) (cf. eq. 2.35).
Testsign l. Sc le
r meters: =1,δa=1,a top=100. tart
0.5 0 −0.5 −1 1
1.5
2
2.5
3
3.5
4
0.02 0.04 a 0.06 0.08 0.1 1 1.5 2 2.5 t 3 3.5 4
Fig. 2.4. CWT of the te t ignal ith the Morlet  avelet 2.34, Pha e plot.

The time here the ignal change occur  i  localize very clearly by traight li
ne  converging to t = π. This qu lit tive beh vior of the
h se re
resent tion (co
nvergence to
oints of sign l ch nges) h s been re
orted lre d ver e rl (cf.
the rticle of A. Grossm nn, R. Kronl nd M rtinet nd J. Morlet in [6]). In ch

. 3 we will introduce the f mous f mil of D ubechies w velets [7].
or more
det iled tre tment we refer to this ch
ter. Here we onl rem rk
2.3 C se Studies
33
th t the members of this f mil re numbered with dbn (n = 1, 2, 3, . . .). An i
m
ort nt s
ect for sign l
n lsis
ur
oses is th t dbn w velets h ve n v nishi
ng moments (cf. eq. 2.31)! igure 2.5 shows the w velet tr nsform of the bove s
ign l with the db2w velet for three incre sing sc le f ctors (cf. gure c
tion; T
S denotes the s m
ling dist nce).
db2−w velet 0.5 f 0 −0.5 1 1.5 2 2.5 3 3.5 4
x 10 1 0 −1 x 10 5 0 −5 −10 d
2
−6
d
1
−6
x 10 4 2 0 −2 −4 1
−5
d
3
1.5
2
2.5
3
3.5
4

ig. 2.5. To
: Sign l. Below: W velet tr nsforms for = TS , = 2TS , = 4TS
(from bove). W velet: db2.

igure 2.6 shows the s me results using the db4 w velet. The gures con rm the qu li
t tive beh vior mentioned lre d in sect. 2.2.2: With incre sing number of v ni
shing moments the res
ective w velet tr nsforms tend to be better loc lized rou
nd signi c nt sign l ch nges. Corres
ondingl, the v nishing in “smooth” sign l region
s is more
ronounced for w velets with l rge number of v nishing moments. Rel
ted to this is the f ct th t the dbn w velets tend to be “smoother” with incre sing
n. We will not stress these to
ics here; for n e s to re d summ r the re der
is referred to [24].
2.3 C se Studies
2.3.1 An lsis of Sensor Sign ls All sign ls discussed in this section were reco
rded in the sensor of robot, c rring out (light) rc welding
rocess. The
rc itself is used s sensor me suring the dist nce from the welding tool to th
e work
iece.
34
2 Continuous An lsis
db4−w velet 0.5 f 0 −0.5 1 1.5 2 2.5 3 3.5 4
x 10 2 0 −2 x 10 5 0 −5 x 10 2
3
−7
d
1
−7
d
2
−6
d
0 −2 1 1.5 2 2.5 3 3.5 4

ig. 2.6. To
: Sign l. Below: W velet tr nsforms for = TS , = 2TS , = 4TS
(from bove). W velet: db4.
The results described in the following
r gr
hs h ve been obt ined with the MA
functions mstft nd mcwt (cf. sections 2.1.3 nd 2.2.3,

TLAB


res
ectivel). ST



T  lic tion The rst ex m le to be resented is the ST T n lsis of such sig
n l. Here the welding robot
erforms n oscill tor motion which is tr nsvers l
to the welding se m. Without going into det ils of the technolog we mention th
t the frequenc of this oscill tion t t ≈ 3 sec should get doubled from ν ≈ 3 Hz to ν ≈ 6
Hz. The sesor sigal is sho  i the upper part of Fig. 2.7. It obviously is h
eavily corrupted ith oise. The lo er part of Fig. 2.7 sho s the absolute value
s of the respective STFT sigal. The STFT is coded ith gray values as described
i sect. 1.2.5. To simplify evaluatio, the STFT as represeted o the t ν plae
i cotrast to Fig. 1.6, here it as represeted o the t ω plae. Obviously, th
e frequecy doublig metioed above clearly is localized ith the STFT. Moreove
r, the STFT sho s ho the corruptio ith oise leads to additioal high freque
cy compoets i the sigal. The ido fuctio used for the STFT computatio
as the Gaussia ido 2.15. Thus, actually the Gabor trasform of the sigal a
s computed. CWT applicatios Before discussig further examples of sesor sigal
s occurrig at a arc eldig process, e remark that e ill aalyze sigals st
emmig from three di er 
2.3 Case Studies
Sesor Sigal. Parameter: δx=5,kmin=20,δk=1,kmax=200,σ=1.0 2 1.5 1 0.5 0 −0.5 −1 0 2 4 6 8
10 12 14
35
2 4 6 ν 8 10 12 0 2 4 6 t 8 10 12 14
Fig. 2.7. STFT example: Sigal (above) ad STFT (belo ). Here the Gabor ido (e
q. 5.11) has bee used.
et techologies, the details of hich caot be explaied here. We oly ote, t
hat they are deoted ith the abbreviatios “MSG”, “MAG” ad “WIG”, respectively. Apart fro
their di ereces they share a commo property: usually the sesor performs a osc
illatory motio trasversally to the eldig seam. For test purposes this oscill
atio may be s itched o (third sigal i Fig. 2.9). All time  ad scale values i
the plots belo are measured i secods. The result of the CWT aalysis of the
MAG  ad MSG sigals, respectively, is depicted i Fig. 2.8. As expected, it sho
s bright stripes at scale factors associated to the avelegths of the respecti
ve oscillatios. This remais true also for the three WIG sigals; the results o
f their respective CWT aalysis are sho  i Fig. 2.9. Note i particular the la
ck of the correspodig stripe for the third sigal, here the oscillatio as s
itched o . Visual ispectio aloe does ot reveal oticeable di ereces bet ee th
e various techologies. Yet CWT aalysis clearly idicates a di erece: WIGsigals
sho bright peaks at scale a ≈ 6 sec. Ideed, a posteriori oe might isolate corr
espodig “peak groups” he ispectig the sigals from Fig. 2.9. This idicates th
at there exists a correspodig correlatio i WIG sigals i cotrast to MSG  a
d MAG sigals. Note that this correlatio is preset, eve if the oscillatio i
s s itched o (cf. third sigal i Fig. 2.9). This example illustrates that the mu
ltiscale ivestigatio of a sigal, performed by the CWT, may o er valuable iform
atio cocerig relevat parameters of the respective process behid the sigal
.
36
2 Cotiuous Aalysis
MAG−sigal. Scale parameters: astart=2,δa=2,a top=500. 1 0.5 0 −0.5 −1 −1.5 0
5
10
15
20
25
30
1 2 a[ ec] 3 4 5 6 7 0 5 10 15 t[ ec]
tart

20 =2,δ =2,a
a top
25 =500.
30
MSG− ignal. Scale parameter : a 2 1.5 1 0.5 0 −0.5 −1 0 2 4 6
8
10
1 2 a[ ec] 3 4 5 6 7 0 2 4 6 t[ ec] 8 10
Fig. 2.8. Above: CWT of a MAG ignal. Belo : CWT of a MSG ignal.
 
A  a nal remark e note that the avelet u e in the tran form  vi ualize in Fig 
. 2.8 – 2.9 a  the Morlet avelet 2.34. 2.3.2 Analy i  an Cla i cationof  Au io
Signal  In thi  ection e ecribe the analy i  of acou  tic ignal  recor e u
ring certain in pection proce ure  in the automotive in u try1 . In ome applica
1

The material pre ente in thi  ub ection re ult  from a cooperation
 ith AUDI A
G, a major German car manufacturer. It i  publi he ith kin permi ion.
 
2.3 Ca e Stu ie 
37
Fig. 2.9. CWT of three WIG ignal .
38
2 Continuou  Analy i 
 
tion  uch a ignal may inicate the pre ence or ab ence of a efect;  thu  in th
i  context it i  u e for   iagno i  purpo e . An
 example
 i  e  cribe in [18]. I
n other ca e  the recor e ignal imply houl oun “plea ant” in or er to ful l com
fort requirement
 . Both ca e  have in common that a humanli tener can i crimin 
ate “goo ” from “ba ” ignal . In thepre ent ection e e cribe the r t tep  to ar  a
trainable cla i cation cheme,  ba e on a local analy i  of uch ignal . Thi c
la
 i cation proce ure houl in principle be able to perform
 the above mentione
i crimination of a human li   tener automatically.
 In or er to compare the re ult
 of both
 technique  e u e the CWT an the STFT.
 We ketch
 no
 the cla i cation
proce ure. Fir t the local  tran form i  ample on
 a gri of i crete value  bo
th ith re pect to time an cale (CWT) or time an frequency (STFT). For the CW
T the ampling
 i  illu
 trate in Fig. 2.10; in the upper half a typical acou tic
ignal i  i playe .
Acou tic Signal. Scale parameter : a 0.1 0.05 0 −0.05 =1,δ =10,a
a
tart
top

=1000.
0 t1
0.5 t2
1 t
3
1.5
0 0.05 0.1 0.15 0.2
a1 a2 a3 a4
a
0
0.5 t
1
1.5

Fig. 2.10. CWT of an acou tic ignal. Sampling for 3 time an 4 cale value .
  
In the next tep from the ample tran form value  a pattern vector i  erive
hich erve  a an input to the ub equent
 cla i cation proceure. The cla i cator 
may be traine ith “goo ” ignal , lea ing to a geometrical e cription of the“goo
cla ” in the pace of pattern vector 2 . A given ignal then i  cla i e ba e on
a certain notion of i tance to  that cla . Figure 2.11 ho  ome r t cla icatio
n re ult . Without going into etail  e mention that 15 ignal  ere pre ente
to the y tem. The r t 5
2
 
For an intro uction to the ba ic principle  of pattern recognition refer to, e.g
., [37]
 
2.3 Ca e Stu ie 
39
  
ignal  ere “goo ”;  in the gure they are enote by io01.mat,
 ..., io05.mat. All oth
er ignal  ere “ba ”. For a ignal to be quali e a  “goo ” it mu t reach the maximum co
re of 9 in the bar graph . The upper re ult a  obtaine
 ith a CWT, the loer r
e ult ith a STFT, the re pective avelet  or in o function  are in icate in
the title line of the plot .
Cla ification re ult . CWT (Morlet− avelet). iO01.mat iO 2.mat 0 iO03.mat iO04.ma
t iO05.mat niO 1.mat 0 niO 2.mat 0 niO03.mat niO04.mat niO06.mat niO07.mat niO08
.mat niO 9.mat 0 niO10.mat niO 1.mat
1
9 8 7 6 5 4 3 2 1 0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15

Cla ification re ult . STFT ith Gau − in o (Gabor−tran form). 9 8 7 6 5 4 3 2 1
0
iO 1.mat 0 iO 2.mat 0 iO 3.mat 0 iO 4.mat 0 iO 5.mat 0 niO 1.mat 0 niO 2.mat 0 n
iO 3.mat 0 niO 4.mat 0 niO 6.mat 0 niO 7.mat 0 niO 8.mat 0 niO 9.mat 0 niO 0.mat
1 niO 1.mat
1
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15
Fig. 2.11. Cla i cation re ult  of acou tic ignal .

The re ult  in icate that both tran form  are u eful
 in the context of our inve 
tigation,
 but  the avelet tran form pre umably
 lea   to a clearer eparation of “g
oo ” from “ba ”. Note that the Fourier ba e proce ure lea 
40
2 Continuou  Analy i 
 
to ome mi cla i cation  (fal
 e po itive
 ): The “ba ” ignal  nio01.mat an nio06.mat 
, re pectively,are cla i e a  “goo”. We nally remark that the re ult  pre ente abo
ve ere obtaine ith linearly ample cale  value  (CWT) or frequency value  (S
TFT), re pectively.
 An option for future evelopment  i  the u e of logarithmica

lly ample cale  or frequencie . The re ulting notion  of “octave  ” an “voice  per
octave” (for a hort urvey of the e topic  cf., e.g.,[38]) prove to be ucce f
ul hen analyzing peech or mu ic. Here e inve tigate ignal  re ultingfrom a
technical proce , neverthele  the cla i cation of the e ignal  houl perform
imilar to the human ear. In [4] a mathematical mo el of the cochlea, a piral 
like organ hich i  e ential for  hearing, i  given, hich relate  the perceptio
n of the human ear to the CWT an incorporate  a logarithmic re pon e to frequen
cie !

2.4 Note  an Exerci e    
Note  For upplementary rea ing on the topic  treate  in thi  chapter
 the reaer
i  referre to the bibliography.  Reference  [7] an [20] provi e athorough i 
cu ion of the mathematical foun ation  of the CWT. In particular, ecay propert
ie  of the CWT, e ential for quantifying
 it  concentration propertyon the “impor
tant” ignal region  are invetigate . The STFT originally a  intro uce by D. Ga
bor [10]. The book [17] provie  a nice overvie on thi   topic. Short an compre
hen ive revie  of the CWT an STFT are al o containe  in [1] an [34]. Exerci e
 1. a) A ume that the Fourier tranform  corre pon ence pair y(t) ◦ − • y (ω) ˆ is give.

Sho that the y(t)tm ◦ − • j m Hin : S ar from
+∞
dm y ˆ (ω). dω m
(2.36)
y (ω) = ˆ
−∞
y(t)e−jωt dt
ad di eretiate both sides ith respect to ω.
2.4 Notes ad Exercises
41
b) Agai assume that the Fourier trasform correspodece pair y(t) ◦ − • y (ω) ˆ is give
. Sho that the dm y (t) ◦ − • (jω)m y (ω). ˆ dtm Hit: Start from 1 y(t) = 2π
+∞
(2.37)
 (ω)e+jωt dω ˆ
−∞
ad di eretiate both sides ith respect to t. 2. Give T ith T > 0, the “box  ido
” of idth T is de ed as (t) = 1 −T ≤ t ≤ 2 0 else
T 2
Compute the respective localizatio parameters 5.6 – 5.9. 3. Cosider a sigal hi
ch may be represeted by a itegrable fuctio f (t) ad deote the rst itegral
ith F (t). Sho that the CWT computed
ith the
Haar 
avelet ψH de ned in eq. 2.32
for > 0 re ds 1 LψH f ( , t) = √ 2 t + − (t) − (t + ) . cψH 2 This n l tic l
result
rovides the
ossibilit to check the f st CWT lgorithm
described in sect
ions 2.2.1 nd 2.2.3: Choose s m
le function f (t) such th t (t) is known. N
ow choose the discretiz tion nd the t discretiz tion from sect. 2.2.1 nd com

ute with MATLAB the m trix LψH f ( i , tk ) using the bove ex
ression. Now visu
lize the bsolute v lue with im gesc nd com
re the result with the visu liz ti
on obt ined from mcwt. Note th t, when gr  v lue coding is used, im gesc utom
tic ll m
s the lowest m trix v lue to bl ck nd the highest to white. Thus fo
r visu liz tion the f ctor cψH , which onl
l s the role of glob l intensit f
ctor, m  be omitted when com
uting LψH f ( i , tk ). M Φ 4. Show th t ψ(t) = d M (t)
h s M v nishing moments. Ste
s: dt ˆ ˆ ) Show th t ψ(ω) = (jω)M Φ(ω) usig 2.37. mˆ m md
ow th t ψ(t)t ◦ − • j dωm (ω) usig 2.36.
ψ c) Conclude th t d m (0) = 0 (m = 0, . . . , M − 1) nd rel te this to the dω momet
coditio 2.31.
m
ˆ
42
2 Cotiuous Aalysis
5. Assume that the avelet ψ h s M v nishing moments. Assume, moreover, th t f (t)
is
iecewise
olnomi l nd the degree of the res
ective


ol nomi
ls is below M
. Show th t then Lψ f ( , t) = 0 for “suit ble” t nd . or the recise formul tion
refer to the section on
r ctic l im
lic tions of the v nishing moments conditio
n (
ge 27 nd following
ges). 


6. Tr
to im lement

our own MATLAB functions f

or com uting the CWT nd the ST T, res ectivel . or ssist nce refer to section
s 2.1.1, 2.1.3, 2.2.1 nd 2.2.3.
3 The Discrete W velet Tr nsform

3.1 Redund nc of the CWT nd the ST T

The loc l tr nsforms tre ted in
ch
. 2 m
continuous time sign l f (t) ˆ to fu
nctions of two v ri bles: The ST T fw (ω, t) ad the CWT Lψ f ( , t), res
ectivel.
Thus the tr nsformed sign l in cert in sense is “l rger” th n the origin l sign l.
This st tement m  be m de more
recise b restricting ourselves to the
r ctic
ll relev nt c se of s m
led sign l nd b

ling the CWT to such sign l.
The rguments

in this introductor section corres
ondingl m  be extended lso
to the ST T. The s m
led sign l re ds f = {fk }N −1 = {f (kTS )}N −1 k=0 k=0 (3.1)
(cf. eq. 2.1). It m  be re
resented s row vector of N re l numbers. A

ling
the f st CWT  lgorithm on
ge 23 for the sc le f ctors 1 , . . . , M to this
vector we obt in the CWT v lues Lψ f ( i , kTS ) (k = 0, . . . , N − 1; i = 1, . .
. , M ). The m  be rr nged s n M × N m trix with i numbering rows nd k numb
ering columns. In f ct, ll CWT im ges discussed so f r re visu liz tions of th
is m trix! Thus the origin l sign l w s re
resented b N re l numbers where s th
e tr nsform is re
resented b M N numbers (which even m  be com
lex, if the w v
elet ψ is com
lex). Storing nd tr nsmitting the tr nsformed sign l on com
uter
network therefore requires roughl M times of the corres
onding c
cities for t
he origin l sign l! As we h ve seen in c
ter 2, this ddition l mount of d t
m  be useful when n lzing the sign l, i.e., looking for sign l ch nges, domin
nt sc les nd/or frequencies etc. When
erforming such sign l n lsis t sks, t
he reconstruction formul e of sect. 2.2.1 usu ll re not needed. The onl serv
e s
44
3 The Discrete W velet Tr nsform
n indic tion th t no sign l inform tion is “lost”, when

ling the CWT. The

li
c tion discussed in ch
. 4 is sign l com
ression. Here the w velet tr nsform c
n be t ken s the rst ste
in com
ression
rocedure nd this ste
h s the follo
wing desir ble
ro
erties: 1. If the origin l sign l consists of N re l v lues,
the tr nsformed sign l lso should consist of onl N re l v lues! 2. The tr nsfo
rmed sign l v lues fter cert in m ni
ul tions should llow for re
resent tion
with s few bits s
ossible. We sh ll not discuss the det ils here, the will
be tre ted in ch
. 4. 3. In the “decom
ression ste
” the sign l should then be reco
nstructed from the N m ni
ul ted tr nsform v lues! Thus, for sign l com
ression

ur
oses we h ve di erent intention; the t sk m  be formul ted s follows: Is i
t
ossible to nd w velets ψ such th t the CWT of f is com
uted on discrete subgri
d of the t  
l ne onl, resulting in (roughl) N tr nsform v lues Lψ f ( i , tk )
such th t the origin l row vector f m  be reconstructed from these v lues? An
ins
ection of the f st CWT reconstruction lgorithm described on
ge 25 shows t
he redund nc of the CWT Lψ f ( , t). If the sc le f ctor nd the w velet ψ re ch
osen such th t in formul 2.28 Ak = 0 for ll k, then the s m
led sign l {f (kTS
)}N −1 m  be recovered b  formul 2.29. The left
rt k=0 of
ig. 3.1 illustr te

s this situ tion: or recovering f not ll v lues of the tr nsform (i.e., the M ×
N m trix mentioned bove) re needed; inste d, one single row corres
onding to
the chosen v lue su ces. In the gure the m trix is visu lized b
the rect ngul r
re , the selected row b the horizont l line. Simil rl, for the ST T

lied t
o f under
ro
er conditions


it su ces to ick out single column from the corres


onding N × N m trix of
ST T v lues in order to reconstruct f . This c n be conclu
ded b ins
ecting the ST Treconstruction

lgorithm – in
rticul r eq. 2.12 – nd is


illustr ted in the right rt of ig. 3.1. In both c ses onl N tr nsform v lue
s re needed to recover the origin l sign l f , which is strong
indic tion for
the f ct th t the inform tion stored in the full CWT m trix or ST T m trix, res

ectivel, is redund nt. In f ct this is true: M them tic ll two di erent CWT tr
nsform v lues Lψ f ( 1 , t1 ) nd Lψ f ( 2 , t2 ) re rel ted with e ch
other 


b the
“re roducing kernel condition”. An n logous st tement holds for the ST T. or det
ils refer to, e.g., [20]. The t sk formul ted bove, i.e., selecting
ro
er di
screte subset of the t 
l ne for the CWT (of the t ω plae for the STFT, respecti
vely) such that f completely ca be recostructed from the respective trasform
values o these subsets oly, ca be solved. I the geeral case it leads to the
otio
3.2 The Haar System t t
45
a
ω
Fig. 3.1. Left: Uder certai coditios the sampled sigal f may be recostruct
ed from the correspodig CWT matrix by selectig oe suitable scale factor. Rig
ht: Uder certai coditios the sampled sigal f may be recostructed from the
correspodig STFT matrix by selectig oe suitable time value.
of “frames” [7], for avelets the most popular versio is the “discrete avelet trasf
orm” (DWT) described i this chapter. I cotrast to the situatio illustrated i
the left part of Fig. 3.1 the DWT ill be a real “multiresolutio aalysis” of the s
igal, sice the DWT values ill ot be computed for a sigle scale factor oly.
I the ext sectio e shall discuss the simplest versio of such a trasform.
We shall ork out the aspects, hich may be geeralized ad describe the geeral
DWT procedure i sect. 3.3. Sectio 3.4 is devoted to fast algorithms ad relat
ed implemetatios ith the MATLAB Wavelet Toolbox. I sect. 3.5 e ill adopt a
vie poit o the developed techiques, hich serves as a preparatio for the co
mpressio procedures described i chap. 4. I sect. 3.6 some further applicatio
s are discussed, otes ad exercises ally are collected i sect. 3.7. Throughout
this chapter e ill cosider sampled sigals as de ed i eq. 3.1. As idicated
there, the k th sigal value for simplicity reasos mostly ill be deoted ith
fk istead of f (kTS ). Thus, sampled sigals are represeted as ite sequeces o
f umbers as idicated i table 5.2 (cf. sect. 5.5). They are al ays assumed to
be sampled ith a samplig distace TS , eve if this is ot ritte do  explic
itly. For simplicity i the ext sectios e shall assume that TS = 1. (3.2)
The results obtaied there may be exteded to a arbitrary samplig distace TS
very easily ad e shall idicate ho .
3.2 The Haar System
I this sectio e shall develop the DWT for the Haar  avelet (cf. eq. 2.32). It
ill tur out that the resultig trasform is liked to the correspodig CWT
46
3 The Discrete Wavelet Trasform
as de ed i eq. 2.17. Sice the CWT ca be applied oly to a cotiuoustime siga
l f (t), hereas e start from a discretely sampled sigal as de ed i eq. 3.1, i
t ill be ecessary to costruct a cotiuous time sigal f (t) from the discret
ely sampled sigal {fk }N −1 . We shall sho i the rst subsectio k=0 ho this ca
be doe ad ho to develop the DWT for f (t). It ill tur out that the otio
of cotiuous time sigals may be omitted ad the DWT may be formulated i a pur
ely discrete maer, i.e., for sequeces of umbers. The resultig facts ad ot
ios are collected i the secod subsectio. As stated already after eq. 2.1, i
order to keep otatio simple e shall use the symbol f both to represet eithe
r a cotiuous time sigal or a discretely sampled sigal. The actual meaig al
ays shall be clear from the cotext. I sect. 3.2.1 e shall itroduce some mor
e cotiuous time fuctios, hich are uiquely coected ith sequeces. Here t
he same covetio holds. 3.2.1 Cotiuous Time Fuctios Cosider the follo ig
discretely sampled sigal of legth 8: f = {fk }7 = {8, 4, 6, 8, 9, 7, 2, 4} k=
0 (3.3)
Moreover, e eed the follo ig fuctio, hich is called “scalig fuctio”: φH (t) :
= 10≤t<1 0 else (3.4)
We shall see later that to each wavelet or which a DWT may be constructed, ther
e belongs a unique scaling unction. This explains the index H in the above expr
ession, since the scaling unction de ned by 3.4 belongs to the Haar-wavelet ψH (cf.
eq. 2.32). We sh ll see l ter how this rel tion m  be formul ted m them tic ll
. The sc ling function is used to build continuous time sign l f (t) from the
sequence {fk }:
7
f (t) :=
k=0
fk φH (t − k).
(3.5)

The to
row of ig. 3.2 shows the sequence {fk } (left) nd the corres
onding fu
nction f (t) (right). It is su
er
osition of shifted versions of the sc ling f
unction 3.4, e ch multi
lied with the corres
onding sequence element. Thus, the
continuous time sign l f (t) will be
iecewise const nt on interv ls of length o
ne, it is discontinuous t the s m
ling times. In the next ste
we sh ll constru
ct “co rser version” f 1 of the sequence f b com
uting the rithmetic me n of nei
ghboring sequence elements s indic ted below:
3.2 The H r Sstem
Sequences 10 5 0 0 10 5 0 −1 2 0 −2 −1 0 1
k 2 3 4 2 k 10 5 0 0 1 k 2 0 −2 0 2 4 t 6 8 2
3 4 0 2 4 t 6 8 4 6 8 10 5 0 0 2 4 t 6 8 unctions
47

ig. 3.2. f , f 1 nd d1 re
resented s sequences (left) nd functions (right).

1 f 1 = {fk }3 = k=0
f0 + f1 f2 + f3 f4 + f5 f6 + f7 , , , 2 2 2 2
= {6, 7, 8, 3}.
(3.6)
Obviousl this com
ut tion ste
le
ds to length of f 1 , which is h lf of the
length of the origin l sign l f . rom this new sequence we sh ll construct co
rres
onding continuous time sign l f 1 (t), which g in is
iecewise const nt, b
ut now on interv ls of length 2! Thus it covers the s me time interv l s f (t).
Simil rl to the
rocedure described for f (t) it is constructed from resc led
nd shifted versions of the sc ling function, multi
lied with the corres
onding
sequence elements: f (t) :=
k=0 1 1 3 1 fk φH
t −k . 2
(3.7)
1 Both the sequence fk nd
the corres
onding continuous time function f (t) re
shown in the second row of ig. 3.2. The co rsening
rocedure described bove fo
r sequences le ds to n n logous
rocedure for the corres
onding continuous tim
e sign ls: f 1 (t) is obt ined from f (t) b combining two neighboring ste
s of
width 1 into one ste
of width 2. The new ste
height is obt ined from the two f
ormer ste
heights b com
uting the rithmetic me n.
48
3 The Discrete W velet Tr nsform

The right
lot in the third row of ig. 3.2 shows the di erence sign l of f (t) n
d f 1 (t), which we sh ll c ll d1 (t): d1 (t) := f (t) − f 1 (t). (3.8)
This sign l lso is c lled “det il sign l”. The re son is th t, b de nition, it re
re
sents the sign l contributions which re needed to recover the origin l sign l f
(t) from the co rsened version f 1 (t). An ins
ection of the
lot shows th t on
the interv ls [0, 2], [2, 4], [4, 6] nd [6, 8] the det il sign l is of me n ze
ro: A ste
of cert in height nd width 1 is followed b ste
of the s me m g
nitude nd width, but with inverted sign. This is n immedi te consequence of th
e co rsening
rocedure discussed bove, le ding from f (t)
to f 1 (t). The H r 
w velet (cf. eq. 2.32) w s dis
l ed in the u

er
rt of ig. 2.2. It consists
of ex ctl two ste
s of width 1 nd the s me m gnitude but with inverted sign. T
herefore, d1 (t) 2 c n be written s su
er
osition of shifted H r w velets, w
hose width is doubled nd which re multi
lied with
ro
er f ctor. These f cto
rs c n be obt ined directl b ins
ecting the gr
h of d1 (t): d1 (t) = 2ψH t 2 − ψH t
− 1 + ψH 2 t − 2 − ψH 2 t −3 . 2
We sh ll formul te this rel tion in slightl more gener l f shion. De ne the seq
uence d1 := {d1 }3 = {2, −1, 1, −1}. k k=0 (3.9)
Then d1 (t) m  be written down simil rl to f 1 (t) (cf. eq. 3.7). But here we
ex
nd with res
ect to ψH inste d of φH and the expansion coe cients 1 are given by th
e sequence {d1 } instead o  { k }: k d1 (t) =
3 k=0
d1 ψH k
t −k . 2
(3.10)

The sequence {d1 }3 is dis
l ed in the left
lot in the third row of ig. k k=0
3.2. We obt ined it b de ning nd investig ting the continuous function d1 (t).
It is im
ort nt to re lize th t this sequence lso c n be obt ined directl from
the origin l discrete sign l {fk }7 . k=0 {d1 }3 = k k=0 f0 − f1 f2 − f3 f4 − f5 f6 − f
7 , , , 2 2 2 2 = {2, −1, 1, −1}.
(3.11)
Note the simil rit of this
rescri
tion to eq. 3.6! We re now re d to formul
te the connection of the co rsening nd det iling
rocedure discussed so f r to
the CWT s de ned in eq. 2.17:
3.2 The H r Sstem
49
LψH f (2, 2k) =
2 cψH
d1 (k = 0, . . . , 3). k
(3.12)
The re der is invited to check this in the exercises. It should be cle r how the
results nd formul e of this section m  be extended to gener l discrete sign
l of length N . Note th t vi eq. 3.12 the CWT v lues for the H r w velet m 
be com
uted b sim
le summ tion formul s given b eq. 3.11, i.e., without ev
lu ting nd

roxim ting the integr l in formul 2.17! But, s we h ve seen, i
n this w  we obt in LψH f ( , t) onl for = 2 nd t =
2k (k = 0, 1, . . .). We
conclude this section with collection of the results. st lgorithm for the c
om
ut tion of the CWT with the H rw velet: 1. St rt from the discrete sign l {f
k }, the H r w velet ψH (cf. 2.32) nd the corres
onding sc ling function φH (c . 3
.4). De ne the continuous-time signal  (t) as in 3.5. 1 2. Compute the sequences
{ k } and {d1 } as in eqs. 3.6 and 3.11, k respectively. 3. Then the CWT with th
e Haar-wavelet, applied to  (t), or a = 2 and t = 2k (k = 0, 1, . . .) may be
evaluated via eq. 3.12. 4.  (t) can be reconstructed as ollows: De ne the coarse
ned signal  1 (t) and the detail signal d1 (t) as in eqs. 3.7 and 3.10, respect
ively. Then  (t) =  1 (t) + d1 (t).
This procedure is a one-step-version o  the “discrete wavelet trans orm” (DWT), carr
ied out with the Haar-wavelet. We note already here that the computation may be
iterated, starting with  1 (t) instead o   (t). This explains why we speak o 
a one-step-version. We shall describe the iteration procedure in some more detai
l later. In Fig. 3.3 we visualize the subset o  the t-a-plane, on which the CWTv
alues are computed via the DWT. Note that  (t) may be reconstructed rom the CW
T-values on this subset together with the coarsened signal  1 (t)! 3.2.2 Sequen
ces
1 The computation procedure given in eq. 3.6, leading rom { k } to { k }, may b
e represented by the diagram shown in Fig. 3.4. The top row symbolizes the 1 inp
ut sequence { k } (c . 3.3), the bottom row the output sequence { k }. 1 Sequenc
e  is obtained as ollows: The elements o  the input sequence pointing toward a
certain element o  the output sequence with an arrow are multiplied with the co
rresponding numbers and added at the node, where 1 the arrows meet. Thus, e.g. 
0 = 0 1 + 1 1 . The diagram illustrates the 2 2
50
3 The Discrete Wavelet Trans orm 0 2 4 6 8
Et
2r
r
r
r
r
a
c
Fig. 3.3. One-step-version o  the DWT: Subset o  the t-a-plane. 0 = 8
r
r   1 2         ©   c r
1 2
1 = 4

2 = 6
r   1 2         ©   c r
1 2
r
3 = 8
1 0 = 6
1 1 = 7
Fig. 3.4. Visualization o  eq. 3.6.
1 computation o  the rst two elements o  { k }. It is clear how to proceed with t
he remaining elements. Comparing Fig. 3.4 with Fig. 5.6 we conclude that the com
putation o   1 may be written as
 1 = H .

(3.13)
As explained in sect. 5.3 (c . in particular eqs. 5.17 and 5.20) the sequence 
1 is obtained rom  by a two-step procedure: First a digital lter with coe  cients
h0 = 1 , h1 = 1 is applied, subsequently the signal is downsampled, 2 2 i.e., o
nly every second sequence element is kept. This procedure is denoted with the sy
mbol H, note in particular, that the number o  sequence elements (the length o 
the signal) is divided by two. The computation o  sequence d1 was illustrated in
eq. 3.11. It is now obvious that similarly to above we may write d1 = G . (3.14
)
Again G denotes ltering ollowed by subsampling, but now the corresponding lter co
e cients read g0 = 1 , g1 = − 1 . 2 2 Equ tions. 3.13 nd 3.14 ct on sequences onl
. The corres
onding continuoustime sign ls f 1 (t) nd d1 (t), res
ectivel, re
not needed t ll. But if we w nt
3.2 The H r Sstem
51
to reconstruct the discrete sign l f = {fk } from these sequences b the scheme
described on
ge 49 (cf. in
rticul r ste
4 of this scheme), we must introduc
e the continuous time functions

f 1 (t) nd d1 (t) in order to recover the conti
nuous time sign l f (t). rom this sign l the origin l discrete sign l then m 
be recovered b
ro
erl ev lu ting the corres
onding ex
nsion formul 3.5. We
will show now th t this is not necess r. Inste d, it will be
ossible to recove
r the origin l sequence b

ling du l lters s described in sect. 5.3.
1 We st
rt from sequence {fk


}3 (cf. eq. 3.6) nd com ute new sequence k=0 17 { k }k=
0 s illustr ted in ig. 3.5.
3
r
3
1 2
r
7 2
r
1 T 2
7 2 1 2
r
           r  
1 0 = 6
1 T 2
           r  
1 1 = 7
Fig. 3.5. Computation o  the rst our elements o  H  1 .
1 1 The computation o  F0 , . . . , F3 is indicated in the diagram, the remainin
g elements are computed analogously. Thus nally we obtain 1 {Fk }7 = k=0
33 77 3, 3, , , 4, 4, , 22 22
.
The interpretation o  diagram 3.5 is similar to diagram 3.4. The di erence is that
now the input sequence is symbolized by the bottom row and the output sequence
by the top row. Obviously the length o  the input sequence is doubled. The indic
ated procedure is exactly the application o  a dual lter as explained in sect. 5.
3 (c . Fig. 5.7). Thus we may write compactly F 1 = H  1, where the lter coe cients
associated to the dual lter H read h0 = 1 , h1 = 1 . 2 2 Analogously, starting 
rom {d1 }3 (c . eq. 3.11) we may compute k k=0 D1 = G d1 , where the lter coe cients
associated to the dual lter G read g0 = 1 , g1 = − 1 . The re der m  verif th t
2 2

(3.15)
(3.16)
52
3 The Discrete W velet Tr nsform
1 {Dk }7 = k=0
111 1 11 1, −1, − , , , − , − , 222 2 22 97 4, 2, 3, 4, , , 1, 2 22
.

Adding both sequences we obt in {( 1 + D1 )k }7 = k=0 = 1 {fk }7 . k=0 2

Thus,
rt from the f ctor 1 ,
we h ve reconstructed the origin l dis2 crete si
gn l {fk } b dding sequences 1 nd D1 . Remembering th t both sequences were
obt ined b

ling du l lters we rewrite this equ tion s f = 2(H f 1 + G d1 ).
(3.17)
This is the nnounced reconstruction formul : We obt in the origin l sequence di
rectl from sequences f 1 nd d1 , res
ectivel, b

ling du l lters! Thus the
discrete sign l m  be reconstructed without intermedi tel com
uting the corre


s onding continuous time sign l f (t). Remembering th t f 1 = Hf nd d1 = Gf (cf
. eqs. 3.13 nd 3.14) we m  rewrite 3.17 even more com
ctl: 2(H H + G G) = Id.
The Id smbol denotes the identit, le ving sequences unch nged. We n ll summ r
ize the results. One ste
version of the DWT with the H r w velet Let {fk } den
ote the origin l discrete sign l. H nd G denote lters with subs m
ling nd coe cie
nts h0 = nd g0 = 1 1 , g1 = − 2 2 , 1 1 , h1 = 2 2 (3.18)
res
ectivel. H nd G denote the corres
onding du l lters. 1. Decom
osition ) Com

ute the sequences f 1 = Hf nd d1 = Gf . b) f 1 then is co rser version of th
e origin l sign l, d1 stores the det ils needed to recover f . 2. Reconstruction
) Com
ute nd dd the sequences H f 1 nd G d1 . b) Multi
l the resulting sequ
ence with 2.
3.3 Gener liz tion to D ubechies W velets
53
Let us close this section with some concluding rem rks: 1. The consider tions of
this section were c rried out with the ex m
le sign l 3.3 but m  be gener lize
d to rbitr r discrete sign ls of length N . 2. The
rocedure formul ted bove
is
urel discrete, i.e., does not rel on continuous time sign ls. Thus it su ces
for digit l sign l
rocessing
ur
oses. The rel tion to the CWT m  be est blis
hed b introducing the continuoustime sign l f (t) corres
onding to eq. 3.5 nd
considering eq. 3.12. 3.
In order to sim
lif terminolog, we sh ll s
e k of H,
H , G, G s “ lters”. rom the res
ective smbol it will be cle r th t we ctu ll me n
ltering combined with down  or u
s m
ling, res
ectivel.
3.3 Gener liz tion to D ubechies W velets
Before gener lizing the results of the
receding section to w velets di erent from
the H r w velet we slightl modif the lters H, H , G, G introduced bove. Let u
s ssume th t the corres
onding coe cients re d √ √ 2 2 , h1 = h0 = 2 2 nd √ √ 2 2 g0 = ,
g1 = − 2 2 , (3.20) (3.19)
inste d of the v lues used so f r. This me ns th t we multi
l e ch of √ the lters
with 2 without ch nging the smbols. Then the reconstruction equ tion 3.18 gets
even more sim
le: H H + G G = Id. (3.21)
Corres
ondingl, the n l multi
lic tion with 2 in the reconstruction
rocedure (s
te
2 b in the scheme on
ge 52) c n be omitted. Decom
osition nd reconstructi
on usu ll re re
resented b the di gr ms in eq. 3.22.
G 1
f −→ d H f1
f ⊕ ←− d1 H f1
G
(3.22)
The left di gr m re
resents decom
osition, the right di gr m reconstruction. The
⊕ smbol denotes the ddition of the in
ut sequences G d1 nd H f 1.
54
3 The Discrete W velet Tr nsform
We summ rize the results obt ined so f r: 1. St rting from coe cients 3.19 nd 3.2
0, res
ectivel, we constructed lter
irs H, H nd G, G such th t hk =
k
√ 2
(3.23) (3.24) (3.25)
gk = 0
k
H H + G G = Id
Note th t, though never mentioned ex
licitl, eqs. 3.23  3.24 obviousl re s t
is ed. 2. Given discrete sign l f = {fk } one ste
version of DWT m  be
er
formed s indic ted in eq. 3.22. 3. De ning the continuous time sign l f (t) = k f
k φH (t − k), the sequence d1 = {d1 } is rel ted to the the CWT of f (t), c rried ou
t for the H rk w velet ψH , simil rl to eq. 3.12. Note th t with coe cients 3.20 t
his equ tion must be modi ed to LψH f (2, 2k) = 11 d cψH k (3.26)

since, s ex
l ined bove, the origin l g coe cients were multi
lied with √ 2. or s
im
licit nd future reference we sh ll denote this connection between
urel
discrete view
oint (sign ls re sequences) nd continuous view
oint (sign ls
re continuous time functions) b (H, G) ↔ (φH , ψH ).

(3.27)
The du l lters (H , G ) re omitted in this shorth nd not tion, since the re un
iquel rel ted to (H, G). In sect. 2.2.2 we noted th t the H r w velet h s onl
one v nishing moment. We lso g ve rguments there th t for sign l n lsis
ur

oses it is desir ble to h ve s m n v nishing moments s
ossible:
or sign
l which m  be modeled b
iecewise
olnomi l functions we noted th t the CWT v
nishes, if the w velet h s M v nishing moments nd the highest degree k of the

olnomi ls s tis es k < M . So the CWT is concentr ted round sign l regions whic
h c n not be modeled in this w . This is im
ort nt for detecting nd loc lizing
sign l ch nges. It moreover is useful for
ro
ert c lled “energ com
ction” [38
], b sic
rerequisite for

ling w velets to sign l com
ression. L ter we sh
ll discuss rel ted ex m
les 1 .
1

or the v nishing moments condition refer to eq. 2.31. The
r ctic

l im
lic tion
s of this condition

re discussed in more det il on
ges 27 – 29. or n illustr
tion refer to ig. 2.3.
3.3 Gener liz tion to D ubechies W velets
55
Therefore it is n tur l question to sk if the
rocedure given bove c n be ge
ner lized: Is it
ossible to develo
one ste
DWT, i.e., to design lters s tisf
ing eqs. 3.23 – 3.25, which re rel ted to sc ling function φ and a wavelet ψ s in
dic ted in 3.27, such th t ψ h s more v nishing moments th n ψH ? This
roblem h s b
een solved com
letel b Ingrid D ubechies [7]. She showed th t equ tions 3.23 – 3
.25 together with ddition l restrictions on the lter coe cients, re ecting in
rtic
ul r the v nishing moments condition, le d to f mil of lters nd corres
onding
sc ling functions nd w velets. This f mil usu ll is c lled the “f mil of D ub
echies w velets” nd e ch member of this f mil is l belled b n tur l number n
= 1, 2, 3, . . .. It turns out th t for e ch member of this f mil it is su cient
to s
ecif onl the lter coe cients of H. The coe cients of G then c n be com
uted c
cording to eq. 3.28, in
rticul r the number of g coe cients will be identic l wi
th the number of h coe cients. An logousl, it will be su cient to s
ecif the
corre
s
onding sc ling function φ, since then ψ lso is determined (cf. eq. 3.29). or g
iven n the sc ling functions nd w velets re denoted with dbn, the rel ted lters
will h ve 2n coe cients. After these
relimin ries concerning not tion nd b sic

ro
erties of the f mil of D ubechies w velets we c n give the nswer to the qu
estion from bove: dbn w velets h ve n v nishing moments! In T ble 3.1 we indic
te the coe
cients for the rst three members of the f mil.
T ble 3.1. ilter coe cients rel ted to D ubechies w velets for n = 1, 2, 3. n {hk
} 1 {hk }1 = k=0 2 {hk }3 = k=0
√ √ 2 , 22 2 √ √ √ √ 1− 3 3− 3 3+ 3 1+ 3 √, √, √, √ 42 42 42 42
3 {hk }5 = {0.332671, 0.806892, 0.459878, −0.135011, −0.085441, 0.035226} k=0
The corres
onding g coe cients m  be com
uted ccording to gk = (−1)k h1−k (cf. [7]).
(3.28)
56
3 The Discrete W velet Tr nsform
The sim
lest member of the f mil of D ubechies w velets is db1. The corres
ondi
ng sc ling function nd w velet re given b the H r sc ling function φH and the
Haar-wavelet ψH , res
ectivel. Indeed, the rel ted lters h ve 2 coe cients √ eqs 3.19
nd eqs 3.20) nd the g coe cients s tisf the rel (cf. √ tions g0 = 22 = h1 nd g1
= − 22 = −h0 in ccord nce with eq. 3.28. Likewise, for n = 2 eq. 3.28 im
lies √ √√ √ 3−1 1
3+ 3 3− 3 1 √ ,− √ , √ , √ {gk }k=−2 = , 42 42 42 42 s e sil m  be veri ed. Rel tions 3
nd 3.24 re lso immedi tel checked for n = 2. The w velets m  be com
uted fro
m the res
ective sc ling functions b the “sc le equ tion” (cf. [7]): ψ(t) = √ 2
k
gk φ(2t − k).
(3.29)
Ag in this rel tion is e sil veri ed for the H r w velet. With coe cients 3.20 the
equ tion m  be sim
li ed s follows: √ 2
1
ψH (t) =
gk φH (2t − k)
k=0
√ =2
√ √ 2 2 φH (2t) − φH (2t − 1) 2 2

= φH (2t) − φH (2t − 1). Indeed (cf. ig. 2.2) the H r w velet is com
osed from two shr
unk versions of the sc ling function on the interv ls [0,


1
) nd [ 1 ,
1), res
ectivel , the 2 2 l tter one with inverted sign. 3.3.1 rom ilters to unctions
In the
receding section we s
eci ed the lter coe cients for the rst three members of
the f mil of D ubechies w velets. But u
to now onl for db1, i.e., for the H
r sstem, we know both the lter coe cients nd the corres
onding w velet nd sc li
ng function. In this section we describe n iter tive
rocedure (“c sc de lgorith
m”), which en bles us to com
ute the sc ling function φ(t) or any member dbn o  the
amiliy o  Daubechies-wavelets, provided the h-coe cients are given [8]. The corr
esponding wavelet results rom 3.29. We shall give a detailed description o  the
procedure or the Haar-system, i.e., or db1. So the H - lter mentioned below bel
ongs to the coe cients 3.19. The generalization to arbitrary dbn then should be ob
vious.
3.3 Generalization to Daubechies-Wavelets
57
1. De ne the sequence e = {ei } with ei = 1i=0 0 else
Represent this sequence by a piecewise constant continuous-time unction as oll
ows: a) Subdivide the t-axis in intervals o  length 1 such that t = 0 is the cen
ter o  one o  these intervals. Denote this interval with I0 and the neighboring
intervals with Ii (i = 1, 2, 3, . . .) and Ii (i = −1, −2, −3, . . .), res
ectivel. b
) Re
resent the sequence e = {ei } b continuous time function e(t), where e(t
) = 1 t ∈ I0 0 else

The to
row of ig. 3.6 shows the sequence e = {ei } nd the rel ted function e(
t).
i.e.,

l du l lter simil rl s i

√ 2. Now com ute the sequence e1 = 2H e,

n ig. 3.5, where the coe cient v lues re re l ced b 1 inste d of 1 2 nd in the
bottom row we insert the sequence v lues ek
. The resulting sequence is re
rese
nted b the left gr
h in the second row of ig. 3.6. In
rticul r, we h ve e1
= e1 = 1. 0 1 Simil rl to bove, continuous time function e1 (t) is ssoci te
d to this sequence with the following di erences: The length of the interv ls used
for subdividing the t  xis is now 1 inste d of 1. Moreover 2 ⎧1 ⎨ e0 = 1 t ∈ I0 e1 (t
) = e1 = 1 t ∈ I1 ⎩1 0 else This function is re
resented by the right gra
h in the s
econd row of Fig. 3.6. √ 3. √ Continue the ste
s described above, i.e., com
ute e2 =
2H e1 , e3 = 2H e2 , . . .. Associate the cores
onding continuous time function
s e2 (t), 3 e (t), . . . where each function is
iecewise constant on intervals,
whose length is divided by two when
roceding from ei (t) to ei+1 (t). Sequence
e2 = {e2 } and the corres
onding continuous time function e2 (t) k are shown in
the last row of Fig. 3.6 Figure 3.6 shows how the “initial estimates” e(t), e1 (t),
e2 (t), . . . converge to the correct scaling function φH (t) de ned in eq. 3.4. Li
kewise, or general n, the procedure described above leads to an approximative s
cheme or computing the dbn-scaling unction. The quality o  the approximation i
s controlled by the number o  iterations. As mentioned above, the corresponding
wavelet may be obtained rom eq. 3.29.
58
3 The Discrete Wavelet Trans orm
Cascade algorithm: Sequences 1.5 1 0.5 0 −0.5 −2 1.5 1 0.5 0 −0.5 −2 1.5 1 0.5 0 −0.5 −2
1 2

C sc de lgorithm: unctions 1.5 1 0.5 0

0
2
4
−0.5 1.5 1 0.5 0
−1.5
−0.5
0.5
1.5
0
2
4
−0.5 1.5 1 0.5 0
−1.25
−0.25
0.75
1.75
0
2
4
−0.5
−1.125 −0.125
0.875
1.875

ig. 3.6. e, e , e re
resented s sequences (left) nd functions (right). The gr

id lines in the function
lots indic te the res
ective subdivision of the t  xis
.

or the db2 c se this is illustr ted in ig. 3.7. The

roxim tion e1 (t) corre
s
onding to the rst iter tion is dis
l ed in the u

er left di gr m. The re der
m  verif th t the v lues of the nonzero ste
heights re given √ b the corres
o
nding coe cients from T ble 3.1, multi
lied with 2. The lower left di gr m shows t
he w velet

roxim tion derived from e1 (t); the

roxim tions to sc ling func
tion nd w velet fter 7 iter tions re dis
l ed in the u

er right nd lower r
ight di gr ms.
φdb2. Number o  iterations: 1. 1 0.8 0.6 0.4 0.2 0 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 φdb
2. Number o  iterations: 7.
1
0.5
0 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4
ψdb2. Number of iter tions: 1. 1 1 0.5 0 −0.5 −1 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 0.5 0 −0
5 −1 −1.5 −1 −0.5 0
ψdb2. Number of iter tions: 7.
0.5
1
1.5
2
2.5
3
3.5
4

ig. 3.7. Iter tive construction of db2 sc ling function nd w velet. One iter t
ion (left), 7 iter tions (right).
3.3 Gener liz tion to D ubechies W velets
59
We note n “im
rovement in smoothness”: The db1 w velets nd sc ling functions re d
iscontinuous, where s in the db2 c se both functions re continuous. Still these
functions look r ther rugged nd in f ct higher order dbn w velets nd sc ling
functions get smoother. M them tic ll this st tement
c n be m de
recise using
the notion of “regul rit” [24], it is illustr ted in ig. 3.8 for db5.
φdb5. Number o  iterations: 7. 1 0.8 0.6 0.4 0.2 0 −0.2 −4 −3 −2 −1 0 1 2 3 4 5 6 7
ψdb5. Number of iter tions: 7. 1 0.5 0 −0.5 −1 −4
−3
−2
−1
0
1
2
3
4
5
6
7

ig. 3.8. db5: sc ling function nd w velet fter 7 iter tions.

So, in f ct, for incre sing n we not onl obt in w velets with n incre sing num
ber of v nishing moments but lso w velets with incre sing regul rit. Both
ro

erties re desir ble for sign l n lsis
ur
oses2 . 3.3.2 Tr nsfer Pro
erties
  


or n dbn  lter the H s mbol ctu ll denotes the lic tion of digit l lter
H
with the
coe cients {hk }, followed b subs m
ling
rocedure (cf. sect. 5.3, ig


. 5.5). or
the G s mbol nd the
corres onding
coe cients {gk } n logous conventi
ons hold. rom digit l sign l rocessing view oint it is quite instructive to
stud the frequenc res
onse of H nd G, res
ectivel. In this section we sh ll
consider

onl the m
litude res
onse, the
h se res
onse will be investig ted l
ter. requenc res
onse, m
litude

res
onse nd
h se res
onse re de ned in eqs.
5.18 nd 5.19, res ectivel . or short summ r of these to
ics the re der is r


eferred to sect. 5.3.
2

We remind the re der to the com
rison of db2 with db4 shown in igs. 2.5 nd 2.


6, res ectivel .
60
3 The Discrete W velet Tr nsform

or
db1 to db4 the m
litude res
onse of the cores
onding H nd G lters re shown
in ig. 3.9. The gr
hs re rr nged in clockwise order with db1 dis
l ed in th
e u

er left di gr m. In e ch di gr m both the m
litude res
onses of H (solid)
nd G (broken) re
lotted.
db1, m
litude res
onse (H: solid, G: broken) 1.5 1.5 db2, m
litude res
onse (H
: solid, G: broken)
1
1
0.5
0.5
0 0 1.5
0.5
1
1.5
2
2.5
3
3.5
0 0 1.5
0.5
1
1.5
2
2.5
3
3.5
db4, m
litude res
onse (H: solid, G: broken)
db3, m
litude res
onse (H: solid, G: broken)
1
1
0.5
0.5
0 0
0.5
1
1.5
2
2.5
3
3.5
0 0
0.5
1
1.5
2
2.5
3
3.5

ig. 3.9. Am
litude res
onse of db1 db4 lters (clockwise from bove left).
The
lots indic te th t the res
ective H  lters exhibit low 
ss ch r cteristic,
where s the G  lters re high 
ss lters. Moreover, obviousl with incre sing n th
e res
ective lter
erform nce im
roves: The cuto frequencies re more sh r
l loc
lized nd tr nsmission s well s su

ression of the res
ective frequenc r nges
works better nd better. This im
rovement in lter beh vior corres
onds to the di
scussion in the
receding section, where we demonstr ted, how w velets nd sc li
ng functions belonging to the lters “im
rove” when n is incre sed.
3.4 Multisc le An lsis
Let us ssume th t for cert in n the corres
onding dbn  lter coe cients re given.
So we h ve
ir of lters (H, G) whose coe cients s tisf eqs. 3.23 –
3.4 Multisc le An lsis
61

3.25. or discretel s m
led sign l f one ste
version of DWT c n be c rri

ed out s de
icted in eq. 3.22. In this section we sh ll show how this
rocedure
m  be iter ted, le ding to multisc le n lsis of f . It will turn out th t
the lgorithm in quite str ightforw rd m nner m  be gener lized to two dimens
ion l sign ls. So we will extend the DWT techniques to im ge
rocessing. 3.4.1 O
ne Dimension l Sign ls Let us rst consider the decom
osition com
onent of eq. 3.2
2. St rting from f , the sign ls f 1 nd d1 re com
uted b

ling the lter
i
r (H, G) to f . We m  re
e t this
rocedure beginning with f 1 nd obt ining f
2 nd d2 . If we
roceed in this w , i.e., if we

l (H, G) successivel to f
2 , f 3 , . . ., fter J ste
s the sign l is decom
osed s indic ted b the lef
t di gr m of eq. 3.30.
G
H
f −→ d
G
1
f ⊕ ←− d1 H ⊕ ←− d2 H . . . ⊕ ←− dJ H fJ
G G G
1 f −→ d2 H . . .
(3.30)
f J−1 −→ dJ H fJ
G
Usu ll f J is c lled “

roxim tion sign l”, the rem ining sign ls dj (j = 1, . . .
, J) “det il sign ls”. Remember th t f J results from re
e ted

lic tion of l
ow 
ss lter, where s for the det il sign ls the n l com
ut tion ste
lw s is hi
gh 
ss ltering. Below we sh ll discuss these sign ls in more det il nd show som
e ex m
les. Note th t in e ch


ltering
ste the sign l length is divided b 2, whi


ch m be re resented s in ig. 3.10. rom this di gr m it is obvious th t ther
e exists n u

er limit for the number of decom
osition ste
s J: If the origin l
sign l is of length 2L , the m ximum v lue of J will be L. Since e ch decom
osi
tion ste
m  be inverted s indic ted in the right di gr m of eq. 3.22, J ste

DWT decom
osition of f is inverted b the lgorithm shown in the right di gr m
of eq. 3.30. Ag in, the ⊕ smbol re
resents ddition of the res
ective in
ut sequ
ences. Equ tion 3.26, est blishing the link from sequences to continuous time fu
nctions m  be gener lized to J ste
DWT s follows: De ning the continuous time
sign l f (t) = k fk φ(t − k), the corres
ond 
62
3 The Discrete W velet Tr nsform f H  d G 1 H  d G 2
©  d
d1
step 1 step 2 . . .
©d  
d2
H  d G ©d   . . .
Fig. 3.10. J-step DWT-decomposition o  a signal 
ing CWT-values or a = 2j and t = 2j k (with integer k) can be computed accordin
g to Lψ f (2j , 2j k) = 1j d (j = 1, . . . , J). cψ k (3.31)
Ag in, (φ, ψ) denote sc ling

function nd w velet, ssoci ted to lters (H, G) s disc
ussed
bove. Simil rl to ig. 3.3 the bove subset of the t  
l ne is visu liz
ed in ig. 3.11 for J = 3. It is cle r how to extend this subgrid for gener l J.
This
lot ex
l ins wh the J ste
DWT decom
osition lso is c lled “multisc le n
lsis”.
0 2 4 6 8
Et
2r 4r
r
r r
r
r r
8r
r
c

ig. 3.11. 3 ste
version of the DWT: Subset of the t  
l ne.

Note th t from ig. 3.10 it follows th t J ste
DWT le ves the tot l number of
d t unch nged: A

ling J ste
DWT to the N sequence elements of
3.4 Multisc le An lsis
63
the origin l sign l f = {fk }N −1 , we obt in the sequences f J , dJ , dJ−1 , . . .
, d1 , k=0 where the tot l number of d t resulting from dding the lengths of t
hese sequences, g in3 will be N . Moreover, the decom
osition
rt of eq. 3.30

rovides f st 4 lgorithm for com
uting w velet tr nsform v lues vi the seque

nces dj . Thus, when com
ring with the situ tion visu lized in ig. 3.1, we con
clude th t the J ste DWT le ds to sim le nd quick lgorithm for com
uting w



velet tr nsforms without redund nc on subgrid of the t  
l ne, which t kes i
nto ccount v ring sc le f ctors in contr st to the discretized CWT. U
to now
(cf. eq. 3.2) we ssumed TS = 1. As nnounced on
ge 45
the extension to n rb
itr r s m
ling dist nce TS is ver sim
le: Looking t ig. 3.11 re
l ce on the
t  xis the l bels 0, 2, 4, 6, . . . b 0, 2TS , 4TS , 6TS , . . . nd corres
ond
ingl on the  xis the l bels 0, 2, 4, 8, . . . b 0, 2TS , 4TS , 8TS , . . ..
So, e.g., Lψ f (4TS , 4TS ) = 12 d. cψ 1

Ex m
les In ig. 3.12 the result of 2 ste
DWT with db1  lters is dis
l ed. ro



m bove, the sign
l f , the det il sign ls d1 nd d2 nd the roxim tion sign
l f 2 re shown. igure 3.13 shows the results of the s me tr nsform, but
erfor
med with db2  lters inste d of db1. The results re in ccord nce with the
ro
ert
ies of dbn  lters collected in sections 3.3 nd 3.3.1. Since the db2 w velet is sm
oother nd h s more v nishing moments th n the db1 w velet, one ex
ects the db2 
det il sign ls to be better concentr ted round signi c nt sign l ch nges th n the
db1 det ilsign ls. Since f h s regul r

e r nce, i.e., signi c nt sign l ch n
ges re l cking, this sim
l me ns th t det il sign l v lues should be r ther sm
ll. Obviousl this is true for both lter t
es nd indeed the db2 det il sign ls
re closer to zero th n the corres
onding db1 det il sign ls. The concentr tion

ro
ert discussed bove h s nother useful consequence. Remember th t the sign
l c n be reconstructed from

roxim tion nd det il sign ls b the reconstruct
ion
rt of eq. 3.30. Therefore  due to the concentr tion
ro
ert  the J ste

DWT is method to concentr
te the full inform tion stored in the sign l in few
signi c nt coe cients. In igs. 3.12 nd 3.13 these coe cients b sic ll re the sequ
ence elements of f 2 nd the concentr tion mech nism works more e cientl for db2
th n for db1. This
3 4
To be
recise, for rbitr r N the tot l number of tr nsformed d t c n be sligh
tl l rger th n N . Refer to sect. 3.4.3 for short discussion. In f ct, since
the lgorithm reduces to discrete ltering, the number of o
er tions needed for co
m
uting the dj sequences incre ses line rl with N .
64
3 The Discrete W velet Tr nsform
2−ste
−DWT with db1 2 f 0 −2 0 2 d1 0 −2 0 2 d2 0 −2 0 2 f2 0 −2 0 10 20 30 40 50 60 70 10
0 30 40 50 60 70 10 20 30 40 50 60 70 10 20 30 40 50 60 70

ig. 3.12. 2 ste
DWT with db1.

2−ste
−DWT with db2 2 f 0 −2 0 2 d1 0 −2 0 2 d2 0 −2 0 2 f2 0 −2 0 10 20 30 40 50 60 70 10
0 30 40 50 60 70 10 20 30 40 50 60 70 10 20 30 40 50 60 70

ig. 3.13. 2 ste
DWT with db2.

concentr tion fe ture is c lled “energ com
ction” [38]
nd b sic
rerequisite fo
r d t com
ression. We will return to it in ch
. 4. In ig. 3.14 it is illustr
ted with histogr ms. The to
gure shows the histogr m of f com
uted s follows: T
ke the sm llest nd the l rgest sequence element nd subdivide the interv l bet
ween these v lues in 20 equ ll s
ced bins. Now for e
ch bin count the number o
f sequence elements whose v lues re cont ined in it. or the tr nsformed sign l
we
roceeded s follows: Merge sequences f 2 , d2 nd d1 into one sequence, the
n continue s described for f . In this w  we obt ined the db1 histogr m (botto
m left) nd the db2 histogr m
3.4 Multisc le An lsis
65

(bottom right). irst we note th t both histogr ms re more sh r
l
e ked (with


the l rgest e k round zero) th n the histogr m of the origin l sign l. This i
s in ccord nce with our discussion from bove concerning the concentr tion fe t
ures of the J ste
DWT in gener l. Moreover, the db2 histogr m is more sh r
l

e ked th n the db1 histogr m, which lso m tches our ex
ect tion.
Histogr m of sign l v lues 12
10
8
6
4
2
0 −1.5 14
−1
−0.5
0
0.5
1
1.5
Histogr m of tr nsformed v lues (db1)
Histogr m of tr nsformed v lues (db2) 25
12
20
10
15
8
6
10
4
5
2
0 −2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
0 −2
−1.5
−1
−0.5
0
0.5
1
1.5
2

ig. 3.14. Above: Histogr m of the origin l sign l. Below left: Histogr m of the
tr nsformed sign l (db1); below right: Histogr m of the tr nsformed sign l (db2
).
3.4.2 Two Dimension l Sign ls (Im ges) Until here, the sign ls considered were s
equences, usu ll recorded s time series. Digit l im ges, in contr st, re re

resented s m trices f = {fmn } (m = 0, . . . , M − 1; n = 0, . . . , N − 1). (3.32
)
Simil rl to the 1 d c se, f denotes the m trix, fmn the element in row m nd co
lumn n. M nd N denote the tot l number of rows nd columns, res
ectivel. Since
f re
resents
icture, fmn will be c lled the “
ixel” in row m nd column n. The i
m ge is gener ted b re
resenting e ch
ixel b gr  v lue such th t d rk corr
es
onds to sm ll nd bright to l rge
ixel
66
3 The Discrete W velet Tr nsform

v lues5 . An ex m
le is shown in the u

er
rt of ig. 3.15, here M = 240 nd N

= 320. The lters discussed so f r tr nsformed sequences into sequences. The c n
be extended to im ges b

ling them to rows, or columns, res
ectivel. The k
ind of

lic tion is indic ted b corres
onding index. So, e.g., the lter Hr i
s de ned s follows: St rting with the origin l digit l im
ge f new im ge Hr f i
s gener ted b ltering ever row of f se
r tel with H. rom the de ning equ tion
5.20 it follows th t (Hr f )mn =
k
hk−2n fmk .
(3.33)
Note th t, since the length of sequence is divided b 2 when

ling H, corre
s
ondingl the length of the rows, i.e., the number of columns,
will be divided
b two when

ling Hr . As n ex m
le, the lower
rt of ig. 3.15 shows Hr f
, the origin l f is dis
l ed in the u

er
rt. Here we used the db2 H  lter with
coe cients given in T ble 3.1. Com
letel n logousl, the smbol Hc is used if H
is

lied to columns. Corres
ondingl, in this c se the number of rows will be
divided b 2. It should be cle r now, how to

l the rem ining lters, i.e., th
e G  lter nd the du l lters H nd G , to rows or columns, res
ectivel, nd how to
denote this. Given digit l im ge f , one ste
DWT is de ned b

ling ll fo
ur
ossible combin tions resulting from ltering long rows nd columns with H nd
G, res
ectivel: f → {Hr Hc f, Hr Gc f, Gr Hc f, Gr Gc f } (3.34)
Note th t, for ex m
le, Hr Hc f will h ve tot l number of
ixels, which is div
ided b 4, when com
red with the tot l number of
ixels of f . The re son is th
t the number of columns nd the number of rows is divided b 2. The s me re son
ing

lies to the rem ining im ges Hr Gc f, Gr Hc f nd Gr Gc f . Therefore, if
we dd the number of
ixels of ll gener ted im ges, we see th t  s in the on
e dimension l c se  the tot l number of d t rem ins unch nged, when one ste

DWT is

lied6 . The im ges Hr Gc f, Gr Hc f nd Gr Gc f , re c lled “horizont
l”, “vertic l” nd “di gon l” det il sign ls, res
ectivel. To motiv te this terminolog w
e consider Hr Gc f , the rgument for the other im ges is com
letel n logous.
A

ling Gc to f , i.e.,

ling G to the columns of f , me ns th t we com
ute
the w velet tr nsform of e ch column. As we h ve seen, we com
ute sign l,
5
6
We sh ll restrict ourselves to this kind of im ges, lso c lled “gr  level im ges”.
The techniques develo
ed in this section e sil c n be d
ted to other colorm

s nd to colored im ges.
or surve on im ge re
resent tions refer to, e.g.,
[14]. An logousl to the rem rks m de in the footnote on
ge 63, the tot l numb
er of tr nsformed d t c n be slightl incre sed. Refer to sect. 3.4.3 for sho
rt discussion.
3.4 Multisc le An lsis
67
H −filtering with db2
r
50 100 150 200 100 200 300
50 100 150 200 50 100 150

ig. 3.15. Above: Origin l im ge. Below: Result of

ling H (here: db2) to e c
h row.
68
3 The Discrete W velet Tr nsform
which b sic ll “re cts” on sign l ch nges. In other words: We look long the column
s for sign l ch nges. On the other h nd, horizont l edges will give rise to sign
l ch nges when sc nning columns, therefore the im ge Hr Gc f will re ct on hori
zont l edges. Moreover, since (cf. eq. 3.26) the w velet tr nsform is com
uted f
or sc le 21 , the horizont l det il sign l Hr Gc f lso is denoted with d1h . Co
m
letel n logousl, vertic l nd di gon l det il im ges re ct on corres
onding
l directed edges nd therefore Gr Hc f is denoted with d1v , Gr Gc f with d1d .
We n ll rem rk th t  n logousl to the 1 d c se tre ted bove the low 
ss lte
red im ge Hr Hc f is c lled “

roxim tion sign l”, denoted with f 1 . Usu ll the f
our subim ges {f 1 , d1h , d1v , d1d } gener ted b one ste
DWT re rr nged l
ike f 1 d1h d1v d1d (3.35)

igure 3.16 shows one ste
DWT c rried out with H r  lters. The resulting subim
ges re rr nged s in eq. 3.35.
Origin l
One−ste
−DWT with db1
50 100 150 200 250 50 100 150 200 250
50 100 150 200 250 50 100 150 200 250

ig. 3.16. Left: Origin l im ge. Right: Result of the one ste
DWT (here: db1).
In
rticul r for horizont l edges, the bove mentioned direction l sensitivit
of the det il sign l m  be veri ed. Ag in, n logousl to the
rocedure described
in sect. 3.4.1, “multisc le n lsis” of digit l im ge f c n be
erformed b


ling one ste
DWT 3.34 successivel to f 1 , f 2 = Hr Hc f 1 , f 3 = Hr Hc f
2 , . . .. Thus the equiv lent of eq. 3.30 re ds
3.4 Multisc le An lsis
69
Gr Hc
−→ d1v
f
Hr G c
f Hr Gc d1h −→
1d −→ d Hr Hc Gr Hc Gr Gc
←− d1h ←− d1d
⊕ Gr Hc d1v ←−
G G rc
f
1 Hr Gc Gr Gc
−→ d2v −→ d2h
2d
Hr Hc
Hr G c
−→ d Hr Hc . . .
Gr Hc
←− d2h ←− d2d (3.36)
⊕ Gr Hc d2v ←−
G G rc
Hr Hc . . .
−→ dJv −→ dJh
Jd
f
J−1 Hr Gc Gr Gc
−→ d Hr Hc fJ
Hr G c G Hc r
←− dJh ←− dJd
⊕ ←− dJv
G G rc
Hr Hc fJ
Ag in, the left di gr m re
resents decom
osition, the right di gr m reconstructi
on. The smbol ⊕ now denotes ddition of m trices r ther th n sequences. Gener liz
ing the re
resent tion from eq. 3.35, the result of twoste
DWT, for ex m
le,
c n be re
resented s ⎞ ⎛ 2 2h

fd 1h 2v 2d d ⎠ ⎝d d (3.37) 1v 1d d d So, if we continue
to decom
ose the
origin l of ig. 3.16, we obt in decom
osition s shown in th
e right
rt of ig. 3.17. 3.4.3 Im
lement tions with the MATLAB W velet Toolbox
In this section we give short descri
tion of functions cont ined in the MATLA
B W velet Toolbox, which im
lement the J ste
DWT for 1 d nd 2 d sign ls, res
e
ctivel. Rel ted functions which m ke use of the toolbox
rocedures h ve been tes
ted with MATLAB 6.5, rele se 13 nd m  be down 
70
3 The Discrete W velet Tr nsform
Two−ste
−DWT with db1
50 100 150 200 250 50 100 150 200 250
50 100 150 200 250 50 100 150 200 250

ig. 3.17. Left: One Ste
DWT (cf. ig. 3.16). Right: Two Ste
DWT, obt ined b
decom osing the u er left subim ge of the one ste
DWT.

lo ded from the URL given in sect. 1.3. Ag in, s mentioned e rlier, b sic und
erst nding of MATLAB snt x is required. 1 d functions The decom
osition
rt of
the one dimension l J ste
DWT (cf. eq. 3.30) is im
lemented in the toolbox fun
ction w vedec. A
rotot
e function c ll re ds [f_tr ns,L] = w vedec(f,J,’dbn’); The
origin l sign l is stored in the vector f, the result of the tr nsform in the v
ector f_tr ns. The l tter is org nized s follows : f2 Length: L(1) d2 Length: L
(2) d1 Length: L(3)

Here for illustr tion
ur
oses J = 2 w s chosen, refer lso to ig. 3.10. As ind
ic ted, the lengths of the individu l subsequences re stored in the com
onents
of “book kee
ing vector” L. Thus, e.g., in MATLAB not tion d2 m  be extr cted from
f_tr ns s follows: d_2=f_tr ns((L(1)+1):(L(1)+L(2))); Note th t, if N denotes t
he length of the origin l sign l, then in our ex m
le L(1) ≈ N N N , L(2) ≈ , L(3) ≈ .
4 4 2
3.4 Multisc le An lsis
71
The re son for the

roxim te equ lit is th t N not necess ril is
ower of
two; moreover, for dbn  lters with n > 1 the tre tment of sign l bound ries requir
es the subsequences to be slightl longer th n the fr ctions com
uted bove. As
n illustr tion we include the following MATLAB di logue, demonstr ting 2 ste

DWT with sign l of length 1367: >> f=r nd(1367,1); >> [f_tr ns,L]=w vedec(f,2
,’db2’); >> sum(L(1:3)) ns = 1373 >> [f_tr ns,L]=w vedec(f,2,’db3’); >> sum(L(1:3)) ns
= 1376 We see th t f_tr ns in gener l will be slightl longer th n f nd this “le
ngthening e ect” incre ses with lter length. So the book kee
ing vector is essenti l
for the correct h ndling of the individu l subsequences merged in the vector f_t
r ns. It is cle r, therefore, th t it is n im
ort nt in
ut into the toolbox fun
ction w verec im
lementing the reconstruction
rt of the one dimension l J ste

DWT (cf. eq. 3.30). A
rotot
e c ll re ds f = w verec(f_tr ns,L,’dbn’); Note th t
the decom
osition de
th J needs not to be s
eci ed, since it follows immedi tel f
rom the length of the book kee
ing vector L, gener ted b the
receding decom
os
ition. 2 d functions The decom
osition
rt of the two dimension l J ste
DWT (c
f. eq. 3.36) is im
lemented in the toolbox function w vedec2. A
rotot
e functi
on c ll re ds [f_tr ns,L] = w vedec2(f,J,’dbn’); Here the origin l im ge is stored i
n the m trix f with M rows nd N columns. The result of the tr nsform is stored
in the vector f_tr ns, which is org nized s follows : f1 d1h d1v d1d
72
3 The Discrete W velet Tr nsform
Here for illustr tion
ur
oses J = 1 w s chosen, refer lso to eq. 3.35. Subsequ
ence f 1 , for ex m
le, cont ins ll
ixels belonging to the

roxim tion im ge
, but rr nged s vector obt ined b sc nning the

roxim tion im ge row b r
ow. Simil rl the rem ining subsequences re constructed. Inste d of book kee

ing vector in te 2 d c se “book kee
ing m trix” L is gener ted, which stores the m
trix dimensions of the

roxim tion sign l nd the det il sign ls s follows:
L h s 2 columns, the rst row
rovides the number of rows nd of columns, res
ecti
vel, for f 1 . The second row
rovides the dimensions of d1h , d1v nd d1d (whi
ch re ll equ l). Simil rl for J > 1 subsequent rows of L re gener ted. The M
ATLAB di logue shown below illustr tes the use of this function for n im ge wit
h M = N = 16: >> f=r nd(16,16); >> [f_tr ns,L]=w vedec2(f,2,’db1’); >> L(1:3,:) ns
= 4 4 8 4 4 8
>> length(f_tr ns) ns = 256 >> [f_tr ns,L]=w vedec2(f,2,’db2’); >> L(1:3,:) ns = 6
6 9 6 6 9
>> length(f_tr ns) ns = 387 Note th t – n logousl to the 1 d c se – with incre si
ng lter order the tr nsformed sign l will be l rger th n the origin l! Since the
tr nsformed sign l is stored in vector, where the bound ries of the individu l
subsequences m  be determined from L, these subsequences c n be extr cted usin
g L. To build the corres
onding im ge, however, the must be re rr nged to m t
rix!
3.4 Multisc le An lsis
73
Let’s continue the ex m
le from bove with the t sk to extr ct the det il im ge d1
d : >> >> >> >> >> lower=L(1,1)*L(1,2)+3*L(2,1)*L(2,2)+2*L(3,1)*L(3,2)+1; u

er=
L(1,1)*L(1,2)+3*L(2,1)*L(2,2)+3*L(3,1)*L(3,2); d1d=f_tr ns(lower:u

er); d1d=res
h
e(d1d,L(3,1),L(3,2)); size(d1d)

ns = 9 9 As in the 1 d c se the sign l c n be reconstructed from f_tr ns,
rovi

ded the “building reci
e” of f_tr ns is known, which is stored in L. Thus the toolbo
x function w verec2 im
lementing the reconstruction
rt of the twodimension l J
ste
DWT (cf. eq. 3.36) m  be envoked s follows: f = w verec2(f_tr ns,L,’dbn’); d
bn coe cients In sect. 3.3 we summ rized the
ro
erties of dbn  lters without s
ecif
ing how to com
ute the corres
onding h coe cients. Of course, there exists n lg
orithm for th t
ur
ose (cf., e.g., [20]). In the MATLAB W velet Toolbox the fun
ction wfilters does the job. So, e.g., h=wfilters(’dbn’); returns the h coe cients bel
onging to dbn  lters
in the vector h. The m ximum v lue of n is 45. 3.4.4 Gener li
z tion: Biorthogon l ilters As noted in the

endix, 

s mmetric lters h ve lin



e r h se res onse, which reduces sign l distortion. or more det ils the re der
is referred to sect. 5.3. Ph se res
onse
is de ned in eqs. 5.18 – 5.19, ex m
les fo
r smmetric lters re shown on
ge 132. rom T ble 3.1, we conclude th t the h c
oe cients belonging to db1 re smmetric, for n = 2 nd n = 3, however, this is no
t true. In f ct, ll dbn lters with n > 1 re
not smmetric, therefore we c nnot e
x
ect line r
h se res
onse for these lters. igure 3.18 illustr tes this f ct b

lotting
h se res
onses for db1 nd db2. Simil rl to
ig. 3.9 the
h se res
o
nses of H nd G re
lotted s solid nd broken lines, res
ectivel. So the ques
tion rises, if lters m int ining the “good”
ro
erties of dbn with n > 1 nd being s
mmetric c n be constructed. This
roblem h s been
74
2
3 The Discrete W velet Tr nsform
db1,
h se res
onse (H: solid, G: broken)
db2,
h se res
onse (H: solid, G: broken) 4
1.5
3
1
2
0.5
1
0
0
−0.5
−1
−1
−2
−1.5
−3
−2 0
0.5
1
1.5
2
2.5
3
3.5
−4 0
0.5
1
1.5
2
2.5
3
3.5

ig. 3.18. Left: Ph se res
onse for db1. Right: Ph se res
onse for db2.

solved b A. Cohen [5]. or n e s to re d summ r see [24]. We sh ll brie  re
o
rt the m in results for the 1 d c se. The gener liz tion to 2 d sign ls then is
obvious. It turns out th t eq. 3.27 re
resenting the correl tion between discret
e lters nd w velets needs to be extended s follows: ˜ ˜ ˜ ˜ (H, H, G, G) ↔ (φ, φ, ψ, ψ).
r liz tion of eq. 3.25 re ds ˜ ˜ H H + G G = Id. (3.39) (3.38)
In other words: We h ve
ir of w velets nd sc ling functions, res
ectivel,
ssoci ted to corres
onding
ir of high  nd low 
ss lters, such ˜ th t singl
e decom
osition ste
is
erformed b

ling H nd G. This ˜ ste
m  be inverted
b

ling the du l lters H nd G , res
ectivel, nd dding the results. So the
J ste tr nsform 3.30 now m be re
resented s


˜ G
H
f −→ d
˜ G
1
f ⊕ ←− d1 ˜ H ⊕ ←− d2 ˜ H . . . ⊕ ←− dJ ˜ H fJ
G G G
1 f −→ d2 H . . .
(3.40)
f J−1 −→ dJ H fJ
˜ G
The gener liz tion of this equ tion to 2 d sign ls is str ightforw rd.
3.5 A Unifing View
oint: B sis Sstems
75
In im ge com
ression biorthogon l w velet lters re of
rticul r im
ort nce: The
lossless mode of the JPEG 2000 still im ge com
ression st nd rd ˜ [36] uses bio
rthogon l lter
ir H, H with 9 nd 7 coe cients, res
ectivel. The v lues re list
ed below:

T ble 3.2. ilter coe cients of the biorthogon l 9 7  lter
ir. {hk } √ h−4 = 0.0267487
57411 √ 2 h−3 = −0.016864118443√ 2 h−2 = −0.078223266529 2 √ h−1 = 0.266864118443 2 √ h0 =
9018236√ 2 h1 = 0.266864118443 √ 2 h2 = −0.078223266529√ 2 h3 = −0.016864118443 2 √ h4 = 0.
26748757411 2 ˜ {hk } √ ˜ h−3 = −0.045635881557√ 2 ˜ h−2 = −0.028771763114 2 √ ˜ h−1 = 0.29
h0 = 0.557543526229√ 2 ˜ 2 h1 = 0.295635881557 √ ˜ h2 = −0.028771763114√ 2 ˜ h3 = −0.045635
7 2
Note the smmetr of the lters! The corres
onding G  lters re constructed simil rl
 to eq. 3.28. We end this section with
lot of m
litude nd
h se res
onse o
f H (solid) nd G (broken), res
ectivel (both h ving 9 coe cients).
9 coefficients: Am
litude res
onse (H: solid, G: broken) 1.5
3.5 9 coefficients: Ph se res
onse (H: solid, G: broken)
3
2.5
1
2
1.5
1
0.5
0.5
0
0 0
0.5
1
1.5
2
2.5
3
3.5
−0.5 0
0.5
1
1.5
2
2.5
3
3.5

ig. 3.19. Left: Am
litude res
onse of H (solid) nd G (broken); both lters h ve

9 coe cients. Right: Corres
onding
h se res
onse.
3.5 A Unifing View
oint: B sis Sstems
E rlier

in this book (cf. sect. 1.2.6) we
resented qu lit tive com
rision of
the ourier tr nsform with the w velet tr nsform. In this section we sh ll
76
3 The Discrete W velet Tr nsform

consider the discrete versions, i.e., the D T nd the J ste
DWT, nd discuss n


s ect which both tr nsforms h ve in common. It is shown th t both tr nsforms
re ex
nsions of the sign l with res
ect to cert in “b sis sstems” nd these sstem
s
rovide insight into the qu lit tive di erences of both tr nsforms. A
rt from s
ect. 3.6.1, the m teri l covered in this section is not required for the rest of
the book. So it m  be ski

ed if the re der m inl is interested in quick n
d

lic tion oriented w velet introduction. 3.5.1 One Dimension l Sign ls Let u
s st rt with one dimension l discretel s m
led sign l of length N : f = {fk }
N −1 . k=0 φ0 natural φ1 natural . . . (3.41)
We introduce N sequences, called “natural basis system”, as ollows: = = . . . {1, 0
, 0, . . . , 0} {0, 1, 0, . . . , 0} . . .
(3.42)
φN −1 = {0, 0, 0, . . . , 1} n tur l Then f m  be ex
nded s follows:
1 f = f0 φ0 natural + 1 φnatural + . . . N −1
=
i=0
fi φi natural
(3.43)
Note that in these ormulae the summation o  sequences is de ned by ele3 mentwise
addition! For N = 4 the natural basis system φ0 natural , . . . , φnatural is plotte
d in the le t column o  Fig. 3.20. Thus, when starting with the basis system 3.4
2, the signal values k may be viewed as expansion coe cients when expanding the s
ignal with respect to this basis. It is a well known act rom linear algebra th
at basis systems are not unique: There exist other basis systems φ0 , . . . , φN −1 su
ch th t
N −1
f=
i=0
ci φi
(3.44)


rovide
with proper expansion coe cients c0 , . . . , cN −1 . In f ct, the ID T 5.14


s such ch nge of b sis s stems. De ne the following sequences, c lled “ ourier b s
is”:
3.5 A Unifing View
oint: B sis Sstems
77
φ0 ourier = F φ1 ourier = F φ2 ourier F . . . −1 φNourier F
= .. .. .. j2π(N −1)—0 j2π(N −1)—1 j2π(N −1)—2

j2π(N −1)—(N −1) N N N N = √1 {e ,e ,e ,...,
5.14 m  be rewritten s follows: 1 f = √ ( 0 φ0 ourier + F1 φ1 ourier + . . .) F F N
1 =√ N
N −1
1 √ {1, 1, 1, . . . , 1} N j2π1—(N −1) j2π1—0 j2π1—1 j2π1—2 1 √ {e N , e N , e N , . . . ,
−1) j2π2—0 j2π2—1 j2π2—2 1 √ {e N , e N , e N , . . . , e N } N

i φi ourier . F
i=0
(3.46)
In other words: When expanding  with respect to the Fourier basis system −1 φ0 ouri
er , . . . , φNourier , the corresponding expansion coe cients are given by F F Fi √ ,
where Fi denotes the i-th DFT-coe cient computed according to eq. N 5.13. For N =
4 the Fourier basis system is plotted in the middle column o  Fig. 3.20. Note t
hat the sequences belonging to the Fourier basis are complex! The plots show the
real part o  the respective basis sequences. Now assume that a certain dbn-wave
let and a decomposition depth J has been chosen and per orm a J-step-DWT corresp
onding to eq. 3.30. Assume that  J , dJ , . . . , d1 are arranged as indicated
in Fig. 3.10 and merge these ˜ sequences into a new sequence called  . For J = 2
the procedure is indicated below: (  2 d2 d1 )
˜ 
(3.47)
˜ Expanding  with respect to the corresponding natural basis system we may write ˜
=
i
˜˜ i φi natural ,
(3.48)
˜ ˜ where i denotes the i-the element o  the sequence  . Now decompose each ˜i basis
sequence φnatural into subsequences corresponding to the preceding Jstep-DWT deco
mposition. Again, we indicate the procedure or J = 2:
2 ( φ d2 d1 ) φφ ˜ φi natural
(3.49)
78
3 The Discrete Wavelet Trans orm
J Apply the reconstruction part o  eq. 3.30 to φ , dJ , . . . , d1 and denote the
φ φ i reconstructed sequence with φwavelet . Then rom eq. 3.48 it ollows that
=
i
˜ i φi wavelet .
(3.50)
So similarly to the DFT-case (eq. 3.46) a J-step-DWT may be viewed as a change o
 basis: Now the signal is expanded with respect to the wavelet basis instead o 
the Fourier basis and the expansion coe cients are given by the entries o  the tr
ans ormed sequences  J , dJ , . . . , d1 . This is in complete analogy to the D
FT where the expansion coe cients were given by the DFTvalues. In contrast to the
DFT-case, however, the wavelet basis is not unique: Obviously, the basis sequenc
es φi wavelet (i = 0, 1, . . .) depend on both the chosen wavelet and the chosen d
ecomposition depth J. For N = 4 a wavelet basis system is plotted in the right c
olumn o  Fig. 3.20. As mentioned above, this system depends on the chosen wavele
t as well as on the chosen decomposition depth J. Here db1, i.e., the Haar-wavel
et with the corresponding lters, and J = 1 were selected.
φnatural 1.5 1 0.5 0 −0.5 1.5 1 0.5 0 −0.5 1.5 1 0.5 0 −0.5 1.5 1 0.5 0 −0.5 1 1 0 −1 1 1 0
1 1 0 −1 1 1 0 −1 0 −1 0 2 4 0 2 4 0 2 4 0 −1 0 2 4 0 2 4 0 2 4 0 −1 0 2 4 0 2 4 0 2 4 0 −
0 2 4 0 2 4 0 2 4 φwavelet
Real(φFourier)
Fig. 3.20. Basis systems or N = 4. Le t column: φk natural . Middle column: Re(φk o
urier ). Right column: φk F wavelet . From above: k = 0, . . . , 3.
From this gure, together with the above computations, we collect the ollowing ob
servations:
3.5 A Uni ying Viewpoint: Basis Systems
79
The signal  = { k } may be expanded with respect to 3 di erent basis systems: The
natural basis is “strictly local” in the sense that the corresponding basis sequenc
es are nonzero at exactly one position, the expansion coe cients are given by the
original signal values k . The Fourier basis is “global” in the sense that the corr
esponding basis sequences may be nonzero at all positions, the expansion coe  cien
ts are proportional to the DFT-values Fk . The wavelet basis is “local” in the sense
that the corresponding basis sequences are concentrated around several position
s, the expansion coe cients are given by the entries o  the sequences  J , dJ , .
. . , d1 . Thus, in a sense, the wavelet basis system interpolates between the
strictly local natural basis and the global Fourier basis. We will return to thi
s qualitative di erence in the behavior o  the trans orm later (c . sect. 3.6.1 an
d chap. 4). 3.5.2 Two-Dimensional Signals What we have done or one-dimensional
signals may be extended easily to the 2-d-case. We shall not bother the reader w
ith detailed ormulae but rather sketch the basic acts in eqs. 3.51 – 3.53 below.
As in sect. 3.4.2 the signal will be a matrix  = { mn } with m numbering rows
and n numbering columns. Expanding the signal with respect to the natural basis
we obtain: =
mn
mn φmn natural

(3.51)
Here the matrices φmn natural are nonzero at exactly one pixel position. The expan
sion o   with respect to the Fourier basis reads: =
mn
Amn φmn F ourier
(3.52)
Here the expansion coe cients Amn are proportional to Fmn , i.e., the values o  th
e two-dimensional DFT7 . The matrices φmn F ourier may be nonzero at all pixel pos
itions. Finally, we may expand  with respect to a wavelet basis: =
mn
7
˜ mn φmn wavelet
(3.53)
In sect. 5.2 the DFT was de ned only or sequences. We remark that it may be exten
ded to matrices by applying the DFT row- and columnwise. For details re er to, e
.g. [33].
80
3 The Discrete Wavelet Trans orm
˜ Here the expansion coe cients mn result rom merging the outcome o  a two-dimensi
onal J-step-DWT into one matrix analogously to eq. 3.37. The matrices φmn wavelet
may be nonzero at several pixel position. Note that, as in the 1-d-case, the wav
elet basis system will depend on the chosen dbn and decomposition step J. We com
pare the qualitative di erences o  the basis systems in Fig. 3.21, where we used a
gray value coding or the displayed matrices, with black representing the small
est value and white the largest. For the Fourier basis matrix the real part is d
isplayed, or matrices belonging to natural and wavelet bases, respectively, abs
olute values are shown. Note that the respective eatures o  the basis systems,
recorded on page 79 or the one-dimensional case, obviously are valid also or 2
-d-signals.
φ (3,5) 0 Re(φ(3,5) )
Fourier
natural
0
1
1
2
2
3
3
4
4
5
5
6
6
7 0 1 2 3 4 5 6 7
7 0 1 2 3 4 5 6 7
φ123 . J=2. Wavelet:db2 Wavelet 0
1
2
3
4
5
6
7 0 1 2 3 4 5 6 7
Fig. 3.21. Selected basis matrices or 16 × 16-images. Above le t: A natural basis
matrix, above right: Real part o  a Fourier basis matrix, below: A wavelet basi
s matrix (db2, J = 2).
3.6 Case Studies
81
3.5.3 Computation and Visualization with MATLAB Again, or this section basic a
miliarity with MATLAB is required. We shall discuss implementations generating t
he basis systems discussed above. The corresponding MATLAB-m- les have been tested
with MATLAB 6.5, release 13 and may be downloaded rom the URL given in sect. 1
.3. Note that the m- les discussed in this section require the MATLAB Wavelet Tool
box! As we have seen, or one-dimensional signals o  length N each basis system
is constructed by speci ying N basis sequences φ0 , . . . , φN −1 . The function b ses
1d returns the three sstems discussed in sect. 3.5.2 nd visu lizes the res
ect
ive subsets φ0 , . . . , φkmax by the sample call [phi_natural,phi_Fourier,phi_wavel
et]=bases1d(kmax,N); Here 0 ≤ kmax ≤ N − 1 must be s tis ed nd
hi_n tur l, for ex m
le
, N −1 is N × N m trix with the sequences φ0 natural , . . . , φnatural arranged roww
ise. Likewise, the sequences constituting the Fourier and the wavelet basis syst
em, respectively, are collected in the matrices phi_Fourier and phi_wavelet. Fig
ure 3.20 was created with this unction. For quadratic images, i.e., N × N -images
, representative basis matrices may be selected, plotted and returned with bases
2d. A sample call reads [phi_natural,phi_Fourier,phi_wavelet]=bases2d; The selec
tion o  phi_natural and phi_Fourier, respectively, is done by speci ying the ind
ex pair mn corresponding to ormulae 3.51 and 3.52. The selection o  phi_wavelet
is slightly more complicated since the toolbox unctions wavedec2 and waverec2
are used, which reshape the outcome o  a twodimensional J-step-DWT as a vector (
c . sect. 3.4.3). Figure 3.21 was created with this unction.
3.6 Case Studies
3.6.1 Energy Compaction and Compression In this section more examples or the en
ergy compaction property discussed above (c . Figs. 3.12 – 3.14) are presented. Re
call that “energy compaction” means that, a ter applying a J-step-DWT to the signal
 , a large portion o  the trans ormed signal values will be close to zero. In o
ther words: The in ormation stored in the signal will be concentrated in relativ
ely ew signi cant trans orm values. It is obvious that this property is essential
or signal compression and we shall discuss a na¨ compression model. ıve The DFT al
so shows energy compaction. There ore, the sample signals we discuss will be tra
ns ormed both with the DFT and the J-step-DWT and characteristic eatures o  bot
h trans orms will be worked out.
82
3 The Discrete Wavelet Trans orm
One-dimensional signals The top graph o  Fig. 3.22 shows an acoustic signal. It
was sampled with a sampling rate νS = 8192 Hz (for the de itio of the samplig rat
e refer to Table 5.2). The legth of the sigal f is N = 73113. For completeess
e metio that f represets the rst 9 secods of G.F. H¨del’s famous a “Hallelujah” ch
oir from the “Messiah” oratory. For the rest of this sectio e shall deote ith F
the sequece F = {Fk } of DFT values (cf. eq. 5.13), the outcome f J , dJ , dJ−1 ,
. . . , d1 of a J step DWT ˜ ill be merged ito a sequece f as sho  i eq. 3.4
7.
From above: Origial, trucated origial, Fourier ad avelet recostructios.
1 0 −1 0 1 0 −1 0 1 0 −1 0 1 0 −1 0
1
2
3
4
5
6
7 x 10
8
4
1
2
3
4
5
6
7 x 10
8
4
1
2
3
4
5
6
7 x 10
8
4
1
2
3
4
p fF ourier
5
p f avelet .
6
7 x 10
8
4
Fig. 3.22. From above: Plot of f , f , J = 5, db20.
p
ad
Parameters: p = 1%.
As before (cf. Fig. 3.14) e illustrate eergy compactio ith histograms. Simil
arly to the procedure described already, e compute the respective his˜ tograms of
f , F ad f as follo s: Take the smallest ad the largest absolute value of the
correspodig sequece elemets ad subdivide the iterval be 
3.6 Case Studies
83
t ee these values i 100 equally spaced bis. No for each bi cout the umber
of sequece elemets, hose absolute values are cotaied i it. The result of
this procedure is sho  i Fig. 3.23.
From above: Histograms of origial, Fourier ad avelet coefficiets. 6000 4000
2000 0 0 4 x 10 2
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1
0 0 15000 10000 5000 0 0
100
200
300
400
500
600
700
800
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
˜ Fig. 3.23. Histograms of f , F ad f (from above). Wavelet parameters as i Fig.
3.22.
We see that both trasforms lead to histograms hich are sharply peaked close to
zero. Obviously, the “mai iformatio cotet” of the origial sigal is stored i
relatively fe sigi cat values. For completeess e metio that the parameters
belogig to the J step DWT ere J = 5 (decompositio depth) ad db20 (chose
avelet). Observe that the histograms of both trasforms are i sharp cotrast to
the histogram of the origial, hich is relatively uiform o its rage of data
! We ote already here (for more details refer to sect. 4.1.2) that for compress
io purposes the follo ig properties are ecessary/desirable:
84
3 The Discrete Wavelet Trasform
Trasform the origial sigal such that 1. The trasform is ivertible (ecessar
y coditio) ad 2. The histogram of the trasform values is sharply peaked (des
irable coditio). I fact, the achievable compressio rate of the procedure des
cribed i 4.1.2 ill icrease if the histogram is cocetrated more ad more o
oly fe values. Obviously the DFT ad the J step DWT ful ll both requiremets. I
order to provide a feelig for the respective degree of eergy compactio, he
compared ith the origial, the follo ig experimet as carried out (refer als
o to sect. 1.2.6): 1. Choose a percetage value p (0 < p ≤ 100). ˜ 2. Startig from
the histogram of the J step DWT f compute a trucated p ˜ versio f p ad a corres
podig recostructio f avelet as follo s: ˜ a) Sort the sequece elemets fk i
descedig order ith respect to their absolute values. The resultig list e de
ote ith l = {lk }. No ˜ choose a limit f limit such that the absolute values of
the rst p% of ˜ l etries are above or equal to f limit , the rest belo . b) The ˜p
fk = ˜ ˜ ˜ fk if |fk | ≥ f limi 0 else
      
In o herwords: If he absolu e value  of a given  ransform  coe cien  belongs o h
epercen age  p of ransform
 coe
 cien s having  he larges  absolu e values,  i is r
e ained, o herwise i is “dele ed”,  i.e., pu equal
 o zero. p c) Compu e fwavele b
y applying he  recons ruc ion par
   of eq. 3.30
  o p ˜ ˜ f p = {fk }. 3. Proceed anal
ogously wi h he DFT leading o he runca ed ransform p p sequence  F p = {Fk }
and
 o he corresponding recons ruc ion
  fF ourier by applying   he IDFT (eq.
 5.1
4) o F p . 4. Proceed analogously wi h he original leading o he runca ed ve
rsion f p .     
p p The runca
 ed signals f p , fF ourierand  fwavele
 will
 be such ha heir r
espec ive his ograms are  much more concenra ed a  zero han he his ograms  from
Fig. 3.23: They are
 ob ained
 from he la er by pu ing
 all coe cien s wi h absol 
u e values below he respec ive  limi s from
 above in he “zero bin”. Corresponding
o he remarks  made on page84 herefore heir compression  performance
 will incre
ase. We no ealready here ha p provides a rough es i˜ ma e for he achievable  co
mpression ra e if f p, f p or F p are coded and s ored as described    in sec . 4.
1.2 [1].p p How are he signals f p , fF ourier  and fwavele
 dis or  ed when com
pared wi h he original f ? As we have seen in he sec ion before, he signal ma
y
 
3.6 Case S udies
85
     
be expanded wi  h respec o he na ural basis  (eq. 3.43), he Fourier  basis Fi (
eq. 3.46) or he wavele
 basis (eq. 3.50) wi h expansion coe
  cien  s fi , √N ˜ and fi ,
respec ively.
  The runca ion procedure from above means ha in  hese expansion 
s all con ribu ions from basis  sequences
  whose
 expansion
  coe cien s are mapped o
zero will vanish. Then by he s ric locali  y of hena ural basis  sys em only p
 percen of f p will be nonzero.
  On he  o her  hand, his will no p be  rue for
he Fourier
 basis, due o i s global
  na ure,  herefore  fF ourier
 migh 8 have no
nzero
 en ries everywhere.
  As s a ed before, he locali y of he wavele basis sy 
sem will lead o an “in  ermedia e behavior”:   For small p values
 one p may expec h
a some par s of fwavele  will be zero,
  bu he “nonzero por ion” p will be larger
han for f . p These fac sare illus  ra ed  in Fig. 3.22. Here from
 above  f , f p
,fF ourier
 p and
 fwavele
 are plo  ed wi h p  value and wavele parame ers
 indic
a ed in he cap ion. I is clear ha a sudden  change
 from  zero
 loudness o soun
d and vice versa  will no be pleasing o he lis ener. No e ha f pwill  be zer
o over 99% of he playing ime! Thus a “compression”  resul  ing from  he runca ion p
rocedure discussed above wi h p = 1% will lead o unaccep able resul  s. Never he
less, he basic mechanisms
  responsible
 for losses   in signal  quali y when compres
sing i by concen ra ing his   ograms can be
    illus ra ed wi h hese
 examples.  For
compressing
 audio signals i urns
 ou ha he use of “Coi e s” (a varian
  of dbn wav
ele s which ensures a close ma ch of he approxima ion signal  f J wi h he origi

nal f ) is promising  [38]. Moreover,  he desired concen  ra  ion process of his og
rams
 should resul from a “percep uallylossless quan  iza
 ion procedure” (see also s
ec . 4.3), which is more involved han hesimple runca ion procedure  from abov
e9 . Two dimensional
 signals The experimen
 described above
 in comple e analogy

was carried ou also for images.  The erminology   will be he same, i.e., he ori
ginal image is de˜  no ed wi h f  , F and f deno e he corresponding wo dimensional

 and p p J s epDWT, respec
DFT  ively10 . Again, f p , fF ourier  andfwavele de 
no e he recons ruc ions resul ing from ransforms which were  runca ed in exac
ly he same way as described above.  All images
 are subjec  o gray levelcoding,
i.e.: zero is coded as  black, he larges value as whi e. The upper lef  subima
ge of Fig. 3.24 shows he original. The number of rows and columns of he corres
ponding pixel ma rix were M = 240 and
8 9 10
   
For he noions of s ric ly local, global or local behavior, which are used here
we refer he reader  o page79. The successful  MP3 compression
 scheme of audio 
signals combines he  “discre
 e cosine  ransform”, a varian  of he DFT, wih percep
ually lossless quan iza ion. ˜ The ma rices F and f were in roduced in sec . 3.5.2
, pages 79 .
86
 
3 The Discre e Wavele
 Transform 
Original Recons ruc ion from 10% larges pixels
50
50
100
100
150
150
200
200
50
100
150
200
250
300
50
100
150
200
250
300
   
Recons ruc ion from 10% larges Fourier coefficien s
    
Recons ruc ion from 10% larges wavele coefficien s
50
50
100
100
150
150
200
200
50
100
150
200
250
300
50
100
150
200
250
300
 
Fig. 3.24. In clockwise
 order from above lef : Plo of f , f , Image dimensions:
240 × 320. Percen age: p = 10%. J = 4, db4.
p

p fwavele
and
p fF ourer .
      
N = 320, respec
 ively. Figure  3.25 implies ha he ransforms  lead
 o a concen
ra ion in he respec ive his ograms, which is even   dras ic han for 1 d sig
more
nals. The remarks made before  concerning
    locali y of he respec ive basis sys em
s apply
 also here.
 In par icular, no e ha he chosen
 percen age of p = 10% imp
lies ha exac ly 90% of f p will be black,  since he
 corresponding  pixels are m
apped o zero. The  locali y proper ies of he respec
  ive basis ma rices (cf. Fig 
. 3.21) are re ec ed in he corresponding  recons ruc ions:
 The global p periodici
y of Fourier basis
 ma rices
 leads o  ickering ar ifac s in
 fF ourier . p The loca
liy of wavele
   basis
 ma rices leads
 o regions in
 fwavele which are of near  co
ns an in ensi y ( hey  correspond o regions
  in he audio
  signal having cons an

loudness),
 whereas he  zooming
  proper y of he wavele ransform (refer also o
sec . 1.2.6) ensures ha de ailslike edges  are rela ively well reproduced. Th
e visual recep
 ion here is di eren from lis ening: As explained
 before,
 changing

from cons an loudness o meaningful  sound  will  be unaccep able o he
 lis ener,
 whereas changing
 from smoo hened regions o de ailed regions will no be so dis
urbing for he observer.
 
3.6 Case S udies
x 10
4
87
  
From above: His ograms of original, Fourier and wavele coefficien s.
2
1
0 0 4 x 10 10
50
100
150
200
250
5
0 0 4 x 10 10
1
2
3
4
5
6 x 10
7
6
5
0 0
500
1000
1500
2000
2500
3000
3500
4000
  
˜ Fig. 3.25. His ograms of f , F and f (from above). Wavele parame ers as in Fig.
3.24.
    
MATLAB
 Implemen
 a ions The runca ion procedure described above is implemen ed i
n he func ions CompAudioToolbox(one dimensional signals) and CompImageToolbox

(wodimensional signals),respec  ively.
 These func ions require he MATLAB Wavel
e Toolbox  and have been es ed wi h MATLAB   6.5, release
 13. They maybe downloa
ded from he URL given in sec . 1.3. Pro o ype func
 ion
 calls read [y rek,yfrek,
yw rek]=CompAudioToolbox(y);
 and [rek_pix,rek_f
  ,rek_w  ]=CompImageToolbox(y); re
spec ively. The
 original signal is deno ed wi h y, in he one dimensional
 p p ca
se he  recons ruc ions f p, fF  ourier and fwavele are s ored  in y rek, yfrek
 a
nd
 yw rek, analogously
 in he
 wo  dimensional case hese ma rices are deno ed wi
h rek_pix, rek_f and rek_w .
88
 
3 The Discre e Wavele Transform
     
All required
 parame ers (percen age p, reques ed dbn wavele , decomposi ion dep
h J) mus beprovided  in a user  dialogue. 3.6.2 Denoising  a Sensor  Signal / Real
Time Proper ies of he Algori  hm Removing
  noise is a s andard opic
 insignal p
rocessing. Usually noise  is a ribu
 ed o high frequency componen s
 of he signa
l, which correspond osmall de ail  sizes when performing a wavele  analysis (cf
.
 eq. 1.4).
  Therefore he idea is
  o remove high frequency/small de ail componen
s such ha underlying signals ruc ures, which may be obscured by  noise,
 are
 p
reserved and will emerge  from he denoising process. We have
 seen ha
  a J ep 
s
DWT3.30 performs a mul iscale analysis of a signalf in he sense ha de ails
up o ≈ TS 2J areremoved from f J , whereas  he de ail signals  dJ , dJ−1, . . . ,
d1 sore con ribu ions from decreasing de ail sizes in a way ha makes i possi 
ble o recover f comple ely from f J ,dJ , dJ−1 , .. . , d1 by he recons  ruc io
n par from eq. 3.30. Therefore,  a J s ep DWT seems o be a goodcandida e for d
enoisingby he following hree s ep  procedure: 1. Perform aJ s ep  DWT.
 2. Thre
shold de ail signals
 corresponding
  o scales, which may
  be a ribu ed o noise.
3. Recons ruc wi h manipula  ed de ail signals. Due o  he pioneering work of D.
L. Donoho and his collabora  ors (see, e.g. [9]) for cer ain noise models hresho

lding prescrip
 ions
 for de ail
  signal coe cien
 s can be given, which remove
 he
 no
ise wi hou disurbing impor an signal fea ures. Fora brief survey  of  hese e
chniquesrefer o [1], he MATLAB Wavele  Toolbox con ains rela  ed func ions [24
]. From he sec ion before, where da a compressionwas linked o hresholding,  w
e see ha denoising  and compression are based on he
 same basic
 proper  ies of
he DWT, namely
 mul iscale analysis and energy compac ion. In  his sec  ion we sha 
ll illus ra edenoising by a crude  example. The purpose is  o demons  ra e he r
ansfer
 proper ies  of wavele l ers (see  sec . 3.3.2) and o work ou an aspec im
por an for prac ical applica   ions. Le us again consider he sensor signal f fr
om Fig. 2.7. Remember ha his signal was recorded during an oscilla ory proces
s whose frequency changes from ν ≈ 3 Hz to ν ≈ 6 Hz at t ≈ 3 sec. Figure 2.7 sho s that th
is trasitio may be recogized from the STFT, but f is obviously heavily corrup
ted ith oise, hich is re ected by ozero high frequecy cotributios. I sect
. 3.3.2 e sa that each H  lterig step i a J step DWT 3.30 correspods to lo p
ass lterig. Sice lo pass lterig removes high frequecy compoets (small detai
ls) of f , oe expects that f J , hich results from applyig J lo pass lterig
steps, ill be a smootheed versio of
3.6 Case Studies
89
f ad the degree of smoothig ill icrease ith J. Note, ho ever, that the leg
th of f J ill be equal to the legth of f , divided by 2J , due to subsamplig
i every decompositio step! Therefore, a smootheed versio f J,smooth of f is
obtaied by applyig the recostructio part of eq. 3.30 to f J , dJ,smooth , dJ−1
,smooth , . . . , d1,smooth , here the etries of all dj,smooth sequeces are z
ero. Referrig to the three step procedure from above, e apply a “brute force thr
esholdig” by mappig all detail coe ciets to zero, regardless of the respective le
vel j, i.e., the respective scale sizes. For J = 2, for example, the deoised si
gal f 2,smooth is computed as follo s: f −→ H −→ H −→ H −→ H −→ f 2,smooth .
2,smooth
(3.54)
I Fig. 3.26 e sho the resultig f together ith its STFT (cf. sect. 2.1.3). H
ere db1  lters ere chose, i.e., the Haar  avelet as used. Figure 3.27 sho s the
same results for db20  lters istead of db1  lters.
Parameters: J=2, L=1 (db1) 1 0.5 0 −0.5 0
2
4
6
8
10
5 ν [Hz]
10
15 0 2 4 t 6 8 10
Fig. 3.26. Deoisig the sesor sigal from Fig. 2.7 ith J=2 ad L = 1 (db1).
Comparig both gures ith the origial ad its STFT (Fig. 2.7) e see that both p
rocedures lead to very satisfyig results: The frequecy trasitio is o clear
ly visible i the STFT plot ad may eve be localized by visual ispectio of th
e deoised sigals. Moreover, as is to be expected, the suppressio of high freq
uecies orks better ith icreasig order of db (see Fig.
90
3 The Discrete Wavelet Trasform
Parameters: J=2, L=39 (db20) 1 0.5 0 −0.5 0 2 4 6 8 10
5 ν [Hz]
10
15 0 2 4 t 6 8 10
Fig. 3.27. Deoisig the sesor sigal from Fig. 2.7 ith J=2 ad L = 39 (db20).
3.9). Also the smoothess of the deoised sigals improves ith icreasig order
of db i perfect accordace ith the correspodig avelet properties (cf. sec
t. 3.3.1). This simple example sho s the potetial of avelet methods for deois
ig sigals. Deoisig a sesor sigal ca be importat for obtaiig reliable i
formatio from the sesor. Here, the sigal stems from a light arc sesor of a
eldig robot ad is eeded for obtaiig the actual distace of the eldig too
l from the eldig seam. This distace iformatio is of fudametal importace
for cotrollig the positio of the eldig tool. I these kids of applicatios
a desirable feature of deoisig is to compute the deoised sigal “olie”: As soo
 as the k th sigal value fk is recorded, the correspodig J,smooth of the de
oised sigal should be available. value fk A olie feature as de ed above cao
t be realized ith avelet lters. As a example e choose db1 ad J = 2. The lter
coe ciets read {h0 , h1 } ad e illustrate the t o fold H  lterig from eq. 3.54 b
y extedig Fig. 5.6 to t o decompositio steps (Fig. 3.28). The diagram is plot
ted for TS = 1. For a geeral samplig time TS e ill 2 have a miimum delay ti
me of 3TS i order to compute the fk values11 . The reaso for this delay is th
at the coe ciet h1 of the db1  lter is “o causal”:
11
Observe that this delay time ill be a miimum value sice additio ad multipli
catio also cosume processor resources.
f0
r
r h1   h0         ©   c r
1

2
r   h0         ©   c r
h1
r
3.7 Notes and Exercises 3
91
1 0
  h1  h0  c  A  r
1 1
2 0
Fig. 3.28. Two- old H-application or a lter with coe cients {h0 , h1 }.
1 In order to compute , e.g., 0 , the “ uture signal value” 1 is needed and this d
elay propagates rom decomposition step to decomposition step. In act, it is no
t hard to obtain a general delay ormula. With L denoting the number o  non-caus
al lter coe cients (i.e., the number o  hk -coe cients with k > 0) the minimum delay
tdelay reads
tdelay = TS L(2J − 1).
(3.55)

Indeed, for db1 with L = 1 nd J = 2 we obt in the del  time re d o from ig. 3.
 
28. Note th t there is tr deo between “denoising
qu lit ” nd del time! A dbn H  lt
er h s L = 2n − 1 non c us l coe cients. We s w in igs. 3.26 nd 3.27, res
ectivel
, th t the fe tures of the denoised sign l im
rove with incre sing n, but this i
m
rovement corres
onds to deterior tion of onlinec
bilit. The corres
onding
L v ues re given in the gure c
tions.
3.7 Notes nd Exercises
Notes There re numerous w velet to
ics not tre ted t ll in this book. We ex
l
iclitl mention w velet
ckets nd multiw velets nd encour ge the re der to co
nsult the references. The

lic tion oriented re der might
refer [38], [24] or
[1]. Re ders interested in rese rch re referred to [7], [20], [26], [3] or [23
]. In reference [22] n exh ustive discussion of w velet

lic tions in sign l

rocessing is given. The two dimension l J ste
DWT develo
ed in sect. 3.4.2 is
not the onl w  to de ne discrete w velet tr nsform for im
ges. “Isotro
ic w vele
ts”, s constructed b P. M ß, re n ltern tive

ro ch. or review refer to [
21].
92
3 The Discrete W velet Tr nsform
Another to
ic not covered in this book is the lifting scheme develo
ed for the D
WT. It ddresses in
rticul r im
lement tions of the decom
osition nd reconstr
uction lgorithm, which ver e cientl reduce stor ge nd com
uting time requireme
nts. The lifting scheme is described in [16]. Exercises 1. Verif eq. 3.12 in th
e following ste
s: ) Sketch √1 ψH u−t for = 2 nd t = 2k (k = 0, . . . , 3). b) D
ecom
ose f (t) in the form f (t) = f 1 (t) + d1 (t) nd show th t LψH f 1 (2, 2k)
= 0 (k = 0, . . . , N − 1). 2 c) Show th t LψH d1 (2, 2k) = cψ d1 (k = 0, . . . , N − 1)
. k
H
|a|
  
2. Compu
 e frequency
 response,
 ampli ude response and phase  response for  he Haa
r  l er H wi h coe cien s √√ 2 2 , h1 =. h0 = 2 2 These   ransfer func
 ions are de ned in
eqs. 5.18 and 5.19,
 respec ively. Compu e hese quan i ies also for he corresp
onding
  G  l er wi h coe cien s √
 √ 2 2 g0 = , g1 = − . 2 2 3. For N = 4, db1 and J = 1 com
pu e he one dimensional wavele basis
1 2 3 φ0 wavelet , φwavelet , φwavelet , φwavelet .
Steps: a) Choose the natural basis ˜ ˜1 ˜2 ˜3 φ0 natural , φnatural , φnatural , φnatural a
compose each o  these sequences analogously to eq. 3.49, where the decomposition
or J = 2 was sketched. ˜ b) For each basis sequence φi natural apply the reconstru
ction part o  eq. 3.30 to the corresponding decomposition. For the coe cients belo
nging to the dual lters H and G , respectively, c . 3.19 and 3.20! 4. Some MATLAB-
exercises. a) Develop a MATLAB- unction with the ollowing prototype call: trans
ers(coe ); Here the row-vector coe  is constructed as ollows: coe =[coe ic
ients, lower, upper].
3.7 Notes and Exercises
93
The vector coe icients stores the hk -coe cients o  a dbn-H- lter, lower denotes th
e lowest, upper the highest k-value. As an example, or db1 we have coe =[0.707
1, 0.7071, 0, 1.0000]. The vector coe icients can be obtained by the MATLAB Wav
elet Toolbox unction w ilters (c . sect. 3.4.3), note that or a dbn- lter lower=
0 and upper=2*n-1. Then trans ers should compute and plot the corresponding ampl
itude and phase trans er unctions (c . eq. 5.18). Furthermore, the gk coe cients
should be determined according to eq. 3.28 and their trans er unctions also sho
uld be computed and plotted. b) Develop a MATLAB- unction with the ollowing pro
totype call: out=imdec(original,J,’dbn’); Here the matrix original stores a gray lev
el image, J denotes the decomposition depth o  a J-step-DWT and dbn chooses the
dbn- lters. Then out should store the output matrix arranged as indicated in eq. 3
.37 (see also Fig. 3.17). The unction should invoke the MATLAB Wavelet Toolbox
unction wavedec2 with speci cation given in sect. 3.4.3. c) (Involved!) Develop y
our own MATLAB- unction realizing the iterative construction and plotting o  sca
ling unction φ and wavelet ψ, when the lter coe cients {hk } re given. The
rocedure
(“c sc de lgorithm”)
c n b

is described in sect. 3.3.1. Solutions to these roblems
e found in folders ilters ( ), TwoDim (b) nd W veIter (c). These folders m  b
e downlo ded the URL given in sect. 1.3. 5. (Involved!) In order to solve the
r
oblem formul ted below we h ve to extend theorem 5.1 from sect. 5.2. Let {f (kTS
)} be s m
ling of b nd limited nite energ sign l f (t) such th t the Sh nno
n condition 5.12 is ful lled. Then not onl the s
ectrum ˆ f (ω) ad the DFT are relat
ed by eq. 5.16, moreover, the cotiuous time sigal f (t) may be recovered comp
letely from the sample values {f (kTS )}: f (t) =
k
f (kTS )φSh (t − kTS ).
(3.56)
Here, the Sh nnon inter
ol tion function is de ned s follows: sin π φSh (t) = π
t TS t TS
.
(3.57)
In wh t follows, denote with φ(t) the Shannon interpolation unction φSh (t) or TS
= 1, i.e. φ(t) = sin(πt) . πt (3.58)
94
3 The Discrete W velet Tr nsform
Show th t φ(t) = Steps: a) Show that ˆ φ(ω) = 1 |ω| ≤ π 0 else √ 2
k
φ( k ) hk φ(2t − k) where hk = √2 . 2
(3.59)
Hint: Use reconstruction formul 5.2! b) Show th t the m ximum frequenc ωmax belo
gig to φ(t) satis es the Shannon condition 5.12 or TS = 1 . 2 c) Put  (t) = φ(t) a
nd write down eq. 3.56 or TS = 1 . 2 Supplementary in ormation: Remember that w
avelet and scaling unction are related by eq. 3.29, which is called a “scale-equa
tion”. It may be shown that the scaling unction ul lls a similar scale equation: φ(t
) = √ 2
k
hk φ(2t − k).
As n ex m
le, the re der m  verif this rel tion for the H r w velet nd the
corres
onding lter coe cients {hk }. Then from eq. 3.59 follows th t φ, as de ned in eq
. 3.58, may be viewed as a scaling unction related to the above lter coe cients. T
hese lters in turn lead to a general J-step-DWT 3.30. In contrast to dbn-wavelets
, however, the coe cient set {hk } = φ( k ) √2 2
will be in nite! Thus, any implementation on a computer necessarily will lead to t
runcation errors.
4 More Applications
In this chapter we shall consider two DWT-applications: data compression and the
use o  wavelets or automatic indexing o  images in image databases. In the nex
t section the general structure o  the trans orm compression scheme is presented
. We shall sketch the basic components and in particular will ocus on the mecha
nism responsible or distortions when compressing data. In sect. 4.1.3 we descri
be a procedure or reducing these distortions. This procedure is implemented in
MATLAB. Section 4.2 is devoted to wavelet applications or image databases.
4.1 The Trans orm Compression Scheme
Any data compression scheme aims at reducing storage requirements. The data cons
idered in this section are digital images1 , which are represented as matrices 
= { mn } (m = 0, . . . , M − 1; n = 0, . . . , N − 1) (4.1)
(cf. sect. 3.4.2). Here, M nd N denote the number of rows nd columns, res
ecti
vel, the m trix entries fmn re c lled “
ixels”. If single
ixel requires b bits
of stor ge, the digit l im ge f will require Sf bits of stor ge, where Sf = N M
b. (4.2)
Inste d of stor ge we sh ll lso use the
hr se “ le size” from now on. So the le size
of the origin l im ge f is com
uted ccording to eq. 4.2.
1
Note, however, th t this restriction is m de onl for kee
ing not tion consisten
t. The
resented lgorithms nd techniques re lso

lic ble to one dimension
l sign ls, i.e., sequences f = {fk }N−1 . k=0
96
4 More A

lic tions
To motiv te the need of com
ression techniques,

we st rt with sim
le ex m
le.


A wide s re d im ge form t is the “SI form t” with resolution of M = 352 rows nd
N = 288 columns. It is used, for ex m
le, in low cost web c mer s. A true color

im ge requires 24 bits for e ch ixel.
So true color im ge f recorded with SI

resolution will h
ve le size Sf com uted s Sf = 352 × 288 × 24 bits = 2433024 bi
ts. Note th t the SI resolution is r ther
oor! It is cle r th t with incre sin
g resolution dem nds the volume of digitized im ges will incre se dr stic ll. I
t is not onl consum
tion of disk s
ce which c uses
roblems when
rocessing di
git l im ges or video sequences. Downlo ding im ges from the internet, tr nsmitt
ing im ges over loc l or wide re networks will be extremel time consuming if
the d t volume is too l rge. Therefore, there is n urgent need for com
ression
techniques le ding to reduced mount of d t with onl sm ll or even insigni c
nt distortions of the com
ressed im ges. We close these introductor rem rks b
de ning com
ression r te, formul ting the go l of n e ective com
ression
rocedure
nd introducing two commonl used distortion me sures. Ever com
ression
rocedu
re tr nsforms the origin l im ge f to sign l f c with reduced le size. The “com
r
ession r te” me sures this reduction nd is de ned s C= Sf . Sf c (4.3)
We sh ll not restrict ourselves to lossless com
ression
rocedures. So in gener
l from f c sign l will be recovered (“decom
ressed”), which we sh ll denote with f
dec . This sign l should be “close” to the origin l f in suit ble sense (see belo
w). Therefore, n e cient com
ression scheme is ch r cterized b l rge com
ressi
on r te combined with onl sm ll sign l distortions: Com
ression go l: C 1 where
f dec ≈ f. (4.4)
The following distortion me sures give
recise me ning to f dec ≈ f : The “norm di
st nce” of f dec with res
ect to f is de ned s follows: f dec − f =
mn dec (fmn − fmn )2 .
(4.5)
It is cle r th t lossless com
ression
rocedure will le d to f dec −f = 0. Anoth
er qu ntit, frequentl used in
rticul r in the engineering communit [36], is
the “
e k sign l to noise r tio” (PSNR), which is me sured in Dezibel nd re ds
4.1 The Tr nsform Com
ression Scheme
97
PSNR(f, f dec ) = 10 log10
M N (2b − 1)2 f dec − f 2
[dB].
(4.6)
Here, s mentioned bove, M nd N denote the number of rows nd columns, res
ect
ivel, b is the number of bits required for storing single
ixel
of the origin
l f , f dec − f denotes the norm dist nce introduced in eq. 4.5. or re ders f mi
li r with the notion of sign l energ we give short ex
l n tion of the
hr se “

e k sign l to noise r tio”: When considering gr  sc le im ges with resolution o
f b bits
er
ixel, 2b − 1 will be the m ximum intensit v lue of
ixel. Thus th
e bove fr ction rel tes the m ximum energ, which m  be stored in M × N im ge
to the energ stored in the error sign l f dec − f . This error sign l m  be tt
ributed to noise, coming from the com
ression
rocedure. Summ rizing, n e cient c
om
ression scheme in the sense of eq. 4.4 will tend to minimize f dec − f , or, eq
uiv lentl, m ximize PSNR(f, f dec ), t the s me time kee
ing the com
ression r
te C l rge. A wides
re d com
ression
rocedure is the “tr nsform com
ression sche
me” nd we sh ll describe its b sic com
onents in the next section. 4.1.1 The Gene
r l Procedure A “tr nsform com
ression scheme” is three ste

rocedure s sketched
below: Com
ression: f −→ T −→ Q −→ EC −→ f c
fT fQ
(4.7)
Decom
ression: f dec ←− T −1 ←− Q−1
fQ
−1
f EC
←−
−1
EC −1 ←− f c
(4.8)
The
rocessing ste
s T , Q nd EC must be chosen nd d
ted to e ch other such
th t requirement 4.4 for n e cient com
ression scheme is ful lled. The re ex
l in
ed below: 1. T denotes line r invertible tr nsform f ↔ fT . f T g in will be
m trix of the s me2 size M × N s the origin l f . In this section we sh ll consid
er the J ste
DWT 3.36, since it is included in the ctu l JPEG 2000 st nd rd fo
r still im ge com
ression [36]. The rem ining com
onents of tr nsform com
ress
ion scheme, however, work
2

or re sons ex
l ined in sect. 3.4.3 this onl

roxim tel will be true, if f
T is gener ted b J ste
DWT.
98
4 More A

lic tions
with n line r nd invertible tr nsform. In the “old” version of the JPEG still im
ge com
ression st nd rd [39], for ex m
le, f T resulted
from blockwise

lic
tion of the “discrete cosine tr nsform” (DCT) [39]. In ig. 4.1 we dis
l  both f T
v ri nts. The result of J ste
DWT g in is rr nged s indic ted in eq. 3.37
. Here J = 2 w s chosen.
DCT
DCT
DCT
———
f2 d2v
d2h d1h d2d
d1v
d1d

ig. 4.1. Tr nsformed im ges f T for JPEG “old” (left) nd JPEG 2000 (right).
As indic ted lre d in sect. 3.4.4, JPEG 2000 recommends the use of ˜ biorthogo
n l lter
ir H, H with 9 nd 7 coe cients, res
ectivel. 2. Q denotes “qu ntiz tion”.
Here interv ls of the v lues of f T entries re re
resented b single v lue (
so c lled “smbol”) Sj . If the interv ls
h ve const nt length, the qu ntiz tion i
s c lled “uniform”. An ex m
le is shown in ig. 4.2. Ever f T entr is subject to
such qu ntiz tion le ding to m trix f Q of size M × N . Since the m
[qj , qj
+1 ) → Sj is m n to one, qu ntiz tion is loss ste
. In f ct, the “reconstruction
error” f dec − f ctu ll is qu ntiz tion error, since qu ntiz tion is the onl l
oss ste
t tr nsform com
ression. Thus in eq. 4.8 the smbol Q−1 does not denote
ex ct inversion of the q ntiz tion ste
Q but r ther is suit ble
rescri
tion
, to choose v lue in the interv l [qj , qj+1 ), when Sj is given. Note th t th
e trunc tion
rescri
tion discussed in sect. 3.6.1 (see
ge ˜ ˜ 84) ctu ll is q
u ntiz tion, re
resenting the interv l (−f limit , f limit ) b the smbol 0. 3. E
C denotes n “entro
 coder”, which encodes the smbol m trix f Q with s few bits
s
ossible resulting in the com
ressed sign l f c . Note th t, whichever w  the
entro
 coder is im
lemented, entro
 coding is invertible, therefore the tr ns
ition from f Q to f c nd vice vers is lossless: f Q ↔ f c.
4.1 The Tr nsform Com
ression Scheme
99
TQ f
mn
Sj
qj
qj+1
E
T fmn

ig. 4.2. A uniform qu ntiz tion rule.

Moreover, the ctu l le size reduction is
erformed in this ste
nd the
ur
ose
of the
receding ste
s is to gener te histogr m of the smbols {Sj } such th t
entro
 coding m  be

lied e ectivel. B sic notions of entro
 coding

re int
roduced below, for review of entro
 coding schemes see [36], [21]. in ll we
note th t the three ste
scheme de
icted in equ tions 4.7 nd 4.8, res
ectivel
, is oversim
li ed. Cert in “modes” of JPEG 2000 do not show strict se
r tion of qu
ntiz tion nd entro
 coding. R ther, these ste
s re merged to n ent ngled
r
ocedure [36]. Moreover, com
ression e cienc c n be incre sed b combining qu ntiz
tion with run length coding [1]. There exist m n more v ri nts of tr nsform co
m
ression which will en ble high com
ression r tes t low distortion. Neverthele
ss, in this section we sh ll st  with the three ste
scheme 4.7, since it llow
s for sstem tic investig tion of qu ntiz tion errors (cf. sect. 4.1.3). The r
e der should kee
in mind, however, th t re l im
lement tions of com
ression sch
emes re more involved nd we refer to the quoted references for more det iled i
nform tion. 4.1.2 Entro
 Coders Let us ssume th t qu ntiz tion h s been
erf
ormed le ding to n smbols. Therefore, fter qu ntiz tion, the qu ntized m trix
f Q will h ve entries with v lues t ken from the set {S1 , . . . , Sn }. Assume
th t ni denotes the number of Si occurrences in f Q nd denote with
i = ni MN
(4.9)
100
4 More A

lic tions
the corres
onding rel tive frequenc. Then the smbols Si (i = 1, . . . , n) wil
l occur with rel tive frequencies
i (i = 1, . . . , n) s tisfing
n

i = 1.
i=1
(4.10)
The “entro
” of f Q is de ned s [36]
n
H(
1 , . . . ,
n ) = −
i=1

i log2
i

(4.11)
H s tis es the inequ lities 0 ≤ H(
1 , . . . ,
n ) ≤ log2 n. for
1 =
2 = . . . =
n
= the lower bound for
i = 1i=k 0i=k (4.14) 1 , MN (4.13) (4.12)
We ex
licitl note two im
ort nt f cts: The u

er bound will be re ched
Both equ tions illustr te n im
ort nt s
ect of entro
: If ll smbols occur w
ith equ l frequenc (eq. 4.13), H will re ch its m ximum; if ctu ll onl smbo
l Sk occurs in f Q (eq. 4.14), H will re ch its minimum. This m  be gener lized
b the following observ tion: H me sures the concentr tion
ro
erties of the hi
stogr m counting the occurrences of the smbols Si (i = 1, . . . , n). If the hi
stogr m is t, H will be l rge, if the histogr m is sh



r l e ked, i.e., concent
r ted round rel tivel few s mbols, H will be sm ll. or storing nd
rocessing
 
f Q on digit l com
uter, we need “bin r code”, i.e., n invertible m
of the
s mbols to sequences of two v lued bits. Usu ll the two
ossible v lues of bi

t re re
resented s 0 nd 1, res
ectivel. A bin r sequence re
resenting sm
bol will be c lled “codeword”. The length of the codeword re
resenting smbol Si w
e denote with li . A bin r coding of three smbols is shown in T ble 4.1. As in
this ex m
le, the length of the codewords in gener l m  v r, code with this

ro
ert is c lled “v ri ble length code”. We s w bove th t e ch smbol Si will


e r in f Q with rel tive frequenc
i resulting from eq. 4.9. So we c n com
u
te n “ ver ge codeword length” l corres
onding to
4.1 The Tr nsform Com
ression Scheme T ble 4.1. A bin r code for three smbols.
Smbol codeword S1 S2 S3 1 01 00
101
n
l=
i=1

i li .

(4.15)
A fund ment l theorem of inform tion theor [29] st tes th t for n rbitr r bi
n r code the following inequ lit will be true: l ≥ H(
1 , . . . ,
n ). (4.16)
Thus, the entro
 com
uted ccording to eq. 4.11 is lower bound for the ver g
e codeword length nd bin r code, which h s the
ro
ert th t l ≈ H(
1 , . . .
,
n ), is c lled “entro
 codec”. Such codecs3 lso re c lled “o
tim l codecs”, since
it is not
ossible to re
resent the smbols (in the ver ge) b shorter codeword
s. We sh ll see below th t lgorithms for entro
 coders c n be given, so we m 
extr ct “com
ression reci
e” from wh t we h ve collected so f r:
1. Tr to reduce entro
 b choosing tr nsform T such th t the T histogr

m of
fmn v lues is concentr ted bout rel tivel few v lues (cf., e.g., ig. 3.25).
2. Tr to reduce entro
 even more b

ling
ro
er qu ntiz Q tion rule such
th t the histogr m of fmn v lues is sh r
er
e ked T th n the fmn histogr m.
3. A

l n entro
 codec to f Q resulting in the com
ressed sign l f c. As co
nsequence, the com
ression r te C de ned in eq. 4.3 m  be estim ted s C≈ b . H(
1
, . . . ,
n ) (4.17)
The re son is th t the le size of Sf c of f c m  be estim ted s Sf c = M N l ≈ M
N H(
1 , . . . ,
n ), where s the le size of the origin l re ds Sf = M N b (cf.
eq. 4.2, b denotes the number of bits needed to store
ixel of the origin l im
ge).
3
The term “codec” is com
osed from coding nd decoding stressing code invertibilit.
102
4 More A

lic tions
We sh ll now sketch two entro
 codecs frequentl used in im ge com
ression. Bot
h lgorithms re described in, e.g., [36]. The re der m  consult [36] lso for
further rel ted references. The Hu m n codec Given the smbols {S1 , . . . , Sn }
nd the corres
onding rel tive frequencies {
1 , . . . ,
n } the lgorithm cons
tructs bin r tree, whose le ves corres
ond to the smbols. The construction

roceeds s follows: 1. De ne the “le ves” of the tree b
utting ll smbols in row.
Inste d of le f we will use the term “node” from now on. E ch node will be l belled
with the corres
onding rel tive frequenc
i . 2. Merge the two nodes with the
sm llest
rob bilities (“children”) into one node (“
rent”), l belled with the sum of t
he two rel tive frequencies. Re
resent the merging
rocess b gr
h connecting
both children with the
rent node. 3. Re
e t ste
2 with the
rent node(s) ge
ner ted so f r nd the rem ining le ves. 4. Sto
, when the l st node h s been ge
ner ted b the merging
rocess. This node is c lled “root”, obviousl it will be l b
elled with 1, since it corres
onds to the sum over ll rel tive frequencies. 5.
Code e ch smbol b the
th from the root to the corres
onding le f, thereb co
ding e ch “left turn” with 0, e ch “right turn” with 1. We illustr te the
rocedure b c
onstructing the Hu m n code for the smbol string ABBC. The rel tive frequencies r
e d
A = 1 1 1 ,
B = ,
C = 4 2 4
The tree building
rocess discussed bove m  be illustr ted s follows: C 1 A 1
B 2
2
4
4.1 The Tr nsform Com
ression Scheme
103
Note th t here the nodes re l belled with bsolute r ther th n rel tive frequen
cies. This m kes no di erence, the root node for this ex m
le then necess ril is
l belled with 4. The resulting code re ds:
T ble 4.2. Hu m n code for ABBC. Smbol codeword C A B 00 01 1
We n ll check th t for this ex m
le the ver ge codeword length is determined b
the entro
. The entro
 is com
uted s 1 H(
A ,
B ,
C ) = − log2 4 = 1.5 1 4 1
log2 2 1 2 1 log2 4 1 4
−
−
The coded string re ds 011100. Therefore, the ver ge codeword length results fr
om 6 4 = 1.5
lHuf f m n =
So indeed the Hu m n code re ches the lower bound from eq. 4.16. Obviousl e ch Hu m
n code will le d “codebook” ssoci ting to e ch smbol the corres
onding codeword
s in T ble 4.2. So the minimum length of e ch codeword will be 1 nd therefore
, reg rdless of the smbol
rob bilities, the ver ge codeword length of n rbi
tr r Hu m n code will ful ll lHuf f m n ≥ 1. (4.18)
On the other h nd, the smbol
rob bilities {
1 , . . . ,
n } m  be such th t
entro
 s tis es H(
1 , . . . ,
n ) < 1. (4.19)
In f ct, if in tr
nsform com
ression scheme the tr nsform T is J ste
DWT, or
two dimension l D T,
fter qu ntiz tion the resulting histogr m m  be so sh


r l concentr ted (cf. ig. 3.25) such th t eq. 4.19 will hold.
104
4 More A

lic tions
In this c se obviousl Hu m n coding c nnot be o
tim l, since it will not re ch th
e lower bound from eq. 4.16. An coding scheme re ching this bound, therefore ne
cess ril will ssoci te the smbol string stored in f Q com
letel to bin r
codeword. Below we sh ll describe such coding
rocedure. Arithmetic Coding Giv
en the smbols {S1 , . . . , Sn } nd the corres
onding rel tive frequencies {
1
, . . . ,
n } the lgorithm will code the given smbol string (in tr nsform
com
ression scheme this string m  result, e.g., from rowwise sc n of f Q ) b
bin r sequence, which we sh ll c ll “string codeword” below. We will describe no
w the
rocedure. 1. St rt : Subdivide the interv l [0, 1) into non overl

ing i
nterv ls of length
i (i = 1, . . . , n). Associ te e ch smbol Si with the corr
e
onding subinterv l. 2. Selection: Select the subinterv l corres
onding to the r
st smbol of the string. 3. Ex
nsion: Decom
ose the selected interv l into subi
nterv ls ssoci ted to the individu l smbols com
letel n logousl to ste
1.
4. Re
e t selection nd ex
nsion until the l st smbol of the string h s been

rocessed. In this w  we will obt in n l interv l [u, o). Now choose number i
n this interv l such th t its bin r ex
nsion4 is s short s
ossible. This bi
n r ex
nsion will be the string codeword. Ag in we sh ll illustr te the
roced
ure b determining the rithmetic code of ABBC with rel tive frequencies
A = St
rt : B 0 Selection: B 0 Ex
nsion: B
1 2
4
1 1 1 ,
B = ,
C = . 4 2 4 A C
3 4
1 2
1 C
A
1 2 3 4
1 C
A
5 8 11 16
3 4
Re ders not f mili r with com
uting the bin r ex
nsion of decim l number q w
ith 0 ≤ q < 1 re referred to sect. 4.3, eq. 4.35, for n ex m
le.
4.1 The Tr nsform Com
ression Scheme
105
Selection: B
1 2 5 8
A
11 16
C
3 4
Ex
nsion: B
1 2 18 32
A
19 32
C
5 8
Selection: B
1 2 18 32
A
19 32
C
5 8
Ex
nsion: B
1 2 34 64
A
35 64
C
18 32
Selection: B
1 2 34 64
A
35 64
C
18 32
We obt in the n l interv l [u, o) = 35 18 , 64 32 .
The bin r ex
nsion of u = 35 re ds .100011. An logousl, the bin r 64 ex
nsi
on of o = 18 is given b .1001. 32 So the string codeword re
resenting ABBC re d
s 100011. The ver ge codeword length is com
uted s 6 4 = 1.5
lAC =
Here, obviousl rithmetic coding nd Hu m n coding both re o
tim l, since their
res
ective ver ge codeword lengths re given b the entro
 (see bove). We sh
ll now consider the following string: A B...B
n−1
(4.20)
times
106
4 More A

lic tions
The string will h ve tot l number of n smbols nd therefore the rel tive freq
uencies re d
A = 1 , n
B = n−1 . n
If n is l rge enough, entro
 will ful ll H(
A ,
B ) < 1 nd we sh ll investig te
how rithmetic coding
erforms. To do so, we need the following result: Arithme
tic coding of the smbol string 4.20 will le d to the n l interv l [u, o), where
u= 1 n 1− 1− 1 n
n−1
,
o=
1 . n
(4.21)
To
rove this, we consider T ble 4.3, where we st rt with u = 0 nd o = 1. E ch
selection will u
d te u, o nd the width of the ctu l interv l; in the t ble th
ese u
d tes re tr cked.
T ble 4.3. Arithmetic coding of string 4.20. Smbol u A B B B . . . 0
1 n2 1 n2 1 n2
o Interv l width
1 n 1 n 1 n (n−1)2 1 n n4 1 n n−1 n2 (n−1)2 3 n (n−1)3 n4
+ +
. . .
n−1 n3 n−1 n3
+
. . .
. . .
Thus n ll 1 n2 1 n2 (n − 1)n−2 n − 1 (n − 1)2 + + ...+ n n2 nn−2 1 n + 1− 1 n
2
u= =
1+
1+ 1−
+ ...+ 1−
1 n
n−2
1 1− 1− =2 1 n n = 1 n
1 n−1 n
1− 1−
1 n
n−1

rom the second to the third line we used the geometric l ex
nsion formul
4.1 The Tr nsform Com
ression Scheme
n
107
qk =
k=0
1 − q n+1 (q = 1). 1−q
As n ex m
le we stud the smbol string ABBBBBBB with rel tive frequencies 1 7

A = ,
B = . 8 8 The entro
 re ds 1 H(
A ,
B ) = − log2 8 = 0.5436 1 8 − 7 log2 8
7 8

rom eq. 4.21 the bound ries of the n l interv l re com
uted s: [u, o) = 185 1
, 2437 8
185 The bin r ex
nsion of u = 2437 is given

b .000100110110111100001001. 1 Co
rres ondingl , o = 8 is ex nded s .001. rom both ex
nsions we obt in for the



string codeword the bin r sequence 00011. Therefore the ver ge codeword lengt
h re ds
lAC =
5 8 = 0.6250
In contr st, since the Hu m n code encodes A with 1 nd B with 0 (or vice vers ),
we h ve lHuf f m n = 1. So rithmetic coding will be much closer to the lower bo
und from eq. 4.16 th n Hu m n coding. Since the interv ls gener ted throughout the
coding
rocedure re nested, decoding works s follows: Subdivide the interv l
[0, 1) s described bove nd determine the subinterv l cont ining the number s
soci ted with the string codeword. The corres
onding smbol will be the rst eleme
nt of the decoded smbol string. Then t ke this interv l, subdivide it g in, n
d g in determine the subinterv l cont ining the bove number. This will
roduce
the second element of the decoded string. Proceed in this w  nd
roduce the n
ext smbols. Since this
rocess will not termin te, the tot l number of smbols
must be tr nsmitted in order to sto
the decoding
rocess fter the l st smbol.
108
4 More A

lic tions
4.1.3 O
tim l Qu ntiz tion nd Ex m
les We return now to the tr nsform com
ressi
on scheme described in eqs. 4.7 nd 4.8. In sect. 4.1.1 we s w th t sign l disto
rtions exclusivel c n be ttributed to qu ntiz tion nd in this section we sh l
l investig te how qu ntiz tion distortion m  be reduced. Given the origin l f ,
we re interested in the distortion of the decom
ressed sign l f dec . We s w
bove th t entro
 coding is reversible, therefore we m  combine equ tions 4.7
nd 4.8 such th t we obt in f dec from f s follows: Com
ression: f −→ T −→ Q −→ Q−1 −→ T −1
“norm dist nce” D := f dec − f
dec
(4.22)
(4.23)
with res
ect to f , which w s introduced in eq. 4.5, me sures disof f tortion. A
high D v lue corres
onds to l rge, low
D v lue to sm ll sign l distortion. Wh
t re the
r m ters, D de
ends on? In ig. 4.2 we s w th t qu ntiz tion rule
corres
onds to
iecewise const nt function, which uniquel is determined when
the qu ntiz tion
r meters qj nd the levels Sj on the res
ective interv ls [q
j , qj+1 ) re known. This function will be c lled “qu ntiz tion function”. Then we
observe the following: Assume th t there re n + 1 qu ntiz tion
r meters q1 ,
. . . , qn+1 . Assume moreover th t there exists
rescri
tion, how the levels

Si re com
uted from the qu ntiz tion
r meters (for n ex m
le see below). ur
thermore, ssume th t cert in choice of the tr nsform T h s been m de. Then di
stortion s determined in eqs. 4.22 nd 4.23 will de
end onl on the qu ntiz tio
n
r meters: D = D(q1 , . . . , qn+1 ) (4.24)
In this section we sh ll use the following
rescri
tion to com
ute the levels (“s
mbols”) Sj : qj + qj+1 (j = 1, . . . , n) (4.25) 2 The qu ntiz tion function is co
nstructed s follows: On the interv ls [qj , qj+1 ) (j = 1, . . . , n) the funct
ion v lue will be Sj (j = 1, . . . , n), on [qn+1 , ∞) the function will rem in on
Sn , on (−∞, q1 ) the function v lue Sj =
4.1 The Tr nsform Com
ression Scheme
109
is S1 . So fter qu ntiz tion the m trix elements of f Q will h ve v lues5 t ken
from the smbol set S1 , . . . , Sn . As mentioned in the
receding section, ro
wwise sc nning of f Q will
roduce smbol string, where e ch Si will

e r wi
th rel tive frequenc
i . Since the smbols uniquel re com
uted from the qu
ntiz tion
r meters s in eq. 4.25, the resulting entro
 H (cf. eq. 4.11) ct
u ll will de
end on these
r meters: H = H(q1 , . . . , qn+1 ) (4.26)
Distortion is minimized, if q1 , . . . , qn+1 re chosen such th t D(q1 , . . .
, qn+1 ) is minimized. On the other h nd, eq. 4.17 tells us th t t the s me tim
e H(q1 , . . . , qn+1 ) should be ke
t s sm ll s
ossible in order to obt in
l rge com
ression r te C. So, the over ll o
timiz tion go l m  be formul ted
s follows: Determine q1 , . . . , qn+1 such th t the t rget function t(q1 , . .
. , qn+1 ) is minimized, where t(q1 , . . . , qn+1 ) = λD(q1 , . . . , qn+1 ) + (1
− λ)H(q1 , . . . , qn+1 ) (4.27) The parameter λ is ca ed “distortion weight”, (1−λ) corr
onding y “entropy weight”. So, the target function is a weighted average of distorti
on and entropy and it depends on the choice of λ, which of the con icting “sub goa s” (d
istortion reduction vs. compression e ciency) is favored. It is c ear that there i
s no simp e ana ytica  expression for t. The optimization goa  formu ated above,
therefore, must be so ved numerica y. A numeric 0 0 procedure a ways wi  star
t from an initia  set (q1 , . . . , qn+1 ) and iterative y k k wi  produce (q1
, . . . , qn+1 ) (k = 1, 2, . . .) converging to the desired va ues (q1 , . . .
, qn+1 ), where t is minima . In sect. 4.1.4 we brie y indicate which procedure ha
s been used. Before discussing Figs. 4.3 – 4.5, where resu ts of numeric optimizat
ion are disp ayed, we make two remarks: 1. The initia  set is computed according
to
0 T 0 0 T q1 = min(fmn ) < q2 < . . . < qn+1 = max(fmn ) mn mn
5
(4.28)
For comp eteness we mention that in eq. 4.22 we sha  omit dequantization Q−1 . Th
us, the inverse transform T −1 direct y wi  be app ied to f Q . Since Sj is compu
ted according to eq. 4.25, actua y we perform a dequantization by se ecting the
midpoints of each quantization interva  [qj , qj+1 ).
110
4 More App ications
q0 −q0
0 0 with qj+1 − qj = n+1 1 (j = 1, . . . , n). So we take the sma est and the n 
argest e ement of the transformed image f T (cf. eq. 4.7) and subdivide the resu
ting interva  in n subinterva s of equa  ength. Therefore the initia  quantiza
tion wi  be uniform (cf. sect. 4.1.1). 2. For T we choose a J step DWT. Thus, t
he transformed image f T is constructed as depicted in the right diagram in Fig.
4.1 for J = 2. We sha  a ow for separate quantization ru es for the approxima
tion signa  f J and the detai  signa s. So, in genera  the numbers of quantizati
on eve s for the approximation image f J and the detai  signa s, respective y,
wi  di er.
The upper row of Fig. 4.3 shows the origina  f ( eft) and the decompressed image
f dec (right). The ower eft diagram disp ays the quantization function for th
e approximation signa  f J , the ower right diagram the quantization function f
or the detai  signa s. Here four quantization eve s were chosen for the approxi
mation signa , three quantization eve s for the detai  signa s, respective y. T
he gure shows the resu ts for the uniform start quantization.
Origina 
N
app
: 4, N
detai s
: 3, J: 2, db2, PSNR: 15.39dB, bpp: 0.3019.
50 100 150 200 100 200 300
50 100 150 200 100 200 300
800 600 400 200 0 0 200 400 600 800
300 200 100 0 −100 −200 −200 0 200
Fig. 4.3. Origina  (above eft) and reconstruction (above right) for start quant
ization. Be ow eft: Uniform initia  quantization of approximation signa , Be ow
right: Uniform initia  quantization of detai  signa s.
4.1 The Transform Compression Scheme
111
It is c ear that due to the ow number of quantization eve s distortion wi  be
arge. Distortion was measured by the PSNR (cf. eq. 4.6) (remember that arge d
istortion corresponds to a ow PSNR va ue). Since entropy estimates the average
codeword ength of a symbo , H a so is disp ayed. It is speci ed as the number of “b
its per pixe ” (bpp). The respective va ues in Fig. 4.3 are PSNR(f, f dec ) = 15.3
9 dB and H = 0.3019 bpp. The origina  was represented with b = 8 bits per pixe .
So, we have an estimated compression rate of b C = H = 26.5. Before discussing
Figs. 4.4 and 4.5, we note that the respective subimages are arranged as in Fig.
4.3. Moreover, PSNR  and bpp va ues are determined ana ogous y. Figure 4.4 show
s the resu ts obtained when starting from the initia  quantization from Fig. 4.3
and stopping after 60 iterations. In the target function 4.27 λ = 1 was chosen, i
.e., the entropy weight was zero. So we on y aimed at minimizing distortion with
out taking entropy into account.
Origina 
Napp: 4, Ndetai s: 3, J: 2, db2, PSNR: 18.20dB, bpp: 0.3139. 50 100 150 200
50 100 150 200 100 200 300
100
200
300
800 600 400 200 0 0 200 400 600 800
300 200 100 0 −100 −200 −200 0 200
Fig. 4.4. Origina  (above eft) and reconstruction (above right) after 60 functi
on eva uations, entropy weight: 0. Be ow eft: Resu ting non uniform quantizatio
n of approximation signa , Be ow right: Resu ting non uniform quantization of de
tai  signa s.
112
4 More App ications
Obvious y, the visua  impression of f dec is much better now than in Fig. 4.3. T
his is re ected a so by the PSNR: Now we have PSNR(f, f dec ) = 18.20 dB. As expec
ted, the bpp va ue is s ight y arger than in Fig. 4.3: It reads H = b 0.3139 bp
p corresponding to an estimated compression rate C = H = 25.5. Note that the res
u ting quantization functions deviate from uniform quantization both for the app
roximation image and the detai  signa s! In the na  examp e we again started from
the initia  quantization from Fig. 4.3 and stopped after 60 iterations. But now
in the target function 4.27 λ = 0 was chosen, i.e., the distortion weight was zer
o. So we on y aimed at minimizing entropy (maximizing compression rate) without
taking distortion into account. The resu ts are disp ayed in Fig. 4.5.
Origina 
Napp: 4, Ndetai s: 3, J: 2, db2, PSNR: 15.19dB, bpp: 0.1572. 50 100 150 200
50 100 150 200 100 200 300
100
200
300
800 600 400 200 0 0 200 400 600 800
300 200 100 0 −100 −200 −200 0 200
Fig. 4.5. Origina  (above eft) and reconstruction (above right) after 60 functi
on eva uations, entropy weight: 1. Be ow eft: Resu ting non uniform quantizatio
n of approximation signa , Be ow right: Resu ting non uniform quantization of de
tai  signa s.
Indeed now a ow bit per pixe va ue is obtained: We have H = 0.1572 bpp b corre
sponding to an estimated compression rate C = H = 50.9! But for the dec PSNR va 
ue we now obtain PSNR(f, f ) = 15.19 dB. So it decreased with respect to the ini
tia  quantization. As expected, the decompressed image f dec deteriorates when c
ompared with the initia  quantization. It is instructive to
4.1 The Transform Compression Scheme
113
study the quantization curves. Above we saw that ow entropy va ues correspond t
o sharp y concentrated histograms. Such histograms can be reached by generating
wide quantization interva s. The corresponding symbo s then wi  appear frequent
y eading to concentrated histograms. Compare the quantization curves for the d
etai  signa s in Fig. 4.3 and Fig. 4.5, respective y! Here this broadening e ect i
s c ear y visib e. Note a so that the origina  4 eve quantization of the approx
imation signa  has turned basica y to a 3 eve  quantization, i.e., essentia y
on y three symbo s remained! This again is in accordance with the above remarks
about histogram concentration. 4.1.4 MATLAB Imp ementation In this section we b
rie y describe a MATLAB imp ementation of the quantization optimization out ined i
n the preceding section. The re ated functions require at minimum the MATLAB Wav
e et Too box and have been deve oped and tested with MATLAB 6.5, re ease 13. The
y may be down oaded from the URL given in sect. 1.3. Again, as mentioned ear ier
, a basic understanding of MATLAB syntax is required. Quantization optimization
is invoked by the fo owing prototype function ca : [q_approx_signa ,q_detai _s
igna ]=optshe (f); The origina  image matrix is stored in f. It must be provide
d as a M × N matrix with doub e entries. The parameters characterizing the transfo
rm T are se ected interactive y. These are decomposition depth J and se ected wa
ve et string ’dbn’, where n = 1, 2, . . . (refer to sect. 3.4.3). Moreover, for the
approximation signa  and the detai  signa s the respective number of quantizatio
n eve s (“symbo s”) must be speci ed. The respective initia  quantization parameters
are then computed corresponding to eq. 4.28. In this formu a f T must be rep ace
d by the approximation image f J in order to obtain the start quantization of th
e approximation signa ; for the start quantization of the detai  signa s a corre
sponding rep acement has to be done. The quantization parameters qi resu ting fr
om optimization are returned in vectors q_approx_signa  and q_detai _signa  for
approximation signa  and detai  signa s, respective y. As mentioned in the prece
ding section, beginning from the start quantization iterative y new quantization
parameters are computed such that the resu ting va ues of the target function 4
.27 are essentia y descending towards a minima  va ue (if it exists). The funct
ion optshe  a ows for choosing from two iteration methods. One method is the “Ne
der Mead” a gorithm [19]. It is imp emented in the MATLAB bui t in function fmins
earch, which is invoked if Ne der Mead is se ected. The other method which can b
e se ected, is the “Quasi Newton” a gorithm [2]. The se ection invokes the function
fminunc, which imp ements this a gorithm. This function requires the insta atio
n of the
114
4 More App ications
MATLAB Optimization Too box. It therefore cannot be used if the MATLAB Optimizat
ion Too box is not avai ab e. Figures 4.3 – 4.5 were obtained by invoking fminsear
ch. Be ow we show a resu t invoking fminunc. Here we chose ve quantization eve s
for the approximation signa , four quantization eve s for the detai  signa s.
Figure 4.6 shows the resu ts for the uniform start quantization.
N : 5, N : 4, J: 1, db2, PSNR: 16.99dB.
Origina 
app
detai s
50 100 150 200 100 200 300
50 100 150 200 100 200 300
400 100 300 200 100 0 0 100 200 300 400 0 −100 −200 −200
−100
0
100
Fig. 4.6. Origina  (above eft) and reconstruction (above right) for start quant
ization. Be ow eft: Uniform initia  quantization of approximation signa , Be ow
right: Uniform initia  quantization of detai  signa s.
Figure 4.7 shows the resu ts after a certain number of iterations. As in Fig. 4.
4, entropy weight was set to zero. Referring to the target function 4.27, theref
ore, we aimed at reducing distortion without taking entropy into account. The vi
sua  improvement corresponds to a c ear improvement in PSNRva ues: In Fig. 4.6 w
e have PSNR(f, f dec ) = 16.99 dB, in Fig. 4.7 PSNR(f, f dec ) = 19.87 dB was ob
tained. The bpp va ues were as fo ows: For Fig. 4.6 H = 1.6637 bpp was measured
, corresponding to an estimated compression rate b C = H = 4.81. In Fig. 4.7 we
have H = 0.7182 bpp corresponding to a b compression rate C = H = 11.14.
4.1 The Transform Compression Scheme
Origina  50 100 150 200 100 200 300 N
app
115
: 5, N
detai s
: 4, J: 1, db2, PSNR: 19.87dB.
50 100 150 200 100 200 300
400 100 300 200 100 0 0 100 200 300 400 0 −100 −200 −200
−100
0
100
Fig. 4.7. Origina  (above eft) and reconstruction (above right) after optimizat
ion run, entropy weight: 0. Be ow eft: Resu ting non uniform quantization of ap
proximation signa , Be ow right: Resu ting non uniform quantization of detai  si
gna s.
Thus, in this experiment not on y distortion was reduced, optimization even ead
to an improved compression performance! We c ose this section with some genera 
remarks concerning the optimization imp ementation. As stated above, the numeri
ca  procedures discussed in this section iterative y wi  produce quantization p
arameters such that the corresponding target function va ues wi  be descending,
hopefu y towards a oca  minimum situated in a neighborhood of the start quant
ization. Since no a priori information about the “shape” of the target function t(q1
, . . . , qn+1 ) (cf. eq. 4.27) is given, it is not guaranteed that such a mini
mum exists. If it exists, it is not guaranteed that it wi  be reached within re
asonab e computing time. So, one usua y imits the maximum number of iterations
and wi  be satis ed if, within this number of iterations, the target function va
ue wi  decrease reasonab y. In sect. 4.3 we sha  make some na  comments on the
optimization subject.
116
4 More App ications
4.2 Wave et Based Simi arity Retrieva  in Image Archives
In this section we report on an industria  project performed by the author at te
cmath AG. The work has been pub ished in [31, 32]. In the fo owing we summarize
the main resu ts. Conventiona  digita  image database management systems usua 
y are structured as fo ows: Each stored image e is described by a text document
, which contains administrative data ike date and time of image capture. Moreov
er, this document inc udes we de ned terms and phrases describing image contents
. When a user of the database queries for images re ated to a certain topic, he
or she might enter corresponding keywords. These keywords can then be used for q
uery mechanisms ike, e.g., fu  text retrieva . So the resu t of the research w
i  ead to a ist of documents, whose content description parts contain the key
words. The retrieved documents are inked to the corresponding images, which the
n may be inspected. It is we  known that a verba  description of images often i
s inadequate, since the description stored in the database management system mig
ht not match the expectations of the user. Moreover, it is not possib e to “presen
t” an examp e image to the system and to ook for a  images “simi ar” to the presente
d image when conventiona  image archives as described above are used. Image data
base management systems a owing for such retrieva  mechanisms are ca ed “content
based access systems” (CBA systems). It is c ear that in view of mu timedia techn
o ogy CBA systems are of growing importance. In order to bui d systems “ ooking fo
r simi ar images”, it is necessary to give a precise meaning to the notion of “simi 
arity” and we sha  describe a simp e mode  based on a J step DWT. As a preparatio
n, we sha  exp ore in some more detai , how a J step DWT of an image may be use
d for measuring the appearance of edges. Remember that the resu t of a J step DW
T of an image (cf. eq. 3.36) usua y is arranged as indicated in eq. 3.37. As an
examp e consider the origina  f shown in Fig. 4.8. A 7 step DWT, performed with
db2 is visua ized in Fig. 4.9. The detai  signa s djh , djv , djd (j = 1, . . .
, 7) c ear y con rm the directiona  sensitivity mentioned in sect. 3.4.2: Since t
he origina  shows a damped periodic pattern of horizonta  stripes, vertica  and
diagona  detai  signa s are c ose to zero. Moreover, we note that the detai  sig
na s wi  “react” in particu ar for j va ues such that the associated sca e factors
2j (measured in pixe s) approximate y correspond to stripe width6 . The reaction
may be measured by computing the detai  signa  variance7 . This number estimate
s the average deviation of the detai  signa  from its mean va ue and therefore w
i  be arge for
6 7
As anexamp e consider d3h in Fig. 4.9 We remind
 the reader that the variance σ 2
of a ata et {y1 , . . . , yn } i  compute n n 1 1 a  follo : σ 2 = n−1 i=1 (yi − y
)2 , here y = n i=1 yi .
 
4.2 Wavelet
 Ba e Similarity Retrieval in Image Archive 
Perio : 16 Pixel 
117
50 100 150 200 250 300 350 400 450 500 100 200 300 400 500

Fig. 4.8. 512 × 512 Original. Stripe i th: 16 pixel .

etail
 ignal  ith large o cillation  about their mean. For a J tep DWT e ha
ll enote variance  ith  
2 2 2 2 2 2 σ1h , σ1v , σ1 , . . . , σJh , σJv , σJ .
(4.29)
 
For three te t image  ith
 increa ing tripe
 i th a 7 tep DWT ith the 2 b2 
  been performe . Thecorre
avelet ha  pon ing σjh value  are plotte in Fig. 4.10.
The rea er may con rm that in ee the maximum of the e value  belong  to uch a j
value that the corre poning cale factor
 2ji  approximately equal to the  re p
ective tripe i th. In a ition, cale  epen ent ani otropy mea ure  are e ne a
 follo : aj =
2 σ1v 2 (j = 1, . . . , J) σ1h
(4.30)

They in icate, if tructure of ize ≈ 2j are preferably vertically (aj > 1) or ho
rizontally (aj < 1) oriente .
118
4 More Application 

7− tep−DWT. Wavelet: b2
50 100 150 200 250 300 350 400 450 500 100 200 300 400 500
  
Fig. 4.9. 7 tep DWT of the image i playe in 4.8. Wavelet: b2.
  
Summarizing, the follo ing proce ure i  ugge te : Given an image f , e ne the “pat
tern vector”  
2 2 2 2 2 2 xf = σ1h , σ1v , σ1 , a1 , . . . , σJh , σJv , σJ , aJ .
(4.31)
 
2 The entrie  σjh in icate hether preferably horizontally oriente tructure  of
ize ≈ 2j (in pixel ) are pre ent in f .Analogou ly, the remaining  variance  are
interprete . Moreover, the  entrie
  aj in icate ani otropy a  tate above. T o i
mage  f1 an f2 are con i ere to be “ imilar”, if xf1 i  clo e to xf2 in a uitable
en e ( ee eq. 4.32).
  
The “ i tance” of t o pattern vector  xf1
 an xf1 , extracte from the original imag 
e  f1 an f2 , re pectively, i  mea ure by the norm xf1 − xf2 , hich i  compute
a  follo  ( ee al o eq. 4.5):
 
4.2 Wavelet
 Ba e Similarity Retrieval in Image Archive 
Perio : 4 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 30
0 400 500 Perio : 16 100 200 300 400 500 Perio : 64
119
100 200 300 400 500
0.4 0.3 0.2 0.1 0 0
4 3 2
50 40 30 20
1 0 0
10 5 10 0 0 5 10
5
10
 
Fig. 4.10. Top ro : Original image  ith (fromleft) a tripe i th of 4, 16 an
2 2 64 pixel , re pectively. Belo
 : corre pon ing plot  of σ1h , . . . , σ7h re ult
ing from a 7 tep DWT ith the b2  avelet.
4J
xf1 − xf2 =
i=1
(xf1 − xf2 )2 . i i
(4.32)

Here xf1 (i= 1, . . . , 4J) enote  the  i th entry of the pattern vector xf1 ,
i corre
 pon ingly xf2 i  e ne . i Ba e on thi  imilarity mea ure, in [31], [32]
a igitalimage archive i  propo e that allo  for  retrieving image  “ imilar” to
a pre ente
 image.
  In the re t of thi  ection e e cribe archiving an retriev
al proce ure  an i cu   example 8 . When enteringan image f into the ataba e
management
 y tem, in a ition to manual input
 of e criptive
 an /or a mini tra
tive ata the pattern
 vector xf i  compute , tore an linke
 to the image. Whe
n querying the ataba e for image  imilar to a pre ente image g ( imilarity re
trieval ), the follo ing tep  are performe :
8
 
Figure  4.11 – 4.13 publi he ith kin permi ion of tecmath AG.
120
4 More Application 

1. Compute the pattern vector xg . 2. Generate an “an  er li t” by orting the tore
image  fi uch that xg − xf1 ≤ xg − xf2 ≤ xg − xf3 ≤ . . . 3. Pre ent the an  er li t f1
f2 , f3 , . . . to the u er. An example i  ho n in Fig. 4.11 (4.33)
100
200
300
400
500
600
700 100 200 300 400 500 600 700 800 900 1000
Fig. 4.11. Similarity retrieval.

Here, the top ro  ho  (from left) g an fi , analogou ly the bottom ro ho 
from left fi+1 an fi+2 .The u er  may croll through thi  li t tarting ith f1
. Thi  i  the ituation i playe here: Ro i e from left to right image  g, f1
, f2 an f3 are ho n. The te t archive un erlying Fig. 4.11 con i t  of ≈ 300 im
age . The archive exhibit  a large variety
  concerning image content : The image
ho natural cene , have been recor e in a televi  ion tu io, re ult from me
ical application  (magnetic re onance imaging) an o on. A re ult of a query fo
r magnetic re onance image imilar to a given image
 g i  ho n in Fig. 4.12. It
again ho  g, f1 , f2 an f3 hich are arrange a  in Fig. 4.11.
 
4.2 Wavelet Ba e Similarity Retrieval in Image Archive 
121
50 100 150 200 250 300 350 400 450 100 200 300 400 500 600
Fig. 4.12. Another retrieval re ult.

It i  clear that there might be top ranke image  among the r t po ition  of the
an  er et, hich are not relevant for the  u er. In thi  ca e the retrieval iter

atively can be improve by “relevance  fee back” [28]. We e cribe no the proce ure,
again the given image ill be enote ith g. 1. Let the u er qualify the r t po
ition  of the an  er li t a  “relevant” or “irrelevant”. Denote ith fmi  the “mo t mi le
a ing” irrelevant image, i.e., the irrelevant image having the topmo t po ition in
the an  er li t. Furthermore, enote ith Arel the et of relevant image . 2. S
tarta ne imilarity retrieval run by replacing pattern vector xg ith xne com
pute accor ing to xne = xfmi  xg − fmi  + g x x xfi . xfi (4.34)
f i ∈Arel
  
3. In pect the ne ly generate an  er li t an procee
 ith tep 1, if nece ary
. In formula 4.34, for example, xg i  compute accor ing to
122
4 More Application 
4J
xg =
i=1
(xg )2 i
 
(cf. eq. 4.32). Analogouly the other norm quantitie
  are etermine . Thi  proce
ure ha  been implemente
 in the y tem.Ba e on the retrieval re ult from Fig.
4.12 relevance fee back ha  been carrie out. To generate the ne retrieval run
image  f 1 an
 f 4 (not ho n in Fig. 4.12)
 have been marke a  relevant. Thi 
can be veri e by checking the corre pon ing tick boxe  in the an  er li t. So in
thi  example fmi  = f 2 .
50 100 150 200 250 300 350 400 450 100 200 300 400 500 600

Fig. 4.13. Improvement by relevance fee back.
   
The ne ly generate
 an  er li t i  i playe inFig.
 4.13 ith g, f 1 , f 2 an
f 3 a  arrange before. Obviou ly the re ult in ee improve  retrieval performan
ce ith re pect to imilarity. We clo e thi  ection ith ome nal comment . Of c
our e, given t o image
  f an g, it oul be po ible to mea ure their
 imilarit
y by putting the ra ata, i.e., the matrix element  fmnan gmn , irectlyinto
formula 4.5. The rea on for u ing pattern vector  xf an xg
 in tea , i  re ucti
on of computing time. If the image , for example, are tore in SIF format (i.e.
,
 
4.3 Note  an Exerci e 
123

352 × 288) the um in 4.5 ha  to be taken over 101376 umman . In contra t, hen
choo ing, e.g., J = 4, the pattern vector  ill have 16 component . So the um i
n formula 4.32 exten  over only 16 umman . Obviou ly, it i  a nece ary prere
qui ite that the pattern vector  capture image feature  e ential for imilarity
. Moreover, the computing e ort to obtain the pattern vector from the original mu 
t be lo . Thi  i  true for the t o  imen ional J tep DWT 3.36. The e ort for comp
uting thi  tran form ill increa e only linearly ith the number of matrix eleme
nt  of the original. On a PC platform   a Pentium II proce or a imilarity r
ith
etrieval
 run on the te
 t archive e  cribe above (con i ting of ≈ 300 image ) requ
ire only a fe econ .

4.3 Note  an Exerci  e   
Note  The
 Hu man  co ec r t a  publi he in [13]. It i  the entropy co ing cheme  
upporte by the “ba eline ver ion” of the “ol ” JPEG till image compre ion tan ar . J
PEG 2000 u e  a pecial variant  of the arithmetic co ing cheme. Thi  variant in 
particular allo  for enco ing ymbol tring   “online”:
  It i  not nece ary to rea
the total tringbefore the tring co e or i  pro uce . Moreover,  a progre iv
e tran mi ion
 mo e, i.e., image tran mi  ion ith better  an better re olution, 
i  upporte . In [40] a realization  of an arithmetic
  co er on the
 ba i  of a el
programmable
 gate array (FPGA) i  e cribe . The co er i  a apte to a J tep D
WT. “Rate  i tortion
 theory” [15] tate
   that for a given i tortion any compre
 ion
cheme ill pro uce a bpp value boun  e belo by a function calle
 “rate  i torti
on function”,
 hich may be compute theoretically. Thu , metho  like the optimiza
tion proce ure from  ect. 4.1.3 implement  a practical approach to thi  theoretic
al limit. The
 rea er houl keep in min
 , ho ever, that the optimization algorit
hm e cribe above primarily  i  inten e a  an eaily acce ible  tool for illu t
rating ba ic fact  of co ing like the  tra eo of i tortion re uction v . compre 
ion eciency. So it mainly erve  i actic purpo e . In ect. 3.6.1 e alrea y men
tione the nee for “perceptually lo le  quantization”. Thi  mean  that one look 
for quantization function uch that all element  of a quantization  interval[qi
, qi+1 ) are vi ually in i tigui hable (thi   applie  accor ingly al o to au io
compre ion). It i  till an open i ue to e ne a quantitative meaure for“percept
ual imilarity”
 of image . Di tortion mea ure  like the PSNR are i ely u e , but
in or er to eci e on quantization error on a phy iological ba i, it till i 
common practice to pre ent image   (or  au io ignal ) to human “ju ge ”. Thu , automa
tic optimizer like  the one e cribe in  ect. 4.1.3 are  only one tep in a comp
lex te ting
 an a aptation
 cenario. For a itional rea ing about  imilarityret
rieval an content ba e acce  to multime ia archive  the rea er i  referre to
[25],[27].
124
4 More Application 
 
Exerci e  1. Con tructa Human co e for the ymbol tring ABCCDDEE an compare th
e re ulting average co e or length  lHuf  f man ith the entropyH(pA , pB , pC ,
pD , pE ). 2. For generating
 a co e or ith the arithmetic co er it ill be ne
ce ary to convert a ecimal number q ith 0 < q < 1 to it  binary repre entatio
n. For q = 0.7 the algorithm i  emon trate belo . 0.7 2 0.4 2 0.8 2 0.6 2 0.2
2 0.4 2 . . . = 1.4 = 0.8 = 1.6 = 1.2 = 0.4 = 0.8 → → → → → → .1 .10 .101 .1011 .10110 .1
00 . . .
(4.35)
 
So the binary expan ion of  0.7 rea  .10110 here the overline enote  the perio
ic pattern to be repeate  in nitely. Write a MATLAB function  ( ample call:DecBin
Conv(q,n)), hich rea  a ecimal number q ith 0 < q < 1 an ucce ively i pl
ay  the ingle bit  of it  binary expan ion on the creen!Ue the algorithm ke
tche in eq. 4.35 an limit  the maximum number of i playe  igit  to n. 3. Dete
rmine the arithmetic
  co e for the ymbol tring ABCCDDEE an compare the re ulti
ng average
 co e or length  lAC ith the entropy H(pA , pB , pC , pD , pE ). Hint
: To etermine the electe interval  MATLAB’  number format format rat i  u eful.
It i play  number  a  fraction  of mall integer . 4. Write a MATLAB function
ith ample call ArithBoun ( tr, ymb,freq);  String tr i  the input tring. It c
on i t  of ymbol  hich are tore in the tring ymb. Each element of ymb app
ear  in tr ith a frequency,
  hich i  a corre pon ing element  of array freq. Th
en the function
  houl i play the
 binary expan ion  of boun arie  of the nal int
erval pro uce by arithmetic co ing of tr on  the  creen! For computing the bina
ry expan ion  u e function
 DecBinConv  ketche above. A  an example, the co ing o
f ABBC coul be invoke by ArithBoun (’ABBC’,’BAC’,[2 1 1]); (cf. page 104).
 
5 Appen ix

5.1 Fourier Tran form an Uncertainty  Relation
In
 thi  ection ba ic e nition  an fact  concerning Fourier tran form are recall
e
 . The  main purpo e i  to provi e the Fourier tran form notion   require forun
er tan
 ing the material
 covere in chap. 2. For
 more etaile
 information an f
or a eeper intro uction to the topic the rea er i  referre , e.g., to [34]. Let
f (t) enote a continuou time ignal. Moreover, a ume  that f (t) i  a nite ene
rgy ignal. Thi  mean  that the follo ing relation hol :
+∞
 
|f ( )|2 d < ∞.
−∞
     
For any prac  ically relevan signal he ni e energy assump ion is ful lled. Then h
e “Fourier ransform” is de ned as follows:
+∞
ˆ f (ω) =
−∞
f (t)e−jωt dt
(5.1)
Thus, the Fourier trasform is a fuctio depedig o the real variable ω, ˆ called
the “frequecy”. f (ω) agai is a ite eergy fuctio o the frequecy domai. It som
etimes also is called “spectrum” of f (t). The sigal may be recostructed from its
spectrum: 1 f (t) = 2π
+∞
ˆ f (ω)e+jωt dω
−∞
(5.2)
Equatios 5.1 ad 5.2 establish a oe to oe correspodece bet ee a sigal ad
its Fourier trasform. A useful otatio for this fact is to coect both etit
es to a “correspodece pair”:
126
5 Appedix
ˆ f (t) ◦ − • f (ω)
(5.3)
No let f (t) ad g(t) deote t o ite eergy sigals, such that the “covolutio” of
f ith g, de ed by
+∞
(f g)(t) :=
−∞
f (u)g(t − u) du
(5.4)
leads to a fuctio f g, hich also has ite eergy. The the “covolutio theorem” s
tates that ˆg (f g)(t) ◦ − • f (ω)ˆ(ω). (5.5) Thus, covolutio i the time domai corres
to the product of the respective spectra i the frequecy domai! The covoluti
o theorem is used i sectios 2.2.1 ad 2.2.3, respectively, for developig a r

ecostructio formula for the CWT ad a fast algorithm for the computatio of th
e CWT. As a simple example illustratig Fourier trasforms cosider the fuctio
χT (t) = 1 −T ≤ t ≤ 2 0 else
T 2
It easily may be veri ed that its Fourier transform reads χT (ω) = ˆ Figure 5.1 sho s pl
ots of χT (t) and T = 6.
1.5 1 0.5 0 −0.5 −5
sin( ωT 2
ω 2
)
.
χT (ω) ˆ T,
respe tively, for T = 1 and
−4
−3
−2
−1
0 t
1
2
3
4
5
1.5 1 0.5 0 −0.5 −15
−10
−5
0 ω
5
10
15
Fig. 5.1. Above: χ1 (t) (solid) and χ6 (t) (dashed). Belo : Corresponding Fourier ˆ (ω)
transforms χ1 (ω) (solid) and χ66 (dashed). ˆ
5.1 Fourier Transform and Un ertainty Relation
127
Obviously the behavior of the fun tion and its Fourier transform is in a ertain
sense omplementary: To a fun tion, hi h is “ ide spread” in the time domain, belo
ngs a Fourier transform, hi h is “narro ” in the frequen y domain and vi e versa. T
his rather qualitative statement shall no be sharpened. Given a nite energy sign
al f (t), denote ith E its total energy, i.e., E :=
+∞ −∞
    
|f ( )|2 d . Then de ne he following “localiza ion parame ers”:
+∞

f :=
1 E
  
|f ( )|2 d
−∞ +∞
(5.6)
∆tf :=
1 E
 
(t − tf )2 |f ( )|2 d
−∞ +∞
(5.7)
1 ωf := 2πE
ˆ ω|f (ω)|2 dω
−∞ +∞
(5.8)
∆ω f :=
1 2πE
ˆ (ω − ωf )2 |f (ω)|2 dω
−∞
(5.9)
tf indi ates here on the time domain f (t) is lo ated, ∆tf provides iformatio a
bout ho f (t) is localized aroud tf : For large ∆tf values f (t) ill be ide s
pread aroud tf , for small ∆tf values f (t) ill be arro ly ˆ cocetrated aroud
tf . The same argumets apply to f (ω), ωf ad ∆ω f . Both aspects may be ui ed by repre
setig f i the t ω plae (“phase plae”, cf. sect. 1.2.5). Here f (t) essetially “liv
es” i a rectagular regio ith area ∆tf ∆ω f as idicated i Fig. 5.2.
tf
∆ω f
t
ωf ∆tf
?
ω
Fig. 5.2. Cell i phase plae, occupied by f .
128
5 Appedix
The “Heiseberg ucertaity relatio” the tells us that this area caot be arbitra
rily small: 1 (5.10) 2 This relatio is of fudametal importace. It’s cosequec
es reach from quatum mechaics to sigal aalysis. Ideed, Werer Heiseberg di
scovered that this relatio led to the famous ucertaity priciple, statig tha
t positio ad mometum of a elemetary particle ever together ca be measured
ith arbitrary precisio. I sigal aalysis, relatio 5.10 implies that o i
do fuctio ca be chose, hich is arbitrarily sharply cocetrated both i ti
me ad frequecy domai. Fortuately, there exists a fuctio hose localizatio
parameters yield the lo er boud i the iequality 5.10: Let g(t) be de ed as i
eq. 5.11 ∆tf ∆ω f ≥
t2 1 g(t) := √ e− 2 . 4 π
(5.11)
Then ∆tg ∆ω g = 1 2
5.2 Discrete Fourier Trasform (DFT)
Startig from a correspodece pair as give i eq. 5.3 e shall o cosider th
e follo ig task: Choose a “samplig distace” TS ad deote the correspodig circu
lar 2π frequenc with ωS := TS . Assume no that f is kno n only for t = . . . , −2TS
, TS , 0, TS , 2TS , . . ., i.e., the data form a sequen e {f (kTS )}. For simpl
i ity e also ill use the symbol fk instead of f (kTS ). An example ould be a
sampled a ousti signal on your omputer or your CD player. Is it possible ˆ to ga
in some information about the spe trum f (ω) from that sequen e? Provided ertain
restri tions are met, hi h ill be given belo , the ans er ill be “yes”. We shall
assume that the nite energy signal f (t) is “band limited”, i.e., ˆ f (ω) = 0 outside som
e nite frequen y interval. Let the “maximum frequen y” ωmax > 0 be de ned su h that [−ωmax
ax ] is the smallest possible interval ˆ outside of hi h f (ω) vanishes. A band lim
ited fun tion ill be said to satisfy the “Shannon ondition” if ωS > 2ωmax. (5.12)
No onsider a nitely sampled signal of length N , i.e., a sequen e {fk }N −1 . The
n the “dis rete Fourier Transform” (“DFT”) reads k=0
5.2 Dis rete Fourier Transform (DFT)
N −1
129
Fk =
l=0
fl e−
j2πlk N
(k = 0, 1, . . . , N − 1)
(5.13)

The origin l d t {fk

}N −1 m  be recovered
from the D T sequence k=0 { k }N −1 b
the “inverse discrete ourier Tr nsform” (“ID T”): k=0 fk = 1 N
N −1

l e
l=0
j2πlk N
(k = 0, 1, . . . , N − 1)
(5.14)

ˆ Before ex
l ining how the D T is rel ted with f (ω), e make t o remarks: 1. There
exists a fast algorithm for omputing the DFT, namely the famous “Fast Fourier Tr
ansform” (“FFT”). 2. If the sampled signal {fl }N −1 is real valued, the follo ing relat
ion holds: l=0 FN −k = Fk (k = 1, . . . , N − 1) As mentioned already, the overline
denotes omplex onjugation. From this relation follo s that the DFT sequen e {F
k }N −1 is ompletely deterk=0 mined if roughly the rst half of the sequen e is kno
n. To be pre ise: {Fk }N −1 is ompletely determined, provided the oe ients Fk ar
e kno n k=0 for k = 0, . . . , mN ith mN =
N 2 N −1 2
(N even) (N odd)
(5.15)
The follo ing theorem establishes the announ ed relation bet een {Fk }N −1 k=0 ˆ and
f (ω): Theorem 5.1. Let f (t) be a band limited nite energy signal ith f (t) = 0
for t < 0 and t > (N − 1)TS . Let {f (kTS )}N −1 be a nite sampling of the k=0 signal
su h that ωS = 2π TS
s tis es the Sh nnon condition 5.12. De ne the frequenc s m
ling ωk = k ωS (k = 0, . .
. , mN ). N
No ompute the DFT sequen e orresponding to eq. 5.13 ith fl = f (lTS ). Then
e have ˆ f (ωk ) ≈ TS Fk (k = 0, . . . , mN ). We lose this se tion ith some remark
s (5.16)
130
5 Appendix
1. Sin e the DFT may be omputed very qui kly ith the FFT algorithm, relation 5
.16 provides a fast algorithm for omputing the ontinuous spe ˆ trum f (ω) at the ωk
values given above. 2. If f (t) is nonzero only on the interval [0, (N − 1)TS ],
band limitation stri tly annot be ful lled [34]. This means that the Shannon ond
ition ˆ must be understood su h that f (ω) is “su iently small” outside the interval [−ωma
ωmax ]. If this is not the ase, one has to be a are that, in parti ular for high
ˆ ωk values, the estimation of f (ωk ) via eq. 5.16 leads to distortions by “aliasing”.
This explains, hy in eq. 5.16 the ≈ symbol as used instead of stri t equality. 3
. Readers interested in more details of the topi s overed in this se tion are r
eferred to textbooks on digital signal pro essing and its mathemati al foundatio
ns, respe tively. Examples are in luded in the bibliography ([33], [34]).
5.3 Digital Filters
Filtering In this se tion e shall introdu e basi notions of digital ltering. Th
is se tion is intended only for providing rst prin iples and, parti ularly, for xi
ng the notation. For more details on the topi the reader is referred to, e.g.,
[33]. We shall onsider only dis retely sampled signals using the same onventio
ns as in se t. 5.2. A digital lter generates from a dis retely sampled signal f =
{fk } a ne signal Hf = {(Hf )k }. Usually this pro ess is visualized as sho n
belo :
f
E
H
Hf
E
Fig. 5.3. A digital lter.
The lter is hara terized by a nite set of “ lter oe ients” {hi } and the ne dis rete
ignal Hf is omputed as follo s: (Hf )k =
k
hi−k fi .
(5.17)
It is useful to visualize eq. 5.17 as in Fig. 5.4. Here the lter is spe i ed by thr
ee lter oe ients {h−1 , h0 , h1 }.
fi−2
r
fi−1
r r r d h−1  h1 h0 d   d   d   dc   © d  r
(H  )i

i
5.3 Digital Filters i+1 i+2
131
r
Fig. 5.4. Visualization o  eq. 5.17 or a lter with coe cients {h−1 , h0 , h1 }.
The top ro symbolizes the input sequen e {fk }, the bottom ro the output seque
n e {(Hf )k }. The omputation of the i th element of the output sequen e is ill
ustrated as follo s: The elements of the input sequen e, hi h point to ard the
i th element of the output sequen e ith an arro , are multiplied ith the orre
sponding lter oe ients and added at the node, here the arro s meet. Thus in our
ase (Hf )i = fi−1 h−1 + fi h0 + fi+1 h1 . For ompleteness e note that a lter of th
is kind is alled FIR  lter [33], provided h1 = h2 = h3 = . . . = 0. Sin e e allo
also for nonzero values of these oe ients, e shall onsider the general ase
of “non  ausal lters”1 : If, e.g., h1 = 0 the output signal value orresponding to ti
me iTS may not be omputed before time (i + 1)TS sin e e need also the “future va
lue” fi+1 . Moreover, e also note that the indexing of the lter oe ients is slight
ly di erent from engineering standard [33], but this does not a e t the main aspe ts
of the transfer fun tions dis ussed belo . The transfer properties of a lter are
determined by the orresponding “frequen y response”: H(Ω) =
k
hk e−jkΩ (0 ≤ Ω < π)
(5.18)

The s
ectrum of Hf , s me sured, e.g., b the D T (cf. sect. 5.2), is obt ined
from the s ectrum of f b multi lic tion with the frequenc res
onse. Det ils



sh ll be omitted here, g in we refer the interested re der to [33]. The frequen
c res
onse is com
lex qu ntit. We m  decom
ose it with res
ect to bsolute
v lue nd
h se, thus obt ining “ m
litude res
onse” A(Ω) and “phase response” Φ(Ω): H(Ω) =
jΦ(Ω) . (5.19)
Again, ithout going into details e note here that the amplitude response A(Ω) is
responsible for transmitted or suppressed frequen ies. Moreover, it is desirabl
e to have a phase response Φ(Ω), hi h is linear ith respe t to Ω, sin e in this ase
the ltering pro ess leads to a redu ed signal distortion.
1
This feature is of relevan e for online appli ations, see se t. 3.6.2.
132
5 Appendix
Without proof e remark that “symmetri lters” lead to linear phase response. Here sy
mmetry is de ned as follo s: If the number of oe ients is odd, then the oe ient mu
st be symmetri ith respe t to the enter oe ient. If the number of oe ients is
even, then the oe ients must be symmetri ith respe t to the “ enter line” separat
ing the rst half from the se ond half. For example, lters ith oe ients {h−1 , h0 ,
h1 } = {−1, 2, −1} or {h0 , h1 , h2 , h3 } = {1, 2, 2, 1}, respe tively, both are sy
mmetri . As stated above, the main purpose of this se tion is to introdu e the b
asi notions and to x the notation. Examples related to amplitude and frequen y r
esponse, repe tively, are given in se t. 3.3. Filtering ith subsampling “Subsampl
ing” a sequen e means that only every se ond sequen e element is kept from the ori
ginal sequen e. This pro ess usually is represented by a box ontaining a ↓ 2 symb
ol ( f. Fig. 5.5 belo ). Obviously at this step the length of the original seque
n e is divided by t o. The graphi al representation of a ltering pro ess and a su
bsequent do nsampling pro edure is sho n in Fig. 5.5.
f
E
H
Hf
E
↓2
Hf
E
Fig. 5.5. A digital lter follo ed by do nsampling.
Thus, the ne sequen e obtained after ltering and do nsampling is denoted ith Hf
= {(Hf )k }. Mathemati ally it is omputed as follo s: (Hf )k =
k
hi−2k fi .
(5.20)
For a sequen e {fk }3 of length 4 and a lter ith oe ients {h0 , h1 } k=0 the pro
ess is visualized in Fig. 5.6. The pro edure dis ussed so far leads to sequen e
s ith half as many entries as the original sequen e, here ea h ne sequen e el
ement is omputed as a linear ombination of the original sequen e elements. Fil
tering ith upsampling, dual lters We ill need also the omplementary pro edure:
From a given sequen e a ne sequen e ill be generated ith t i e as may elemen
ts as the original,
f0
r
r h1   h0         ©   c r
1

2
r   h0         ©   c r
h1
r
5.3 Digital Filters 3
133
(H  )0
(H  )1
Fig. 5.6. Filtering with downsampling: Visualization o  eq. 5.20 or a lter with
coe cients {h0 , h1 }. The element o  H  omitted by the downsampling procedure is
indicated with dotted lines.
the new sequence elements also will be linear combinations o  the old sequence e
lements. We again start with a set o  lter coe cients {hi }. Remember that we may a
ssociate a ltering and subsampling procedure, denoted with H, as described above.
The “dual lter” H , applied to a sequence, will produce a sequence, whose length wil
l be doubled, as compared with the length o  the original sequence. The simplest
way to introduce H is a proper visualization. For the case o  three lter coe cient
s {h−1 , h0 , h1 } it is given in Fig. 5.7.
(H f )−2
r
(H f )−1
r
(H f )0
r
(H f )1
r
(H f )2
r
h1           r   −1
T0 h
s dh−1 d
h1  T0    h d   d   d   dr   0
s dh−1 d
T0 h d d d dr
f1
Fig. 5.7. Visualization of a dual lter H ith oe ients {h−1 , h0 , h1 }.
The interpretation is very simple: Imagine the diagram of Fig. 5.6 ritten do n
for three lter oe ients {h−1 , h0 , h1 } and invert it. This means that no the top
ro represents the ne sequen e, the bottom ro the original. Again, if arro s
meet at an element of the ne sequen e, the respe tive old sequen e elements mus
t be multiplied ith the orresponding lter oe ients and added. Thus, e.g., (H f
)−1 = f−1 h1 + f0 h−1 . It is lear from Fig. 5.7 that the signal length is doubled at
this pro ess. The summation formula asso iated ith dual lters looks as follo s:
134
5 Appendix
(H f )k =
k
hk−2i fi .
(5.21)
5.4 Solutions to Sele ted Problems
5.4.1 Problems from Se t. 2.4 1. a) Given y(t) ◦ − • y(ω) e have ˆ
+∞
y(ω) = ˆ
−∞
y(t)e−jωt dt.
Di erentiating this equation on both sides ith respe t to ω e obtain ⎞ ⎛ +∞ d⎝ dˆ  (t)e
(ω) = dω dω
−∞ +∞
=
−∞
y(t)
+∞
de−jωt dt dω
=
−∞
y(t)(−jt)e−jωt dt
Here on the right hand side e inter hanged di erentiation and integration, hi h
may be justi ed. Iterating this pro ess e obtain ˆ dm y (ω) = dω m
+∞
y(t)(−jt)m e−jωt dt.
−∞
Re alling de
nition 5.1
 of the Fourier transform,
 e on lude that y(t)(−jt)m ◦ − • This m
ay be rewri en as y( ) m ◦ − • j m which is he desired resul . dm y ˆ (ω), dω m dm y ˆ (ω
dω
5.4 Solutions to Sele ted Problems
135
b) Pro eed analogously as above starting ith 1 2π
+∞
(t) =

(ω)e+jωt dω. ˆ
−∞
2. In order to ompute all lo alization parameters (ω) is needed. Therefore ˆ e in
sert (t) = in eq. 5.1 and obtain
+∞ +T 2
1 −T ≤ t ≤ 2 0 else
T 2
(ω) = ˆ
−∞ +T 2
(t)e−jωt dt =
−T 2
e−jωt dt
+T 2
=
−T 2
( os(ωt) − j sin(ωt)) dt = 2
0 ωT 2
os(ωt) dt

=
2 sin ω
Here e applied the fa t that os(ωt) is an even and sin(ωt) an odd fun +∞
tion. Moreover, E =
−∞
   
|w( )|2 d = T . Then from eq. 5.6 follows ha
+∞ +T 2

1 w = E =0
−∞
  
1 |w( )|2 d = T
 
d
−T 2
 
Similarly, eq. 5.8 implies ha
+∞ +∞
ω =
1 2πE
ω|w(ω)|2 dω = ˆ
−∞
1 2πT
−∞
4 sin2 ω
ωT 2
dω
=0
136
5 Appendix
Here e made use of the fa t that From eq. 5.7 e obtain
+∞
4 sin2 ( ωT 2 ω
)
is an odd fun tion.
∆t =
1 E
 
(t − t )2 |w( )|2 d
−∞ +T 2 +T 2
=
1 T
 
2 d =
−T 2
2 T
 
2 d
0
T =√ 23 Finally, eq. 5.9 implies
+∞
∆ω =
1 2πE 1 2πT
(ω − ω )2 |w(ω)|2 dω ˆ
−∞ +∞
= =∞
4 sin2
−∞
ωT 2
dω

3. We start ith ψH s de ned in eq. 2.32. or com
uting the CWT 2.17 with the H r 


w velet, we rst ev lu te the ex ression ψH u−t , considered s function of u. Sin
ce the H r w velet is re l v lued nd we ssume > 0 this le ds to u−t u−t = ψH ⎧ ⎨
1 0 ≤ u−t < 1 a 2 = −1 1 ≤ u−t < 1 2 a ⎩ 0 else ⎧ ⎨1 t ≤ u < t + a 2 = −1 t + a ≤ u < t + a
ψH
Inserting this result in formul 2.17 we obt in
5.4 Solutions to Selected Problems
137
∞
ψH
−∞
u−t
t+ 2
t+
f (u) du =
t
f (u) du −
t+ 2
f (u) du

= t+

− (t) − (t + ) + t + 2 2

Thus
1 2 t + − (t) − (t + ) . LψH f ( , t) = √ cψ 2
ψ 4. or n w velet ψ the corres
ondence
ir ψ(t)tm ◦ − • j m d m (ω) ( f. dω problem 1)
be true. M Φ When, moreover, ψ is such th t ψ(t) = d M (t) for some suit ble function
dt Φ(t), in ddition
m
ˆ
ψ(t) =
dM Φ ˆ ˆ (t) ◦ − • (jω)M Φ(ω) = ψ(ω) dtM
( f. problem 1). Combining both relations e on lude that ψ(t)tm ◦ − • j m dm ˆ (jω)M Φ(ω)
+∞
(m = 0, . . . , M − 1). ψ(t)tm dt. Rewriting it s
A moment integr l (cf. eq. 2.31) re ds
−∞ +∞ +∞ m
ψ(t)t dt =
−∞ −∞
ψ(t)tm e−j0t dt

nd rec lling

de nition 5.1 of the ourier tr nsform we conclude th t it is identi
c l to the ourier tr nsform of ψ(t)tm , ev lu ted for ω = 0. Therefore
+∞
ψ(t)tm dt = j m
−∞
dm ˆ (jω)M Φ(ω) dω m
(m = 0, . . . , M − 1).
ω=0
ˆ ˆ ˆ ˆ If Φ(0) nd deriv tives Φ (0), Φ (0), . . . , ΦM−1 (0) exist nd re nite, dm dω m
+∞
ˆ (jω)M Φ(ω)
ω=0
= 0 (m = 0, . . . , M − 1).
Therefore
−∞
ψ(t)tm dt = 0 (m = 0, . . . , M − 1).
138
5 A

endix
5. Let f (t) s tisf the ssum
tion m de on
ge 27 nd following
ges: On I( ,
t) it m  be written s
olnomi l of degree k with k < M :
k
f (u) =
m=0
m um (u ∈ I( ,t) ).

Then
∞
ψ
−∞
u−t
k
∞
f (u) du =
m=0 k
m
−∞ ∞
um ψ
u−t
du 1
=
m=0
m
−∞
(t + u)m ψ(u) du
0
Proceeding from the rst to the second line we substituted u = u−t s new v ri b
le of integr tion. The v nishing of the integr ls follows from ex
nding (t + u
)m with the binomi l ex
nsion formul nd from the v nishing moments condition.
So n ll we obt in Lψ f ( , t) = 0. 5.4.2 Problems from Sect. 3.7 1. We consider
1 √ ψH 2 u−2k 2

. rom eq. 2.32 follows th t for k = 0, . . . , 3 ⎧ ⎨ 1 0 ≤ u−2k < 1 2 2 = −1 1 ≤ u−2k < 1

⎩ 0 else ⎧ ⎨ 1 2k ≤ u < 2k + 1 = −1 2k + 1 ≤ u < 2k + 2 ⎩ 0 else
ψH
u − 2k 2
Thus, this function will be nonzero onl on the interv l [2k, 2k + 2). On the ot
her h nd, on [2k, 2k + 2) the function f 1 (t) will be const nt with v lue f2k +
f2k+1 . Therefore 2 LψH f 1 (2, 2k) = 1 cψH 2
∞
ψH
−∞
u − 2k 2
2k+2
f 1 (u) du u − 2k 2
=
f2k + f2k+1 1 2 cψH 2
ψH
2k
du = 0
5.4 Solutions to Selected Problems
139
On [2k, 2k + 2) the function d1 (t), however, will t ke the following v lues: ⎧ f2
k −f2k+1 2k ≤ u < 2k + 1 ⎨ 2 d1 (t) = − f2k −f2k+1 2k + 1 ≤ u < 2k + 2 2 ⎩ 0 else ⎧1 ⎨ dk 2
k + 1 = −d1 2k + 1 ≤ u < 2k + 2 ⎩k 0 else Therefore
∞
LψH d (2, 2k) =
1
1 cψH 2
ψH
−∞
u − 2k 2
d1 (u) du ⎞
⎛
=
⎜ ⎜ 1 ⎜1 ⎜dk cψH 2 ⎜ ⎝
2k+1
ψH
2k
u − 2k 2
1
2k+2
du −d1 k
2k+1
ψH
u − 2k 2
−1
⎟ ⎟ ⎟ du⎟ ⎟ ⎠
1 = 2d1 k cψH 2 2 =√ d1 cψH k
√
2. Inserting h0 =
2 2 , h1
√
= hk e
2 2
in eq. 5.18 we obt in √ 2 (1 + e−jΩ ) (0 ≤ Ω < π). = 2
H(Ω) =
k
−jkΩ
Observe that (1 + e−jΩ ) = (ej 2 + e−j 2 )e−j 2 = 2 os So, H(Ω) may be ritten as follo s
: H(Ω) = √ 2 os Ω 2 e−j 2 .
Ω Ω Ω Ω
Ω 2
e−j 2 .
Ω
Comparing ith eq. 5.19 e may dire tly read o amplitude and phase response:
140
5 Appendix
A(Ω) =
√
2 os
Ω 2
,
Φ(Ω) = −
Ω . 2
Note that amplitude and phase response asso iated ith the Haar  lter have been de
pi ted in Figs. 3.9 and 3.18, respe tively! The transfer fun tions of the g  lter
are omputed analogously. 3. As an example e onsider the omputation of φ0 wavel
et . We start with ˜ the natural basis sequence φ0 = {1, 0, 0, 0}. In analogy to eq.
3.49 natural where the decomposition o  such a natural basis sequence was sketc
hed or J = 2, we decompose this sequence into two subsequences according to ˜ φ0 na
tural = {{1, 0}, {0, 0}}
1 φ
d1 φ
We now apply the reconstruction part o  eq. 3.30 or J = 1 to these sequences. T
he coe cients belonging to the dual lters H and G , respectively, are given by 3.19
and 3.20. 1 As an example, the computation o  H φ may be visualized as ollows (s
ee also Fig. 3.5):
1 √ 2 1 √ 2 1 √ 2     
r
r
0 r
√ T12
      r   1
√ T12
        r 0
1 √ 2     
0 r
Similarly we obtain G d1 = {0, 0, 0, 0}. Adding both sequences leads to φ φ0 wavelet
= 1 1 √ , √ , 0, 0 . 2 2
As a nal example we illustrate the computation o  φ4 wavelet . Here we start with ˜ φ4
natural = {{0, 0}, {0, 1}}
1 φ
d1 φ
and obtain G d1 via φ
5.4 Solutions to Selected Problems
1 √ 2 1 − √2 r
141
0 r
√ T12
0 r
r
  1   − √2       r   0
  1   − √2       r   1
√ T12
1 Here H φ = {0, 0, 0, 0}. Adding both sequences results in
φ4 wavelet =
1 1 0, 0, √ , − √ 2 2
.
3 Correspondingly, φ2 wavelet and φwavelet are computed. 4. For solutions to problem
s 4a)-c) we re er the reader to the olders mentioned on page 93. As an example
we include the solution to problem 4a):

unction []=trans ers(a) close all; i  (size(a,1)>size(a,2)), a=a’; end %Extractio

n o  h-coe icients uh=a(size(a,2)-1); oh=a(size(a,2)); h=a(1:(size(a,2)-2)); %C
omputation o  g-coe icients ug=1-oh; og=1-uh; index=length(a)-2; sign=(-1)^(ug)
; g=sign*h(index); or i=(ug+1):og, index=index-1; sign=-sign; g=[g,sign*h(index
)]; end %Plots do=input(’Step size: ’); om=0:do:pi; H=h*exp(-j*(uh:oh)’*om); G=g*exp(-
j*(ug:og)’*om);
142
5 Appendix
igure(1); plot(om,abs(H),’k-’,om,abs(G),’k--’); title(...); igure(2); plot(om,angle(H)
,’k-’,om,angle(G),’k--’); title(...); 5. Applying reconstruction ormula 5.2 to ˆ φ(ω) = e
tain
∞ π
1 |ω| ≤ π 0 else
1 2π
−∞
1 ˆ φ(ω)ejωt dω = 2π = 1 π
ejωt dω
−π π
cos(ωt) dω =
0
sin(πt) πt
= φ(t)
with φ(t) as de ned in eq. 3.58. Thus, obviously the maximum requency ωmax belonging
to φ(t) is given by ωmax = π. Choosing the s m
ling dist nce TS = 1 we obt in for the
corres
onding 2 2π s m
ling circul r frequenc ωS = TS = 4π. Thus, obviousl ωS > 2ωmax .
Comparing ith eq. 5.12 e on lude that the Shannon ondition is ful lled hen sa
mpling φ(t) with TS = 1 . The corresponding sample values 2 read {φ k } and the reco
nstruction equation 3.56 implies that 2 φ(t) =
k
φ √ 2
k
k 2
φSh t −
k 2
=
k
φ
k 2
sin(π(2t − k)) π(2t − k)
=
φk √2 φ(2t − k) 2
5.4.3 Problems from Se t. 4.3 1. Given the string ABCCDDEE e ompute relative f
requen ies, entropy, tree and ode as follo s:
5.4 Solutions to Sele ted Problems
143
Relative frequen ies: pA = pB = Entropy: 2 H(pA , pB , pC , pD , pE ) = − log2 8 =
2.25 Tree: A 1 2 B 1 C 2 D 2 4 E 2 1 8 3 log2 4 1 4 1 1 , pC = pD = pE = 8 4
−
4
8 Code:
Table 5.1. Hu man ode for ABCCDDEE. Symbol ode ord A B C D C 000 001 01 10 11
Average ode ord length: 3+3+2+2+2+2+2+2 8 18 = 8 = 2.25
lHuf f man =
144
5 Appendix
So in this example the Hu man ode rea hes the lo er bound from eq. 4.16. 2. We sh
o here a solution hi h is slightly more general than the solution to the origi
nal problem: The produ ed digits are stored in a string hi h is returned by the
fun tion. fun tion out=De BinConv(q,n) %Preparations q=2*q; out=num2str(floor(q
)); q=q floor(q); k=1; %Conversion loop hile((q~=0)&(k<n)), q=2*q; out=str at(o
ut,num2str(floor(q))); q=q floor(q); k=k+1; end 3. To generate the string ode o
rd e start ith the follo ing subdivision of the unit interval [0, 1): B C D E
A 0, 1 1 , 1 1 , 1 1 , 3 3 , 1 8 84 42 24 4 Sele tions and subsequent expansions
lead to the follo ing list of sele ted intervals: A 01 8 1 1 B 64 , 32 5 3 C 25
6 , 128 21 11 C 1024 , 512 43 87 D 2048 , 4096 173 347 D 8192 , 16384 1387 347 E
65536 , 16384 138 347 E 6517 , 16384 So the nal interval reads [u, o) = 138 347
, 6517 16384 .
Computing the respe tive binary expansions e obtain
5.4 Solutions to Sele ted Problems
145
u = .000001010110101111000000000011001001000111111000101100011 o = .000001010110
11
So, nally, the string ode ord reads 0000010101101011111. From this the average
ode ord length may be omputed: 19 8 = 2.375
lAC =
4. Again e sho here a solution, hi h is slightly more general than the soluti
on to the original problem: The binary expansions of the interval boundaries are
returned by the fun tion. fun tion[lo erbin,upperbin]=ArithBound(str,symb,freq)
%Preliminaries format rat; format ompa t; freq=[0, umsum(freq)/sum(freq)]; dis
p(’Start: ’), disp(freq), n=length(str); lo er=0; upper=1; %String loop k=1; hile(k
<=n), %Expansion expanded=lo er+freq*(upper lo er); m=1; hile(str(k)~=symb(m)),
m=m+1; end %Sele tion lo er=expanded(m); upper=expanded(m+1); disp(symb(m)), di
sp([lo er upper]), k=k+1; end lo erbin=De BinConv(lo er,100); upperbin=De BinCon
v(upper,100);
146
5 Appendix
5.5 Notations and Symbols
In this se tion frequently used symbols and notations are olle ted. The rst olu
mn des ribes the meaning, the orresponding symbol used throughout this book is
given in the se ond olumn. The third olumn displays alternative symbols used p
arti ularly in the engineering ommunity.
Table 5.2. Symbols and Notations Explanation Set of real numbers Set of omplex
numbers Continuous

time signal Wavelet Imaginary

unit Complex onjugate of a om
plex number α ourier tr nsform of f (t) ourier tr nsform
ir S m
ling dist nce
S m
ling r te S m
ling circul r frequenc Sequence of numbers Smbol in this tex
t Altern tive smbol R C f (t) ψ(t) j α ˆ f (ω) ˆ f (t) ◦ − • f (ω) TS νS = T1 S 2π ωS = TS

i α f (ω)
f = {f [k]}
Referen es
1. W. B¨ni (2002) Wavelets, eine Einf¨hrung f¨r Ingenieure. Oldenbourg, a u u M¨n hen u
2. C.G. Broyden (1970) The Convergen e of a Class of Double Rank Minimization Al
gorithms. Journal Inst. Math. Appli . 6: 76–90 3. C.S. Burrus, R.A. Gopinath, H. G
uo (1998) Introdu tion to Wavelets and Wavelet Transforms, A Primer. Prenti e Ha
ll, Upper Saddle River (NJ) 4. C.K. Chui (1997) Wavelets: A Mathemati al Tool fo
r Signal Analysis. SIAM, Philadelphia 5. A. Cohen (1992) Ondelettes, analyses mu
ltir´solution et traitement num´rique e e du signal. Ph. D. Thesis, University of Pa
ris IX, Dauphine 6. J.M. Combes et al (eds) (1989) Wavelets, Time Frequen y Meth
ods and Phase Spa e. Springer, Berlin Heidelberg Ne York 7. I. Daube hies (1992
) Ten Le tures on Wavelets. SIAM, Philadelphia 8. I. Daube hies (1989) Orthonorm
al Bases of Wavelets ith Finite Support Conne tion ith Dis rete Filters. In: J
.M. Combes et al (eds) Wavelets, TimeFrequen y Methods and Phase Spa e. Springer
, Berlin Heidelberg Ne York 9. D.L. Donoho (1995) De Noising by Soft Thresholdi
ng. IEEE Trans. on Inf. Theory 41: 613–627 10. D. Gabor (1946) Theory of Communi a
tion. J. Inst. Ele tr. Engrg. 93(III): 429–457 11. P. Goupillaud, J. Morlet, A. Gr
ossmann, (1984/1985) Cy le O tave and Related Transforms in Seismi Signal Analy
sis. Geoexploration 23: 85–102 12. F. Grupp, F. Grupp (2002) MATLAB 6 f¨r Ingenieure
. Oldenbourg, M¨n hen u u 13. D. Hu man (1952) A Method for the Constru tion of Mini
mum Redundan y Codes. Pro eedings of the IRE 40: 1098–1101 14. B. Jaehne (1995) Di
gital Image Pro essing. Springer, Berlin Heidelberg Ne York 15. N.S. Jayant, P.
Noll (1984) Digital Coding of Waveforms. Prenti e Hall, Upper Saddle River (NJ)
16. A. Jensen, A. laCour Harbo (2001) Ripples in Mathemati s  The Dis rete Wav
elet Transform. Springer, Berlin Heidelberg Ne York 17. G. Kaiser (1993) A Frie
ndly Guide to Wavelets. Birkh¨user, Boston Basel Berlin a 18. M. Kobayashi (1996)
Listening for Defe ts: Wavelet Based A ousti al Signal Pro essing in Japan. SIAM
Ne s 29, No. 2
148
Referen es
19. J.C. Lagarias, J. A. Reeds, M. H. Wright, P. E. Wright (1998) Convergen e Pr
operties of the Nelder Mead Simplex Method in Lo Dimensions. SIAM Journal of Op
timization 9:112–147 20. A.K. Louis, P. Maaß, A. Rieder (1997) Wavelets, Theory and
Appli ations. Wiley, Ne York 21. P. Maaß, H. G. Stark (1994) Wavelets and Digital
Image Pro essing. Surveys on Mathemati s for Industry 4: 195–235 22. S. Mallat (1
998) A Wavelet Tour of Signal Pro essing. A ademi Press, Ne York (NY) 23. Y. M
eyer (1993) Wavelets, Algorithms and Appli ations. SIAM, Philadelphia 24. M. Mis
iti Y. Misiti, G. Oppenheim, J. M. Poggi (2000) Wavelet Toolbox for Use ith MAT
LAB  User’s Guide. The MATH WORKS In . 25. A.H.H. Ngu, Q.Z. Sheng, D.Q. Huynh, R.
Lei (2000) Combining Multi visual Features for E ient Indexing in a Large Image
Database. The VLDB Journal 9:1–15 26. Y. Nievergelt (2001) Wavelets Made Easy. Bir
kh¨user, Boston Basel Berlin a 27. S. Santini, R. Jain (1997) Similarity is a geom
eter. Multimedia Tools Appl. 5:277 306 28. G. Salton, C. Bu kley (1990) Improvin
g Retrieval Performan e by Relevan e Feedba k. Journal of the ASIS 41:288–297 29.
C.E. Shannon (1948) A Mathemati al Theory of Communi ation. Bell System Te hni a
l Journal 27 30. K. Sigmon (1994) MATLAB Primer. Chapman & Hall/CRC, Bo a Raton
London Ne York Washington D.C. 31. H. G. Stark (1996) On Image Retrieval ith W
avelets. International Journal of Imaging Systems and Te hnology 7:200–210 32. H. 
G. Stark (1997) Wavelets und Bildar hive. at – Automatisierungste hnik 45:577–584 33
. S.D. Stearns, R.A. David (1996) Signal Pro essing Algorithms in MATLAB. Prenti
e Hall, Upper Saddle River (NJ) 34. W. Strampp, E.V. Vorozhtsov (2004) Mathemat
is he Methoden der Signalverarbeitung. Oldenbourg, M¨n hen u 35. R.F. Streater, A.
S. Wightman (1980) PCT, Spin and Statisti s and All That. Benjamin/Cummings, Rea
ding (MA), London, Amsterdam 36. T. Strutz (2000) Bilddatenkompression, Grundlag
en, Codierung, MPEG, JPEG. Vie eg, Brauns h eig 37. J.T. Tou, R.C. Gonzalez, (19
74) Pattern Re ognition Prin iples. AddisonWesley Publishing Company, Reading (M
A) 38. J.S. Walker (1999) A Primer on Wavelets and Their S ienti Appli ations. C
hapman & Hall/CRC, Bo a Raton London Ne York Washington D.C. 39. G.K. Walla e (
1991) The JPEG Still Pi ture Compression Standard, Comm. of the ACM 34: 31–44 40.
D. Ziener (2002) FPGA Implementierung eines arithmetis hen Codierers. Diploma Th
esis, As ha enburg University of Applied S ien es
Index
Admissibility ondition, 21 Approximation signal, 61 Arithmeti oding, 104 Bits
per pixel, 111 bpp, 111 Cas ade algorithm, 56 Co hlea, 40 Coi ets, 85 Compression
bit per pixel (bpp), 111 ompression rate, 96 na¨ s heme, 84 ıve peaked histograms,
84 transform ompression s heme, 95 CWT, 4, 21 fast algorithm, 23 re onstru tio
n, 24 transform, 21 visualization, 7 Daube hies  avelets, 53 Denoising, 88 Detai
l signal, 61 diagonal details, 66 horizontal details, 66 verti al details, 66 DF
T, 128 Digital ltering, 130 Dis rete Fourier transform, 128 Distortion measures n
orm distan e, 96 PSNR, 96
DWT, 43 one dimensional signals, 61 real time properties, 88 t o dimensional sig
nals, 68 Energy ompa tion, 64, 81 Entropy, 100 Entropy oding, 99 Fourier trans
form ontinuous transform, 3, 125 dis rete Fourier transform, 128 un ertainty re
lation, 128 Gabor transform, 19 Hu man oding, 102 Iterative optimization Nelder M
ead, 114 Quasi Ne ton, 114 JPEG, 98 JPEG 2000, 75, 98 progressive mode, 123 Lo a
l analysis, 2 Lo alization parameters, 127 MATLAB, 11 Optimization Toolbox, 114
STFT fun tions myistft, 20 mystft, 19
150
Index spe trum estimation, 129 Short time Fourier transform (STFT), 4, 14 fast a
lgorithm, 15 Signal ontinuous time, 2, 125, 146 dis rete (sampled), 13, 43, 128
, 146 nite energy, 125 Signal analysis, 1 Signal lassi ation, 36 Signal ompressi
on, 1 Similarity retrieval, 116 Spe trum estimation, 129 STFT, 4, 14 Transfer fu
n tion amplitude response, 131 biorthogonal lters, 75 dbn  lters, 60, 74 frequen y
response, 131 phase response, 131 Un ertainty relation, 128 Vanishing moments o
ndition, 26 pra ti al impli ations, 27 Varian e, 116 Wavelet transform ontinuou
s transform (CWT), 4, 21 dis rete transform (DWT), 43 Wavelets admissibility on
dition, 21 biorthogonal avelets, 73 Coi ets, 85 Daube hies  avelets, 53 dbn  avel
ets, 55 generation by derivation, 26 Haar  avelet, 26 mexi an hat  avelet, 27 Mo
rlet  avelet, 28 vanishing moments, 26 WFT, 4, 14 Windo ed Fourier transform (WF
T), 4, 14
avelet fun tions CompAudioToolbox, 87 CompImageToolbox, 87 bases1d, 81 bases2d,
81 my t, 29 myi t, 31 optshell, 113 Wavelet Toolbox, 11 t, 31 avede , 70
avede 2, 71 avere , 71 avere 2, 73 filters, 73 Multis ale analysis, 60 Norm d
istan e, 96 Pattern re ognition, 38 Pattern ve tor, 38, 118 Peak signal to noise
ratio, 96 Phase plane, 8, 127 lo alization parameters, 127 Phase spa e represen
tation, 8 PSNR, 96 Quantization non uniform, 111 optimal, 108 per eptually lossl
ess, 85, 123 uniform, 99 Rate distortion fun tion, 123 Re onstru tion from sampl
ed signals, 93 Relevan e feedba k, 121 Sampling ir ular frequen y, 146 Sampling
distan e, 13, 146 Sampling rate, 146 S ale plane, 8 S ale spa e representation,
8 Shannon sampling ondition, 128 re onstru tion from sampled signals, 93