Wavelets and Signal Processing
Wavelets and Signal Processing
Wavelets and Signal Processing
Hans-Georg Stark
Wavelets and Signal Processing
An Application-Based Introduction
With 67 Figures and 7 Tables
123
Professor Dr. Hans-Georg Stark FH Aschaffenburg - University of Applied Sciences
FB Ingenieurwissenschaften Würzburger Str. 45 63743 Aschaffenburg Germany hans-ge
[email protected]
Library of Congress Control Number: 2005921923
ISBN 3-540-23433-0 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or
part of the material is concerned, speci cally the rights of translation, reprint
ing, reuse of illustrations, recitation, broadcasting, reproduction on micro lm or
in any other way, and storage in data banks. Duplication of this publication or
parts thereof is permitted only under the provisions of the German Copyright La
w of September 9, 1965, in its current version, and permission for use must alwa
ys be obtained from Springer. Violations are liable to prosecution under the Ger
man Copyright Law. Springer is a part of Springer Science+Business Media springe
ronline.com © Springer-Verlag Berlin Heidelberg 2005 Printed in The Netherlands Th
e use of general descriptive names, registered names, trademarks, etc. in this p
ublication does not imply, even in the absence of a speci c statement, that such n
ames are exempt from the relevant protective laws and regulations and therefore
free for general use. Typesetting: By the author Production: LE-TEX Jelonek, Sch
midt & Vöckler GbR, Leipzig Cover design: medionet AG, Berlin Printed on acid-free
paper 7/3142/YL - 5 4 3 2 1 0
To Fabi and Paula
Preface
“The idea of this book arose in a conversation with H. A. Bethe, who remarked that
a little book about modern eld theory which contained only memorable results wou
ld be a good thing”1 . As a graduate student working in theoretical physics, in pa
rticular desperately needing a readable and compact treatment of quantum eld theo
ry, I came across a beautifully written book: “PCT, Spin and Statistics and all th
at” [35]. Here we are dealing with signal processing instead of quantum eld theory
and the present book does not claim to reach a level of clarity and deepness com
parable to the respective achievement in [35]. Nevertheless, the basic intention
of this book perfectly is described by the above quotation when replacing “ eld the
ory” with “wavelet analyis”: It should provide a quick and readable introduction to th
e wavelet topic for anyone interested in learning about wavelets and/or working
with them and hopefully will contain only memorable results. What does the pro le
of the “ideal reader” of this little book look like? The largest portion of the cove
red material should be readable for anyone with an elementary technical backgrou
nd and familiar with basic notions of calculus like integrals and complex number
s. Thus these parts should be accessible for readers with an engineering or scie
nce education on a bachelor level who are interested in a short and compact intr
oduction into wavelet applications. The target group includes academia and (in p
articular) practitioners in industry. Some parts of the book are of interest mai
nly for readers with a specialization on, e.g., electrical engineering or commun
ications engineering. For these parts basic familiarity with notions from signal
analysis like Fourier transforms and digital ltering will be helpful. Elementary
information about these topics is collected in the appendix. There are some exc
ellent introductory books on wavelets and wavelet applications mainly focussing
on the discrete wavelet transform (cf., e.g. [38]).
1
Quotation from the preface of [35]
VIII
Preface
This book will contain a fairly large part dealing with the continuous transform
, since in my opinion the continuous transform provides a very intuitive insight
into the essence of wavelet techniques and since the continuous version is clos
e to the Fourier transform, a standard tool in engineering. As with all books, s
election and presentation of the material re ect the author’s view of the topic. My
hope is that after reading the book the reader will have gained an intuitive ins
ight into what wavelets are. This insight should then provide a basis on which t
o decide, whether wavelets are interesting for the particular needs of the reade
r and, if yes, how he or she wants to apply them. Therefore this book is intende
d to serve as an introductory guide to the topic. Moreover, the wavelet story is
a story about a fascinating and exciting scienti c development. If this fascinati
on can be felt during reading, this is among the best I can hope for. It is a pl
easure to thank Eva Hestermann-Beyerle and Monika Lempe from Springer-Verlag for
their patience and professional support. Their comments had substantial in uence
on sharpening the pro le of this book. The work reported on in section 4.2 would n
ot have been possible without the collaboration and support of Gernod Laufk¨tter,
Andreas Divivier and Thomas o Goseberg, my friends and colleagues at tecmath AG.
I am very grateful for the opportunity to work with them. Dr. Hans-J¨ rgen Stahl
checked the English of chap. 1, a copyeditor from u Springer-Verlag did so with
the complete manuscript. I am indebted to both for valuable hints which helped m
e to improve spelling and style.
Ascha enburg, spring 2005
Hans-Georg Stark
Contents
1
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 1 1.1 Signals and Signal Processing . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . 1 1.2 Local Analysis . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2.1 Transf
orms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 2 1.2.2 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . 3 1.2.3 Short Time Fourier Transform (STFT) . . . . . . . . . . .
. . . . 4 1.2.4 Wavelet Transform . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 4 1.2.5 Visualization . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 7 1.2.6 Fourier vs. Wavelet Transform - A Com
parison Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 9 1.3 A Roadmap for the Book . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 11 Continuous Analysis . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 The Short Time Fourier
Transform (STFT) . . . . . . . . . . . . . . . . . 2.1.1 De nition, Computation an
d Reconstruction . . . . . . . . . . 2.1.2 Phase Space and Localization Paramete
rs . . . . . . . . . . . . 2.1.3 Implementation with MATLAB and Visualization .
. . . . 2.2 The Continuous Wavelet Transform (CWT) . . . . . . . . . . . . . . .
. . 2.2.1 De nition, Computation and Reconstruction . . . . . . . . . . 2.2.2 Wav
elet Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2.3 Implementation with MATLAB and Visualization . . . . . 2.2.4 Application:
Detection of Signal Changes . . . . . . . . . . . . . 2.3 Case Studies . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
.3.1 Analysis of Sensor Signals . . . . . . . . . . . . . . . . . . . . . . . .
. . 2.3.2 Analysis and Classi cation of Audio Signals . . . . . . . . . . 2.4 Note
s and Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . The Discrete Wavelet Transform . . . . . . . . . . . . . . . . . . . .
. . . . . . . 3.1 Redundancy of the CWT and the STFT . . . . . . . . . . . . .
. . . . . . 3.2 The Haar-System . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 3.2.1 Continuous-Time Functions . . . . . . . .
. . . . . . . . . . . . . . . . . 13 14 14 18 19 21 21 26 29 32 33 33 36 40 43 4
3 45 46
2
3
X
Contents
3.2.2 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . 3.3 Generalization to Daubechies-Wavelets . . . . . . . . . . .
. . . . . . . . . . 3.3.1 From Filters to Functions . . . . . . . . . . . . . .
. . . . . . . . . . . . 3.3.2 Transfer Properties . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . 3.4 Multiscale Analysis . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.1 One-Dimensional Sign
als . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4.2 Two-Dimensional
Signals (Images) . . . . . . . . . . . . . . . . . . . 3.4.3 Implementations wit
h the MATLAB Wavelet Toolbox . . 3.4.4 Generalization: Biorthogonal Filters . .
. . . . . . . . . . . . . . . 3.5 A Unifying Viewpoint: Basis Systems . . . . .
. . . . . . . . . . . . . . . . . 3.5.1 One-Dimensional Signals . . . . . . . .
. . . . . . . . . . . . . . . . . . . 3.5.2 Two-Dimensional Signals . . . . . .
. . . . . . . . . . . . . . . . . . . . . 3.5.3 Computation and Visualization wi
th MATLAB . . . . . . . 3.6 Case Studies . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 3.6.1 Energy Compaction and Com
pression . . . . . . . . . . . . . . . . 3.6.2 Denoising a Sensor Signal / Real-
Time Properties of the Algorithm . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . 3.7 Notes and Exercises . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . 4
49 53 56 59 60 61 65 69 73 75 76 79 81 81 81 88 91
More Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 95 4.1 The Transform Compression Scheme . . . . . . . . . .
. . . . . . . . . . . . . 95 4.1.1 The General Procedure . . . . . . . . . . . .
. . . . . . . . . . . . . . . . 97 4.1.2 Entropy Coders . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . 99 4.1.3 Optimal Quantization and
Examples . . . . . . . . . . . . . . . . . 108 4.1.4 MATLAB Implementation . .
. . . . . . . . . . . . . . . . . . . . . . . . 113 4.2 Wavelet-Based Similarity
Retrieval in Image Archives . . . . . . . . 116 4.3 Notes and Exercises . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Appendix
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . 125 5.1 Fourier Transform and Uncertainty Relation . . . . .
. . . . . . . . . . . 125 5.2 Discrete Fourier Transform (DFT) . . . . . . . .
. . . . . . . . . . . . . . . . . 128 5.3 Digital Filters . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 5.4 Solution
s to Selected Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
34 5.4.1 Problems from Sect. 2.4 . . . . . . . . . . . . . . . . . . . . . . . .
. . . . 134 5.4.2 Problems from Sect. 3.7 . . . . . . . . . . . . . . . . . . .
. . . . . . . . . 138 5.4.3 Problems from Sect. 4.3 . . . . . . . . . . . . . .
. . . . . . . . . . . . . . 142 5.5 Notations and Symbols . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . 146
5
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . 147 Index . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. 149
1 Introduction
1.1 Signals and Signal Processing
Wavelet analysis had its origins in the mid-eighties. From the very beginning it
was driven by application needs: The desire to analyze seismic signals more sen
sitively than with Fourier techniques led to the rst appearance of the continuous
wavelet transform formula. In parallel it turned out that the new technique cou
ld be applied successfully to certain problems in theoretical physics as well as
in pure mathematics. For one of the earliest collections of research and survey
papers documenting the state of the art the reader is referred to [6]. It soon
turned out that wavelet analysis successfully could be applied to many types of
signal processing problems: In signal analysis the detection of discontinuities
or irregularities was tackled with wavelets. The analysis of medical signals lik
e electrocardiograms of the heart is one of the rst reported examples of disconti
nuity detection (see [6]). For more applications, like the analysis of sensor si
gnals in robotics, cf. sect. 2.3.1. Signal compression is another impressive exa
mple of wavelet applications. JPEG 2000, the present version of the internationa
l standard on still image compression is based on wavelet techniques (see, e.g.,
[36]). Wavelet applications both in signal analysis and signal compression shal
l be described in more detail in later sections. This chapter serves as a brief
introduction to the main features of the wavelet transform by comparing wavelet
transform with Fourier transform, the standard tool of signal analysis. For that
purpose we shall work out the common aspects of wavelet and Fourier transforms
and point out the main di erences. For understanding the following section, the kn
owledge of the Fourier transform is not a necessary prerequisite. On the other h
and, of course, it would be useful, if the reader already had some experience wi
th applications of the Fourier transform. Basic facts about the Fourier transfor
m are collected in the appendix, sections 5.2 and 5.3.
2
1 Introduction
Mathematical symbols, used throughout this book, are explained upon their rst app
earance. They are collected in sect. 5.1 of the appendix.
1.2 Local Analysis
In this section we will deal with signals which may be represented by a function
f (t) depending on time t. We shall assume that t is a continuously varying par
ameter; thus f (t) is called a “continuous-time signal”. We shall try to transform f
(t) into a representation, which encorporates the desired information about the
signal as compactly as possible. The Fourier transform (cf. sections 1.2.2 and
1.2.3) supplies information about the contribution of certain frequencies to the
signal, the wavelet transform (cf. sect. 1.2.4) indicates whether details of a
certain size are present in a signal and quanti es their respective contribution.
Both transforms are called “local” if they not only globally measure frequencies and
detail sizes, respectively, but also indicate where they are located in the sig
nal f (t). There are many applications for the kind of signal information descri
bed above – we explicitly mention signal classi cation and data compression. These a
pplications are described in more detail later, in the subsections below we indi
cate how frequencies and detail sizes may be measured. Furthermore, we will work
out the aspects which are common to both transforms and illuminate the respecti
ve di erences. The transform results shall be visualized and we will give an examp
le which serves as an illustration for the above-mentioned compactness of the re
spective signal representations. The purpose of this chapter is to introduce the
ideas underlying Fourier and wavelet transforms, respectively. For more - in pa
rticular for mathematical - details the reader is referred to the following chap
ters. 1.2.1 Transforms All transforms of the signal f (t) described in this sect
ion share a common computation principle: The signal is multiplied with a certai
n “analysis function” and integrated about the time domain. In a symbolic notation t
he prescription for performing a transform reads
+∞
f (t)
transf orm
−→
f (u)g(u) du
−∞
(1.1)
The “analysis function” g(u) characterizes the chosen transform. In general it may b
e a complex function, the overline denotes the complex conjugate entity. g(u) in
a certain way depends on the parameters, i.e. frequencies or detail sizes, to b
e measured (see below). Thus, by the computation principle given above the trans
formed entity will depend on these parameters. In other
1.2 Local Analysis
3
words: the transformed entity again will be a function. These functions we shall
denote with “transform” or “transformed signal”. Another common aspect of all transform
s discussed in this section is invertibility: From the transformed signal the or
iginal signal f (t) may be reconstructed. This is essential for understanding th
e comparison experiment carried out in sect. 1.2.6. 1.2.2 Fourier Transform The
parameter relevant for the Fourier transform is the circular frequency ω, the anal
ysis function reads gω (u) = ejuω . Thus the transformed signal is a ˆ function depend
ing on ω and it is denoted ith f (ω):
+∞
ˆ f (ω) =
−∞
f (u)gω (u) du
(1.2)
Figure 1.1 illustrates the above computation recipe by plotting both curves ˆ requ
ired for computing f (π). The signal is shown as a solid curve, the real art of t
he analysis function gπ (u) = ejuπ is dashed. Obviously, it is an harmonic oscillati
on with circular frequency ω = π.
4 f(u) 3
2 Re(g (u)) π 1
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.1. Fourier transform: signal and analysis function
ˆ Why does f (π) measure the aearance of ω = π in the signal? The qualitative argument
is as follows: If in some time interval the signal oscillates with circular fre
quency ω = π, the signal and the analysis function have a
4
1 Introduction
constant mutual hase shift in this interval and therefore rovide a nonzero ˆ con
tribution to f (π). Yet there is no ossibility to localize the aearance of the
circular freˆ quency: If (the absolute value of) f (π) is “large”, we only know that the
signal contains the circular frequency π, but we do not know where it aears, si
nce the analysis function extends over the whole real axis. The only label aram
eterizing the analysis function is circular frequency. 1.2.3 Short Time Fourier
Transform (STFT) This transform sometimes also is called “Windowed Fourier Transfo
rm” (WFT). The STFT looks for the aearance of the circular frequency ω at a certai
n time t. The corresponding analysis function reads: g(ω,t) (u) = ejuω (u − t). Here
(u) is a “ indo function”, usually centered about the origin (for an example see b
elo ). In the expression (u − t) this indo is shifted to the desired time t. No
the transformed signal depends on ω and t! Since it also ill depend ˆ on the shap
e of the indo function, it is denoted ith f (ω, t):
+∞
ˆ f (ω, t) =
−∞
f (u)g(ω,t) (u) du
(1.3)
For a box indo of idth 2, centered symmetrically about 0, ω = π and t = 8, the
comutation rincile is illustrated in Fig. 1.2. Again the dashed curve shows t
he real art of the analysis function g(π,8) (u); it is obviously now localized at
t = 8, since w(u − 8) denotes the box window, shifted by 8 units to the right. In
general, the analysis function will be localized at the resective “analysis time”
t . Therefore the transform rovides not just global information about the aea
rance of a certain circular frequency, but in addition the time of this aearan
ce. The rocedure described so far has a disadvantage: If in the above examle o
ne is interested in small details of the signal around t = 8, the corresonding
frequency of the analysis function must be increased. As an examle Fig. 1.2 is
redrawn for ω = 6π in Fig. 1.3. Obviously the window width is constant and non adat
ive: If one is interested in very tiny signal details (high frequencies) in only
a small neighborhood of t = 8, eventually signal arts, which actually are “not o
f interest”, also will be analyzed. Zooming into small details analogously to a
microscoe is not suorted. 1.2.4 Wavelet Transform The wavelet transform has
such a zooming roerty. In contrast to the Fourier transform, the wavelet tran
sform does not look for circular frequencies but
1.2 Local Analysis
4 f(u) 3
5
2 Re(g 1
(π,8)
(u))
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.2. STFT: signal and analysis function for ω = π
4 f(u) 3
2 Re(g 1
(6π,8)
(u))
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.3. STFT: signal and analysis function for ω = 6π
rather for detail sizes a at a certain time t. Instead of detail sizes we also w
ill seak of “scale factors”, both notions will be used equivalently. As mentioned a
lready, high frequencies corresond to small details and vice versa, thus, when
comaring wavelet with Fourier transforms we have to take into account that freq
uencies and detail sizes are inversely roortional to each other: There exists
a constant β such that
6
1 Introduction
β . (1.4) ω We shall no rie y indicate, ho the avelet transform is computed. Consi
der a (real or complex) analysis function g, oscillating around the a=
+∞
u-axis (mathematically:
−∞
g(u) du = 0) and decreasing raidly for u → ±∞.
Such a function is called a “wavelet”. In eq. 1.4, relating scale factors with frequ
encies, the constant β depends on g. Starting from g consider the follo ing family
of functions: g(a,t) (u) = 1 √ g u−t . The members of this family are generated fro
m g by shifting the a a function to t followed by shrinking (a < 1) or dilating
(a > 1) the width of the function. The wavelet transform now reads:
+∞
Lg f (a, t) =
−∞
f (u)g(a,t) (u) du
(1.5)
For the “Haar wavelet”
⎧ ⎨1 0 ≤ u < 1 2 g(u) = −1 1 ≤ u < 1 2 ⎩ 0 else
1 2
the comutation of Lg f (a, t) with a =
4 f(u) 3
and t = 8 is illustrated in Fig. 1.4.
2 g((u−t)/a) 1
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.4. Wavelet transform: signal and analysis function for a =
1 2
The reader may note that the Haar wavelet, originally situated in the interval [
0, 1) now has been shifted to the right by 8 units and its width has
1.2 Local Analysis
7
shrunk by the factor 1 , corresonding to the chosen values of t and a. For 2 a
= 1 and t = 8 we obtain Fig. 1.5. 4
4 f(u) 3
2 g((u−t)/a) 1
0
−1
−2
−3 0
1
2
3
4
5 u
6
7
8
9
10
Fig. 1.5. Wavelet transform: signal and analysis function for a =
1 4
Comare Figs. 1.4 and 1.5 with Figs. 1.2 and 1.3 and note that the wavelet trans
form shows the desired zooming roerty in contrast to the STFT: When searching
for smaller and smaller details (higher and higher frequencies) with the wavelet
transform, the corresonding analysis function is oscillating faster and is con
tracted. 1.2.5 Visualization ˆ Both the STFT fw (ω, t) andthe avelet transform Lg
f (a, t) are functions depending on t o varia les. A suita le visualization of t
hese functions is of essential importance in signal analysis. A ide-spread grap
hical representation of t o-dimensional functions is the use of contour lines. I
n signal analysis one usually
prefers the visualization of the a solutevalues o
f the respective transforms y gray values. High values are coded ith right, l
o values
ith dark gray values. Figure1.6 sho s such a visualization
for the S
TFT (a ove) and the avelet
transform ( elo ). As a signal in oth cases the “chir
p” f (t) = sin(t2 ) has een used. The chirp is an harmonic oscillation sin(ωt), ho
se circular frequency
increases ith t: ω = t. The linear increase of frequency is
clearly visi le ith the STFT (see the upper part of Fig. 1.6). Since (cf. eq.
1.4) detail size a and frequency ω are inversely proportional ith respect to each
other, for the avelet transform one ould expect a
8
1 Introduction
f(t)=sin(t2). Parameters: δ =5,k
x min
=1,δ =1,k
k
max
=50,σ=0.2
0.5 0 −0.5
0
1
2
3
4
5
6
7
8
9
10
0 5 10 15 20 25 30 0 1 2 3 4 5 t 6 7 8 9 10 ω
f(t)= in(t2). Scale parameter : a tart=1,δa=1,a top=150.
0.5 0 −0.5
0
1
2
3
4
5
6
7
8
9
10
0.2 0.4 0.6 a 0.8 1 1.2 1.4 0 1 2 3 4 5 t 6 7 8 9 10
Fig. 1.6. Above: STFT of the chirp ignal f (t) = in(t2 ). Belo : Wavelet tran
form of the chirp ignal f (t) = in(t2 ).
behavior corre pon ing to a hyperbola (i.e. proportional to 1 ). The lo er t par
t of Fig. 1.6 ho exactly thi behavior. Since the STFT epen on t an ω, the
gray value co ing of the STFT ha been performe
on the tω plane. Thi plane i a
l o calle “pha e plane”, the corre pon ing gray value co ing “pha e pace
repre entat
ion” of the STFT.Analogou ly the t a plane i calle “ cale plane” an the corre pon
ing gray value co ing of the avelet tran form “ cale pace repre entation” of the
avelet tran form.
1.2 Local Analy i
9
1.2.6 Fourier v . Wavelet Tran form A Compari on Experiment In a certain en e
, the zooming property
of the avelet tran form en ure that characteri tic feat
ure of the analyze ignal on a certain cale are ell repre ente by the tran
form value corre pon ing to thi cale factor, i.e. not i tribute among other
cale factor . Moreover, the e tran form value
ill be localize at the re pec
tive ignal part , here the above mentione feature are pre ent. The e concent
ration propertie both ith re pect to cale an time may be formulate math
ematically in a more rigorou ay; the purpo e of thi ection i , to give a pla
u ibility argument for the above tatement by performing
a compari on experiment
ith the Fourier tranform. The ignal i playe in Fig. a ection from
1.7 i
the
beginning of an au io ignal.
Roughly in the mi le, the u en tart of ou
n clearly can be recognize .
0.15
0.1
0.05
0
−0.05
−0.1
−0.15
−0.2
−0.25 0
1000
2000
3000
4000
5000
6000
7000
8000
9000
Fig. 1.7. “Attack ignal”
Such ignal (“attack
ignal ”) hall locally, i.e. in a mall neighborhoo of the p
oint, here the oun tart , contain high frequencie , equivalently, there ill
be ra tic change on a mall cale.
In uch a ituation the
zooming property o
f the avelet tran form houl be a vantageou hen compare ith the Fourier tr
an form. To con rm thi conjecture, the follo ing experiment ha been carrie out:
1. Compute the Fourier tran form of the ignal, keep tho e 4% of the value of
the tran forme ignal, having the large t ab olute value . Put the remaining
tr
an form value equal to zero
an recon truct the ignal from thi mo
i e tran for
m (remember that, a tate in ect. 1.2.1, all tran form i cu e here are in
vertible).
10
1 Intro uction
2. Perform the ame proce ure ith the avelet tran form in tea of the Fourier
tran form. The re ult of the experiment are ho n in Fig. 1.8. The a he curve
ho the re ult of the Fourier
recon truction, the avelet recon truction i
i playe by + ymbol , the oli line repre ent the original
ignal. When compa
ring ith Fig. 1.7, ob erve that the curve ho an enlarge ection of the ign
al from Fig. 1.7 in a neighborhoo of the point here the oun tart .
0.1
0.05
0
−0.05
−0.1
−0.15
−0.2
−0.25 3680 3700 3720 3740 3760 3780 3800 3820 3840
Fig. 1.8. “Attack ignal”: recon truction
Obviou ly, hen u ing the Fourier tran form, the uppre ion of 96% of the tran
form ignal value namely
tho e tran
form ignal value having lo er ab olute
value
than the retaine one lea to a global moothing (lo pa ltering) an
therefore the local peak uring the attack phae are not repro uce any more.
t, applying the
In contra ame uppre ion proce ure to the avelet tran form
oe not i turb the repro uction of the e peak . Thi i a clear in ication for
the above mentione concentration
propertie of the avelet tran form value .A
a nal remark e houl in icate that the original ignal
a not repre ente a
a continuou time ignal, but it a i cretely ample . For uch ignal there
exi t variant of the continuou formulae
1.2 an 1.5, re pectively, ith hich
the above experiment ha been carrie out. The e “ i crete tran form ” hall be e c
ribe in later ection . Moreover, a ith any avelet tran form application, th
e re ult
of the above experiment epen on the cho
en avelet g. There ult i
playe in Fig. 1.8 have been obtaine u ing the b4 avelet e cribe in a late
r ection.
1.3 A Roa map for the Book
11
1.3 A Roa map for the Book
The topic ketche in thi intro uction ill be e cribe in more etail in cha
p. 2. Que tion like “ hat i an optimal ino function for the Short Time Fourie
r Tran form?” or “ho may vi ualization a i playe in Fig. 1.6 be accompli he ?” i
ll be treate there. Moreover, e ill in icate ho the original ignal may be r
econ tructe from the re pective tran form . Thi i of crucial importance for a
pplication like ignal compre ion: There, tran form of the original ignal u
ually are compute in a r t tep. Sub equently the e tran form are mo in uch
i e
a ay that the require torage pace for the tran forme ignal i re uce con
i erably.
Finally, in the “ ecompre ion” tep the ignal are recon tructe from t
he mo i e tran form . In chap. 2 al o a brief urvey i given of typical ignal a
naly i application of the Short Time Fourier Tran form an the avelet
tran fo
rm, re pectively. T o in u trial ca e tu ie are e cribe in more etail. Sect
ion 2.4 contain ome exerci e . Partly the e exerci e ill be of “paper an pen
type”, but they al o ill con i t in riting computer program . A a programming
p
latform e u e the tool MATLAB, hich i i e prea an a e facto tan ar in t
he engineering community. Of cour e it i po ible to u e the book ju t a a ref
erence gui eto avelet technique ithout performing any programming.
For rea e
r intere te in u ing the oft are e cribe in thi book an in eveloping the
ir o n program
, ho ever, the u e of MATLAB ill be a prerequi
ite. Rea er havi
ng alrea y ome familiarity ith programming language an looking for a compact
ly ritten intro uction to MATLAB are referre to [12]. Thi book provi e a ver
y nice an e cient e cription of MATLAB’ main feature . It i ritten in German; i
f thi turn out to be an ob tacle, reference [30]
i highly recommen e . The MA
TLAB oft are hich
i i cu e in thi book an
ha been ritten by the author
may be o nloa e from . pringeronline.com/ e/3 540 23433 0 Mo t of it requi
re only the ba i ver ion of MATLAB. Some program , ho ever, make u e of the MA
TLAB Wavelet Toolbox,
a collection of aveletrelate
ignal proce ing algorithm
. It u e i e cribe in the u er’ gui e [24], hich i not ju t a oft are han
book. It i moreover a beautifully ritten intro uction to avelet technique .
Since in the author’ opinion MATLAB together ith the avelet toolbox ill turn o
ut to be a tan ar oft are platform for avelet relate ignal proce ing, the
pre ent book al o ill provi e a hort intro uction to the mo t important compo
nent of the MATLAB Wavelet Toolbox. In chapter 1 an 2, re pectively, ignal
are con i ere tobe continuou time ignal , even though computerize ver ion o
f the algorithm e cribe there nece arily involve a i cretization. A i erent
per pective i given in chap. 3: Here from the very beginning ignal are icre
te equence of number an the avelet tran form e cribe there i e igne fo
r uch equence .
12
1 Intro uction
The corre pon ing notion , avelet con truction , ignal tran form an recon tru
ction formulae are given in thi chapter. Mo t important for practical applicati
on i the exi tence of fat algorithm both for tran formation an recon tructi
on. They are al o e cribe in thi chapter together ith MATLAB implementation
. Thereafter again application an ca e tu ie are pre ente . In thi context
e hall alo comment on real time propertie of fa t avelet
algorithm
. Sectio
n 3.7 provi
e exerci e . Chapter 4 i evote to a more etaile e cription of
ome a itional application of the avelet tran form. Fir t e hall focu on
the mo t popular
application, namely ata compre ion. Sub equently, an applicat
ion relate to retrieving image from ataba e management ytem i e cribe .
Again at the en of thi chapter ome exerci e are pre ente . A far a mathema
tic i concerne , e a ume ome familiarity of the rea er ith ba ic calculu
like integral calculu an complex number . We hall give no rigorou proof of
mathematical tatement , rather e ant to provi e ome intuitive in ight intot
he e ence of the e tatement an their practical meaning. More mathematical e
tail relate to backgroun
an application of avelet are collecte in the
ap
pen ix. It i inten e to upport the rea er, if he or he feel that ome a it
ional information oul be helpful to un er tan the main bo y of the text. More
over,
the
appen ix contain olution to electe problem from the exerci e an
provi e a li t of frequently u e ymbol an notation .
2 Continuou Analy i
In chap. 1 e intro uce local tranform of continuou time ignal . The e tran
form no ill be tu ie in more etail. In particular the concept of pha e p
ace localization ill lea to the election of a proper in o function for the
STFT. Moreover, e ill ketch fa t algorithm an their implementation in MATLA
B for computing both the STFT an the continuou avelet tran form. For both tra
nform recon truction
formulae ill be
provi e . Finally t o in u trial ca e t
u ie are pre ente . The r t ca e tu y eal ith application of the avelet tr
an form an the STFT, re pectively, to the analy i of ignal occurring at a (l
ight) arc el ing proce . The econ ca e tu y e cribe , ho the above tran f
orm may be u e for cla ifying au io ignal
occurring at certain in pection p
roce ure in the automotive
in u try. Rea er primarily intere
te in applicatio
n may r t tu y the e nition part in ection 2.1.1 an 2.2.1, re pectively, the
n kip the re t of the e ection an procee irectly to ection 2.2.2,2.2.4
an 2.3. Throughout
thi ection the continuou time ignal to be analyze hall
be enote ith f (t). For completene e al o require f (t) to have nite energ
y (cf. ect. 5.1). A note there, e entially all practically relevant ignal
ful ll thi requirement.
When numerical algorithm an application enter the pict
ure, e ill not eal ith continuou time ignal any more. In tea , e hall c
on i er equence obtaine by ampling thecontinuou time ignal f (t) for t =
0, TS , 2TS , . . . , (N− 1)TS . Here TS enote the “ ampling i tance” an the equ
ence element are enote ith fk := f (kTS ) (k = 0, . . . , N −1). Note, that in
the engineering community al o f [k] i u e in tea of fk . Putting everything
together, e hall a here to the follo ing notation: f = {fk }N −1 = {f (kTS )}N −1
k=0 k=0 (2.1)
hall u e the ymbol f both to repre ent either a continuou time ignal or a
We
i cretely ample ignal. The actual meaning al ay hall be
14
2 Continuou Analy i
clear from the re pective
context.
Our notation concerning i cretely ample
ignal are complete ith the e nition for the ampling rate νS a d the correspod
ig circular frequecy ωS : 1 TS 2π ωS := TS νS :=
(2.2) (2.3)
2.1 The Short Time Fourier Trasform (STFT)
2.1.1 De itio, Computatio ad Recostructio De itio As described i sect. 1.2.
3, the STFT performs a local frequecy aalysis by shiftig a ido fuctio
to time t ad subsequetly computig the Fourier trasform (cf. eq. 5.1) of the
product of the sigal ad the ido :
+∞
ˆ f (ω, t) =
−∞
f (u) (u − t)e−jωu du
(2.4)
As idicated, this formula is also valid he the ido fuctio is complex val
ued. I this case, the overlie deotes complex cojugatio. The ˆ subscript i
the expressio f (ω, t) idicates the depedecy of the trasformed sigal o the
chose ido fuctio . Usig the Fourier trasform correspodece pairs itr
oduced i sect. 5.1, eq. 2.4 also may be ritte as ˆ f (t) (t − τ ) ◦ − • fw (ω, τ ) (2.5)
This equa ion is he key boh for designinga fas algorihm forcompu ing he S
TFT and for recons ruc ing he signal from he
STFT. For
he res of his sec io
n basic knowledge
of Fourier
ransforms and he discre
e Fourier ransform as pr
ovided in heappendix, sec ions 5.1 and 5.2, respec ively, is required.
Readers
primarily
in eres
edin applica ions may proceed from here o sec . 2.3.1 and
hen con inue wi h sec . 2.2.1.
Compu
a ion Given a sampled signal {f (kTS )}N −1 (
cf. eq.
2.1), he ask is o compu e a k=0 ˆ sampled version of he STFT, i.e. o
compu e he sequence {fw (ω, kTS )}N −1 . k=0
2.1 The Short Time Fourier Trasform (STFT)
15
It ill tur out, that a fast algorithm ca be give for suitably sampled ωvalues.
We describe o both the algorithm ad the restrictios uder hich it is valid
. f Assume f ad to be bad limited ad let ωmax ad ωmax deote the maximum fre
quecies of f ad , respectively (cf. sect. 5.2). Aalogously, letωmax de
ote th
e maximum frequecy of f (t) (t − τ ). Then, as implied already by he no a ion, ωmax
ideed does ot deped o τ and, moreover,
f w ωmax ≤ ωmax + ωmax .
(2.6)
Without goig ito details, e metio that this iequality essetially is a co
sequece from the fact that the spectra of f ad are covolved ith each other
. No de e ωS correspodig to eq. 2.3 ad assume that the samplig is such that th
e iequality
f ωS > 2(ωmax + ωmax ).
(2.7)
is ful lled. The eq. 2.6 implies that ωS > 2ωmax . Thus,
the
Sha o co ditio 5.12
i
s ful lled
for f (t) (t − τ ).
Correspondence
pair 2.5
oge her wi h heorem 5.1 hen
leads o he following ˆ Fas algori hm for he compu a ion of fw (ω, t): 1. De e the
frequecy samplig ωk = here mN =
N 2 N −1 2
k ωS (k = 0, . . . , mN ) N
(2.8)
(N eve) (N odd)
(2.9)
2. For τ = 0, TS , 2TS , . . . , (N − 1)TS perform he following s eps: a) De ne he s
equence {f (kTS
)w(kTS
− τ )}N −1 and compu e k=0 i s DFT via he FFT algori hm. Le
he DFT be deno ed wi h {Gk }N −1 . k=0 b) Then
ˆ fw (ωk , τ ) = TS Gk (k = 0, . . . , mN
). Some commen
s are in place: 1. Usually he sampled signal – in par icular he
sampling ra e ωS – is give. Thus, if the Shao coditio 5.12 is true for f , e
q. 2.7 implies that ωmax should be as small as possible i order to avoid distorti
os by aliasig e ects (cf. sect. 5.2). (2.10)
16
2 Cotiuous Aalysis
2. The remarks made after theorem 5.1 apply also here. Sice itely sampled siga
ls ever are bad limited i the strict sese, at least the spectra of f ad m
ust be “su cietly small” outside the respective itervals f f [−ωmax , ωmax ] ad [−ωm
ax ] such that eq. 2.10 is approximately true. Reco structio I this subsectio
e shall cosider the questio ho to recostruct the origial cotiuous time
ite e ergy sig al f (t) from the tra
sformed sig al ˆ f (ω, τ ). Again we s ar from
correspondence
pair 2.5. Applying he Fourier inversion formula 5.2 o 2.5 we ob
ain 1 f ( )w( − τ ) = 2π
+∞
ˆ fw (ω, τ )e+jωt dω
−∞
o both sides ith (t − τ ) and inegraing wih respec
Multiplyi g this equatio
o τ we ob ain for he lef hand side:
+∞ +∞ 2
f ( )|w( − τ )| dτ = f ( )
−∞ −∞
|w( − τ )|2 dτ
+∞
= f ( )
−∞
|w(u)|2 du
In proceeding
from
he rs o he second line he subs i u ion u = − τ has been mad
e. For he righ hand side we ob ain 1 2π
+∞ +∞
ˆ fw (ω, τ )w( − τ )e+jωt dωdτ
−∞ −∞
Equaingbo h sides and resolving for f ( ) we nally ob ain he “con inuous STFT re
cons ruc ion formula”
+∞ +∞
f ( ) =
−∞ −∞
ˆ fw (ω, τ )w( − τ )e+jωt dωdτ
+∞
(2.11) |w(u)|2 du
2π
−∞
Note that the denominator of this formula does only deend on the chosen window
function w. Thus it may be recomuted and stored. But even then
2.1 The Short Time Fourier Transform (STFT)
17
for ractical alications the usefulness of eq. 2.11 is limited. For samled si
gnals and the corresonding STFT the numerical aroximation of the double integ
ral in the numerator by some quadrature formula leads to an unaccetable comuta
tional e ort. Instead, assume that the samling {f (kTS )}N −1 (cf. eq. 2.1) of a co
ntinuousk=0 time nite energy signal f (t) is given, which ful lls eq. 2.7. Assume,
moreover,
that thealgorithm
described in eqs. 2.8 – 2.10 has been performed, resu
l ing in he compu a ion of 1 mN ˆ ωS ) f (ω, τ ) (τ = 0, TS , . . . , (N − 1)TS ; ω = 0,
. . . , N N ith mN give by eq. 2.9. The e use the correspodece pair eq. 2.
5 agai to recostruct {f (kTS
)}N −1 as follo s: k=0 Fast STFT recostructio:
k 1. For a give τ de ne wi h ωk = N ωS (k = 0, .. . , mN ) the seˆ quece Gk := T1 f (ωk
, τ )(k = 0, . . . , N − 1), where he coe S cien s Gk (k = mN + 1, . . . , N − 1) resul
from
GN −1 = G1
, GN −2 = G2 , . .. (cf. sec . 5.2). 2. Then apply he inverse DFT (eq.
5.14) o his sequence, resul ing in he sequence {f (kTS )w(kTS − τ )}N −1 . Schema i
cally: k=0 {Gk }N −1 −→ {f (kTS )w(kTS − τ )}N −1 k=0 k=0
IDF T
!
!
(2.12)
3. Perform he s eps above for a sui able subse {τi }I of τ values, i=1 such ha τ1
= 0, τI = (N −1)TS and w(kTS − τi ) is nonzero for any kTS in he in erval [τi , τi+1 ].
hen on hese in
ervals f (kTS ) may be recons ruc
ed by dividing he sequence ob
ained above
hrough w(kTS − τi ). We remark ha he signal can be recovered
by per
forming
s eps
1 and 2 above for every
τ value. S ep 3 aims a reducing
he compu
a ional e or , since he IDFT compu a ionneeds o be carried ou only
fora subse
of τ values. MATLAB implemen
a ions of he STFT algori hms presen ed in his sec
ion are discussed in sec . 2.1.3.
18
2 Con inuous Analysis
2.1.2
Phase Space
and
Localiza ion Parame ers In his sec ion weadop and deepe
n he viewpoin
on he STFT described
in sec . 1.2.3.
The concep of localiza
io
n parameers in roduced in sec . 5.1
will lead us o he choice of a cer ain win
dow func ion w. Thus rs we rewri e he STFT formula 2.4 as
+∞
ˆ fw (ω, t) =
−∞
f (u)g(ω,t) (u) du
(2.13)
here g(ω,t) (u) = ejuω (u − t) (2.14)
(cf. sect. 1.2.3). Assume that the ido fuctio has the follo ig localizatio
parameters (for the correspodig de itios cf. eqs. 5.6 – 5.9): t = 0, ω = 0, ∆t
, ∆ω . The it is a easy exercise to verify that the localizatio parameters of g(ω,
t) read tg(ω,t) = t, ωg(ω,t) = ω, ∆tg(ω,t) = ∆t , ∆ω g(ω,t) = ∆ω . The correspodig cell
e plae (cf. sect. 5.1) are dra i Fig. 2.1.
t
∆ω
ω ∆t
?
Fig. 2.1. Cells i phase plae, occupied by (solid) ad g(ω,t) (dashed) , respec
tively.
Thus, the trasitio = g(0,0) → g(ω,t) correspods to a pure shift of the cell i
the phase pla e. Shape ad size remai uchaged! Obviously, a optimal resoluti
o both i time ad frequecy ca be reached if ∆t ad ∆ω both are as small as poss
ible. Havig the fast STFTalgorithm 2.8 – 2.10 i mid, e remark that coditio 2
.6 is aother strog argumet for havig a as small as possible ∆ω value.
2.1 The Short Time Fourier Trasform (STFT)
19
Because of the Heiseberg ucertaity priciple 5.10 ∆t ad ∆ω caot both get arbi
trarily small. Therefore the optimal choice ith respect to time a d frequecy r
esolutio is the Gauss fuctio g give i 5.11. It miimizes the product ∆t ∆ω . T
he STFT ith the correspodig ido
t2 1 (t) = √ e− 2 . 4 π
(2.15)
is also called “Gabor transform”. 2.1.3 Imlementation with MATLAB and Visualization
For this section we assume the reader to have basic familiarity with MATLAB. In
articular, knowledge of matrix and vector maniulations and functions like fft
(f) (comutation of the DFT of the sequence f) and ifft(F) (comutation of the I
DFT of the sequence F) is required. We turn now to a short descri tion of a MATL
AB im lementation ofthe STFT algorithm
2.8– 2.10. Subsequen ly, he implemen a i
on of he recons ruc ion algori hm leading
o 2.12 is discussed. The correspondi
ng MATLAB m les have been es ed wi h MATLAB 6.5,release
13 and may be downloade
d from he URL given
in sec
. 1.3. Fas
STFT Compu a ion The algori hm is implem
en ed in he func ion mys f . A pro o ype call
of his func
ion reads [ ,y,ma ri
x]=mys f (signal,T,s
ep ,ommin,sepom,ommax, ex ); We rs give a shor explana io
n of heinpu parame ers. The s ring ex allowsfor a commen o be included i
n he i le of he graphics genera edby he func ion. The sampled signal is rep
resen ed by he vec or signal, T
deno es he sampling dis
ance, ommin, s epom, o
mmax are minimum index, s epwid h and maximum index in he frequency sampling 2.
8, for which he ˆ desired fw (ω, t) values are computed. Thus, i particular, the u
ser must take care of the restrictio ommax ≤ mN ith mN de ed by 2.9. N deotes th
e sigal legth. I order to reduce the computig e ort, stept as itroduced. For
, e.g., stept=3, steps 2a) ad 2b) of the algorithm are performed oly for every
3rd value of the time vector ˆ t=(0:(N 1))*T. The f (ω, t) values for itermediate
times are obtaied by replicatig the last computed values. ˆ As a result of the
algorithm the computed f (ωk , ti ) values are stored i the matrix matrix, ith
k umberig ro s ad i umberig colums. This
20
2 Cotiuous Aalysis
matrix is visualized as, e.g., i Fig. 1.6 usig the imagesc fuctio. Together
ith the sigal y ad the time vector t it is retured by the fuctio. The basi
c computatio ad replicatio code is listed belo : %Loop for icreasig time fo
r idex=1:stept:N, y=feval( ido ,(t t(idex))/sigma).*sigal; colum=coj((fft(
y))’); colum=colum(ommi:stepom:ommax); ifill=mi(stept,N idex+1); for i=1:ifil
l, matrix(:,idex+i 1)=colum; ed ed The ido fuctio must be provided as a
fuctio le, its ame is stored i the strig ido , sigma deotes a additioa
l user cotrol of the ido idth. Fast STFT Recostructio The correspodig f
uctio reads myistft. A prototype call: [t,reco,matrix]=myistft(matrix,T,stept)
; Here matrix deotes the result of a STFT computatio ith mystft. Sice for t
he recostructio algorithm the complete frequecy samplig 2.8 is required, i
the precedig mystft call oly the sigal, the samplig distace ad the time i
cremet should be speci ed, sice the complete frequecy samplig the is computed
by default. Moreover, for the STFT computatio ad the recostructio obviously
the same ido fuctio ad correspodig idth must be used, mystft ad myist
ft request these etities from the user. stept deotes the legth of the recost
ructio itervals i multiples of the samplig time T (cf. step 3 of the recost
ructio algorithm). It is recommeded to choose it idetical ith the time step
parameter i the precedig mystftcall, provided the shifted ido values do ot
vaish o this iterval (cf. step 3 of the recostructio algorithm). Thus a ty
pical dialogue is as follo s: >> [t,y,matrix]=mystft(sigal,.017,10); Eter id
o (i quotes): ’gauss’ Wido idth: 1 >> [t,recostft,matrix]=myistft(matrix,.017,1
0); Eter ido (i quotes): ’gauss’ Wido idth: 1 >> For more implemetatio det
ails the reader is referred to the source le.
2.2 The Cotiuous Wavelet Trasform (CWT)
21
2.2 The Cotiuous Wavelet Trasform (CWT)
2.2.1 De itio, Computatio ad Recostructio De itio The (cotiuous) avelet t
rasform has bee listed already i sect. 1.2.4, formula 1.5. We shall o give
the precise formulatio usig the symbol ψ for the wavelet in accordance with the
general use (cf., e.g., [7]). Choose a nite energ function ψ(t) ful lling the “admissi
bilit condition”
∞
cψ := 2π
−∞
ˆ |ψ(ω)|2 dω < ∞. |ω|
(2.16)
Any ni eenergy
func ion sa isfying
2.16 will
be called
a “wavele
”. Then he “con inuo
us wavele ransform” (CWT) of he signal f ( ) is deno ed wi h Lψ f (a, t) and read
s
∞
1 Lψ f (a, t) = √ cψ
1 |a|
ψ
−∞
u−t a
f (u) du (a = 0), (t ∈ R).
(2.17)
Again, the overline denotes comlex conjugation if ψ(t) is comlex valued. Some re
marks might be aroriate here: 1. When comaring formula 2.17 with formula 1.5
, note that in the latter formula for simlicit reasons the constant factor √1 ψ ,
related to the c admissibilit condition 2.16, has been omitted. 2. For ractica
ll relevant wavelets admissibilit condition 2.16 is ful lled, when
+∞
ψ(t) dt = 0.
−∞
(2.18)
Thus, as stated in sect. 1.2.4, ψ will oscillate around the t axis, since the cont
ributions of ositive and negative function values to the total area, bounded b
the function grah and the t axis, must cancel each other. Since, moreover, ψ(t)
is of nite energ, for t → ±∞ the function ψ(t) will decrease raidl. Both facts taken t
ogether exlain the term “wavelet” for the function ψ(t). The “Haar wavelet” has alread b
een mentioned in sect. 1.2.4. In sect. 2.2.2 it once more will be treated togeth
er with further wavelet examles.
22
2 Continuous Analsis
3. As exlained in sect. 1.2.4, the CWT 2.17 is a kind of multiresolution anals
is, since Lψ f (a, t) rovides information about signal details of size ≈ a. Corres
ondingl, a will be called “detail size” or “scale factor”. As noted in 1.2.4, scale fac
tors and frequencies are inversel roortional to each other. Since the roort
ionalit constant in eq. 1.4 deends on the wavelet, the correct relation reads βψ .
(2.19) ω Readers maily iterested i applicatios may skip the rest of this sect
io ad proceed from here to sect. 2.2.2 ad the to sect. 2.2.4. The rest of th
is sectio ad sect. 2.2.3 require basic ko ledge of Fourier trasforms ad the
discrete Fourier trasform (cf. appedix, sectios 5.1 ad 5.2, respectively).
Equatio 2.17 may be re ritte as a covolutio: With a= ψa (t) := ψ we obtain 1 Lψ f
(a, t) = √ cψ 1 |a|
∞
− a
(2.20)
ψa (t − u)f (u) du.
−∞
Then from the convolution theorem (eq. 5.5) we ma conclude that 1 Lψ f (a, t) ◦ − • √ cψ 1
|a| ˆ ˆ ψa (ω)f (ω). (2.21)
Similarly to eq. 2.5 i the STFT case, eq. 2.21 is the key both for a fast CWT c
omputatio algorithm ad for recostructig the sigal from the CWT. Computatio
Give a sampled sigal {f (kTS )}N −1 (cf. eq. 2.1), e ill describe o a k=0
algorithm to compute a sampled versio of the CWT. As a result, e ill compute
the sequece {Lψ f (a, kTS )}N −1 for a = 0. We describe now both k=0 the algorithm
and the restrictions under which it is valid. f ψ ψa Assume f and ψ to be band limited
and let ωmax , ωmax ad ωmax deote the maximum frequecies of f , ψ and ψa , resectivel
(cf. sect. 5.2). Analogousl, let ωmax deote the maximum frequecy of Lψ f (a, t)
. Then it is not hard to show that
ψa ωmax =
1ψ ω . |a| max
2.2 The Con inuous Wavele Transform (CWT)
23
Now assume ha he sampling is such ha he Shannon condi ion 5.12 is valid
bo
h for f ( )and ψa (t), i.e. 1ψ ω . (2.22) |a| max Then he Shannon condi ion au oma
ically is sa is ed for Lψ f (a, t):
f ωS > 2 max ωmax ,
ωS > 2ωmax.
(2.23)
This is true, sice by 2.21 the spectrum of Lψ f (a, t) will vanish if the sectru
m of ψa (t) or the sectrum of f (t) vanishes. Therefore
f ωmax = mi ωmax ,
1ψ ω |a| max
and 2.22 hen implies 2.23. Thus, if 2.22 is valid, he spec ra of f ( ), ψa (t) a
nd Lψ f (a, t) can be com uted with the DFT as described in theorem 5.1, eq. 5.16.
This leads to the following Fast algorithm for the comutation of Lψ f (a, t) (a
= 0): 1. De ne the sequence {ψa (kTS )}N −1 , where ψa is de ned in eq. k=0 2.20. 2. Comu
te the DFT of this sequence resulting in a sequence {Ak }N −1 . Moreover, comute
the DFT {Fk }N −1 of k=0 k=0 {f (kTS )}N −1 . k=0 3. Al the IDFT to the roduct s
equence √ 1 {Fk Ak }N −1 to k=0 obtain {Lψ f (a, kTS )}N −1 . Schematicall: k=0
cψ |a|
1 IDF T {Fk Ak }N −1 −→ {Lψ f (a, kTS )}N −1 . k=0 k=0 cψ |a| We conclude his sec ion wi h
some remarks:
(2.24)
1.
The algori
hm can be performed
for
any scale fac or a = 0. Prac ically releva
n are posi ive scale fac ors; for he sampling considered here, one usually ak
es a = TS , 2TS , . . . , (N − 1)TS . 2. Again,
usually he sampling
dis ance TS i
s given
a priori. Thus for small scale fac ors one mus
be aware ha
eq. i
2.22
s no valid any more. Therefore
he algori hm inevi
ably will lead o or ion
dis
s for a → 0. 3. In analogy o he STFT algori hm, he remarks made afer heorem 5
.1 apply also here. Since ni ely sampled signals
never are band limi ed
in he s
ric sense,
again “maximum
frequencies”
mus be unders ood such f f ha he respec
ive spec ra are su cien ly small ou side [−ωmax , ωmax ] ψ ψ and [−ωmax , ωmax ], respectiv
24
2 Cotiuous Aalysis
Recostructio I this subsectio e shall cosider the questio ho to recostr
uct the origial cotiuous time ite eergy sigal f (t) from the CWT Lψ f (a, t).
Similarl to the STFT case we start from the corresondence air eq. 2.21. An e
as calculation shows how the Fourier transform of ψa (t) ma be exressed in term
s of the Fourier transform of ψ. Inserting the result in the corresondence air 2
.21 one obtains Lψ f (a, t) ◦ − • |a| ˆ ˆ ψ(ωa)f (ω). cψ (2.25)
Recall (cf. sect. 5.1) that this relation is onl a shorthand notation for
+∞
Lψ f (a, t)e−jωt dt =
−∞
|a| ˆ ˆ ψ(ωa)f (ω). cψ
(2.26)
The detailed calculations in the following rocedure are left to the reader. We
shall onl write down the basic stes and the ideas behind them: 1. Multil bot
h sides of eq. 2.26 with ˆ ψ(ωa) . 3√ |a| 2 cψ and integrate with resect to a. Recalling
de nition 2.16 of cψ , the urose ˆ of this rocedure is to eliminate all terms deen
ding on ψ on the righthand side. After the multilication and integration rocedur
e, the new right hand side will read 1ˆ f (ω). 2π Equating this with the corresonding
new left hand side, we obtain a ˆ formula which comutes f (ω) from the CWT Lψ f (t,
a). 2. Aling now the Fourier inversion formula 5.2 to both sides we end u wi
th the desired reconstruction formula. It reads 2π f (t) = √ cψ
∞ ∞
Lψ f (a, u)
−∞ −∞
1 ψ |a|
−u a
duda a2
(2.27)
Wi
h he same reasoning
as for
he STFT recons rucion formula
2.11 we no e ha
he above reconsruc ion in egral is only of limi ed prac ical relevance if sam
pled signals and he corresponding CWT are given.
2.2 The Con inuous Wavele Transform (CWT)
25
Insead inanalogy o he STFTprocedure one may ob
ain a recons ruc ion alg
ori hm essen ially
by reversing he s eps of he compu a ion algori hm given abo
ve. Assume ha he sampling {f (kTS )}N −1
(cf. eq. 2.1) of a con inuous ime k=0
ni e
energy signal
f ( ) is given such ha eq. 2.22 is ful lled.
Assume, moreover
, ha he algori hm described above has been performed. Thus af er evalua ing e
q. 2.24, he sequence {Lψ f (a, kTS )}N −1 . k=0 is given. Then we use the correson
dence air eq. 2.21 again to reconstruct {f (kTS )}N −1 as follows: k=0 Fast CWT r
econstruction: 1. Comute the sequence {ψa (kTS )}N −1 , where ψa is de ned in k=0 eq. 2
.20 and comute the corresonding DFT denoted with {Ak }N −1 . k=0 2. Comute the
DFT of the sequence {Lψ f (a, kTS )}N −1 . Let us k=0 denote it with {Bk }N −1 . k=0 3
. Comute the sequence {Fk }N −1 where k=0 Fk = cψ |a| Bk . Ak (2.28)
4. Then apply he inverse DFT (eq. 5.14) o his sequence, resul ing in he sequ
ence {f (kTS )}N −1 . Schema ically: k=0 {Fk }N −1 −→ {f (kTS )}N −1 k=0 k=0
IDF T
(2.29)
Noe ha eq. 2.28 requires Ak o be nonzero for all k. If his is no he case,
s eps 1 – 3 mus be repea ed forsui able a values, such ha {Fk }N −1 k=0 comple
ely may be recovered before compu ing 2.29. Never heless,
if he scale fac or a
is such ha Ak= 0 (k = 0, . . . , N −1), he algori hm described above leads o
a recovery of he sampled
signal
from
hecorresponding
CWT from a single
a val
ue! This is an indica ion for he fac ha he informa ion s ored in he Lψ f (a,
t) coe cients is highl redundant. We will reduce this redundanc later (cf. cha
. 3). MATLAB imlementations of the CWT algorithms resented in this section are
discussed in sect. 2.2.3.
26
2 Continuous Analsis
2.2.2 Wavelet Examles As mentioned in sect. 2.2.1, most racticall relevant wa
velets satisf the admissibilit condition 2.16 if relation 2.18 is ful lled. Gene
ralizing this relation we shall construct “wavelets ψ(t) with M vanishing moments” (cf
. eq. 2.31). The ractical imlications of condition 2.31 will be illustrated be
low. Construction rincile Let Φ(t) denote a piecewise smooth function being M -t
imes di erentiable and satisfying some additional slight technical restrictions. D
e ne the wavelet ψ(t) b dM Φ (t). dtM Then one may show that ψ(t) has M vanishing momen
ts, i.e. ψ(t) :=
+∞
(2.30)
ψ(t)tm dt = 0 (m = 0, 1, . . . , M − 1).
−∞
(2.31)
The interested reader is encouraged to rove this in the exercises. M = 1: The H
aar wavelet Consider ⎧ 0≤t< 1 ⎨t 2 ΦH (t) = 1 − t 1 ≤ t < 1 2 ⎩ 0 else
The corresonding wavelet is called “Haar wavelet” and reads ⎧ ⎨1 0 ≤ t < 1 2 dΦH (t) = −1
< 1 ψH (t) := 2 ⎩ dt 0 else Condition 2.31 for the Haar wavelet reads
+∞
(2.32)
ψH (t) dt = 0.
−∞
2.2 The Continuous Wavelet Transform (CWT)
27
M = 2: The Mexican hat wavelet Here
1 t2 2 ΦMH (t) = − √ π − 4 e− 2 3
The corresonding wavelet is called “Mexican hat wavelet” and reads ψMH (t) :=
1 t2 d2 ΦMH 2 (t) = √ π − 4 (1 − t2 )e− 2 2 dt 3
(2.33)
Condition 2.31 for the Mexican hat wavelet reads
+∞ +∞
ψMH (t) dt =
−∞ −∞
ψMH (t)t dt = 0.
Plots Both wavelets treated so far are shown in Fig. 2.2. The wavelets are lott
ed as solid lines, the resective Φ-functions are dashed.
1.5 1 0.5 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 2 ψH(t) Φ (t) H
1 ψ 0.5 0 −0.5 −1 −4
MH
(t)
ΦMH(t) −3 −2 −1 0 1 2 3 4
Fig. 2.2. Haar wavelet and Mexican hat wavelet (below).
Practical imlications of the vanishing moments condition When for a given scale
a and a given time t the CWT
28
2 Continuous Analsis
1 Lψ f (a, t) = √ cψ
1 |a|
∞
ψ
−∞
u−t a
f (u) du
is comuted, onl those u values will contribute, where ψ u−t is nonzero. a Assume t
hat these values constitute an interval I(a,t) . Now assume that ψ(t) has M vanish
ing moments and the signal f (u) on I(a,t) ma be modeled b a olnomial of deg
ree k with k < M . Then one ma show that Lψ f (a, t) = 0. Again, the interested r
eader ma work on the roof in the exercises. This result can be interreted as
follows: Assume that ever “smooth” art of the signal ma be reresented b a certa
in olnomial and signal changes are modeled b switching the olnomial to a ne
w one. In other words: The signal is modelled b iecewise olnomial functions.
Then signal changes will be well localized b the CWT rovided the wavelet ψ(t) h
as enough vanishing moments. The reason is that in the smooth arts the CWT will
vanish if the number of vanishing moments is large enough and the CWT will be c
oncentrated around the time values, where signal changes (i.e. changes in the o
lnomials) occur. This is illustrated in Fig. 2.3. Here the signal consists of t
wo di erent straight lines (i.e. olnomials of degree k = 1) connected together a
t t = 5000. Recall (see above) that the Mexican hat wavelet has 2 vanishing mome
nts, whereas the Haar wavelet onl has 1 vanishing moment. Thus, one exects tha
t resective CWT should be better concentrated around the connection time for th
e Mexican hat wavelet than for the Haar wavelet. This is clearl visible in Fig.
2.3. Note also that the Haar wavelet leads to nonzero CWT values in the “smooth” si
gnal arts in contrast to the Mexican hatwavelet. The Morlet wavelet We close th
is section with the “Morlet wavelet” which was among the rst wavelets to be exlored
for signal analsis uroses [11]. It is comlex valued and reads ψM (t) = π − 4 (e−jαt − e
1 α2 2
)e− 2 (α = π
t2
2 ). ln 2
(2.34)
Since the corresonding w velet tr nsform LψM f ( , t) will be comlexv lued fun
ction, it m be decomosed into bsolute v lue nd h se corresonding to LψM f (
, t) = |LψM f ( , t)|ejΦM ( ,t) . (2.35)
or ossible lic tions of the h se function ΦM ( , t) the re der is referred t
o sect. 2.2.4.
2.2 The Continuous W velet Tr nsform (CWT)
4 2 0 Sign l
29
1000
2000
3000
4000 5000 6000 CWT with H r−W velet
7000
8000
50 100 150 200 1000 2000 3000 4000 5000 6000 CWT with Mexic n−h t−W velet 7000 8000
50 100 150 200 1000 2000 3000 4000 5000 6000 7000 8000
ig. 2.3. To: Sign l. Middle: CWT with the H r w velet. Bottom: CWT with the M
exic n h t w velet.
2.2.3 Imlement tion with MATLAB nd Visu liz tion At the beginning of sect. 2.1
.3 some relimin r st tements concerning MATLAB h ve been m de. The l lso
to this section nd will not be ree ted here. In this section we sh ll describ
e MATLAB imlement tions both of the CWT lgorithm le ding to 2.24 nd of the co
rresonding reconstruction lgorithm le ding to 2.29. Ag in, the resective MATL
AB m les h ve been tested with
MATLAB 6.5, rele se 13 nd m be downlo ded from
the URL given in sect. 1.3. st CWT comut tion The corresonding function le re
ds mcwt. A s mle function c ll is shown in the next line: [t,,m trix]=mcwt(
sign l,T,st rt,ste,sto,comment); An logousl to mstft the vector sign l denot
es the s mled sign l, T the s mling dist nce, the string comment is included i
n the title of the gr hics gener ted b the function. The r meters st rt,ste
nd sto denote minimum index, increment nd m ximum index of the sc le f ctors
for which the CWT is comuted, me sured in multiles of T. Thus in MATLAB not t
ion the resulting sc le v lues re given b the vector (st rt:ste:sto)*T. As
result of the lgorithm the comuted Lψ f ( i , kTS ) v lues re stored in the m
trix m trix, with k numbering columns nd i numbering rows. The i v lues re t
he entries of the bove vector cont ining the sc le v lues. This
30
2 Continuous An lsis
m trix g in is visu lized s, e.g., in ig. 1.6 using the im gesc function. Sin
ce for com lex v lued w velets the h se lot of this m trix might be of interes
t (cf. sect. 2.2.4), the function llows both for visu liz tion of the bsolut
e v lue of m trix nd of its h se. The
b sic com ut tion code is listed below %
Initi liz tions of sc le f ctors nd ourier tr nsform =(st rt:ste:sto)*T; ro
ws=length( ); h t=fft(); %M trix Initi liz tion m trix=zeros(ze,n); %Loo for
incre sing sc le f ctors for i=1:rows, si_sc le=conj(fev l(w v, t./ (i))); si_
sc le_h t=fft(si_sc le); %Time
tr nsl tion such th t minim l time=0 tr ns=ex((
j*t(1)*(0:(N 1))*oms/N)); % ourier tr nsform of w velet tr nsform; conv_h t=((
h t.*si_sc le_h t).*tr ns)/sqrt( (i)); m trix(i,:)=ifft(conv_h t); end The w ve
let must be rovided s function le, whose n me is stored in the string w v. Th
e code should be f irl self exl n tor for re der used to MATLABnot tion nd
h ving worked through sect. 2.2.1. A comment on the time tr nsl tion vector tr
ns should be dded, however. As mentioned in sect. 2.2.1 the CWT will be comute
d for the s mled time v lues 0, TS , 2TS , . . . , (N − 1)TS . Thus, in rticul
r, ll time v lues will be ≥ 0. On the other h nd, the comut tion of ψ (t) := ψ −t
is required. This imlies th t for w velets, which re zero for t < 0 ψ (t) will
v nish for t > 0, thus in rticul r for the bove time s mling. The H r w vel
et 2.32 is simle ex mle for such w velet. Therefore in mcwt the time vect
or t is designed such th t t = 0 corresonds to the middle of the sign l. Hence,
the resulting CWT must be shifted with resect to the time dom
in such th t ft
er te shift the sm llest t v lue corresonds to t = 0. In the ourier dom in thi
s time shift m be re lized b multiic tion with
2.2 The Continuous W velet Tr nsform (CWT)
31
h se f ctors, which re collected in the vector tr ns. In the de nition of this v
ector oms denotes the s mling circul r frequenc. st CWT reconstruction The r
econstruction lgorithm is im lemented in the function le micwt. A rotote c l
l to this function re ds [t,reco,m trix]=micwt(m trix,T,st rt,ste,sto); m tri
x is the result from receding c ll of mcwt, T g in denotes the s mling dis
t nce. As with the mcwt function from the resective inut r meters the sc le
vector =(st rt:ste:sto)*T is constructed. The entries of these vectors re s
c nned in order to select
comonent i such th t the sequence {Ak }N −1 resultin
g from l ing the D T to the sequence k=0 {ψ i (kTS )}N −1 is nonzero for ll k (c
f. comut tion ste 2.28 in the reconk=0 struction lgorithm). If such n i doe
s not exist, the rogr m will not work roerl! It is recommended to choose the
sc le rel ted inut r meters st rt, ste nd sto identic l to the resective
r meters in the receding mcwtc ll. Thus, tic l di logue re ds: >> [t,,
m trix]=mcwt(,.01,1,1,200,’’); Enter w velet (in quotes): ’mex_h t’ >> [t,reco,m trix]
=micwt(m trix,.01,1,1,200); Enter w velet (in quotes): ’mex_h t’ Note th t obviousl
both in the mcwt nd the micwt c ll the s me w velet must be selected! In b
oth functions the n me of the w velet function le is requested from the user. The
reconstructed sign l is stored in the vector reco. A note on the MATLAB W velet
Toolbox The MATLAB W velet Toolbox [24] rovides the cwt function in order to
erform CWT. Below tic l function c ll is shown: m trix=cwt(,1:1:200,’mexh’);
The me ning of the resective r meters should be cle r from the discussion bo
ve. Note th t the st rt , ste nd sto r meters from bove must be merged to
n inut vector of the kind st rt:ste:sto. Note lso th t the chosen w velet
must be seci ed b corresonding inut string, in the bove ex mle ’mexh’ selects
the Mexic n h t w velet 2.33. As with ll other fe tures of the MATLAB W velet T
oolbox the CWT lso m be invoked from gr hic l user interf ce which is l un
ched
b the w vemenu
comm nd. The other loc ltr nsforms tre ted in this ch ter
(ST T, inverse ST T, inverse CWT) re not su orted b the MATLAB W velet Toolb
ox.
32
2 Continuous An lsis
2.2.4 Alic tion: Detection of Sign l Ch nges In sect. 2.2.2 we modeled sign ls
b iecewise olnomi l functions. “Smooth” rts of the sign l corresonded to tim
e interv ls, where the sign l w s reresented b xed olnomi l. The time v lue
s where two di erent olnomi ls re connected corresonded to “sign l ch nges”. In th
is section we will consider more subtle ex mle nd discuss CWT exeriments wi
th v ring w velets. The model sign l re ds f (t) = cos(t − π) t < π cos(2t − 2π) t ≥ π
Obviousl, the rst deriv tive df (t) is continuous, where s the second dt deriv t
ive is discontinuous t t = π. Thus, we h ve kind of “hidden” sign l ch nge occurrin
g not in the sign l itself but in some higher deriv tive. In the rst ex
eriment w
e comuted the CWT of this sign l with the comlex Morlet w velet 2.34. igure 2
.4 visu lizes the h se ΦM ( , t) of LψM f ( , t) (cf. eq. 2.35).
Testsign l. Sc le r meters: =1,δa=1,a top=100. tart
0.5 0 −0.5 −1 1
1.5
2
2.5
3
3.5
4
0.02 0.04 a 0.06 0.08 0.1 1 1.5 2 2.5 t 3 3.5 4
Fig. 2.4. CWT of the te t ignal ith the Morlet avelet 2.34, Pha e plot.
The time here the ignal change occur i localize very clearly by traight li
ne converging to t = π. This qu lit tive beh vior of the h se reresent tion (co
nvergence to oints of sign l ch nges) h s been reorted lre d ver e rl (cf.
the rticle of A. Grossm nn, R. Kronl nd M rtinet nd J. Morlet in [6]). In ch
. 3 we will introduce the f mous f mil of D ubechies w velets [7].
or more
det iled tre tment we refer to this ch ter. Here we onl rem rk
2.3 C se Studies
33
th t the members of this f mil re numbered with dbn (n = 1, 2, 3, . . .). An i
mort nt sect for sign l
n lsis uroses is th t dbn w velets h ve n v nishi
ng moments (cf. eq. 2.31)! igure 2.5 shows the w velet tr nsform of the bove s
ign l with the db2w velet for three incre sing sc le f ctors (cf. gure c tion; T
S denotes the s mling dist nce).
db2−w velet 0.5 f 0 −0.5 1 1.5 2 2.5 3 3.5 4
x 10 1 0 −1 x 10 5 0 −5 −10 d
2
−6
d
1
−6
x 10 4 2 0 −2 −4 1
−5
d
3
1.5
2
2.5
3
3.5
4
ig. 2.5. To: Sign l. Below: W velet tr nsforms for = TS , = 2TS , = 4TS
(from bove). W velet: db2.
igure 2.6 shows the s me results using the db4 w velet. The gures con rm the qu li
t tive beh vior mentioned lre d in sect. 2.2.2: With incre sing number of v ni
shing moments the resective w velet tr nsforms tend to be better loc lized rou
nd signi c nt sign l ch nges. Corresondingl, the v nishing in “smooth” sign l region
s is more ronounced for w velets with l rge number of v nishing moments. Rel
ted to this is the f ct th t the dbn w velets tend to be “smoother” with incre sing
n. We will not stress these toics here; for n e s to re d summ r the re der
is referred to [24].
2.3 C se Studies
2.3.1 An lsis of Sensor Sign ls All sign ls discussed in this section were reco
rded in the sensor of robot, c rring out (light) rc welding rocess. The
rc itself is used s sensor me suring the dist nce from the welding tool to th
e workiece.
34
2 Continuous An lsis
db4−w velet 0.5 f 0 −0.5 1 1.5 2 2.5 3 3.5 4
x 10 2 0 −2 x 10 5 0 −5 x 10 2
3
−7
d
1
−7
d
2
−6
d
0 −2 1 1.5 2 2.5 3 3.5 4
ig. 2.6. To: Sign l. Below: W velet tr nsforms for = TS , = 2TS , = 4TS
(from bove). W velet: db4.
The results described in the following r gr hs h ve been obt ined with the MA
functions mstft nd mcwt (cf. sections 2.1.3 nd 2.2.3,
TLAB
resectivel). ST
T lic tion The rst ex m le to be resented is the ST T n lsis of such sig
n l. Here the welding robot erforms n oscill tor motion which is tr nsvers l
to the welding se m. Without going into det ils of the technolog we mention th
t the frequenc of this oscill tion t t ≈ 3 sec should get doubled from ν ≈ 3 Hz to ν ≈ 6
Hz. The sesor sigal is sho i the upper part of Fig. 2.7. It obviously is h
eavily corrupted ith oise. The lo er part of Fig. 2.7 sho s the absolute value
s of the respective STFT sigal. The STFT is coded ith gray values as described
i sect. 1.2.5. To simplify evaluatio, the STFT as represeted o the t ν plae
i cotrast to Fig. 1.6, here it as represeted o the t ω plae. Obviously, th
e frequecy doublig metioed above clearly is localized ith the STFT. Moreove
r, the STFT sho s ho the corruptio ith oise leads to additioal high freque
cy compoets i the sigal. The ido fuctio used for the STFT computatio
as the Gaussia ido 2.15. Thus, actually the Gabor trasform of the sigal a
s computed. CWT applicatios Before discussig further examples of sesor sigal
s occurrig at a arc eldig process, e remark that e ill aalyze sigals st
emmig from three di er
2.3 Case Studies
Sesor Sigal. Parameter: δx=5,kmin=20,δk=1,kmax=200,σ=1.0 2 1.5 1 0.5 0 −0.5 −1 0 2 4 6 8
10 12 14
35
2 4 6 ν 8 10 12 0 2 4 6 t 8 10 12 14
Fig. 2.7. STFT example: Sigal (above) ad STFT (belo ). Here the Gabor ido (e
q. 5.11) has bee used.
et techologies, the details of hich caot be explaied here. We oly ote, t
hat they are deoted ith the abbreviatios “MSG”, “MAG” ad “WIG”, respectively. Apart fro
their di ereces they share a commo property: usually the sesor performs a osc
illatory motio trasversally to the eldig seam. For test purposes this oscill
atio may be s itched o (third sigal i Fig. 2.9). All time ad scale values i
the plots belo are measured i secods. The result of the CWT aalysis of the
MAG ad MSG sigals, respectively, is depicted i Fig. 2.8. As expected, it sho
s bright stripes at scale factors associated to the avelegths of the respecti
ve oscillatios. This remais true also for the three WIG sigals; the results o
f their respective CWT aalysis are sho i Fig. 2.9. Note i particular the la
ck of the correspodig stripe for the third sigal, here the oscillatio as s
itched o . Visual ispectio aloe does ot reveal oticeable di ereces bet ee th
e various techologies. Yet CWT aalysis clearly idicates a di erece: WIGsigals
sho bright peaks at scale a ≈ 6 sec. Ideed, a posteriori oe might isolate corr
espodig “peak groups” he ispectig the sigals from Fig. 2.9. This idicates th
at there exists a correspodig correlatio i WIG sigals i cotrast to MSG a
d MAG sigals. Note that this correlatio is preset, eve if the oscillatio i
s s itched o (cf. third sigal i Fig. 2.9). This example illustrates that the mu
ltiscale ivestigatio of a sigal, performed by the CWT, may o er valuable iform
atio cocerig relevat parameters of the respective process behid the sigal
.
36
2 Cotiuous Aalysis
MAG−sigal. Scale parameters: astart=2,δa=2,a top=500. 1 0.5 0 −0.5 −1 −1.5 0
5
10
15
20
25
30
1 2 a[ ec] 3 4 5 6 7 0 5 10 15 t[ ec]
tart
20 =2,δ =2,a
a top
25 =500.
30
MSG− ignal. Scale parameter : a 2 1.5 1 0.5 0 −0.5 −1 0 2 4 6
8
10
1 2 a[ ec] 3 4 5 6 7 0 2 4 6 t[ ec] 8 10
Fig. 2.8. Above: CWT of a MAG ignal. Belo : CWT of a MSG ignal.
A a nal remark e note that the avelet u e in the tran form vi ualize in Fig
. 2.8 – 2.9 a the Morlet avelet 2.34. 2.3.2 Analy i an Cla i cationof Au io
Signal In thi ection e ecribe the analy i of acou tic ignal recor e u
ring certain in pection proce ure in the automotive in u try1 . In ome applica
1
The material pre ente in thi ub ection re ult from a cooperation
ith AUDI A
G, a major German car manufacturer. It i publi he ith kin permi ion.
2.3 Ca e Stu ie
37
Fig. 2.9. CWT of three WIG ignal .
38
2 Continuou Analy i
tion uch a ignal may inicate the pre ence or ab ence of a efect; thu in th
i context it i u e for iagno i purpo e . An
example
i e cribe in [18]. I
n other ca e the recor e ignal imply houl oun “plea ant” in or er to ful l com
fort requirement
. Both ca e have in common that a humanli tener can i crimin
ate “goo ” from “ba ” ignal . In thepre ent ection e e cribe the r t tep to ar a
trainable cla i cation cheme, ba e on a local analy i of uch ignal . Thi c
la
i cation proce ure houl in principle be able to perform
the above mentione
i crimination of a human li tener automatically.
In or er to compare the re ult
of both
technique e u e the CWT an the STFT.
We ketch
no
the cla i cation
proce ure. Fir t the local tran form i ample on
a gri of i crete value bo
th ith re pect to time an cale (CWT) or time an frequency (STFT). For the CW
T the ampling
i illu
trate in Fig. 2.10; in the upper half a typical acou tic
ignal i i playe .
Acou tic Signal. Scale parameter : a 0.1 0.05 0 −0.05 =1,δ =10,a
a
tart
top
=1000.
0 t1
0.5 t2
1 t
3
1.5
0 0.05 0.1 0.15 0.2
a1 a2 a3 a4
a
0
0.5 t
1
1.5
Fig. 2.10. CWT of an acou tic ignal. Sampling for 3 time an 4 cale value .
In the next tep from the ample tran form value a pattern vector i erive
hich erve a an input to the ub equent
cla i cation proceure. The cla i cator
may be traine ith “goo ” ignal , lea ing to a geometrical e cription of the“goo
cla ” in the pace of pattern vector 2 . A given ignal then i cla i e ba e on
a certain notion of i tance to that cla . Figure 2.11 ho ome r t cla icatio
n re ult . Without going into etail e mention that 15 ignal ere pre ente
to the y tem. The r t 5
2
For an intro uction to the ba ic principle of pattern recognition refer to, e.g
., [37]
2.3 Ca e Stu ie
39
ignal ere “goo ”; in the gure they are enote by io01.mat,
..., io05.mat. All oth
er ignal ere “ba ”. For a ignal to be quali e a “goo ” it mu t reach the maximum co
re of 9 in the bar graph . The upper re ult a obtaine
ith a CWT, the loer r
e ult ith a STFT, the re pective avelet or in o function are in icate in
the title line of the plot .
Cla ification re ult . CWT (Morlet− avelet). iO01.mat iO 2.mat 0 iO03.mat iO04.ma
t iO05.mat niO 1.mat 0 niO 2.mat 0 niO03.mat niO04.mat niO06.mat niO07.mat niO08
.mat niO 9.mat 0 niO10.mat niO 1.mat
1
9 8 7 6 5 4 3 2 1 0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15
Cla ification re ult . STFT ith Gau − in o (Gabor−tran form). 9 8 7 6 5 4 3 2 1
0
iO 1.mat 0 iO 2.mat 0 iO 3.mat 0 iO 4.mat 0 iO 5.mat 0 niO 1.mat 0 niO 2.mat 0 n
iO 3.mat 0 niO 4.mat 0 niO 6.mat 0 niO 7.mat 0 niO 8.mat 0 niO 9.mat 0 niO 0.mat
1 niO 1.mat
1
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15
Fig. 2.11. Cla i cation re ult of acou tic ignal .
The re ult in icate that both tran form are u eful
in the context of our inve
tigation,
but the avelet tran form pre umably
lea to a clearer eparation of “g
oo ” from “ba ”. Note that the Fourier ba e proce ure lea
40
2 Continuou Analy i
to ome mi cla i cation (fal
e po itive
): The “ba ” ignal nio01.mat an nio06.mat
, re pectively,are cla i e a “goo”. We nally remark that the re ult pre ente abo
ve ere obtaine ith linearly ample cale value (CWT) or frequency value (S
TFT), re pectively.
An option for future evelopment i the u e of logarithmica
lly ample cale or frequencie . The re ulting notion of “octave ” an “voice per
octave” (for a hort urvey of the e topic cf., e.g.,[38]) prove to be ucce f
ul hen analyzing peech or mu ic. Here e inve tigate ignal re ultingfrom a
technical proce , neverthele the cla i cation of the e ignal houl perform
imilar to the human ear. In [4] a mathematical mo el of the cochlea, a piral
like organ hich i e ential for hearing, i given, hich relate the perceptio
n of the human ear to the CWT an incorporate a logarithmic re pon e to frequen
cie !
2.4 Note an Exerci e
Note For upplementary rea ing on the topic treate in thi chapter
the reaer
i referre to the bibliography. Reference [7] an [20] provi e athorough i
cu ion of the mathematical foun ation of the CWT. In particular, ecay propert
ie of the CWT, e ential for quantifying
it concentration propertyon the “impor
tant” ignal region are invetigate . The STFT originally a intro uce by D. Ga
bor [10]. The book [17] provie a nice overvie on thi topic. Short an compre
hen ive revie of the CWT an STFT are al o containe in [1] an [34]. Exerci e
1. a) A ume that the Fourier tranform corre pon ence pair y(t) ◦ − • y (ω) ˆ is give.
Sho that the y(t)tm ◦ − • j m Hin : S ar from
+∞
dm y ˆ (ω). dω m
(2.36)
y (ω) = ˆ
−∞
y(t)e−jωt dt
ad di eretiate both sides ith respect to ω.
2.4 Notes ad Exercises
41
b) Agai assume that the Fourier trasform correspodece pair y(t) ◦ − • y (ω) ˆ is give
. Sho that the dm y (t) ◦ − • (jω)m y (ω). ˆ dtm Hit: Start from 1 y(t) = 2π
+∞
(2.37)
(ω)e+jωt dω ˆ
−∞
ad di eretiate both sides ith respect to t. 2. Give T ith T > 0, the “box ido
” of idth T is de ed as (t) = 1 −T ≤ t ≤ 2 0 else
T 2
Compute the respective localizatio parameters 5.6 – 5.9. 3. Cosider a sigal hi
ch may be represeted by a itegrable fuctio f (t) ad deote the rst itegral
ith F (t). Sho that the CWT computed
ith the
Haar
avelet ψH de ned in eq. 2.32
for > 0 re ds 1 LψH f ( , t) = √ 2 t + − (t) − (t + ) . cψH 2 This n l tic l
result rovides the ossibilit to check the f st CWT lgorithm
described in sect
ions 2.2.1 nd 2.2.3: Choose s mle function f (t) such th t (t) is known. N
ow choose the discretiz tion nd the t discretiz tion from sect. 2.2.1 nd com
ute with MATLAB the m trix LψH f ( i , tk ) using the bove exression. Now visu
lize the bsolute v lue with im gesc nd com re the result with the visu liz ti
on obt ined from mcwt. Note th t, when gr v lue coding is used, im gesc utom
tic ll m s the lowest m trix v lue to bl ck nd the highest to white. Thus fo
r visu liz tion the f ctor cψH , which onl l s the role of glob l intensit f
ctor, m be omitted when comuting LψH f ( i , tk ). M Φ 4. Show th t ψ(t) = d M (t)
h s M v nishing moments. Stes: dt ˆ ˆ ) Show th t ψ(ω) = (jω)M Φ(ω) usig 2.37. mˆ m md
ow th t ψ(t)t ◦ − • j dωm (ω) usig 2.36.
ψ c) Conclude th t d m (0) = 0 (m = 0, . . . , M − 1) nd rel te this to the dω momet
coditio 2.31.
m
ˆ
42
2 Cotiuous Aalysis
5. Assume that the avelet ψ h s M v nishing moments. Assume, moreover, th t f (t)
is iecewise olnomi l nd the degree of the resective
ol nomi ls is below M
. Show th t then Lψ f ( , t) = 0 for “suit ble” t nd . or the recise formul tion
refer to the section on r ctic l imlic tions of the v nishing moments conditio
n ( ge 27 nd following ges).
6. Tr to im lement
s this situ tion: or recovering f not ll v lues of the tr nsform (i.e., the M ×
N m trix mentioned bove) re needed; inste d, one single row corresonding to
the chosen v lue su ces. In the gure the m trix is visu lized b
the rect ngul r
re , the selected row b the horizont l line. Simil rl, for the ST T lied t
o f under roer conditions
it su ces to ick out single column from the corres
onding N × N m trix of
ST T v lues in order to reconstruct f . This c n be conclu
ded b insecting the ST Treconstruction
lgorithm – in rticul r eq. 2.12 – nd is
illustr ted in the right rt of ig. 3.1. In both c ses onl N tr nsform v lue
s re needed to recover the origin l sign l f , which is strong
indic tion for
the f ct th t the inform tion stored in the full CWT m trix or ST T m trix, res
ectivel, is redund nt. In f ct this is true: M them tic ll two di erent CWT tr
nsform v lues Lψ f ( 1 , t1 ) nd Lψ f ( 2 , t2 ) re rel ted with e ch
other
b the
“re roducing kernel condition”. An n logous st tement holds for the ST T. or det
ils refer to, e.g., [20]. The t sk formul ted bove, i.e., selecting roer di
screte subset of the t l ne for the CWT (of the t ω plae for the STFT, respecti
vely) such that f completely ca be recostructed from the respective trasform
values o these subsets oly, ca be solved. I the geeral case it leads to the
otio
3.2 The Haar System t t
45
a
ω
Fig. 3.1. Left: Uder certai coditios the sampled sigal f may be recostruct
ed from the correspodig CWT matrix by selectig oe suitable scale factor. Rig
ht: Uder certai coditios the sampled sigal f may be recostructed from the
correspodig STFT matrix by selectig oe suitable time value.
of “frames” [7], for avelets the most popular versio is the “discrete avelet trasf
orm” (DWT) described i this chapter. I cotrast to the situatio illustrated i
the left part of Fig. 3.1 the DWT ill be a real “multiresolutio aalysis” of the s
igal, sice the DWT values ill ot be computed for a sigle scale factor oly.
I the ext sectio e shall discuss the simplest versio of such a trasform.
We shall ork out the aspects, hich may be geeralized ad describe the geeral
DWT procedure i sect. 3.3. Sectio 3.4 is devoted to fast algorithms ad relat
ed implemetatios ith the MATLAB Wavelet Toolbox. I sect. 3.5 e ill adopt a
vie poit o the developed techiques, hich serves as a preparatio for the co
mpressio procedures described i chap. 4. I sect. 3.6 some further applicatio
s are discussed, otes ad exercises ally are collected i sect. 3.7. Throughout
this chapter e ill cosider sampled sigals as de ed i eq. 3.1. As idicated
there, the k th sigal value for simplicity reasos mostly ill be deoted ith
fk istead of f (kTS ). Thus, sampled sigals are represeted as ite sequeces o
f umbers as idicated i table 5.2 (cf. sect. 5.5). They are al ays assumed to
be sampled ith a samplig distace TS , eve if this is ot ritte do explic
itly. For simplicity i the ext sectios e shall assume that TS = 1. (3.2)
The results obtaied there may be exteded to a arbitrary samplig distace TS
very easily ad e shall idicate ho .
3.2 The Haar System
I this sectio e shall develop the DWT for the Haar avelet (cf. eq. 2.32). It
ill tur out that the resultig trasform is liked to the correspodig CWT
46
3 The Discrete Wavelet Trasform
as de ed i eq. 2.17. Sice the CWT ca be applied oly to a cotiuoustime siga
l f (t), hereas e start from a discretely sampled sigal as de ed i eq. 3.1, i
t ill be ecessary to costruct a cotiuous time sigal f (t) from the discret
ely sampled sigal {fk }N −1 . We shall sho i the rst subsectio k=0 ho this ca
be doe ad ho to develop the DWT for f (t). It ill tur out that the otio
of cotiuous time sigals may be omitted ad the DWT may be formulated i a pur
ely discrete maer, i.e., for sequeces of umbers. The resultig facts ad ot
ios are collected i the secod subsectio. As stated already after eq. 2.1, i
order to keep otatio simple e shall use the symbol f both to represet eithe
r a cotiuous time sigal or a discretely sampled sigal. The actual meaig al
ays shall be clear from the cotext. I sect. 3.2.1 e shall itroduce some mor
e cotiuous time fuctios, hich are uiquely coected ith sequeces. Here t
he same covetio holds. 3.2.1 Cotiuous Time Fuctios Cosider the follo ig
discretely sampled sigal of legth 8: f = {fk }7 = {8, 4, 6, 8, 9, 7, 2, 4} k=
0 (3.3)
Moreover, e eed the follo ig fuctio, hich is called “scalig fuctio”: φH (t) :
= 10≤t<1 0 else (3.4)
We shall see later that to each wavelet or which a DWT may be constructed, ther
e belongs a unique scaling unction. This explains the index H in the above expr
ession, since the scaling unction de ned by 3.4 belongs to the Haar-wavelet ψH (cf.
eq. 2.32). We sh ll see l ter how this rel tion m be formul ted m them tic ll
. The sc ling function is used to build continuous time sign l f (t) from the
sequence {fk }:
7
f (t) :=
k=0
fk φH (t − k).
(3.5)
The to row of ig. 3.2 shows the sequence {fk } (left) nd the corresonding fu
nction f (t) (right). It is suerosition of shifted versions of the sc ling f
unction 3.4, e ch multilied with the corresonding sequence element. Thus, the
continuous time sign l f (t) will be iecewise const nt on interv ls of length o
ne, it is discontinuous t the s mling times. In the next ste we sh ll constru
ct “co rser version” f 1 of the sequence f b comuting the rithmetic me n of nei
ghboring sequence elements s indic ted below:
3.2 The H r Sstem
Sequences 10 5 0 0 10 5 0 −1 2 0 −2 −1 0 1
k 2 3 4 2 k 10 5 0 0 1 k 2 0 −2 0 2 4 t 6 8 2
3 4 0 2 4 t 6 8 4 6 8 10 5 0 0 2 4 t 6 8 unctions
47
The right lot in the third row of ig. 3.2 shows the di erence sign l of f (t) n
d f 1 (t), which we sh ll c ll d1 (t): d1 (t) := f (t) − f 1 (t). (3.8)
This sign l lso is c lled “det il sign l”. The re son is th t, b de nition, it rere
sents the sign l contributions which re needed to recover the origin l sign l f
(t) from the co rsened version f 1 (t). An insection of the lot shows th t on
the interv ls [0, 2], [2, 4], [4, 6] nd [6, 8] the det il sign l is of me n ze
ro: A ste of cert in height nd width 1 is followed b ste of the s me m g
nitude nd width, but with inverted sign. This is n immedi te consequence of th
e co rsening rocedure discussed bove, le ding from f (t)
to f 1 (t). The H r
w velet (cf. eq. 2.32) w s disl ed in the uer rt of ig. 2.2. It consists
of ex ctl two stes of width 1 nd the s me m gnitude but with inverted sign. T
herefore, d1 (t) 2 c n be written s suerosition of shifted H r w velets, w
hose width is doubled nd which re multilied with roer f ctor. These f cto
rs c n be obt ined directl b insecting the gr h of d1 (t): d1 (t) = 2ψH t 2 − ψH t
− 1 + ψH 2 t − 2 − ψH 2 t −3 . 2
We sh ll formul te this rel tion in slightl more gener l f shion. De ne the seq
uence d1 := {d1 }3 = {2, −1, 1, −1}. k k=0 (3.9)
Then d1 (t) m be written down simil rl to f 1 (t) (cf. eq. 3.7). But here we
ex nd with resect to ψH inste d of φH and the expansion coe cients 1 are given by th
e sequence {d1 } instead o { k }: k d1 (t) =
3 k=0
d1 ψH k
t −k . 2
(3.10)
The sequence {d1 }3 is disl ed in the left lot in the third row of ig. k k=0
3.2. We obt ined it b de ning nd investig ting the continuous function d1 (t).
It is imort nt to re lize th t this sequence lso c n be obt ined directl from
the origin l discrete sign l {fk }7 . k=0 {d1 }3 = k k=0 f0 − f1 f2 − f3 f4 − f5 f6 − f
7 , , , 2 2 2 2 = {2, −1, 1, −1}.
(3.11)
Note the simil rit of this rescrition to eq. 3.6! We re now re d to formul
te the connection of the co rsening nd det iling rocedure discussed so f r to
the CWT s de ned in eq. 2.17:
3.2 The H r Sstem
49
LψH f (2, 2k) =
2 cψH
d1 (k = 0, . . . , 3). k
(3.12)
The re der is invited to check this in the exercises. It should be cle r how the
results nd formul e of this section m be extended to gener l discrete sign
l of length N . Note th t vi eq. 3.12 the CWT v lues for the H r w velet m
be comuted b simle summ tion formul s given b eq. 3.11, i.e., without ev
lu ting nd roxim ting the integr l in formul 2.17! But, s we h ve seen, i
n this w we obt in LψH f ( , t) onl for = 2 nd t =
2k (k = 0, 1, . . .). We
conclude this section with collection of the results. st lgorithm for the c
omut tion of the CWT with the H rw velet: 1. St rt from the discrete sign l {f
k }, the H r w velet ψH (cf. 2.32) nd the corresonding sc ling function φH (c . 3
.4). De ne the continuous-time signal (t) as in 3.5. 1 2. Compute the sequences
{ k } and {d1 } as in eqs. 3.6 and 3.11, k respectively. 3. Then the CWT with th
e Haar-wavelet, applied to (t), or a = 2 and t = 2k (k = 0, 1, . . .) may be
evaluated via eq. 3.12. 4. (t) can be reconstructed as ollows: De ne the coarse
ned signal 1 (t) and the detail signal d1 (t) as in eqs. 3.7 and 3.10, respect
ively. Then (t) = 1 (t) + d1 (t).
This procedure is a one-step-version o the “discrete wavelet trans orm” (DWT), carr
ied out with the Haar-wavelet. We note already here that the computation may be
iterated, starting with 1 (t) instead o (t). This explains why we speak o
a one-step-version. We shall describe the iteration procedure in some more detai
l later. In Fig. 3.3 we visualize the subset o the t-a-plane, on which the CWTv
alues are computed via the DWT. Note that (t) may be reconstructed rom the CW
T-values on this subset together with the coarsened signal 1 (t)! 3.2.2 Sequen
ces
1 The computation procedure given in eq. 3.6, leading rom { k } to { k }, may b
e represented by the diagram shown in Fig. 3.4. The top row symbolizes the 1 inp
ut sequence { k } (c . 3.3), the bottom row the output sequence { k }. 1 Sequenc
e is obtained as ollows: The elements o the input sequence pointing toward a
certain element o the output sequence with an arrow are multiplied with the co
rresponding numbers and added at the node, where 1 the arrows meet. Thus, e.g.
0 = 0 1 + 1 1 . The diagram illustrates the 2 2
50
3 The Discrete Wavelet Trans orm 0 2 4 6 8
Et
2r
r
r
r
r
a
c
Fig. 3.3. One-step-version o the DWT: Subset o the t-a-plane. 0 = 8
r
r 1 2 © c r
1 2
1 = 4
2 = 6
r 1 2 © c r
1 2
r
3 = 8
1 0 = 6
1 1 = 7
Fig. 3.4. Visualization o eq. 3.6.
1 computation o the rst two elements o { k }. It is clear how to proceed with t
he remaining elements. Comparing Fig. 3.4 with Fig. 5.6 we conclude that the com
putation o 1 may be written as
1 = H .
(3.13)
As explained in sect. 5.3 (c . in particular eqs. 5.17 and 5.20) the sequence
1 is obtained rom by a two-step procedure: First a digital lter with coe cients
h0 = 1 , h1 = 1 is applied, subsequently the signal is downsampled, 2 2 i.e., o
nly every second sequence element is kept. This procedure is denoted with the sy
mbol H, note in particular, that the number o sequence elements (the length o
the signal) is divided by two. The computation o sequence d1 was illustrated in
eq. 3.11. It is now obvious that similarly to above we may write d1 = G . (3.14
)
Again G denotes ltering ollowed by subsampling, but now the corresponding lter co
e cients read g0 = 1 , g1 = − 1 . 2 2 Equ tions. 3.13 nd 3.14 ct on sequences onl
. The corresonding continuoustime sign ls f 1 (t) nd d1 (t), resectivel, re
not needed t ll. But if we w nt
3.2 The H r Sstem
51
to reconstruct the discrete sign l f = {fk } from these sequences b the scheme
described on ge 49 (cf. in rticul r ste 4 of this scheme), we must introduc
e the continuous time functions
f 1 (t) nd d1 (t) in order to recover the conti
nuous time sign l f (t). rom this sign l the origin l discrete sign l then m
be recovered b roerl ev lu ting the corresonding ex nsion formul 3.5. We
will show now th t this is not necess r. Inste d, it will be ossible to recove
r the origin l sequence b ling du l lters s described in sect. 5.3.
1 We st
rt from sequence {fk
}3 (cf. eq. 3.6) nd com ute new sequence k=0 17 { k }k=
0 s illustr ted in ig. 3.5.
3
r
3
1 2
r
7 2
r
1 T 2
7 2 1 2
r
r
1 0 = 6
1 T 2
r
1 1 = 7
Fig. 3.5. Computation o the rst our elements o H 1 .
1 1 The computation o F0 , . . . , F3 is indicated in the diagram, the remainin
g elements are computed analogously. Thus nally we obtain 1 {Fk }7 = k=0
33 77 3, 3, , , 4, 4, , 22 22
.
The interpretation o diagram 3.5 is similar to diagram 3.4. The di erence is that
now the input sequence is symbolized by the bottom row and the output sequence
by the top row. Obviously the length o the input sequence is doubled. The indic
ated procedure is exactly the application o a dual lter as explained in sect. 5.
3 (c . Fig. 5.7). Thus we may write compactly F 1 = H 1, where the lter coe cients
associated to the dual lter H read h0 = 1 , h1 = 1 . 2 2 Analogously, starting
rom {d1 }3 (c . eq. 3.11) we may compute k k=0 D1 = G d1 , where the lter coe cients
associated to the dual lter G read g0 = 1 , g1 = − 1 . The re der m verif th t
2 2
(3.15)
(3.16)
52
3 The Discrete W velet Tr nsform
1 {Dk }7 = k=0
111 1 11 1, −1, − , , , − , − , 222 2 22 97 4, 2, 3, 4, , , 1, 2 22
.
since, s exl ined bove, the origin l g coe cients were multilied with √ 2. or s
imlicit nd future reference we sh ll denote this connection between urel
discrete viewoint (sign ls re sequences) nd continuous viewoint (sign ls
re continuous time functions) b (H, G) ↔ (φH , ψH ).
(3.27)
The du l lters (H , G ) re omitted in this shorth nd not tion, since the re un
iquel rel ted to (H, G). In sect. 2.2.2 we noted th t the H r w velet h s onl
one v nishing moment. We lso g ve rguments there th t for sign l n lsis ur
oses it is desir ble to h ve s m n v nishing moments s ossible:
or sign
l which m be modeled b iecewise olnomi l functions we noted th t the CWT v
nishes, if the w velet h s M v nishing moments nd the highest degree k of the
olnomi ls s tis es k < M . So the CWT is concentr ted round sign l regions whic
h c n not be modeled in this w . This is imort nt for detecting nd loc lizing
sign l ch nges. It moreover is useful for roert c lled “energ com ction” [38
], b sic rerequisite for ling w velets to sign l comression. L ter we sh
ll discuss rel ted ex mles 1 .
1
= φH (2t) − φH (2t − 1). Indeed (cf. ig. 2.2) the H r w velet is comosed from two shr
unk versions of the sc ling function on the interv ls [0,
1
) nd [ 1 ,
1), res
ectivel , the 2 2 l tter one with inverted sign. 3.3.1 rom ilters to unctions
In the receding section we seci ed the lter coe cients for the rst three members of
the f mil of D ubechies w velets. But u to now onl for db1, i.e., for the H
r sstem, we know both the lter coe cients nd the corresonding w velet nd sc li
ng function. In this section we describe n iter tive rocedure (“c sc de lgorith
m”), which en bles us to comute the sc ling function φ(t) or any member dbn o the
amiliy o Daubechies-wavelets, provided the h-coe cients are given [8]. The corr
esponding wavelet results rom 3.29. We shall give a detailed description o the
procedure or the Haar-system, i.e., or db1. So the H - lter mentioned below bel
ongs to the coe cients 3.19. The generalization to arbitrary dbn then should be ob
vious.
3.3 Generalization to Daubechies-Wavelets
57
1. De ne the sequence e = {ei } with ei = 1i=0 0 else
Represent this sequence by a piecewise constant continuous-time unction as oll
ows: a) Subdivide the t-axis in intervals o length 1 such that t = 0 is the cen
ter o one o these intervals. Denote this interval with I0 and the neighboring
intervals with Ii (i = 1, 2, 3, . . .) and Ii (i = −1, −2, −3, . . .), resectivel. b
) Reresent the sequence e = {ei } b continuous time function e(t), where e(t
) = 1 t ∈ I0 0 else
The to row of ig. 3.6 shows the sequence e = {ei } nd the rel ted function e(
t). i.e., l du l lter simil rl s i
√ 2. Now com ute the sequence e1 = 2H e,
n ig. 3.5, where the coe cient v lues re re l ced b 1 inste d of 1 2 nd in the
bottom row we insert the sequence v lues ek
. The resulting sequence is rerese
nted b the left gr h in the second row of ig. 3.6. In rticul r, we h ve e1
= e1 = 1. 0 1 Simil rl to bove, continuous time function e1 (t) is ssoci te
d to this sequence with the following di erences: The length of the interv ls used
for subdividing the t xis is now 1 inste d of 1. Moreover 2 ⎧1 ⎨ e0 = 1 t ∈ I0 e1 (t
) = e1 = 1 t ∈ I1 ⎩1 0 else This function is reresented by the right grah in the s
econd row of Fig. 3.6. √ 3. √ Continue the stes described above, i.e., comute e2 =
2H e1 , e3 = 2H e2 , . . .. Associate the coresonding continuous time function
s e2 (t), 3 e (t), . . . where each function is iecewise constant on intervals,
whose length is divided by two when roceding from ei (t) to ei+1 (t). Sequence
e2 = {e2 } and the corresonding continuous time function e2 (t) k are shown in
the last row of Fig. 3.6 Figure 3.6 shows how the “initial estimates” e(t), e1 (t),
e2 (t), . . . converge to the correct scaling function φH (t) de ned in eq. 3.4. Li
kewise, or general n, the procedure described above leads to an approximative s
cheme or computing the dbn-scaling unction. The quality o the approximation i
s controlled by the number o iterations. As mentioned above, the corresponding
wavelet may be obtained rom eq. 3.29.
58
3 The Discrete Wavelet Trans orm
Cascade algorithm: Sequences 1.5 1 0.5 0 −0.5 −2 1.5 1 0.5 0 −0.5 −2 1.5 1 0.5 0 −0.5 −2
1 2
or the db2 c se this is illustr ted in ig. 3.7. The roxim tion e1 (t) corre
sonding to the rst iter tion is disl ed in the uer left di gr m. The re der
m verif th t the v lues of the nonzero ste heights re given √ b the correso
nding coe cients from T ble 3.1, multilied with 2. The lower left di gr m shows t
he w velet roxim tion derived from e1 (t); the roxim tions to sc ling func
tion nd w velet fter 7 iter tions re disl ed in the uer right nd lower r
ight di gr ms.
φdb2. Number o iterations: 1. 1 0.8 0.6 0.4 0.2 0 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 φdb
2. Number o iterations: 7.
1
0.5
0 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4
ψdb2. Number of iter tions: 1. 1 1 0.5 0 −0.5 −1 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 0.5 0 −0
5 −1 −1.5 −1 −0.5 0
ψdb2. Number of iter tions: 7.
0.5
1
1.5
2
2.5
3
3.5
4
ig. 3.7. Iter tive construction of db2 sc ling function nd w velet. One iter t
ion (left), 7 iter tions (right).
3.3 Gener liz tion to D ubechies W velets
59
We note n “imrovement in smoothness”: The db1 w velets nd sc ling functions re d
iscontinuous, where s in the db2 c se both functions re continuous. Still these
functions look r ther rugged nd in f ct higher order dbn w velets nd sc ling
functions get smoother. M them tic ll this st tement
c n be m de recise using
the notion of “regul rit” [24], it is illustr ted in ig. 3.8 for db5.
φdb5. Number o iterations: 7. 1 0.8 0.6 0.4 0.2 0 −0.2 −4 −3 −2 −1 0 1 2 3 4 5 6 7
ψdb5. Number of iter tions: 7. 1 0.5 0 −0.5 −1 −4
−3
−2
−1
0
1
2
3
4
5
6
7
erties re desir ble for sign l n lsis uroses2 . 3.3.2 Tr nsfer Proerties
or n dbn lter the H s mbol ctu ll denotes the lic tion of digit l lter
H
with the
coe cients {hk }, followed b subs mling rocedure (cf. sect. 5.3, ig
. 5.5). or
the G s mbol nd thecorres onding coe cients {gk } n logous conventi
ons hold. rom digit l sign l rocessing view oint it is quite instructive to
stud the frequenc resonse of H nd G, resectivel. In this section we sh ll
consider
onl the mlitude resonse, the h se resonse will be investig ted l
ter. requenc resonse, mlitude
resonse nd h se resonse re de ned in eqs.
5.18 nd 5.19, res ectivel . or short summ r of these toics the re der is r
eferred to sect. 5.3.
2
We remind the re der to the com rison of db2 with db4 shown in igs. 2.5 nd 2.
6, res ectivel .
60
3 The Discrete W velet Tr nsform
or
db1 to db4 the mlitude resonse of the coresonding H nd G lters re shown
in ig. 3.9. The gr hs re rr nged in clockwise order with db1 disl ed in th
e uer left di gr m. In e ch di gr m both the mlitude resonses of H (solid)
nd G (broken) re lotted.
db1, mlitude resonse (H: solid, G: broken) 1.5 1.5 db2, mlitude resonse (H
: solid, G: broken)
1
1
0.5
0.5
0 0 1.5
0.5
1
1.5
2
2.5
3
3.5
0 0 1.5
0.5
1
1.5
2
2.5
3
3.5
db4, mlitude resonse (H: solid, G: broken)
db3, mlitude resonse (H: solid, G: broken)
1
1
0.5
0.5
0 0
0.5
1
1.5
2
2.5
3
3.5
0 0
0.5
1
1.5
2
2.5
3
3.5
ig. 3.9. Amlitude resonse of db1 db4 lters (clockwise from bove left).
The lots indic te th t the resective H lters exhibit low ss ch r cteristic,
where s the G lters re high ss lters. Moreover, obviousl with incre sing n th
e resective lter erform nce imroves: The cuto frequencies re more sh rl loc
lized nd tr nsmission s well s suression of the resective frequenc r nges
works better nd better. This imrovement in lter beh vior corresonds to the di
scussion in the receding section, where we demonstr ted, how w velets nd sc li
ng functions belonging to the lters “imrove” when n is incre sed.
3.4 Multisc le An lsis
Let us ssume th t for cert in n the corresonding dbn lter coe cients re given.
So we h ve ir of lters (H, G) whose coe cients s tisf eqs. 3.23 –
3.4 Multisc le An lsis
61
ig. 3.11. 3 ste version of the DWT: Subset of the t l ne.
Note th t from ig. 3.10 it follows th t J ste DWT le ves the tot l number of
d t unch nged: Aling J ste DWT to the N sequence elements of
3.4 Multisc le An lsis
63
the origin l sign l f = {fk }N −1 , we obt in the sequences f J , dJ , dJ−1 , . . .
, d1 , k=0 where the tot l number of d t resulting from dding the lengths of t
hese sequences, g in3 will be N . Moreover, the decomosition rt of eq. 3.30
rovides f st 4 lgorithm for comuting w velet tr nsform v lues vi the seque
nces dj . Thus, when com ring with the situ tion visu lized in ig. 3.1, we con
clude th t the J ste DWT le ds to sim le nd quick lgorithm for comuting w
velet tr nsforms without redund nc on subgrid of the t l ne, which t kes i
nto ccount v ring sc le f ctors in contr st to the discretized CWT. U to now
(cf. eq. 3.2) we ssumed TS = 1. As nnounced on ge 45
the extension to n rb
itr r s mling dist nce TS is ver simle: Looking t ig. 3.11 rel ce on the
t xis the l bels 0, 2, 4, 6, . . . b 0, 2TS , 4TS , 6TS , . . . nd corresond
ingl on the xis the l bels 0, 2, 4, 8, . . . b 0, 2TS , 4TS , 8TS , . . ..
So, e.g., Lψ f (4TS , 4TS ) = 12 d. cψ 1
Ex mles In ig. 3.12 the result of 2 ste DWT with db1 lters is disl ed. ro
m bove, the sign
l f , the det il sign ls d1 nd d2 nd the roxim tion sign
l f 2 re shown. igure 3.13 shows the results of the s me tr nsform, but erfor
med with db2 lters inste d of db1. The results re in ccord nce with the roert
ies of dbn lters collected in sections 3.3 nd 3.3.1. Since the db2 w velet is sm
oother nd h s more v nishing moments th n the db1 w velet, one exects the db2
det il sign ls to be better concentr ted round signi c nt sign l ch nges th n the
db1 det ilsign ls. Since f h s regul r e r nce, i.e., signi c nt sign l ch n
ges re l cking, this siml me ns th t det il sign l v lues should be r ther sm
ll. Obviousl this is true for both lter tes nd indeed the db2 det il sign ls
re closer to zero th n the corresonding db1 det il sign ls. The concentr tion
roert discussed bove h s nother useful consequence. Remember th t the sign
l c n be reconstructed from roxim tion nd det il sign ls b the reconstruct
ion rt of eq. 3.30. Therefore due to the concentr tion roert the J ste
DWT is method to concentr
te the full inform tion stored in the sign l in few
signi c nt coe cients. In igs. 3.12 nd 3.13 these coe cients b sic ll re the sequ
ence elements of f 2 nd the concentr tion mech nism works more e cientl for db2
th n for db1. This
3 4
To be recise, for rbitr r N the tot l number of tr nsformed d t c n be sligh
tl l rger th n N . Refer to sect. 3.4.3 for short discussion. In f ct, since
the lgorithm reduces to discrete ltering, the number of oer tions needed for co
muting the dj sequences incre ses line rl with N .
64
3 The Discrete W velet Tr nsform
2−ste−DWT with db1 2 f 0 −2 0 2 d1 0 −2 0 2 d2 0 −2 0 2 f2 0 −2 0 10 20 30 40 50 60 70 10
0 30 40 50 60 70 10 20 30 40 50 60 70 10 20 30 40 50 60 70
(bottom right). irst we note th t both histogr ms re more sh rl e ked (with
the l rgest e k round zero) th n the histogr m of the origin l sign l. This i
s in ccord nce with our discussion from bove concerning the concentr tion fe t
ures of the J ste DWT in gener l. Moreover, the db2 histogr m is more sh rl
e ked th n the db1 histogr m, which lso m tches our exect tion.
Histogr m of sign l v lues 12
10
8
6
4
2
0 −1.5 14
−1
−0.5
0
0.5
1
1.5
Histogr m of tr nsformed v lues (db1)
Histogr m of tr nsformed v lues (db2) 25
12
20
10
15
8
6
10
4
5
2
0 −2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
2.5
0 −2
−1.5
−1
−0.5
0
0.5
1
1.5
2
ig. 3.14. Above: Histogr m of the origin l sign l. Below left: Histogr m of the
tr nsformed sign l (db1); below right: Histogr m of the tr nsformed sign l (db2
).
3.4.2 Two Dimension l Sign ls (Im ges) Until here, the sign ls considered were s
equences, usu ll recorded s time series. Digit l im ges, in contr st, re re
resented s m trices f = {fmn } (m = 0, . . . , M − 1; n = 0, . . . , N − 1). (3.32
)
Simil rl to the 1 d c se, f denotes the m trix, fmn the element in row m nd co
lumn n. M nd N denote the tot l number of rows nd columns, resectivel. Since
f reresents icture, fmn will be c lled the “ixel” in row m nd column n. The i
m ge is gener ted b reresenting e ch ixel b gr v lue such th t d rk corr
esonds to sm ll nd bright to l rge ixel
66
3 The Discrete W velet Tr nsform
ig. 3.15. Above: Origin l im ge. Below: Result of ling H (here: db2) to e c
h row.
68
3 The Discrete W velet Tr nsform
which b sic ll “re cts” on sign l ch nges. In other words: We look long the column
s for sign l ch nges. On the other h nd, horizont l edges will give rise to sign
l ch nges when sc nning columns, therefore the im ge Hr Gc f will re ct on hori
zont l edges. Moreover, since (cf. eq. 3.26) the w velet tr nsform is comuted f
or sc le 21 , the horizont l det il sign l Hr Gc f lso is denoted with d1h . Co
mletel n logousl, vertic l nd di gon l det il im ges re ct on corresonding
l directed edges nd therefore Gr Hc f is denoted with d1v , Gr Gc f with d1d .
We n ll rem rk th t n logousl to the 1 d c se tre ted bove the low ss lte
red im ge Hr Hc f is c lled “ roxim tion sign l”, denoted with f 1 . Usu ll the f
our subim ges {f 1 , d1h , d1v , d1d } gener ted b one steDWT re rr nged l
ike f 1 d1h d1v d1d (3.35)
igure 3.16 shows one ste DWT c rried out with H r lters. The resulting subim
ges re rr nged s in eq. 3.35.
Origin l
One−ste−DWT with db1
50 100 150 200 250 50 100 150 200 250
50 100 150 200 250 50 100 150 200 250
ig. 3.16. Left: Origin l im ge. Right: Result of the one ste DWT (here: db1).
In rticul r for horizont l edges, the bove mentioned direction l sensitivit
of the det il sign l m be veri ed. Ag in, n logousl to the rocedure described
in sect. 3.4.1, “multisc le n lsis” of digit l im ge f c n be erformed b
ling one ste DWT 3.34 successivel to f 1 , f 2 = Hr Hc f 1 , f 3 = Hr Hc f
2 , . . .. Thus the equiv lent of eq. 3.30 re ds
3.4 Multisc le An lsis
69
Gr Hc
−→ d1v
f
Hr G c
f Hr Gc d1h −→
1d −→ d Hr Hc Gr Hc Gr Gc
←− d1h ←− d1d
⊕ Gr Hc d1v ←−
G G rc
f
1 Hr Gc Gr Gc
−→ d2v −→ d2h
2d
Hr Hc
Hr G c
−→ d Hr Hc . . .
Gr Hc
←− d2h ←− d2d (3.36)
⊕ Gr Hc d2v ←−
G G rc
Hr Hc . . .
−→ dJv −→ dJh
Jd
f
J−1 Hr Gc Gr Gc
−→ d Hr Hc fJ
Hr G c G Hc r
←− dJh ←− dJd
⊕ ←− dJv
G G rc
Hr Hc fJ
Ag in, the left di gr m reresents decomosition, the right di gr m reconstructi
on. The smbol ⊕ now denotes ddition of m trices r ther th n sequences. Gener liz
ing the reresent tion from eq. 3.35, the result of twoste DWT, for ex mle,
c n be reresented s ⎞ ⎛ 2 2h
fd 1h 2v 2d d ⎠ ⎝d d (3.37) 1v 1d d d So, if we continue
to decomose the
origin l of ig. 3.16, we obt in decomosition s shown in th
e right rt of ig. 3.17. 3.4.3 Imlement tions with the MATLAB W velet Toolbox
In this section we give short descrition of functions cont ined in the MATLA
B W velet Toolbox, which imlement the J ste DWT for 1 d nd 2 d sign ls, rese
ctivel. Rel ted functions which m ke use of the toolboxrocedures h ve been tes
ted with MATLAB 6.5, rele se 13 nd m be down
70
3 The Discrete W velet Tr nsform
Two−ste−DWT with db1
50 100 150 200 250 50 100 150 200 250
50 100 150 200 250 50 100 150 200 250
ig. 3.17. Left: One Ste DWT (cf. ig. 3.16). Right: Two Ste DWT, obt ined b
decom osing the u er left subim ge of the one ste DWT.
lo ded from the URL given in sect. 1.3. Ag in, s mentioned e rlier, b sic und
erst nding of MATLAB snt x is required. 1 d functions The decomosition rt of
the one dimension l J ste DWT (cf. eq. 3.30) is imlemented in the toolbox fun
ction w vedec. A rotote function c ll re ds [f_tr ns,L] = w vedec(f,J,’dbn’); The
origin l sign l is stored in the vector f, the result of the tr nsform in the v
ector f_tr ns. The l tter is org nized s follows : f2 Length: L(1) d2 Length: L
(2) d1 Length: L(3)
Here for illustr tion uroses J = 2 w s chosen, refer lso to ig. 3.10. As ind
ic ted, the lengths of the individu l subsequences re stored in the comonents
of “book keeing vector” L. Thus, e.g., in MATLAB not tion d2 m be extr cted from
f_tr ns s follows: d_2=f_tr ns((L(1)+1):(L(1)+L(2))); Note th t, if N denotes t
he length of the origin l sign l, then in our ex mle L(1) ≈ N N N , L(2) ≈ , L(3) ≈ .
4 4 2
3.4 Multisc le An lsis
71
The re son for the roxim te equ lit is th t N not necess ril is ower of
two; moreover, for dbn lters with n > 1 the tre tment of sign l bound ries requir
es the subsequences to be slightl longer th n the fr ctions comuted bove. As
n illustr tion we include the following MATLAB di logue, demonstr ting 2 ste
DWT with sign l of length 1367: >> f=r nd(1367,1); >> [f_tr ns,L]=w vedec(f,2
,’db2’); >> sum(L(1:3)) ns = 1373 >> [f_tr ns,L]=w vedec(f,2,’db3’); >> sum(L(1:3)) ns
= 1376 We see th t f_tr ns in gener l will be slightl longer th n f nd this “le
ngthening e ect” incre ses with lter length. So the book keeing vector is essenti l
for the correct h ndling of the individu l subsequences merged in the vector f_t
r ns. It is cle r, therefore, th t it is n imort nt inut into the toolbox fun
ction w verec imlementing the reconstruction rt of the one dimension l J ste
DWT (cf. eq. 3.30). A rotote c ll re ds f = w verec(f_tr ns,L,’dbn’); Note th t
the decomosition deth J needs not to be seci ed, since it follows immedi tel f
rom the length of the book keeing vector L, gener ted b the receding decomos
ition. 2 d functions The decomosition rt of the two dimension l J ste DWT (c
f. eq. 3.36) is imlemented in the toolbox function w vedec2. A rotote functi
on c ll re ds [f_tr ns,L] = w vedec2(f,J,’dbn’); Here the origin l im ge is stored i
n the m trix f with M rows nd N columns. The result of the tr nsform is stored
in the vector f_tr ns, which is org nized s follows : f1 d1h d1v d1d
72
3 The Discrete W velet Tr nsform
Here for illustr tion uroses J = 1 w s chosen, refer lso to eq. 3.35. Subsequ
ence f 1 , for ex mle, cont ins ll ixels belonging to the roxim tion im ge
, but rr nged s vector obt ined b sc nning the roxim tion im ge row b r
ow. Simil rl the rem ining subsequences re constructed. Inste d of book kee
ing vector in te 2 d c se “book keeing m trix” L is gener ted, which stores the m
trix dimensions of the roxim tion sign l nd the det il sign ls s follows:
L h s 2 columns, the rst row rovides the number of rows nd of columns, resecti
vel, for f 1 . The second row rovides the dimensions of d1h , d1v nd d1d (whi
ch re ll equ l). Simil rl for J > 1 subsequent rows of L re gener ted. The M
ATLAB di logue shown below illustr tes the use of this function for n im ge wit
h M = N = 16: >> f=r nd(16,16); >> [f_tr ns,L]=w vedec2(f,2,’db1’); >> L(1:3,:) ns
= 4 4 8 4 4 8
>> length(f_tr ns) ns = 256 >> [f_tr ns,L]=w vedec2(f,2,’db2’); >> L(1:3,:) ns = 6
6 9 6 6 9
>> length(f_tr ns) ns = 387 Note th t – n logousl to the 1 d c se – with incre si
ng lter order the tr nsformed sign l will be l rger th n the origin l! Since the
tr nsformed sign l is stored in vector, where the bound ries of the individu l
subsequences m be determined from L, these subsequences c n be extr cted usin
g L. To build the corresonding im ge, however, the must be re rr nged to m t
rix!
3.4 Multisc le An lsis
73
Let’s continue the ex mle from bove with the t sk to extr ct the det il im ge d1
d : >> >> >> >> >> lower=L(1,1)*L(1,2)+3*L(2,1)*L(2,2)+2*L(3,1)*L(3,2)+1; uer=
L(1,1)*L(1,2)+3*L(2,1)*L(2,2)+3*L(3,1)*L(3,2); d1d=f_tr ns(lower:uer); d1d=res
h e(d1d,L(3,1),L(3,2)); size(d1d)
ig. 3.18. Left: Ph se resonse for db1. Right: Ph se resonse for db2.
solved b A. Cohen [5]. or n e s to re d summ r see [24]. We sh ll brie reo
rt the m in results for the 1 d c se. The gener liz tion to 2 d sign ls then is
obvious. It turns out th t eq. 3.27 reresenting the correl tion between discret
e lters nd w velets needs to be extended s follows: ˜ ˜ ˜ ˜ (H, H, G, G) ↔ (φ, φ, ψ, ψ).
r liz tion of eq. 3.25 re ds ˜ ˜ H H + G G = Id. (3.39) (3.38)
In other words: We h ve ir of w velets nd sc ling functions, resectivel,
ssoci ted to corresonding ir of high nd low ss lters, such ˜ th t singl
e decomosition ste is erformed b ling H nd G. This ˜ ste m be inverted
b ling the du l lters H nd G , resectivel, nd dding the results. So the
J ste tr nsform 3.30 now m be reresented s
˜ G
H
f −→ d
˜ G
1
f ⊕ ←− d1 ˜ H ⊕ ←− d2 ˜ H . . . ⊕ ←− dJ ˜ H fJ
G G G
1 f −→ d2 H . . .
(3.40)
f J−1 −→ dJ H fJ
˜ G
The gener liz tion of this equ tion to 2 d sign ls is str ightforw rd.
3.5 A Unifing Viewoint: B sis Sstems
75
In im ge comression biorthogon l w velet lters re of rticul r imort nce: The
lossless mode of the JPEG 2000 still im ge comression st nd rd ˜ [36] uses bio
rthogon l lter ir H, H with 9 nd 7 coe cients, resectivel. The v lues re list
ed below:
T ble 3.2. ilter coe cients of the biorthogon l 9 7 lter ir. {hk } √ h−4 = 0.0267487
57411 √ 2 h−3 = −0.016864118443√ 2 h−2 = −0.078223266529 2 √ h−1 = 0.266864118443 2 √ h0 =
9018236√ 2 h1 = 0.266864118443 √ 2 h2 = −0.078223266529√ 2 h3 = −0.016864118443 2 √ h4 = 0.
26748757411 2 ˜ {hk } √ ˜ h−3 = −0.045635881557√ 2 ˜ h−2 = −0.028771763114 2 √ ˜ h−1 = 0.29
h0 = 0.557543526229√ 2 ˜ 2 h1 = 0.295635881557 √ ˜ h2 = −0.028771763114√ 2 ˜ h3 = −0.045635
7 2
Note the smmetr of the lters! The corresonding G lters re constructed simil rl
to eq. 3.28. We end this section with lot of mlitude nd h se resonse o
f H (solid) nd G (broken), resectivel (both h ving 9 coe cients).
9 coefficients: Amlitude resonse (H: solid, G: broken) 1.5
3.5 9 coefficients: Ph se resonse (H: solid, G: broken)
3
2.5
1
2
1.5
1
0.5
0.5
0
0 0
0.5
1
1.5
2
2.5
3
3.5
−0.5 0
0.5
1
1.5
2
2.5
3
3.5
consider the discrete versions, i.e., the D T nd the J ste DWT, nd discuss n
s ect which both tr nsforms h ve in common. It is shown th t both tr nsforms
re ex nsions of the sign l with resect to cert in “b sis sstems” nd these sstem
s rovide insight into the qu lit tive di erences of both tr nsforms. A rt from s
ect. 3.6.1, the m teri l covered in this section is not required for the rest of
the book. So it m be skied if the re der m inl is interested in quick n
d lic tion oriented w velet introduction. 3.5.1 One Dimension l Sign ls Let u
s st rt with one dimension l discretel s mled sign l of length N : f = {fk }
N −1 . k=0 φ0 natural φ1 natural . . . (3.41)
We introduce N sequences, called “natural basis system”, as ollows: = = . . . {1, 0
, 0, . . . , 0} {0, 1, 0, . . . , 0} . . .
(3.42)
φN −1 = {0, 0, 0, . . . , 1} n tur l Then f m be ex nded s follows:
1 f = f0 φ0 natural + 1 φnatural + . . . N −1
=
i=0
fi φi natural
(3.43)
Note that in these ormulae the summation o sequences is de ned by ele3 mentwise
addition! For N = 4 the natural basis system φ0 natural , . . . , φnatural is plotte
d in the le t column o Fig. 3.20. Thus, when starting with the basis system 3.4
2, the signal values k may be viewed as expansion coe cients when expanding the s
ignal with respect to this basis. It is a well known act rom linear algebra th
at basis systems are not unique: There exist other basis systems φ0 , . . . , φN −1 su
ch th t
N −1
f=
i=0
ci φi
(3.44)
rovide
with proper expansion coe cients c0 , . . . , cN −1 . In f ct, the ID T 5.14
s such ch nge of b sis s stems. De ne the following sequences, c lled “ ourier b s
is”:
3.5 A Unifing Viewoint: B sis Sstems
77
φ0 ourier = F φ1 ourier = F φ2 ourier F . . . −1 φNourier F
= .. .. .. j2π(N −1)0 j2π(N −1)1 j2π(N −1)2
j2π(N −1)(N −1) N N N N = √1 {e ,e ,e ,...,
5.14 m be rewritten s follows: 1 f = √ ( 0 φ0 ourier + F1 φ1 ourier + . . .) F F N
1 =√ N
N −1
1 √ {1, 1, 1, . . . , 1} N j2π1(N −1) j2π10 j2π11 j2π12 1 √ {e N , e N , e N , . . . ,
−1) j2π20 j2π21 j2π22 1 √ {e N , e N , e N , . . . , e N } N
i φi ourier . F
i=0
(3.46)
In other words: When expanding with respect to the Fourier basis system −1 φ0 ouri
er , . . . , φNourier , the corresponding expansion coe cients are given by F F Fi √ ,
where Fi denotes the i-th DFT-coe cient computed according to eq. N 5.13. For N =
4 the Fourier basis system is plotted in the middle column o Fig. 3.20. Note t
hat the sequences belonging to the Fourier basis are complex! The plots show the
real part o the respective basis sequences. Now assume that a certain dbn-wave
let and a decomposition depth J has been chosen and per orm a J-step-DWT corresp
onding to eq. 3.30. Assume that J , dJ , . . . , d1 are arranged as indicated
in Fig. 3.10 and merge these ˜ sequences into a new sequence called . For J = 2
the procedure is indicated below: ( 2 d2 d1 )
˜
(3.47)
˜ Expanding with respect to the corresponding natural basis system we may write ˜
=
i
˜˜ i φi natural ,
(3.48)
˜ ˜ where i denotes the i-the element o the sequence . Now decompose each ˜i basis
sequence φnatural into subsequences corresponding to the preceding Jstep-DWT deco
mposition. Again, we indicate the procedure or J = 2:
2 ( φ d2 d1 ) φφ ˜ φi natural
(3.49)
78
3 The Discrete Wavelet Trans orm
J Apply the reconstruction part o eq. 3.30 to φ , dJ , . . . , d1 and denote the
φ φ i reconstructed sequence with φwavelet . Then rom eq. 3.48 it ollows that
=
i
˜ i φi wavelet .
(3.50)
So similarly to the DFT-case (eq. 3.46) a J-step-DWT may be viewed as a change o
basis: Now the signal is expanded with respect to the wavelet basis instead o
the Fourier basis and the expansion coe cients are given by the entries o the tr
ans ormed sequences J , dJ , . . . , d1 . This is in complete analogy to the D
FT where the expansion coe cients were given by the DFTvalues. In contrast to the
DFT-case, however, the wavelet basis is not unique: Obviously, the basis sequenc
es φi wavelet (i = 0, 1, . . .) depend on both the chosen wavelet and the chosen d
ecomposition depth J. For N = 4 a wavelet basis system is plotted in the right c
olumn o Fig. 3.20. As mentioned above, this system depends on the chosen wavele
t as well as on the chosen decomposition depth J. Here db1, i.e., the Haar-wavel
et with the corresponding lters, and J = 1 were selected.
φnatural 1.5 1 0.5 0 −0.5 1.5 1 0.5 0 −0.5 1.5 1 0.5 0 −0.5 1.5 1 0.5 0 −0.5 1 1 0 −1 1 1 0
1 1 0 −1 1 1 0 −1 0 −1 0 2 4 0 2 4 0 2 4 0 −1 0 2 4 0 2 4 0 2 4 0 −1 0 2 4 0 2 4 0 2 4 0 −
0 2 4 0 2 4 0 2 4 φwavelet
Real(φFourier)
Fig. 3.20. Basis systems or N = 4. Le t column: φk natural . Middle column: Re(φk o
urier ). Right column: φk F wavelet . From above: k = 0, . . . , 3.
From this gure, together with the above computations, we collect the ollowing ob
servations:
3.5 A Uni ying Viewpoint: Basis Systems
79
The signal = { k } may be expanded with respect to 3 di erent basis systems: The
natural basis is “strictly local” in the sense that the corresponding basis sequenc
es are nonzero at exactly one position, the expansion coe cients are given by the
original signal values k . The Fourier basis is “global” in the sense that the corr
esponding basis sequences may be nonzero at all positions, the expansion coe cien
ts are proportional to the DFT-values Fk . The wavelet basis is “local” in the sense
that the corresponding basis sequences are concentrated around several position
s, the expansion coe cients are given by the entries o the sequences J , dJ , .
. . , d1 . Thus, in a sense, the wavelet basis system interpolates between the
strictly local natural basis and the global Fourier basis. We will return to thi
s qualitative di erence in the behavior o the trans orm later (c . sect. 3.6.1 an
d chap. 4). 3.5.2 Two-Dimensional Signals What we have done or one-dimensional
signals may be extended easily to the 2-d-case. We shall not bother the reader w
ith detailed ormulae but rather sketch the basic acts in eqs. 3.51 – 3.53 below.
As in sect. 3.4.2 the signal will be a matrix = { mn } with m numbering rows
and n numbering columns. Expanding the signal with respect to the natural basis
we obtain: =
mn
mn φmn natural
(3.51)
Here the matrices φmn natural are nonzero at exactly one pixel position. The expan
sion o with respect to the Fourier basis reads: =
mn
Amn φmn F ourier
(3.52)
Here the expansion coe cients Amn are proportional to Fmn , i.e., the values o th
e two-dimensional DFT7 . The matrices φmn F ourier may be nonzero at all pixel pos
itions. Finally, we may expand with respect to a wavelet basis: =
mn
7
˜ mn φmn wavelet
(3.53)
In sect. 5.2 the DFT was de ned only or sequences. We remark that it may be exten
ded to matrices by applying the DFT row- and columnwise. For details re er to, e
.g. [33].
80
3 The Discrete Wavelet Trans orm
˜ Here the expansion coe cients mn result rom merging the outcome o a two-dimensi
onal J-step-DWT into one matrix analogously to eq. 3.37. The matrices φmn wavelet
may be nonzero at several pixel position. Note that, as in the 1-d-case, the wav
elet basis system will depend on the chosen dbn and decomposition step J. We com
pare the qualitative di erences o the basis systems in Fig. 3.21, where we used a
gray value coding or the displayed matrices, with black representing the small
est value and white the largest. For the Fourier basis matrix the real part is d
isplayed, or matrices belonging to natural and wavelet bases, respectively, abs
olute values are shown. Note that the respective eatures o the basis systems,
recorded on page 79 or the one-dimensional case, obviously are valid also or 2
-d-signals.
φ (3,5) 0 Re(φ(3,5) )
Fourier
natural
0
1
1
2
2
3
3
4
4
5
5
6
6
7 0 1 2 3 4 5 6 7
7 0 1 2 3 4 5 6 7
φ123 . J=2. Wavelet:db2 Wavelet 0
1
2
3
4
5
6
7 0 1 2 3 4 5 6 7
Fig. 3.21. Selected basis matrices or 16 × 16-images. Above le t: A natural basis
matrix, above right: Real part o a Fourier basis matrix, below: A wavelet basi
s matrix (db2, J = 2).
3.6 Case Studies
81
3.5.3 Computation and Visualization with MATLAB Again, or this section basic a
miliarity with MATLAB is required. We shall discuss implementations generating t
he basis systems discussed above. The corresponding MATLAB-m- les have been tested
with MATLAB 6.5, release 13 and may be downloaded rom the URL given in sect. 1
.3. Note that the m- les discussed in this section require the MATLAB Wavelet Tool
box! As we have seen, or one-dimensional signals o length N each basis system
is constructed by speci ying N basis sequences φ0 , . . . , φN −1 . The function b ses
1d returns the three sstems discussed in sect. 3.5.2 nd visu lizes the resect
ive subsets φ0 , . . . , φkmax by the sample call [phi_natural,phi_Fourier,phi_wavel
et]=bases1d(kmax,N); Here 0 ≤ kmax ≤ N − 1 must be s tis ed nd hi_n tur l, for ex mle
, N −1 is N × N m trix with the sequences φ0 natural , . . . , φnatural arranged roww
ise. Likewise, the sequences constituting the Fourier and the wavelet basis syst
em, respectively, are collected in the matrices phi_Fourier and phi_wavelet. Fig
ure 3.20 was created with this unction. For quadratic images, i.e., N × N -images
, representative basis matrices may be selected, plotted and returned with bases
2d. A sample call reads [phi_natural,phi_Fourier,phi_wavelet]=bases2d; The selec
tion o phi_natural and phi_Fourier, respectively, is done by speci ying the ind
ex pair mn corresponding to ormulae 3.51 and 3.52. The selection o phi_wavelet
is slightly more complicated since the toolbox unctions wavedec2 and waverec2
are used, which reshape the outcome o a twodimensional J-step-DWT as a vector (
c . sect. 3.4.3). Figure 3.21 was created with this unction.
3.6 Case Studies
3.6.1 Energy Compaction and Compression In this section more examples or the en
ergy compaction property discussed above (c . Figs. 3.12 – 3.14) are presented. Re
call that “energy compaction” means that, a ter applying a J-step-DWT to the signal
, a large portion o the trans ormed signal values will be close to zero. In o
ther words: The in ormation stored in the signal will be concentrated in relativ
ely ew signi cant trans orm values. It is obvious that this property is essential
or signal compression and we shall discuss a na¨ compression model. ıve The DFT al
so shows energy compaction. There ore, the sample signals we discuss will be tra
ns ormed both with the DFT and the J-step-DWT and characteristic eatures o bot
h trans orms will be worked out.
82
3 The Discrete Wavelet Trans orm
One-dimensional signals The top graph o Fig. 3.22 shows an acoustic signal. It
was sampled with a sampling rate νS = 8192 Hz (for the de itio of the samplig rat
e refer to Table 5.2). The legth of the sigal f is N = 73113. For completeess
e metio that f represets the rst 9 secods of G.F. H¨del’s famous a “Hallelujah” ch
oir from the “Messiah” oratory. For the rest of this sectio e shall deote ith F
the sequece F = {Fk } of DFT values (cf. eq. 5.13), the outcome f J , dJ , dJ−1 ,
. . . , d1 of a J step DWT ˜ ill be merged ito a sequece f as sho i eq. 3.4
7.
From above: Origial, trucated origial, Fourier ad avelet recostructios.
1 0 −1 0 1 0 −1 0 1 0 −1 0 1 0 −1 0
1
2
3
4
5
6
7 x 10
8
4
1
2
3
4
5
6
7 x 10
8
4
1
2
3
4
5
6
7 x 10
8
4
1
2
3
4
p fF ourier
5
p f avelet .
6
7 x 10
8
4
Fig. 3.22. From above: Plot of f , f , J = 5, db20.
p
ad
Parameters: p = 1%.
As before (cf. Fig. 3.14) e illustrate eergy compactio ith histograms. Simil
arly to the procedure described already, e compute the respective his˜ tograms of
f , F ad f as follo s: Take the smallest ad the largest absolute value of the
correspodig sequece elemets ad subdivide the iterval be
3.6 Case Studies
83
t ee these values i 100 equally spaced bis. No for each bi cout the umber
of sequece elemets, hose absolute values are cotaied i it. The result of
this procedure is sho i Fig. 3.23.
From above: Histograms of origial, Fourier ad avelet coefficiets. 6000 4000
2000 0 0 4 x 10 2
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1
0 0 15000 10000 5000 0 0
100
200
300
400
500
600
700
800
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
˜ Fig. 3.23. Histograms of f , F ad f (from above). Wavelet parameters as i Fig.
3.22.
We see that both trasforms lead to histograms hich are sharply peaked close to
zero. Obviously, the “mai iformatio cotet” of the origial sigal is stored i
relatively fe sigi cat values. For completeess e metio that the parameters
belogig to the J step DWT ere J = 5 (decompositio depth) ad db20 (chose
avelet). Observe that the histograms of both trasforms are i sharp cotrast to
the histogram of the origial, hich is relatively uiform o its rage of data
! We ote already here (for more details refer to sect. 4.1.2) that for compress
io purposes the follo ig properties are ecessary/desirable:
84
3 The Discrete Wavelet Trasform
Trasform the origial sigal such that 1. The trasform is ivertible (ecessar
y coditio) ad 2. The histogram of the trasform values is sharply peaked (des
irable coditio). I fact, the achievable compressio rate of the procedure des
cribed i 4.1.2 ill icrease if the histogram is cocetrated more ad more o
oly fe values. Obviously the DFT ad the J step DWT ful ll both requiremets. I
order to provide a feelig for the respective degree of eergy compactio, he
compared ith the origial, the follo ig experimet as carried out (refer als
o to sect. 1.2.6): 1. Choose a percetage value p (0 < p ≤ 100). ˜ 2. Startig from
the histogram of the J step DWT f compute a trucated p ˜ versio f p ad a corres
podig recostructio f avelet as follo s: ˜ a) Sort the sequece elemets fk i
descedig order ith respect to their absolute values. The resultig list e de
ote ith l = {lk }. No ˜ choose a limit f limit such that the absolute values of
the rst p% of ˜ l etries are above or equal to f limit , the rest belo . b) The ˜p
fk = ˜ ˜ ˜ fk if |fk | ≥ f limi 0 else
In o herwords: If he absolu e value of a given ransform coe cien belongs o h
epercen age p of ransform
coe
cien s having he larges absolu e values, i is r
e ained, o herwise i is “dele ed”, i.e., pu equal
o zero. p c) Compu e fwavele b
y applying he recons ruc ion par
of eq. 3.30
o p ˜ ˜ f p = {fk }. 3. Proceed anal
ogously wi h he DFT leading o he runca ed ransform p p sequence F p = {Fk }
and
o he corresponding recons ruc ion
fF ourier by applying he IDFT (eq.
5.1
4) o F p . 4. Proceed analogously wi h he original leading o he runca ed ve
rsion f p .
p p The runca
ed signals f p , fF ourierand fwavele
will
be such ha heir r
espec ive his ograms are much more concenra ed a zero han he his ograms from
Fig. 3.23: They are
ob ained
from he la er by pu ing
all coe cien s wi h absol
u e values below he respec ive limi s from
above in he “zero bin”. Corresponding
o he remarks made on page84 herefore heir compression performance
will incre
ase. We no ealready here ha p provides a rough es i˜ ma e for he achievable co
mpression ra e if f p, f p or F p are coded and s ored as described in sec . 4.
1.2 [1].p p How are he signals f p , fF ourier and fwavele
dis or ed when com
pared wi h he original f ? As we have seen in he sec ion before, he signal ma
y
3.6 Case S udies
85
be expanded wi h respec o he na ural basis (eq. 3.43), he Fourier basis Fi (
eq. 3.46) or he wavele
basis (eq. 3.50) wi h expansion coe
cien s fi , √N ˜ and fi ,
respec ively.
The runca ion procedure from above means ha in hese expansion
s all con ribu ions from basis sequences
whose
expansion
coe cien s are mapped o
zero will vanish. Then by he s ric locali y of hena ural basis sys em only p
percen of f p will be nonzero.
On he o her hand, his will no p be rue for
he Fourier
basis, due o i s global
na ure, herefore fF ourier
migh 8 have no
nzero
en ries everywhere.
As s a ed before, he locali y of he wavele basis sy
sem will lead o an “in ermedia e behavior”: For small p values
one p may expec h
a some par s of fwavele will be zero,
bu he “nonzero por ion” p will be larger
han for f . p These fac sare illus ra ed in Fig. 3.22. Here from
above f , f p
,fF ourier
p and
fwavele
are plo ed wi h p value and wavele parame ers
indic
a ed in he cap ion. I is clear ha a sudden change
from zero
loudness o soun
d and vice versa will no be pleasing o he lis ener. No e ha f pwill be zer
o over 99% of he playing ime! Thus a “compression” resul ing from he runca ion p
rocedure discussed above wi h p = 1% will lead o unaccep able resul s. Never he
less, he basic mechanisms
responsible
for losses in signal quali y when compres
sing i by concen ra ing his ograms can be
illus ra ed wi h hese
examples. For
compressing
audio signals i urns
ou ha he use of “Coi e s” (a varian
of dbn wav
ele s which ensures a close ma ch of he approxima ion signal f J wi h he origi
nal f ) is promising [38]. Moreover, he desired concen ra ion process of his og
rams
should resul from a “percep uallylossless quan iza
ion procedure” (see also s
ec . 4.3), which is more involved han hesimple runca ion procedure from abov
e9 . Two dimensional
signals The experimen
described above
in comple e analogy
was carried ou also for images. The erminology will be he same, i.e., he ori
ginal image is de˜ no ed wi h f , F and f deno e he corresponding wo dimensional
and p p J s epDWT, respec
DFT ively10 . Again, f p , fF ourier andfwavele de
no e he recons ruc ions resul ing from ransforms which were runca ed in exac
ly he same way as described above. All images
are subjec o gray levelcoding,
i.e.: zero is coded as black, he larges value as whi e. The upper lef subima
ge of Fig. 3.24 shows he original. The number of rows and columns of he corres
ponding pixel ma rix were M = 240 and
8 9 10
For he noions of s ric ly local, global or local behavior, which are used here
we refer he reader o page79. The successful MP3 compression
scheme of audio
signals combines he “discre
e cosine ransform”, a varian of he DFT, wih percep
ually lossless quan iza ion. ˜ The ma rices F and f were in roduced in sec . 3.5.2
, pages 79 .
86
3 The Discre e Wavele
Transform
Original Recons ruc ion from 10% larges pixels
50
50
100
100
150
150
200
200
50
100
150
200
250
300
50
100
150
200
250
300
Recons ruc ion from 10% larges Fourier coefficien s
Recons ruc ion from 10% larges wavele coefficien s
50
50
100
100
150
150
200
200
50
100
150
200
250
300
50
100
150
200
250
300
Fig. 3.24. In clockwise
order from above lef : Plo of f , f , Image dimensions:
240 × 320. Percen age: p = 10%. J = 4, db4.
p
p fwavele
and
p fF ourer .
N = 320, respec
ively. Figure 3.25 implies ha he ransforms lead
o a concen
ra ion in he respec ive his ograms, which is even dras ic han for 1 d sig
more
nals. The remarks made before concerning
locali y of he respec ive basis sys em
s apply
also here.
In par icular, no e ha he chosen
percen age of p = 10% imp
lies ha exac ly 90% of f p will be black, since he
corresponding pixels are m
apped o zero. The locali y proper ies of he respec
ive basis ma rices (cf. Fig
. 3.21) are re ec ed in he corresponding recons ruc ions:
The global p periodici
y of Fourier basis
ma rices
leads o ickering ar ifac s in
fF ourier . p The loca
liy of wavele
basis
ma rices leads
o regions in
fwavele which are of near co
ns an in ensi y ( hey correspond o regions
in he audio
signal having cons an
loudness),
whereas he zooming
proper y of he wavele ransform (refer also o
sec . 1.2.6) ensures ha de ailslike edges are rela ively well reproduced. Th
e visual recep
ion here is di eren from lis ening: As explained
before,
changing
from cons an loudness o meaningful sound will be unaccep able o he
lis ener,
whereas changing
from smoo hened regions o de ailed regions will no be so dis
urbing for he observer.
3.6 Case S udies
x 10
4
87
From above: His ograms of original, Fourier and wavele coefficien s.
2
1
0 0 4 x 10 10
50
100
150
200
250
5
0 0 4 x 10 10
1
2
3
4
5
6 x 10
7
6
5
0 0
500
1000
1500
2000
2500
3000
3500
4000
˜ Fig. 3.25. His ograms of f , F and f (from above). Wavele parame ers as in Fig.
3.24.
MATLAB
Implemen
a ions The runca ion procedure described above is implemen ed i
n he func ions CompAudioToolbox(one dimensional signals) and CompImageToolbox
(wodimensional signals),respec ively.
These func ions require he MATLAB Wavel
e Toolbox and have been es ed wi h MATLAB 6.5, release
13. They maybe downloa
ded from he URL given in sec . 1.3. Pro o ype func
ion
calls read [y rek,yfrek,
yw rek]=CompAudioToolbox(y);
and [rek_pix,rek_f
,rek_w ]=CompImageToolbox(y); re
spec ively. The
original signal is deno ed wi h y, in he one dimensional
p p ca
se he recons ruc ions f p, fF ourier and fwavele are s ored in y rek, yfrek
a
nd
yw rek, analogously
in he
wo dimensional case hese ma rices are deno ed wi
h rek_pix, rek_f and rek_w .
88
3 The Discre e Wavele Transform
All required
parame ers (percen age p, reques ed dbn wavele , decomposi ion dep
h J) mus beprovided in a user dialogue. 3.6.2 Denoising a Sensor Signal / Real
Time Proper ies of he Algori hm Removing
noise is a s andard opic
insignal p
rocessing. Usually noise is a ribu
ed o high frequency componen s
of he signa
l, which correspond osmall de ail sizes when performing a wavele analysis (cf
.
eq. 1.4).
Therefore he idea is
o remove high frequency/small de ail componen
s such ha underlying signals ruc ures, which may be obscured by noise,
are
p
reserved and will emerge from he denoising process. We have
seen ha
a J ep
s
DWT3.30 performs a mul iscale analysis of a signalf in he sense ha de ails
up o ≈ TS 2J areremoved from f J , whereas he de ail signals dJ , dJ−1, . . . ,
d1 sore con ribu ions from decreasing de ail sizes in a way ha makes i possi
ble o recover f comple ely from f J ,dJ , dJ−1 , .. . , d1 by he recons ruc io
n par from eq. 3.30. Therefore, a J s ep DWT seems o be a goodcandida e for d
enoisingby he following hree s ep procedure: 1. Perform aJ s ep DWT.
2. Thre
shold de ail signals
corresponding
o scales, which may
be a ribu ed o noise.
3. Recons ruc wi h manipula ed de ail signals. Due o he pioneering work of D.
L. Donoho and his collabora ors (see, e.g. [9]) for cer ain noise models hresho
lding prescrip
ions
for de ail
signal coe cien
s can be given, which remove
he
no
ise wi hou disurbing impor an signal fea ures. Fora brief survey of hese e
chniquesrefer o [1], he MATLAB Wavele Toolbox con ains rela ed func ions [24
]. From he sec ion before, where da a compressionwas linked o hresholding, w
e see ha denoising and compression are based on he
same basic
proper ies of
he DWT, namely
mul iscale analysis and energy compac ion. In his sec ion we sha
ll illus ra edenoising by a crude example. The purpose is o demons ra e he r
ansfer
proper ies of wavele l ers (see sec . 3.3.2) and o work ou an aspec im
por an for prac ical applica ions. Le us again consider he sensor signal f fr
om Fig. 2.7. Remember ha his signal was recorded during an oscilla ory proces
s whose frequency changes from ν ≈ 3 Hz to ν ≈ 6 Hz at t ≈ 3 sec. Figure 2.7 sho s that th
is trasitio may be recogized from the STFT, but f is obviously heavily corrup
ted ith oise, hich is re ected by ozero high frequecy cotributios. I sect
. 3.3.2 e sa that each H lterig step i a J step DWT 3.30 correspods to lo p
ass lterig. Sice lo pass lterig removes high frequecy compoets (small detai
ls) of f , oe expects that f J , hich results from applyig J lo pass lterig
steps, ill be a smootheed versio of
3.6 Case Studies
89
f ad the degree of smoothig ill icrease ith J. Note, ho ever, that the leg
th of f J ill be equal to the legth of f , divided by 2J , due to subsamplig
i every decompositio step! Therefore, a smootheed versio f J,smooth of f is
obtaied by applyig the recostructio part of eq. 3.30 to f J , dJ,smooth , dJ−1
,smooth , . . . , d1,smooth , here the etries of all dj,smooth sequeces are z
ero. Referrig to the three step procedure from above, e apply a “brute force thr
esholdig” by mappig all detail coe ciets to zero, regardless of the respective le
vel j, i.e., the respective scale sizes. For J = 2, for example, the deoised si
gal f 2,smooth is computed as follo s: f −→ H −→ H −→ H −→ H −→ f 2,smooth .
2,smooth
(3.54)
I Fig. 3.26 e sho the resultig f together ith its STFT (cf. sect. 2.1.3). H
ere db1 lters ere chose, i.e., the Haar avelet as used. Figure 3.27 sho s the
same results for db20 lters istead of db1 lters.
Parameters: J=2, L=1 (db1) 1 0.5 0 −0.5 0
2
4
6
8
10
5 ν [Hz]
10
15 0 2 4 t 6 8 10
Fig. 3.26. Deoisig the sesor sigal from Fig. 2.7 ith J=2 ad L = 1 (db1).
Comparig both gures ith the origial ad its STFT (Fig. 2.7) e see that both p
rocedures lead to very satisfyig results: The frequecy trasitio is o clear
ly visible i the STFT plot ad may eve be localized by visual ispectio of th
e deoised sigals. Moreover, as is to be expected, the suppressio of high freq
uecies orks better ith icreasig order of db (see Fig.
90
3 The Discrete Wavelet Trasform
Parameters: J=2, L=39 (db20) 1 0.5 0 −0.5 0 2 4 6 8 10
5 ν [Hz]
10
15 0 2 4 t 6 8 10
Fig. 3.27. Deoisig the sesor sigal from Fig. 2.7 ith J=2 ad L = 39 (db20).
3.9). Also the smoothess of the deoised sigals improves ith icreasig order
of db i perfect accordace ith the correspodig avelet properties (cf. sec
t. 3.3.1). This simple example sho s the potetial of avelet methods for deois
ig sigals. Deoisig a sesor sigal ca be importat for obtaiig reliable i
formatio from the sesor. Here, the sigal stems from a light arc sesor of a
eldig robot ad is eeded for obtaiig the actual distace of the eldig too
l from the eldig seam. This distace iformatio is of fudametal importace
for cotrollig the positio of the eldig tool. I these kids of applicatios
a desirable feature of deoisig is to compute the deoised sigal “olie”: As soo
as the k th sigal value fk is recorded, the correspodig J,smooth of the de
oised sigal should be available. value fk A olie feature as de ed above cao
t be realized ith avelet lters. As a example e choose db1 ad J = 2. The lter
coe ciets read {h0 , h1 } ad e illustrate the t o fold H lterig from eq. 3.54 b
y extedig Fig. 5.6 to t o decompositio steps (Fig. 3.28). The diagram is plot
ted for TS = 1. For a geeral samplig time TS e ill 2 have a miimum delay ti
me of 3TS i order to compute the fk values11 . The reaso for this delay is th
at the coe ciet h1 of the db1 lter is “o causal”:
11
Observe that this delay time ill be a miimum value sice additio ad multipli
catio also cosume processor resources.
f0
r
r h1 h0 © c r
1
2
r h0 © c r
h1
r
3.7 Notes and Exercises 3
91
1 0
h1 h0 c A r
1 1
2 0
Fig. 3.28. Two- old H-application or a lter with coe cients {h0 , h1 }.
1 In order to compute , e.g., 0 , the “ uture signal value” 1 is needed and this d
elay propagates rom decomposition step to decomposition step. In act, it is no
t hard to obtain a general delay ormula. With L denoting the number o non-caus
al lter coe cients (i.e., the number o hk -coe cients with k > 0) the minimum delay
tdelay reads
tdelay = TS L(2J − 1).
(3.55)
Indeed, for db1 with L = 1 nd J = 2 we obt in the del time re d o from ig. 3.
28. Note th t there is tr deo between “denoising
qu lit ” nd del time! A dbn H lt
er h s L = 2n − 1 non c us l coe cients. We s w in igs. 3.26 nd 3.27, resectivel
, th t the fe tures of the denoised sign l imrove with incre sing n, but this i
mrovement corresonds to deterior tion of onlinec bilit. The corresonding
L v ues re given in the gure c tions.
3.7 Notes nd Exercises
Notes There re numerous w velet toics not tre ted t ll in this book. We exl
iclitl mention w velet ckets nd multiw velets nd encour ge the re der to co
nsult the references. The lic tion oriented re der might refer [38], [24] or
[1]. Re ders interested in rese rch re referred to [7], [20], [26], [3] or [23
]. In reference [22] n exh ustive discussion of w velet lic tions in sign l
rocessing is given. The two dimension l J ste DWT develoed in sect. 3.4.2 is
not the onl w to de ne discrete w velet tr nsform for im
ges. “Isotroic w vele
ts”, s constructed b P. M ß, re n ltern tive ro ch. or review refer to [
21].
92
3 The Discrete W velet Tr nsform
Another toic not covered in this book is the lifting scheme develoed for the D
WT. It ddresses in rticul r imlement tions of the decomosition nd reconstr
uction lgorithm, which ver e cientl reduce stor ge nd comuting time requireme
nts. The lifting scheme is described in [16]. Exercises 1. Verif eq. 3.12 in th
e following stes: ) Sketch √1 ψH u−t for = 2 nd t = 2k (k = 0, . . . , 3). b) D
ecomose f (t) in the form f (t) = f 1 (t) + d1 (t) nd show th t LψH f 1 (2, 2k)
= 0 (k = 0, . . . , N − 1). 2 c) Show th t LψH d1 (2, 2k) = cψ d1 (k = 0, . . . , N − 1)
. k
H
|a|
2. Compu
e frequency
response,
ampli ude response and phase response for he Haa
r l er H wi h coe cien s √√ 2 2 , h1 =. h0 = 2 2 These ransfer func
ions are de ned in
eqs. 5.18 and 5.19,
respec ively. Compu e hese quan i ies also for he corresp
onding
G l er wi h coe cien s √
√ 2 2 g0 = , g1 = − . 2 2 3. For N = 4, db1 and J = 1 com
pu e he one dimensional wavele basis
1 2 3 φ0 wavelet , φwavelet , φwavelet , φwavelet .
Steps: a) Choose the natural basis ˜ ˜1 ˜2 ˜3 φ0 natural , φnatural , φnatural , φnatural a
compose each o these sequences analogously to eq. 3.49, where the decomposition
or J = 2 was sketched. ˜ b) For each basis sequence φi natural apply the reconstru
ction part o eq. 3.30 to the corresponding decomposition. For the coe cients belo
nging to the dual lters H and G , respectively, c . 3.19 and 3.20! 4. Some MATLAB-
exercises. a) Develop a MATLAB- unction with the ollowing prototype call: trans
ers(coe ); Here the row-vector coe is constructed as ollows: coe =[coe ic
ients, lower, upper].
3.7 Notes and Exercises
93
The vector coe icients stores the hk -coe cients o a dbn-H- lter, lower denotes th
e lowest, upper the highest k-value. As an example, or db1 we have coe =[0.707
1, 0.7071, 0, 1.0000]. The vector coe icients can be obtained by the MATLAB Wav
elet Toolbox unction w ilters (c . sect. 3.4.3), note that or a dbn- lter lower=
0 and upper=2*n-1. Then trans ers should compute and plot the corresponding ampl
itude and phase trans er unctions (c . eq. 5.18). Furthermore, the gk coe cients
should be determined according to eq. 3.28 and their trans er unctions also sho
uld be computed and plotted. b) Develop a MATLAB- unction with the ollowing pro
totype call: out=imdec(original,J,’dbn’); Here the matrix original stores a gray lev
el image, J denotes the decomposition depth o a J-step-DWT and dbn chooses the
dbn- lters. Then out should store the output matrix arranged as indicated in eq. 3
.37 (see also Fig. 3.17). The unction should invoke the MATLAB Wavelet Toolbox
unction wavedec2 with speci cation given in sect. 3.4.3. c) (Involved!) Develop y
our own MATLAB- unction realizing the iterative construction and plotting o sca
ling unction φ and wavelet ψ, when the lter coe cients {hk } re given. The rocedure
(“c sc de lgorithm”) c n b
is described in sect. 3.3.1. Solutions to these roblems
e found in folders ilters ( ), TwoDim (b) nd W veIter (c). These folders m b
e downlo ded the URL given in sect. 1.3. 5. (Involved!) In order to solve the r
oblem formul ted below we h ve to extend theorem 5.1 from sect. 5.2. Let {f (kTS
)} be s mling of b nd limited nite energ sign l f (t) such th t the Sh nno
n condition 5.12 is ful lled. Then not onl the sectrum ˆ f (ω) ad the DFT are relat
ed by eq. 5.16, moreover, the cotiuous time sigal f (t) may be recovered comp
letely from the sample values {f (kTS )}: f (t) =
k
f (kTS )φSh (t − kTS ).
(3.56)
Here, the Sh nnon interol tion function is de ned s follows: sin π φSh (t) = π
t TS t TS
.
(3.57)
In wh t follows, denote with φ(t) the Shannon interpolation unction φSh (t) or TS
= 1, i.e. φ(t) = sin(πt) . πt (3.58)
94
3 The Discrete W velet Tr nsform
Show th t φ(t) = Steps: a) Show that ˆ φ(ω) = 1 |ω| ≤ π 0 else √ 2
k
φ( k ) hk φ(2t − k) where hk = √2 . 2
(3.59)
Hint: Use reconstruction formul 5.2! b) Show th t the m ximum frequenc ωmax belo
gig to φ(t) satis es the Shannon condition 5.12 or TS = 1 . 2 c) Put (t) = φ(t) a
nd write down eq. 3.56 or TS = 1 . 2 Supplementary in ormation: Remember that w
avelet and scaling unction are related by eq. 3.29, which is called a “scale-equa
tion”. It may be shown that the scaling unction ul lls a similar scale equation: φ(t
) = √ 2
k
hk φ(2t − k).
As n ex mle, the re der m verif this rel tion for the H r w velet nd the
corresonding lter coe cients {hk }. Then from eq. 3.59 follows th t φ, as de ned in eq
. 3.58, may be viewed as a scaling unction related to the above lter coe cients. T
hese lters in turn lead to a general J-step-DWT 3.30. In contrast to dbn-wavelets
, however, the coe cient set {hk } = φ( k ) √2 2
will be in nite! Thus, any implementation on a computer necessarily will lead to t
runcation errors.
4 More Applications
In this chapter we shall consider two DWT-applications: data compression and the
use o wavelets or automatic indexing o images in image databases. In the nex
t section the general structure o the trans orm compression scheme is presented
. We shall sketch the basic components and in particular will ocus on the mecha
nism responsible or distortions when compressing data. In sect. 4.1.3 we descri
be a procedure or reducing these distortions. This procedure is implemented in
MATLAB. Section 4.2 is devoted to wavelet applications or image databases.
4.1 The Trans orm Compression Scheme
Any data compression scheme aims at reducing storage requirements. The data cons
idered in this section are digital images1 , which are represented as matrices
= { mn } (m = 0, . . . , M − 1; n = 0, . . . , N − 1) (4.1)
(cf. sect. 3.4.2). Here, M nd N denote the number of rows nd columns, resecti
vel, the m trix entries fmn re c lled “ixels”. If single ixel requires b bits
of stor ge, the digit l im ge f will require Sf bits of stor ge, where Sf = N M
b. (4.2)
Inste d of stor ge we sh ll lso use the hr se “ le size” from now on. So the le size
of the origin l im ge f is comuted ccording to eq. 4.2.
1
Note, however, th t this restriction is m de onl for keeing not tion consisten
t. The resented lgorithms nd techniques re lso lic ble to one dimension
l sign ls, i.e., sequences f = {fk }N−1 . k=0
96
4 More Alic tions
To motiv te the need of comression techniques,
we st rt with simle ex mle.
A wide s re d im ge form t is the “SI form t” with resolution of M = 352 rows nd
N = 288 columns. It is used, for ex mle, in low cost web c mer s. A true color
or re sons exl ined in sect. 3.4.3 this onl roxim tel will be true, if f
T is gener ted b J ste DWT.
98
4 More Alic tions
with n line r nd invertible tr nsform. In the “old” version of the JPEG still im
ge comression st nd rd [39], for ex mle, f T resulted
from blockwise lic
tion of the “discrete cosine tr nsform” (DCT) [39]. In ig. 4.1 we disl both f T
v ri nts. The result of J ste DWT g in is rr nged s indic ted in eq. 3.37
. Here J = 2 w s chosen.
DCT
DCT
DCT
f2 d2v
d2h d1h d2d
d1v
d1d
ig. 4.1. Tr nsformed im ges f T for JPEG “old” (left) nd JPEG 2000 (right).
As indic ted lre d in sect. 3.4.4, JPEG 2000 recommends the use of ˜ biorthogo
n l lter ir H, H with 9 nd 7 coe cients, resectivel. 2. Q denotes “qu ntiz tion”.
Here interv ls of the v lues of f T entries re reresented b single v lue (
so c lled “smbol”) Sj . If the interv ls
h ve const nt length, the qu ntiz tion i
s c lled “uniform”. An ex mle is shown in ig. 4.2. Ever f T entr is subject to
such qu ntiz tion le ding to m trix f Q of size M × N . Since the m [qj , qj
+1 ) → Sj is m n to one, qu ntiz tion is loss ste. In f ct, the “reconstruction
error” f dec − f ctu ll is qu ntiz tion error, since qu ntiz tion is the onl l
oss ste t tr nsform comression. Thus in eq. 4.8 the smbol Q−1 does not denote
ex ct inversion of the q ntiz tion ste Q but r ther is suit ble rescrition
, to choose v lue in the interv l [qj , qj+1 ), when Sj is given. Note th t th
e trunc tion rescrition discussed in sect. 3.6.1 (see ge ˜ ˜ 84) ctu ll is q
u ntiz tion, reresenting the interv l (−f limit , f limit ) b the smbol 0. 3. E
C denotes n “entro coder”, which encodes the smbol m trix f Q with s few bits
s ossible resulting in the comressed sign l f c . Note th t, whichever w the
entro coder is imlemented, entro coding is invertible, therefore the tr ns
ition from f Q to f c nd vice vers is lossless: f Q ↔ f c.
4.1 The Tr nsform Comression Scheme
99
TQ f
mn
Sj
qj
qj+1
E
T fmn
(4.11)
H s tis es the inequ lities 0 ≤ H(1 , . . . , n ) ≤ log2 n. for 1 = 2 = . . . = n
= the lower bound for i = 1i=k 0i=k (4.14) 1 , MN (4.13) (4.12)
We exlicitl note two imort nt f cts: The uer bound will be re ched
Both equ tions illustr te n imort nt sect of entro: If ll smbols occur w
ith equ l frequenc (eq. 4.13), H will re ch its m ximum; if ctu ll onl smbo
l Sk occurs in f Q (eq. 4.14), H will re ch its minimum. This m be gener lized
b the following observ tion: H me sures the concentr tion roerties of the hi
stogr m counting the occurrences of the smbols Si (i = 1, . . . , n). If the hi
stogr m is t, H will be l rge, if the histogr m is sh
r l e ked, i.e., concent
r ted round rel tivel few s mbols, H will be sm ll. or storing nd rocessing
f Q on digit l comuter, we need “bin r code”, i.e., n invertible m of the
s mbols to sequences of two v lued bits. Usu ll the two ossible v lues of bi
t re reresented s 0 nd 1, resectivel. A bin r sequence reresenting sm
bol will be c lled “codeword”. The length of the codeword reresenting smbol Si w
e denote with li . A bin r coding of three smbols is shown in T ble 4.1. As in
this ex mle, the length of the codewords in gener l m v r, code with this
roert is c lled “v ri ble length code”. We s w bove th t e ch smbol Si will
e r in f Q with rel tive frequenc i resulting from eq. 4.9. So we c n comu
te n “ ver ge codeword length” l corresonding to
4.1 The Tr nsform Comression Scheme T ble 4.1. A bin r code for three smbols.
Smbol codeword S1 S2 S3 1 01 00
101
n
l=
i=1
i li .
(4.15)
A fund ment l theorem of inform tion theor [29] st tes th t for n rbitr r bi
n r code the following inequ lit will be true: l ≥ H(1 , . . . , n ). (4.16)
Thus, the entro comuted ccording to eq. 4.11 is lower bound for the ver g
e codeword length nd bin r code, which h s the roert th t l ≈ H(1 , . . .
, n ), is c lled “entro codec”. Such codecs3 lso re c lled “otim l codecs”, since
it is not ossible to reresent the smbols (in the ver ge) b shorter codeword
s. We sh ll see below th t lgorithms for entro coders c n be given, so we m
extr ct “comression recie” from wh t we h ve collected so f r:
1. Tr to reduce entro b choosing tr nsform T such th t the T histogr
m of
fmn v lues is concentr ted bout rel tivel few v lues (cf., e.g., ig. 3.25).
2. Tr to reduce entro even more b ling roer qu ntiz Q tion rule such
th t the histogr m of fmn v lues is sh rer e ked T th n the fmn histogr m.
3. Al n entro codec to f Q resulting in the comressed sign l f c. As co
nsequence, the comression r te C de ned in eq. 4.3 m be estim ted s C≈ b . H(1
, . . . , n ) (4.17)
The re son is th t the le size of Sf c of f c m be estim ted s Sf c = M N l ≈ M
N H(1 , . . . , n ), where s the le size of the origin l re ds Sf = M N b (cf.
eq. 4.2, b denotes the number of bits needed to store ixel of the origin l im
ge).
3
The term “codec” is comosed from coding nd decoding stressing code invertibilit.
102
4 More Alic tions
We sh ll now sketch two entro codecs frequentl used in im ge comression. Bot
h lgorithms re described in, e.g., [36]. The re der m consult [36] lso for
further rel ted references. The Hu m n codec Given the smbols {S1 , . . . , Sn }
nd the corresonding rel tive frequencies {1 , . . . , n } the lgorithm cons
tructs bin r tree, whose le ves corresond to the smbols. The construction
roceeds s follows: 1. De ne the “le ves” of the tree b utting ll smbols in row.
Inste d of le f we will use the term “node” from now on. E ch node will be l belled
with the corresonding rel tive frequenc i . 2. Merge the two nodes with the
sm llest rob bilities (“children”) into one node (“ rent”), l belled with the sum of t
he two rel tive frequencies. Reresent the merging rocess b gr h connecting
both children with the rent node. 3. Ree t ste 2 with the rent node(s) ge
ner ted so f r nd the rem ining le ves. 4. Sto, when the l st node h s been ge
ner ted b the merging rocess. This node is c lled “root”, obviousl it will be l b
elled with 1, since it corresonds to the sum over ll rel tive frequencies. 5.
Code e ch smbol b the th from the root to the corresonding le f, thereb co
ding e ch “left turn” with 0, e ch “right turn” with 1. We illustr te the rocedure b c
onstructing the Hu m n code for the smbol string ABBC. The rel tive frequencies r
e d A = 1 1 1 , B = , C = 4 2 4
The tree building rocess discussed bove m be illustr ted s follows: C 1 A 1
B 2
2
4
4.1 The Tr nsform Comression Scheme
103
Note th t here the nodes re l belled with bsolute r ther th n rel tive frequen
cies. This m kes no di erence, the root node for this ex mle then necess ril is
l belled with 4. The resulting code re ds:
T ble 4.2. Hu m n code for ABBC. Smbol codeword C A B 00 01 1
We n ll check th t for this ex mle the ver ge codeword length is determined b
the entro. The entro is comuted s 1 H(A , B , C ) = − log2 4 = 1.5 1 4 1
log2 2 1 2 1 log2 4 1 4
−
−
The coded string re ds 011100. Therefore, the ver ge codeword length results fr
om 6 4 = 1.5
lHuf f m n =
So indeed the Hu m n code re ches the lower bound from eq. 4.16. Obviousl e ch Hu m
n code will le d “codebook” ssoci ting to e ch smbol the corresonding codeword
s in T ble 4.2. So the minimum length of e ch codeword will be 1 nd therefore
, reg rdless of the smbol rob bilities, the ver ge codeword length of n rbi
tr r Hu m n code will ful ll lHuf f m n ≥ 1. (4.18)
On the other h nd, the smbol rob bilities {1 , . . . , n } m be such th t
entro s tis es H(1 , . . . , n ) < 1. (4.19)
In f ct, if in tr
nsform comression scheme the tr nsform T is J steDWT, or
two dimension l D T,
fter qu ntiz tion the resulting histogr m m be so sh
r l concentr ted (cf. ig. 3.25) such th t eq. 4.19 will hold.
104
4 More Alic tions
In this c se obviousl Hu m n coding c nnot be otim l, since it will not re ch th
e lower bound from eq. 4.16. An coding scheme re ching this bound, therefore ne
cess ril will ssoci te the smbol string stored in f Q comletel to bin r
codeword. Below we sh ll describe such coding rocedure. Arithmetic Coding Giv
en the smbols {S1 , . . . , Sn } nd the corresonding rel tive frequencies {1
, . . . , n } the lgorithm will code the given smbol string (in tr nsform
comression scheme this string m result, e.g., from rowwise sc n of f Q ) b
bin r sequence, which we sh ll c ll “string codeword” below. We will describe no
w the rocedure. 1. St rt : Subdivide the interv l [0, 1) into non overl ing i
nterv ls of length i (i = 1, . . . , n). Associ te e ch smbol Si with the corr
eonding subinterv l. 2. Selection: Select the subinterv l corresonding to the r
st smbol of the string. 3. Ex nsion: Decomose the selected interv l into subi
nterv ls ssoci ted to the individu l smbols comletel n logousl to ste 1.
4. Ree t selection nd ex nsion until the l st smbol of the string h s been
rocessed. In this w we will obt in n l interv l [u, o). Now choose number i
n this interv l such th t its bin r ex nsion4 is s short s ossible. This bi
n r ex nsion will be the string codeword. Ag in we sh ll illustr te the roced
ure b determining the rithmetic code of ABBC with rel tive frequencies A = St
rt : B 0 Selection: B 0 Ex nsion: B
1 2
4
1 1 1 , B = , C = . 4 2 4 A C
3 4
1 2
1 C
A
1 2 3 4
1 C
A
5 8 11 16
3 4
Re ders not f mili r with comuting the bin r ex nsion of decim l number q w
ith 0 ≤ q < 1 re referred to sect. 4.3, eq. 4.35, for n ex mle.
4.1 The Tr nsform Comression Scheme
105
Selection: B
1 2 5 8
A
11 16
C
3 4
Ex nsion: B
1 2 18 32
A
19 32
C
5 8
Selection: B
1 2 18 32
A
19 32
C
5 8
Ex nsion: B
1 2 34 64
A
35 64
C
18 32
Selection: B
1 2 34 64
A
35 64
C
18 32
We obt in the n l interv l [u, o) = 35 18 , 64 32 .
The bin r ex nsion of u = 35 re ds .100011. An logousl, the bin r 64 ex nsi
on of o = 18 is given b .1001. 32 So the string codeword reresenting ABBC re d
s 100011. The ver ge codeword length is comuted s 6 4 = 1.5
lAC =
Here, obviousl rithmetic coding nd Hu m n coding both re otim l, since their
resective ver ge codeword lengths re given b the entro (see bove). We sh
ll now consider the following string: A B...B
n−1
(4.20)
times
106
4 More Alic tions
The string will h ve tot l number of n smbols nd therefore the rel tive freq
uencies re d A = 1 , n B = n−1 . n
If n is l rge enough, entro will ful ll H(A , B ) < 1 nd we sh ll investig te
how rithmetic coding erforms. To do so, we need the following result: Arithme
tic coding of the smbol string 4.20 will le d to the n l interv l [u, o), where
u= 1 n 1− 1− 1 n
n−1
,
o=
1 . n
(4.21)
To rove this, we consider T ble 4.3, where we st rt with u = 0 nd o = 1. E ch
selection will ud te u, o nd the width of the ctu l interv l; in the t ble th
ese ud tes re tr cked.
T ble 4.3. Arithmetic coding of string 4.20. Smbol u A B B B . . . 0
1 n2 1 n2 1 n2
o Interv l width
1 n 1 n 1 n (n−1)2 1 n n4 1 n n−1 n2 (n−1)2 3 n (n−1)3 n4
+ +
. . .
n−1 n3 n−1 n3
+
. . .
. . .
Thus n ll 1 n2 1 n2 (n − 1)n−2 n − 1 (n − 1)2 + + ...+ n n2 nn−2 1 n + 1− 1 n
2
u= =
1+
1+ 1−
+ ...+ 1−
1 n
n−2
1 1− 1− =2 1 n n = 1 n
1 n−1 n
1− 1−
1 n
n−1
rom the second to the third line we used the geometric l ex nsion formul
4.1 The Tr nsform Comression Scheme
n
107
qk =
k=0
1 − q n+1 (q = 1). 1−q
As n ex mle we stud the smbol string ABBBBBBB with rel tive frequencies 1 7
A = , B = . 8 8 The entro re ds 1 H(A , B ) = − log2 8 = 0.5436 1 8 − 7 log2 8
7 8
rom eq. 4.21 the bound ries of the n l interv l re comuted s: [u, o) = 185 1
, 2437 8
185 The bin r ex nsion of u = 2437 is given
b .000100110110111100001001. 1 Co
rres ondingl , o = 8 is ex nded s .001. rom both ex nsions we obt in for the
string codeword the bin r sequence 00011. Therefore the ver ge codeword lengt
h re ds
lAC =
5 8 = 0.6250
In contr st, since the Hu m n code encodes A with 1 nd B with 0 (or vice vers ),
we h ve lHuf f m n = 1. So rithmetic coding will be much closer to the lower bo
und from eq. 4.16 th n Hu m n coding. Since the interv ls gener ted throughout the
coding rocedure re nested, decoding works s follows: Subdivide the interv l
[0, 1) s described bove nd determine the subinterv l cont ining the number s
soci ted with the string codeword. The corresonding smbol will be the rst eleme
nt of the decoded smbol string. Then t ke this interv l, subdivide it g in, n
d g in determine the subinterv l cont ining the bove number. This will roduce
the second element of the decoded string. Proceed in this w nd roduce the n
ext smbols. Since this rocess will not termin te, the tot l number of smbols
must be tr nsmitted in order to sto the decoding rocess fter the l st smbol.
108
4 More Alic tions
4.1.3 Otim l Qu ntiz tion nd Ex mles We return now to the tr nsform comressi
on scheme described in eqs. 4.7 nd 4.8. In sect. 4.1.1 we s w th t sign l disto
rtions exclusivel c n be ttributed to qu ntiz tion nd in this section we sh l
l investig te how qu ntiz tion distortion m be reduced. Given the origin l f ,
we re interested in the distortion of the decomressed sign l f dec . We s w
bove th t entro coding is reversible, therefore we m combine equ tions 4.7
nd 4.8 such th t we obt in f dec from f s follows: Comression: f −→ T −→ Q −→ Q−1 −→ T −1
“norm dist nce” D := f dec − f
dec
(4.22)
(4.23)
with resect to f , which w s introduced in eq. 4.5, me sures disof f tortion. A
high D v lue corresonds to l rge, low
D v lue to sm ll sign l distortion. Wh
t re the r m ters, D deends on? In ig. 4.2 we s w th t qu ntiz tion rule
corresonds to iecewise const nt function, which uniquel is determined when
the qu ntiz tion r meters qj nd the levels Sj on the resective interv ls [q
j , qj+1 ) re known. This function will be c lled “qu ntiz tion function”. Then we
observe the following: Assume th t there re n + 1 qu ntiz tion r meters q1 ,
. . . , qn+1 . Assume moreover th t there exists rescrition, how the levels
Si re comuted from the qu ntiz tion r meters (for n ex mle see below). ur
thermore, ssume th t cert in choice of the tr nsform T h s been m de. Then di
stortion s determined in eqs. 4.22 nd 4.23 will deend onl on the qu ntiz tio
n r meters: D = D(q1 , . . . , qn+1 ) (4.24)
In this section we sh ll use the following rescrition to comute the levels (“s
mbols”) Sj : qj + qj+1 (j = 1, . . . , n) (4.25) 2 The qu ntiz tion function is co
nstructed s follows: On the interv ls [qj , qj+1 ) (j = 1, . . . , n) the funct
ion v lue will be Sj (j = 1, . . . , n), on [qn+1 , ∞) the function will rem in on
Sn , on (−∞, q1 ) the function v lue Sj =
4.1 The Tr nsform Comression Scheme
109
is S1 . So fter qu ntiz tion the m trix elements of f Q will h ve v lues5 t ken
from the smbol set S1 , . . . , Sn . As mentioned in the receding section, ro
wwise sc nning of f Q will roduce smbol string, where e ch Si will e r wi
th rel tive frequenc i . Since the smbols uniquel re comuted from the qu
ntiz tion r meters s in eq. 4.25, the resulting entro H (cf. eq. 4.11) ct
u ll will deend on these r meters: H = H(q1 , . . . , qn+1 ) (4.26)
Distortion is minimized, if q1 , . . . , qn+1 re chosen such th t D(q1 , . . .
, qn+1 ) is minimized. On the other h nd, eq. 4.17 tells us th t t the s me tim
e H(q1 , . . . , qn+1 ) should be ket s sm ll s ossible in order to obt in
l rge comression r te C. So, the over ll otimiz tion go l m be formul ted
s follows: Determine q1 , . . . , qn+1 such th t the t rget function t(q1 , . .
. , qn+1 ) is minimized, where t(q1 , . . . , qn+1 ) = λD(q1 , . . . , qn+1 ) + (1
− λ)H(q1 , . . . , qn+1 ) (4.27) The parameter λ is ca ed “distortion weight”, (1−λ) corr
onding y “entropy weight”. So, the target function is a weighted average of distorti
on and entropy and it depends on the choice of λ, which of the con icting “sub goa s” (d
istortion reduction vs. compression e ciency) is favored. It is c ear that there i
s no simp e ana ytica expression for t. The optimization goa formu ated above,
therefore, must be so ved numerica y. A numeric 0 0 procedure a ways wi star
t from an initia set (q1 , . . . , qn+1 ) and iterative y k k wi produce (q1
, . . . , qn+1 ) (k = 1, 2, . . .) converging to the desired va ues (q1 , . . .
, qn+1 ), where t is minima . In sect. 4.1.4 we brie y indicate which procedure ha
s been used. Before discussing Figs. 4.3 – 4.5, where resu ts of numeric optimizat
ion are disp ayed, we make two remarks: 1. The initia set is computed according
to
0 T 0 0 T q1 = min(fmn ) < q2 < . . . < qn+1 = max(fmn ) mn mn
5
(4.28)
For comp eteness we mention that in eq. 4.22 we sha omit dequantization Q−1 . Th
us, the inverse transform T −1 direct y wi be app ied to f Q . Since Sj is compu
ted according to eq. 4.25, actua y we perform a dequantization by se ecting the
midpoints of each quantization interva [qj , qj+1 ).
110
4 More App ications
q0 −q0
0 0 with qj+1 − qj = n+1 1 (j = 1, . . . , n). So we take the sma est and the n
argest e ement of the transformed image f T (cf. eq. 4.7) and subdivide the resu
ting interva in n subinterva s of equa ength. Therefore the initia quantiza
tion wi be uniform (cf. sect. 4.1.1). 2. For T we choose a J step DWT. Thus, t
he transformed image f T is constructed as depicted in the right diagram in Fig.
4.1 for J = 2. We sha a ow for separate quantization ru es for the approxima
tion signa f J and the detai signa s. So, in genera the numbers of quantizati
on eve s for the approximation image f J and the detai signa s, respective y,
wi di er.
The upper row of Fig. 4.3 shows the origina f ( eft) and the decompressed image
f dec (right). The ower eft diagram disp ays the quantization function for th
e approximation signa f J , the ower right diagram the quantization function f
or the detai signa s. Here four quantization eve s were chosen for the approxi
mation signa , three quantization eve s for the detai signa s, respective y. T
he gure shows the resu ts for the uniform start quantization.
Origina
N
app
: 4, N
detai s
: 3, J: 2, db2, PSNR: 15.39dB, bpp: 0.3019.
50 100 150 200 100 200 300
50 100 150 200 100 200 300
800 600 400 200 0 0 200 400 600 800
300 200 100 0 −100 −200 −200 0 200
Fig. 4.3. Origina (above eft) and reconstruction (above right) for start quant
ization. Be ow eft: Uniform initia quantization of approximation signa , Be ow
right: Uniform initia quantization of detai signa s.
4.1 The Transform Compression Scheme
111
It is c ear that due to the ow number of quantization eve s distortion wi be
arge. Distortion was measured by the PSNR (cf. eq. 4.6) (remember that arge d
istortion corresponds to a ow PSNR va ue). Since entropy estimates the average
codeword ength of a symbo , H a so is disp ayed. It is speci ed as the number of “b
its per pixe ” (bpp). The respective va ues in Fig. 4.3 are PSNR(f, f dec ) = 15.3
9 dB and H = 0.3019 bpp. The origina was represented with b = 8 bits per pixe .
So, we have an estimated compression rate of b C = H = 26.5. Before discussing
Figs. 4.4 and 4.5, we note that the respective subimages are arranged as in Fig.
4.3. Moreover, PSNR and bpp va ues are determined ana ogous y. Figure 4.4 show
s the resu ts obtained when starting from the initia quantization from Fig. 4.3
and stopping after 60 iterations. In the target function 4.27 λ = 1 was chosen, i
.e., the entropy weight was zero. So we on y aimed at minimizing distortion with
out taking entropy into account.
Origina
Napp: 4, Ndetai s: 3, J: 2, db2, PSNR: 18.20dB, bpp: 0.3139. 50 100 150 200
50 100 150 200 100 200 300
100
200
300
800 600 400 200 0 0 200 400 600 800
300 200 100 0 −100 −200 −200 0 200
Fig. 4.4. Origina (above eft) and reconstruction (above right) after 60 functi
on eva uations, entropy weight: 0. Be ow eft: Resu ting non uniform quantizatio
n of approximation signa , Be ow right: Resu ting non uniform quantization of de
tai signa s.
112
4 More App ications
Obvious y, the visua impression of f dec is much better now than in Fig. 4.3. T
his is re ected a so by the PSNR: Now we have PSNR(f, f dec ) = 18.20 dB. As expec
ted, the bpp va ue is s ight y arger than in Fig. 4.3: It reads H = b 0.3139 bp
p corresponding to an estimated compression rate C = H = 25.5. Note that the res
u ting quantization functions deviate from uniform quantization both for the app
roximation image and the detai signa s! In the na examp e we again started from
the initia quantization from Fig. 4.3 and stopped after 60 iterations. But now
in the target function 4.27 λ = 0 was chosen, i.e., the distortion weight was zer
o. So we on y aimed at minimizing entropy (maximizing compression rate) without
taking distortion into account. The resu ts are disp ayed in Fig. 4.5.
Origina
Napp: 4, Ndetai s: 3, J: 2, db2, PSNR: 15.19dB, bpp: 0.1572. 50 100 150 200
50 100 150 200 100 200 300
100
200
300
800 600 400 200 0 0 200 400 600 800
300 200 100 0 −100 −200 −200 0 200
Fig. 4.5. Origina (above eft) and reconstruction (above right) after 60 functi
on eva uations, entropy weight: 1. Be ow eft: Resu ting non uniform quantizatio
n of approximation signa , Be ow right: Resu ting non uniform quantization of de
tai signa s.
Indeed now a ow bit per pixe va ue is obtained: We have H = 0.1572 bpp b corre
sponding to an estimated compression rate C = H = 50.9! But for the dec PSNR va
ue we now obtain PSNR(f, f ) = 15.19 dB. So it decreased with respect to the ini
tia quantization. As expected, the decompressed image f dec deteriorates when c
ompared with the initia quantization. It is instructive to
4.1 The Transform Compression Scheme
113
study the quantization curves. Above we saw that ow entropy va ues correspond t
o sharp y concentrated histograms. Such histograms can be reached by generating
wide quantization interva s. The corresponding symbo s then wi appear frequent
y eading to concentrated histograms. Compare the quantization curves for the d
etai signa s in Fig. 4.3 and Fig. 4.5, respective y! Here this broadening e ect i
s c ear y visib e. Note a so that the origina 4 eve quantization of the approx
imation signa has turned basica y to a 3 eve quantization, i.e., essentia y
on y three symbo s remained! This again is in accordance with the above remarks
about histogram concentration. 4.1.4 MATLAB Imp ementation In this section we b
rie y describe a MATLAB imp ementation of the quantization optimization out ined i
n the preceding section. The re ated functions require at minimum the MATLAB Wav
e et Too box and have been deve oped and tested with MATLAB 6.5, re ease 13. The
y may be down oaded from the URL given in sect. 1.3. Again, as mentioned ear ier
, a basic understanding of MATLAB syntax is required. Quantization optimization
is invoked by the fo owing prototype function ca : [q_approx_signa ,q_detai _s
igna ]=optshe (f); The origina image matrix is stored in f. It must be provide
d as a M × N matrix with doub e entries. The parameters characterizing the transfo
rm T are se ected interactive y. These are decomposition depth J and se ected wa
ve et string ’dbn’, where n = 1, 2, . . . (refer to sect. 3.4.3). Moreover, for the
approximation signa and the detai signa s the respective number of quantizatio
n eve s (“symbo s”) must be speci ed. The respective initia quantization parameters
are then computed corresponding to eq. 4.28. In this formu a f T must be rep ace
d by the approximation image f J in order to obtain the start quantization of th
e approximation signa ; for the start quantization of the detai signa s a corre
sponding rep acement has to be done. The quantization parameters qi resu ting fr
om optimization are returned in vectors q_approx_signa and q_detai _signa for
approximation signa and detai signa s, respective y. As mentioned in the prece
ding section, beginning from the start quantization iterative y new quantization
parameters are computed such that the resu ting va ues of the target function 4
.27 are essentia y descending towards a minima va ue (if it exists). The funct
ion optshe a ows for choosing from two iteration methods. One method is the “Ne
der Mead” a gorithm [19]. It is imp emented in the MATLAB bui t in function fmins
earch, which is invoked if Ne der Mead is se ected. The other method which can b
e se ected, is the “Quasi Newton” a gorithm [2]. The se ection invokes the function
fminunc, which imp ements this a gorithm. This function requires the insta atio
n of the
114
4 More App ications
MATLAB Optimization Too box. It therefore cannot be used if the MATLAB Optimizat
ion Too box is not avai ab e. Figures 4.3 – 4.5 were obtained by invoking fminsear
ch. Be ow we show a resu t invoking fminunc. Here we chose ve quantization eve s
for the approximation signa , four quantization eve s for the detai signa s.
Figure 4.6 shows the resu ts for the uniform start quantization.
N : 5, N : 4, J: 1, db2, PSNR: 16.99dB.
Origina
app
detai s
50 100 150 200 100 200 300
50 100 150 200 100 200 300
400 100 300 200 100 0 0 100 200 300 400 0 −100 −200 −200
−100
0
100
Fig. 4.6. Origina (above eft) and reconstruction (above right) for start quant
ization. Be ow eft: Uniform initia quantization of approximation signa , Be ow
right: Uniform initia quantization of detai signa s.
Figure 4.7 shows the resu ts after a certain number of iterations. As in Fig. 4.
4, entropy weight was set to zero. Referring to the target function 4.27, theref
ore, we aimed at reducing distortion without taking entropy into account. The vi
sua improvement corresponds to a c ear improvement in PSNRva ues: In Fig. 4.6 w
e have PSNR(f, f dec ) = 16.99 dB, in Fig. 4.7 PSNR(f, f dec ) = 19.87 dB was ob
tained. The bpp va ues were as fo ows: For Fig. 4.6 H = 1.6637 bpp was measured
, corresponding to an estimated compression rate b C = H = 4.81. In Fig. 4.7 we
have H = 0.7182 bpp corresponding to a b compression rate C = H = 11.14.
4.1 The Transform Compression Scheme
Origina 50 100 150 200 100 200 300 N
app
115
: 5, N
detai s
: 4, J: 1, db2, PSNR: 19.87dB.
50 100 150 200 100 200 300
400 100 300 200 100 0 0 100 200 300 400 0 −100 −200 −200
−100
0
100
Fig. 4.7. Origina (above eft) and reconstruction (above right) after optimizat
ion run, entropy weight: 0. Be ow eft: Resu ting non uniform quantization of ap
proximation signa , Be ow right: Resu ting non uniform quantization of detai si
gna s.
Thus, in this experiment not on y distortion was reduced, optimization even ead
to an improved compression performance! We c ose this section with some genera
remarks concerning the optimization imp ementation. As stated above, the numeri
ca procedures discussed in this section iterative y wi produce quantization p
arameters such that the corresponding target function va ues wi be descending,
hopefu y towards a oca minimum situated in a neighborhood of the start quant
ization. Since no a priori information about the “shape” of the target function t(q1
, . . . , qn+1 ) (cf. eq. 4.27) is given, it is not guaranteed that such a mini
mum exists. If it exists, it is not guaranteed that it wi be reached within re
asonab e computing time. So, one usua y imits the maximum number of iterations
and wi be satis ed if, within this number of iterations, the target function va
ue wi decrease reasonab y. In sect. 4.3 we sha make some na comments on the
optimization subject.
116
4 More App ications
4.2 Wave et Based Simi arity Retrieva in Image Archives
In this section we report on an industria project performed by the author at te
cmath AG. The work has been pub ished in [31, 32]. In the fo owing we summarize
the main resu ts. Conventiona digita image database management systems usua
y are structured as fo ows: Each stored image e is described by a text document
, which contains administrative data ike date and time of image capture. Moreov
er, this document inc udes we de ned terms and phrases describing image contents
. When a user of the database queries for images re ated to a certain topic, he
or she might enter corresponding keywords. These keywords can then be used for q
uery mechanisms ike, e.g., fu text retrieva . So the resu t of the research w
i ead to a ist of documents, whose content description parts contain the key
words. The retrieved documents are inked to the corresponding images, which the
n may be inspected. It is we known that a verba description of images often i
s inadequate, since the description stored in the database management system mig
ht not match the expectations of the user. Moreover, it is not possib e to “presen
t” an examp e image to the system and to ook for a images “simi ar” to the presente
d image when conventiona image archives as described above are used. Image data
base management systems a owing for such retrieva mechanisms are ca ed “content
based access systems” (CBA systems). It is c ear that in view of mu timedia techn
o ogy CBA systems are of growing importance. In order to bui d systems “ ooking fo
r simi ar images”, it is necessary to give a precise meaning to the notion of “simi
arity” and we sha describe a simp e mode based on a J step DWT. As a preparatio
n, we sha exp ore in some more detai , how a J step DWT of an image may be use
d for measuring the appearance of edges. Remember that the resu t of a J step DW
T of an image (cf. eq. 3.36) usua y is arranged as indicated in eq. 3.37. As an
examp e consider the origina f shown in Fig. 4.8. A 7 step DWT, performed with
db2 is visua ized in Fig. 4.9. The detai signa s djh , djv , djd (j = 1, . . .
, 7) c ear y con rm the directiona sensitivity mentioned in sect. 3.4.2: Since t
he origina shows a damped periodic pattern of horizonta stripes, vertica and
diagona detai signa s are c ose to zero. Moreover, we note that the detai sig
na s wi “react” in particu ar for j va ues such that the associated sca e factors
2j (measured in pixe s) approximate y correspond to stripe width6 . The reaction
may be measured by computing the detai signa variance7 . This number estimate
s the average deviation of the detai signa from its mean va ue and therefore w
i be arge for
6 7
As anexamp e consider d3h in Fig. 4.9 We remind
the reader that the variance σ 2
of a ata et {y1 , . . . , yn } i compute n n 1 1 a follo : σ 2 = n−1 i=1 (yi − y
)2 , here y = n i=1 yi .
4.2 Wavelet
Ba e Similarity Retrieval in Image Archive
Perio : 16 Pixel
117
50 100 150 200 250 300 350 400 450 500 100 200 300 400 500
Fig. 4.8. 512 × 512 Original. Stripe i th: 16 pixel .
etail
ignal ith large o cillation about their mean. For a J tep DWT e ha
ll enote variance ith
2 2 2 2 2 2 σ1h , σ1v , σ1 , . . . , σJh , σJv , σJ .
(4.29)
For three te t image ith
increa ing tripe
i th a 7 tep DWT ith the 2 b2
been performe . Thecorre
avelet ha pon ing σjh value are plotte in Fig. 4.10.
The rea er may con rm that in ee the maximum of the e value belong to uch a j
value that the corre poning cale factor
2ji approximately equal to the re p
ective tripe i th. In a ition, cale epen ent ani otropy mea ure are e ne a
follo : aj =
2 σ1v 2 (j = 1, . . . , J) σ1h
(4.30)
They in icate, if tructure of ize ≈ 2j are preferably vertically (aj > 1) or ho
rizontally (aj < 1) oriente .
118
4 More Application
7− tep−DWT. Wavelet: b2
50 100 150 200 250 300 350 400 450 500 100 200 300 400 500
Fig. 4.9. 7 tep DWT of the image i playe in 4.8. Wavelet: b2.
Summarizing, the follo ing proce ure i ugge te : Given an image f , e ne the “pat
tern vector”
2 2 2 2 2 2 xf = σ1h , σ1v , σ1 , a1 , . . . , σJh , σJv , σJ , aJ .
(4.31)
2 The entrie σjh in icate hether preferably horizontally oriente tructure of
ize ≈ 2j (in pixel ) are pre ent in f .Analogou ly, the remaining variance are
interprete . Moreover, the entrie
aj in icate ani otropy a tate above. T o i
mage f1 an f2 are con i ere to be “ imilar”, if xf1 i clo e to xf2 in a uitable
en e ( ee eq. 4.32).
The “ i tance” of t o pattern vector xf1
an xf1 , extracte from the original imag
e f1 an f2 , re pectively, i mea ure by the norm xf1 − xf2 , hich i compute
a follo ( ee al o eq. 4.5):
4.2 Wavelet
Ba e Similarity Retrieval in Image Archive
Perio : 4 100 200 300 400 500 100 200 300 400 500 100 200 300 400 500 100 200 30
0 400 500 Perio : 16 100 200 300 400 500 Perio : 64
119
100 200 300 400 500
0.4 0.3 0.2 0.1 0 0
4 3 2
50 40 30 20
1 0 0
10 5 10 0 0 5 10
5
10
Fig. 4.10. Top ro : Original image ith (fromleft) a tripe i th of 4, 16 an
2 2 64 pixel , re pectively. Belo
: corre pon ing plot of σ1h , . . . , σ7h re ult
ing from a 7 tep DWT ith the b2 avelet.
4J
xf1 − xf2 =
i=1
(xf1 − xf2 )2 . i i
(4.32)
Here xf1 (i= 1, . . . , 4J) enote the i th entry of the pattern vector xf1 ,
i corre
pon ingly xf2 i e ne . i Ba e on thi imilarity mea ure, in [31], [32]
a igitalimage archive i propo e that allo for retrieving image “ imilar” to
a pre ente
image.
In the re t of thi ection e e cribe archiving an retriev
al proce ure an i cu example 8 . When enteringan image f into the ataba e
management
y tem, in a ition to manual input
of e criptive
an /or a mini tra
tive ata the pattern
vector xf i compute , tore an linke
to the image. Whe
n querying the ataba e for image imilar to a pre ente image g ( imilarity re
trieval ), the follo ing tep are performe :
8
Figure 4.11 – 4.13 publi he ith kin permi ion of tecmath AG.
120
4 More Application
1. Compute the pattern vector xg . 2. Generate an “an er li t” by orting the tore
image fi uch that xg − xf1 ≤ xg − xf2 ≤ xg − xf3 ≤ . . . 3. Pre ent the an er li t f1
f2 , f3 , . . . to the u er. An example i ho n in Fig. 4.11 (4.33)
100
200
300
400
500
600
700 100 200 300 400 500 600 700 800 900 1000
Fig. 4.11. Similarity retrieval.
Here, the top ro ho (from left) g an fi , analogou ly the bottom ro ho
from left fi+1 an fi+2 .The u er may croll through thi li t tarting ith f1
. Thi i the ituation i playe here: Ro i e from left to right image g, f1
, f2 an f3 are ho n. The te t archive un erlying Fig. 4.11 con i t of ≈ 300 im
age . The archive exhibit a large variety
concerning image content : The image
ho natural cene , have been recor e in a televi ion tu io, re ult from me
ical application (magnetic re onance imaging) an o on. A re ult of a query fo
r magnetic re onance image imilar to a given image
g i ho n in Fig. 4.12. It
again ho g, f1 , f2 an f3 hich are arrange a in Fig. 4.11.
4.2 Wavelet Ba e Similarity Retrieval in Image Archive
121
50 100 150 200 250 300 350 400 450 100 200 300 400 500 600
Fig. 4.12. Another retrieval re ult.
It i clear that there might be top ranke image among the r t po ition of the
an er et, hich are not relevant for the u er. In thi ca e the retrieval iter
atively can be improve by “relevance fee back” [28]. We e cribe no the proce ure,
again the given image ill be enote ith g. 1. Let the u er qualify the r t po
ition of the an er li t a “relevant” or “irrelevant”. Denote ith fmi the “mo t mi le
a ing” irrelevant image, i.e., the irrelevant image having the topmo t po ition in
the an er li t. Furthermore, enote ith Arel the et of relevant image . 2. S
tarta ne imilarity retrieval run by replacing pattern vector xg ith xne com
pute accor ing to xne = xfmi xg − fmi + g x x xfi . xfi (4.34)
f i ∈Arel
3. In pect the ne ly generate an er li t an procee
ith tep 1, if nece ary
. In formula 4.34, for example, xg i compute accor ing to
122
4 More Application
4J
xg =
i=1
(xg )2 i
(cf. eq. 4.32). Analogouly the other norm quantitie
are etermine . Thi proce
ure ha been implemente
in the y tem.Ba e on the retrieval re ult from Fig.
4.12 relevance fee back ha been carrie out. To generate the ne retrieval run
image f 1 an
f 4 (not ho n in Fig. 4.12)
have been marke a relevant. Thi
can be veri e by checking the corre pon ing tick boxe in the an er li t. So in
thi example fmi = f 2 .
50 100 150 200 250 300 350 400 450 100 200 300 400 500 600
Fig. 4.13. Improvement by relevance fee back.
The ne ly generate
an er li t i i playe inFig.
4.13 ith g, f 1 , f 2 an
f 3 a arrange before. Obviou ly the re ult in ee improve retrieval performan
ce ith re pect to imilarity. We clo e thi ection ith ome nal comment . Of c
our e, given t o image
f an g, it oul be po ible to mea ure their
imilarit
y by putting the ra ata, i.e., the matrix element fmnan gmn , irectlyinto
formula 4.5. The rea on for u ing pattern vector xf an xg
in tea , i re ucti
on of computing time. If the image , for example, are tore in SIF format (i.e.
,
4.3 Note an Exerci e
123
352 × 288) the um in 4.5 ha to be taken over 101376 umman . In contra t, hen
choo ing, e.g., J = 4, the pattern vector ill have 16 component . So the um i
n formula 4.32 exten over only 16 umman . Obviou ly, it i a nece ary prere
qui ite that the pattern vector capture image feature e ential for imilarity
. Moreover, the computing e ort to obtain the pattern vector from the original mu
t be lo . Thi i true for the t o imen ional J tep DWT 3.36. The e ort for comp
uting thi tran form ill increa e only linearly ith the number of matrix eleme
nt of the original. On a PC platform a Pentium II proce or a imilarity r
ith
etrieval
run on the te
t archive e cribe above (con i ting of ≈ 300 image ) requ
ire only a fe econ .
4.3 Note an Exerci e
Note The
Hu man co ec r t a publi he in [13]. It i the entropy co ing cheme
upporte by the “ba eline ver ion” of the “ol ” JPEG till image compre ion tan ar . J
PEG 2000 u e a pecial variant of the arithmetic co ing cheme. Thi variant in
particular allo for enco ing ymbol tring “online”:
It i not nece ary to rea
the total tringbefore the tring co e or i pro uce . Moreover, a progre iv
e tran mi ion
mo e, i.e., image tran mi ion ith better an better re olution,
i upporte . In [40] a realization of an arithmetic
co er on the
ba i of a el
programmable
gate array (FPGA) i e cribe . The co er i a apte to a J tep D
WT. “Rate i tortion
theory” [15] tate
that for a given i tortion any compre
ion
cheme ill pro uce a bpp value boun e belo by a function calle
“rate i torti
on function”,
hich may be compute theoretically. Thu , metho like the optimiza
tion proce ure from ect. 4.1.3 implement a practical approach to thi theoretic
al limit. The
rea er houl keep in min
, ho ever, that the optimization algorit
hm e cribe above primarily i inten e a an eaily acce ible tool for illu t
rating ba ic fact of co ing like the tra eo of i tortion re uction v . compre
ion eciency. So it mainly erve i actic purpo e . In ect. 3.6.1 e alrea y men
tione the nee for “perceptually lo le quantization”. Thi mean that one look
for quantization function uch that all element of a quantization interval[qi
, qi+1 ) are vi ually in i tigui hable (thi applie accor ingly al o to au io
compre ion). It i till an open i ue to e ne a quantitative meaure for“percept
ual imilarity”
of image . Di tortion mea ure like the PSNR are i ely u e , but
in or er to eci e on quantization error on a phy iological ba i, it till i
common practice to pre ent image (or au io ignal ) to human “ju ge ”. Thu , automa
tic optimizer like the one e cribe in ect. 4.1.3 are only one tep in a comp
lex te ting
an a aptation
cenario. For a itional rea ing about imilarityret
rieval an content ba e acce to multime ia archive the rea er i referre to
[25],[27].
124
4 More Application
Exerci e 1. Con tructa Human co e for the ymbol tring ABCCDDEE an compare th
e re ulting average co e or length lHuf f man ith the entropyH(pA , pB , pC ,
pD , pE ). 2. For generating
a co e or ith the arithmetic co er it ill be ne
ce ary to convert a ecimal number q ith 0 < q < 1 to it binary repre entatio
n. For q = 0.7 the algorithm i emon trate belo . 0.7 2 0.4 2 0.8 2 0.6 2 0.2
2 0.4 2 . . . = 1.4 = 0.8 = 1.6 = 1.2 = 0.4 = 0.8 → → → → → → .1 .10 .101 .1011 .10110 .1
00 . . .
(4.35)
So the binary expan ion of 0.7 rea .10110 here the overline enote the perio
ic pattern to be repeate in nitely. Write a MATLAB function ( ample call:DecBin
Conv(q,n)), hich rea a ecimal number q ith 0 < q < 1 an ucce ively i pl
ay the ingle bit of it binary expan ion on the creen!Ue the algorithm ke
tche in eq. 4.35 an limit the maximum number of i playe igit to n. 3. Dete
rmine the arithmetic
co e for the ymbol tring ABCCDDEE an compare the re ulti
ng average
co e or length lAC ith the entropy H(pA , pB , pC , pD , pE ). Hint
: To etermine the electe interval MATLAB’ number format format rat i u eful.
It i play number a fraction of mall integer . 4. Write a MATLAB function
ith ample call ArithBoun ( tr, ymb,freq); String tr i the input tring. It c
on i t of ymbol hich are tore in the tring ymb. Each element of ymb app
ear in tr ith a frequency,
hich i a corre pon ing element of array freq. Th
en the function
houl i play the
binary expan ion of boun arie of the nal int
erval pro uce by arithmetic co ing of tr on the creen! For computing the bina
ry expan ion u e function
DecBinConv ketche above. A an example, the co ing o
f ABBC coul be invoke by ArithBoun (’ABBC’,’BAC’,[2 1 1]); (cf. page 104).
5 Appen ix
5.1 Fourier Tran form an Uncertainty Relation
In
thi ection ba ic e nition an fact concerning Fourier tran form are recall
e
. The main purpo e i to provi e the Fourier tran form notion require forun
er tan
ing the material
covere in chap. 2. For
more etaile
information an f
or a eeper intro uction to the topic the rea er i referre , e.g., to [34]. Let
f (t) enote a continuou time ignal. Moreover, a ume that f (t) i a nite ene
rgy ignal. Thi mean that the follo ing relation hol :
+∞
|f ( )|2 d < ∞.
−∞
For any prac ically relevan signal he ni e energy assump ion is ful lled. Then h
e “Fourier ransform” is de ned as follows:
+∞
ˆ f (ω) =
−∞
f (t)e−jωt dt
(5.1)
Thus, the Fourier trasform is a fuctio depedig o the real variable ω, ˆ called
the “frequecy”. f (ω) agai is a ite eergy fuctio o the frequecy domai. It som
etimes also is called “spectrum” of f (t). The sigal may be recostructed from its
spectrum: 1 f (t) = 2π
+∞
ˆ f (ω)e+jωt dω
−∞
(5.2)
Equatios 5.1 ad 5.2 establish a oe to oe correspodece bet ee a sigal ad
its Fourier trasform. A useful otatio for this fact is to coect both etit
es to a “correspodece pair”:
126
5 Appedix
ˆ f (t) ◦ − • f (ω)
(5.3)
No let f (t) ad g(t) deote t o ite eergy sigals, such that the “covolutio” of
f ith g, de ed by
+∞
(f g)(t) :=
−∞
f (u)g(t − u) du
(5.4)
leads to a fuctio f g, hich also has ite eergy. The the “covolutio theorem” s
tates that ˆg (f g)(t) ◦ − • f (ω)ˆ(ω). (5.5) Thus, covolutio i the time domai corres
to the product of the respective spectra i the frequecy domai! The covoluti
o theorem is used i sectios 2.2.1 ad 2.2.3, respectively, for developig a r
ecostructio formula for the CWT ad a fast algorithm for the computatio of th
e CWT. As a simple example illustratig Fourier trasforms cosider the fuctio
χT (t) = 1 −T ≤ t ≤ 2 0 else
T 2
It easily may be veri ed that its Fourier transform reads χT (ω) = ˆ Figure 5.1 sho s pl
ots of χT (t) and T = 6.
1.5 1 0.5 0 −0.5 −5
sin( ωT 2
ω 2
)
.
χT (ω) ˆ T,
respe
tively, for T = 1 and
−4
−3
−2
−1
0 t
1
2
3
4
5
1.5 1 0.5 0 −0.5 −15
−10
−5
0 ω
5
10
15
Fig. 5.1. Above: χ1 (t) (solid) and χ6 (t) (dashed). Belo : Corresponding Fourier ˆ (ω)
transforms χ1 (ω) (solid) and χ66 (dashed). ˆ
5.1 Fourier Transform and Un
ertainty Relation
127
Obviously the behavior of the fun
tion and its Fourier transform is in a
ertain
sense
omplementary: To a fun
tion, hi
h is “ ide spread” in the time domain, belo
ngs a Fourier transform, hi
h is “narro ” in the frequen
y domain and vi
e versa. T
his rather qualitative statement shall no be sharpened. Given a nite energy sign
al f (t), denote ith E its total energy, i.e., E :=
+∞ −∞
|f ( )|2 d . Then de ne he following “localiza ion parame ers”:
+∞
f :=
1 E
|f ( )|2 d
−∞ +∞
(5.6)
∆tf :=
1 E
(t − tf )2 |f ( )|2 d
−∞ +∞
(5.7)
1 ωf := 2πE
ˆ ω|f (ω)|2 dω
−∞ +∞
(5.8)
∆ω f :=
1 2πE
ˆ (ω − ωf )2 |f (ω)|2 dω
−∞
(5.9)
tf indi
ates here on the time domain f (t) is lo
ated, ∆tf provides iformatio a
bout ho f (t) is localized aroud tf : For large ∆tf values f (t) ill be ide s
pread aroud tf , for small ∆tf values f (t) ill be arro ly ˆ cocetrated aroud
tf . The same argumets apply to f (ω), ωf ad ∆ω f . Both aspects may be ui ed by repre
setig f i the t ω plae (“phase plae”, cf. sect. 1.2.5). Here f (t) essetially “liv
es” i a rectagular regio ith area ∆tf ∆ω f as idicated i Fig. 5.2.
tf
∆ω f
t
ωf ∆tf
?
ω
Fig. 5.2. Cell i phase plae, occupied by f .
128
5 Appedix
The “Heiseberg ucertaity relatio” the tells us that this area caot be arbitra
rily small: 1 (5.10) 2 This relatio is of fudametal importace. It’s cosequec
es reach from quatum mechaics to sigal aalysis. Ideed, Werer Heiseberg di
scovered that this relatio led to the famous ucertaity priciple, statig tha
t positio ad mometum of a elemetary particle ever together ca be measured
ith arbitrary precisio. I sigal aalysis, relatio 5.10 implies that o i
do fuctio ca be chose, hich is arbitrarily sharply cocetrated both i ti
me ad frequecy domai. Fortuately, there exists a fuctio hose localizatio
parameters yield the lo er boud i the iequality 5.10: Let g(t) be de ed as i
eq. 5.11 ∆tf ∆ω f ≥
t2 1 g(t) := √ e− 2 . 4 π
(5.11)
Then ∆tg ∆ω g = 1 2
5.2 Discrete Fourier Trasform (DFT)
Startig from a correspodece pair as give i eq. 5.3 e shall o cosider th
e follo ig task: Choose a “samplig distace” TS ad deote the correspodig circu
lar 2π frequenc with ωS := TS . Assume no that f is kno n only for t = . . . , −2TS
, TS , 0, TS , 2TS , . . ., i.e., the data form a sequen
e {f (kTS )}. For simpl
i
ity e also ill use the symbol fk instead of f (kTS ). An example
ould be a
sampled a
ousti
signal on your
omputer or your CD player. Is it possible ˆ to ga
in some information about the spe
trum f (ω) from that sequen
e? Provided
ertain
restri
tions are met, hi
h ill be given belo , the ans er ill be “yes”. We shall
assume that the nite energy signal f (t) is “band limited”, i.e., ˆ f (ω) = 0 outside som
e nite frequen
y interval. Let the “maximum frequen
y” ωmax > 0 be de ned su
h that [−ωmax
ax ] is the smallest possible interval ˆ outside of hi
h f (ω) vanishes. A band lim
ited fun
tion ill be said to satisfy the “Shannon
ondition” if ωS > 2ωmax. (5.12)
No
onsider a nitely sampled signal of length N , i.e., a sequen
e {fk }N −1 . The
n the “dis
rete Fourier Transform” (“DFT”) reads k=0
5.2 Dis
rete Fourier Transform (DFT)
N −1
129
Fk =
l=0
fl e−
j2πlk N
(k = 0, 1, . . . , N − 1)
(5.13)
l e
l=0
j2πlk N
(k = 0, 1, . . . , N − 1)
(5.14)
ˆ Before exl ining how the D T is rel ted with f (ω), e make t o remarks: 1. There
exists a fast algorithm for
omputing the DFT, namely the famous “Fast Fourier Tr
ansform” (“FFT”). 2. If the sampled signal {fl }N −1 is real valued, the follo ing relat
ion holds: l=0 FN −k = Fk (k = 1, . . . , N − 1) As mentioned already, the overline
denotes
omplex
onjugation. From this relation follo s that the DFT sequen
e {F
k }N −1 is
ompletely deterk=0 mined if roughly the rst half of the sequen
e is kno
n. To be pre
ise: {Fk }N −1 is
ompletely determined, provided the
oe
ients Fk ar
e kno n k=0 for k = 0, . . . , mN ith mN =
N 2 N −1 2
(N even) (N odd)
(5.15)
The follo ing theorem establishes the announ
ed relation bet een {Fk }N −1 k=0 ˆ and
f (ω): Theorem 5.1. Let f (t) be a band limited nite energy signal ith f (t) = 0
for t < 0 and t > (N − 1)TS . Let {f (kTS )}N −1 be a nite sampling of the k=0 signal
su
h that ωS = 2π TS
s tis es the Sh nnon condition 5.12. De ne the frequenc s mling ωk = k ωS (k = 0, . .
. , mN ). N
No
ompute the DFT sequen
e
orresponding to eq. 5.13 ith fl = f (lTS ). Then
e have ˆ f (ωk ) ≈ TS Fk (k = 0, . . . , mN ). We
lose this se
tion ith some remark
s (5.16)
130
5 Appendix
1. Sin
e the DFT may be
omputed very qui
kly ith the FFT algorithm, relation 5
.16 provides a fast algorithm for
omputing the
ontinuous spe
ˆ trum f (ω) at the ωk
values given above. 2. If f (t) is nonzero only on the interval [0, (N − 1)TS ],
band limitation stri
tly
annot be ful lled [34]. This means that the Shannon
ond
ition ˆ must be understood su
h that f (ω) is “su
iently small” outside the interval [−ωma
ωmax ]. If this is not the
ase, one has to be a are that, in parti
ular for high
ˆ ωk values, the estimation of f (ωk ) via eq. 5.16 leads to distortions by “aliasing”.
This explains, hy in eq. 5.16 the ≈ symbol as used instead of stri
t equality. 3
. Readers interested in more details of the topi
s
overed in this se
tion are r
eferred to textbooks on digital signal pro
essing and its mathemati
al foundatio
ns, respe
tively. Examples are in
luded in the bibliography ([33], [34]).
5.3 Digital Filters
Filtering In this se
tion e shall introdu
e basi
notions of digital ltering. Th
is se
tion is intended only for providing rst prin
iples and, parti
ularly, for xi
ng the notation. For more details on the topi
the reader is referred to, e.g.,
[33]. We shall
onsider only dis
retely sampled signals using the same
onventio
ns as in se
t. 5.2. A digital lter generates from a dis
retely sampled signal f =
{fk } a ne signal Hf = {(Hf )k }. Usually this pro
ess is visualized as sho n
belo :
f
E
H
Hf
E
Fig. 5.3. A digital lter.
The lter is
hara
terized by a nite set of “ lter
oe
ients” {hi } and the ne dis
rete
ignal Hf is
omputed as follo s: (Hf )k =
k
hi−k fi .
(5.17)
It is useful to visualize eq. 5.17 as in Fig. 5.4. Here the lter is spe
i ed by thr
ee lter
oe
ients {h−1 , h0 , h1 }.
fi−2
r
fi−1
r r r d h−1 h1 h0 d d d dc © d r
(H )i
i
5.3 Digital Filters i+1 i+2
131
r
Fig. 5.4. Visualization o eq. 5.17 or a lter with coe cients {h−1 , h0 , h1 }.
The top ro symbolizes the input sequen
e {fk }, the bottom ro the output seque
n
e {(Hf )k }. The
omputation of the i th element of the output sequen
e is ill
ustrated as follo s: The elements of the input sequen
e, hi
h point to ard the
i th element of the output sequen
e ith an arro , are multiplied ith the
orre
sponding lter
oe
ients and added at the node, here the arro s meet. Thus in our
ase (Hf )i = fi−1 h−1 + fi h0 + fi+1 h1 . For
ompleteness e note that a lter of th
is kind is
alled FIR lter [33], provided h1 = h2 = h3 = . . . = 0. Sin
e e allo
also for nonzero values of these
oe
ients, e shall
onsider the general
ase
of “non
ausal lters”1 : If, e.g., h1 = 0 the output signal value
orresponding to ti
me iTS may not be
omputed before time (i + 1)TS sin
e e need also the “future va
lue” fi+1 . Moreover, e also note that the indexing of the lter
oe
ients is slight
ly di erent from engineering standard [33], but this does not a e
t the main aspe
ts
of the transfer fun
tions dis
ussed belo . The transfer properties of a lter are
determined by the
orresponding “frequen
y response”: H(Ω) =
k
hk e−jkΩ (0 ≤ Ω < π)
(5.18)
The sectrum of Hf , s me sured, e.g., b the D T (cf. sect. 5.2), is obt ined
from the s ectrum of f b multi lic tion with the frequenc resonse. Det ils
sh ll be omitted here, g in we refer the interested re der to [33]. The frequen
c resonse is comlex qu ntit. We m decomose it with resect to bsolute
v lue nd h se, thus obt ining “ mlitude resonse” A(Ω) and “phase response” Φ(Ω): H(Ω) =
jΦ(Ω) . (5.19)
Again, ithout going into details e note here that the amplitude response A(Ω) is
responsible for transmitted or suppressed frequen
ies. Moreover, it is desirabl
e to have a phase response Φ(Ω), hi
h is linear ith respe
t to Ω, sin
e in this
ase
the ltering pro
ess leads to a redu
ed signal distortion.
1
This feature is of relevan
e for online appli
ations, see se
t. 3.6.2.
132
5 Appendix
Without proof e remark that “symmetri
lters” lead to linear phase response. Here sy
mmetry is de ned as follo s: If the number of
oe
ients is odd, then the
oe
ient mu
st be symmetri
ith respe
t to the
enter
oe
ient. If the number of
oe
ients is
even, then the
oe
ients must be symmetri
ith respe
t to the “
enter line” separat
ing the rst half from the se
ond half. For example, lters ith
oe
ients {h−1 , h0 ,
h1 } = {−1, 2, −1} or {h0 , h1 , h2 , h3 } = {1, 2, 2, 1}, respe
tively, both are sy
mmetri
. As stated above, the main purpose of this se
tion is to introdu
e the b
asi
notions and to x the notation. Examples related to amplitude and frequen
y r
esponse, repe
tively, are given in se
t. 3.3. Filtering ith subsampling “Subsampl
ing” a sequen
e means that only every se
ond sequen
e element is kept from the ori
ginal sequen
e. This pro
ess usually is represented by a box
ontaining a ↓ 2 symb
ol (
f. Fig. 5.5 belo ). Obviously at this step the length of the original seque
n
e is divided by t o. The graphi
al representation of a ltering pro
ess and a su
bsequent do nsampling pro
edure is sho n in Fig. 5.5.
f
E
H
Hf
E
↓2
Hf
E
Fig. 5.5. A digital lter follo ed by do nsampling.
Thus, the ne sequen
e obtained after ltering and do nsampling is denoted ith Hf
= {(Hf )k }. Mathemati
ally it is
omputed as follo s: (Hf )k =
k
hi−2k fi .
(5.20)
For a sequen
e {fk }3 of length 4 and a lter ith
oe
ients {h0 , h1 } k=0 the pro
ess is visualized in Fig. 5.6. The pro
edure dis
ussed so far leads to sequen
e
s ith half as many entries as the original sequen
e, here ea
h ne sequen
e el
ement is
omputed as a linear
ombination of the original sequen
e elements. Fil
tering ith upsampling, dual lters We ill need also the
omplementary pro
edure:
From a given sequen
e a ne sequen
e ill be generated ith t i
e as may elemen
ts as the original,
f0
r
r h1 h0 © c r
1
2
r h0 © c r
h1
r
5.3 Digital Filters 3
133
(H )0
(H )1
Fig. 5.6. Filtering with downsampling: Visualization o eq. 5.20 or a lter with
coe cients {h0 , h1 }. The element o H omitted by the downsampling procedure is
indicated with dotted lines.
the new sequence elements also will be linear combinations o the old sequence e
lements. We again start with a set o lter coe cients {hi }. Remember that we may a
ssociate a ltering and subsampling procedure, denoted with H, as described above.
The “dual lter” H , applied to a sequence, will produce a sequence, whose length wil
l be doubled, as compared with the length o the original sequence. The simplest
way to introduce H is a proper visualization. For the case o three lter coe cient
s {h−1 , h0 , h1 } it is given in Fig. 5.7.
(H f )−2
r
(H f )−1
r
(H f )0
r
(H f )1
r
(H f )2
r
h1 r −1
T0 h
s dh−1 d
h1 T0 h d d d dr 0
s dh−1 d
T0 h d d d dr
f1
Fig. 5.7. Visualization of a dual lter H ith
oe
ients {h−1 , h0 , h1 }.
The interpretation is very simple: Imagine the diagram of Fig. 5.6 ritten do n
for three lter
oe
ients {h−1 , h0 , h1 } and invert it. This means that no the top
ro represents the ne sequen
e, the bottom ro the original. Again, if arro s
meet at an element of the ne sequen
e, the respe
tive old sequen
e elements mus
t be multiplied ith the
orresponding lter
oe
ients and added. Thus, e.g., (H f
)−1 = f−1 h1 + f0 h−1 . It is
lear from Fig. 5.7 that the signal length is doubled at
this pro
ess. The summation formula asso
iated ith dual lters looks as follo s:
134
5 Appendix
(H f )k =
k
hk−2i fi .
(5.21)
5.4 Solutions to Sele
ted Problems
5.4.1 Problems from Se
t. 2.4 1. a) Given y(t) ◦ − • y(ω) e have ˆ
+∞
y(ω) = ˆ
−∞
y(t)e−jωt dt.
Di erentiating this equation on both sides ith respe
t to ω e obtain ⎞ ⎛ +∞ d⎝ dˆ (t)e
(ω) = dω dω
−∞ +∞
=
−∞
y(t)
+∞
de−jωt dt dω
=
−∞
y(t)(−jt)e−jωt dt
Here on the right hand side e inter
hanged di erentiation and integration, hi
h
may be justi ed. Iterating this pro
ess e obtain ˆ dm y (ω) = dω m
+∞
y(t)(−jt)m e−jωt dt.
−∞
Re
alling de
nition 5.1
of the Fourier transform,
e
on
lude that y(t)(−jt)m ◦ − • This m
ay be rewri en as y( ) m ◦ − • j m which is he desired resul . dm y ˆ (ω), dω m dm y ˆ (ω
dω
5.4 Solutions to Sele
ted Problems
135
b) Pro
eed analogously as above starting ith 1 2π
+∞
(t) =
(ω)e+jωt dω. ˆ
−∞
2. In order to
ompute all lo
alization parameters (ω) is needed. Therefore ˆ e in
sert (t) = in eq. 5.1 and obtain
+∞ +T 2
1 −T ≤ t ≤ 2 0 else
T 2
(ω) = ˆ
−∞ +T 2
(t)e−jωt dt =
−T 2
e−jωt dt
+T 2
=
−T 2
(
os(ωt) − j sin(ωt)) dt = 2
0 ωT 2
os(ωt) dt
=
2 sin ω
Here e applied the fa
t that
os(ωt) is an even and sin(ωt) an odd fun
+∞
tion. Moreover, E =
−∞
|w( )|2 d = T . Then from eq. 5.6 follows ha
+∞ +T 2
1 w = E =0
−∞
1 |w( )|2 d = T
d
−T 2
Similarly, eq. 5.8 implies ha
+∞ +∞
ω =
1 2πE
ω|w(ω)|2 dω = ˆ
−∞
1 2πT
−∞
4 sin2 ω
ωT 2
dω
=0
136
5 Appendix
Here e made use of the fa
t that From eq. 5.7 e obtain
+∞
4 sin2 ( ωT 2 ω
)
is an odd fun
tion.
∆t =
1 E
(t − t )2 |w( )|2 d
−∞ +T 2 +T 2
=
1 T
2 d =
−T 2
2 T
2 d
0
T =√ 23 Finally, eq. 5.9 implies
+∞
∆ω =
1 2πE 1 2πT
(ω − ω )2 |w(ω)|2 dω ˆ
−∞ +∞
= =∞
4 sin2
−∞
ωT 2
dω
3. We start ith ψH s de ned in eq. 2.32. or comuting the CWT 2.17 with the H r
w velet, we rst ev lu te the ex ression ψH u−t , considered s function of u. Sin
ce the H r w velet is re l v lued nd we ssume > 0 this le ds to u−t u−t = ψH ⎧ ⎨
1 0 ≤ u−t < 1 a 2 = −1 1 ≤ u−t < 1 2 a ⎩ 0 else ⎧ ⎨1 t ≤ u < t + a 2 = −1 t + a ≤ u < t + a
ψH
Inserting this result in formul 2.17 we obt in
5.4 Solutions to Selected Problems
137
∞
ψH
−∞
u−t
t+ 2
t+
f (u) du =
t
f (u) du −
t+ 2
f (u) du
= t+
− (t) − (t + ) + t + 2 2
Thus
1 2 t + − (t) − (t + ) . LψH f ( , t) = √ cψ 2
ψ 4. or n w velet ψ the corresondence ir ψ(t)tm ◦ − • j m d m (ω) (
f. dω problem 1)
be true. M Φ When, moreover, ψ is such th t ψ(t) = d M (t) for some suit ble function
dt Φ(t), in ddition
m
ˆ
ψ(t) =
dM Φ ˆ ˆ (t) ◦ − • (jω)M Φ(ω) = ψ(ω) dtM
(
f. problem 1). Combining both relations e
on
lude that ψ(t)tm ◦ − • j m dm ˆ (jω)M Φ(ω)
+∞
(m = 0, . . . , M − 1). ψ(t)tm dt. Rewriting it s
A moment integr l (cf. eq. 2.31) re ds
−∞ +∞ +∞ m
ψ(t)t dt =
−∞ −∞
ψ(t)tm e−j0t dt
nd rec lling
de nition 5.1 of the ourier tr nsform we conclude th t it is identi
c l to the ourier tr nsform of ψ(t)tm , ev lu ted for ω = 0. Therefore
+∞
ψ(t)tm dt = j m
−∞
dm ˆ (jω)M Φ(ω) dω m
(m = 0, . . . , M − 1).
ω=0
ˆ ˆ ˆ ˆ If Φ(0) nd deriv tives Φ (0), Φ (0), . . . , ΦM−1 (0) exist nd re nite, dm dω m
+∞
ˆ (jω)M Φ(ω)
ω=0
= 0 (m = 0, . . . , M − 1).
Therefore
−∞
ψ(t)tm dt = 0 (m = 0, . . . , M − 1).
138
5 Aendix
5. Let f (t) s tisf the ssumtion m de on ge 27 nd following ges: On I( ,
t) it m be written s olnomi l of degree k with k < M :
k
f (u) =
m=0
m um (u ∈ I( ,t) ).
Then
∞
ψ
−∞
u−t
k
∞
f (u) du =
m=0 k
m
−∞ ∞
um ψ
u−t
du 1
=
m=0
m
−∞
(t + u)m ψ(u) du
0
Proceeding from the rst to the second line we substituted u = u−t s new v ri b
le of integr tion. The v nishing of the integr ls follows from ex nding (t + u
)m with the binomi l ex nsion formul nd from the v nishing moments condition.
So n ll we obt in Lψ f ( , t) = 0. 5.4.2 Problems from Sect. 3.7 1. We consider
1 √ ψH 2 u−2k 2
i α f (ω)
f = {f [k]}
Referen
es
1. W. B¨ni (2002) Wavelets, eine Einf¨hrung f¨r Ingenieure. Oldenbourg, a u u M¨n
hen u
2. C.G. Broyden (1970) The Convergen
e of a Class of Double Rank Minimization Al
gorithms. Journal Inst. Math. Appli
. 6: 76–90 3. C.S. Burrus, R.A. Gopinath, H. G
uo (1998) Introdu
tion to Wavelets and Wavelet Transforms, A Primer. Prenti
e Ha
ll, Upper Saddle River (NJ) 4. C.K. Chui (1997) Wavelets: A Mathemati
al Tool fo
r Signal Analysis. SIAM, Philadelphia 5. A. Cohen (1992) Ondelettes, analyses mu
ltir´solution et traitement num´rique e e du signal. Ph. D. Thesis, University of Pa
ris IX, Dauphine 6. J.M. Combes et al (eds) (1989) Wavelets, Time Frequen
y Meth
ods and Phase Spa
e. Springer, Berlin Heidelberg Ne York 7. I. Daube
hies (1992
) Ten Le
tures on Wavelets. SIAM, Philadelphia 8. I. Daube
hies (1989) Orthonorm
al Bases of Wavelets ith Finite Support Conne
tion ith Dis
rete Filters. In: J
.M. Combes et al (eds) Wavelets, TimeFrequen
y Methods and Phase Spa
e. Springer
, Berlin Heidelberg Ne York 9. D.L. Donoho (1995) De Noising by Soft Thresholdi
ng. IEEE Trans. on Inf. Theory 41: 613–627 10. D. Gabor (1946) Theory of Communi
a
tion. J. Inst. Ele
tr. Engrg. 93(III): 429–457 11. P. Goupillaud, J. Morlet, A. Gr
ossmann, (1984/1985) Cy
le O
tave and Related Transforms in Seismi
Signal Analy
sis. Geoexploration 23: 85–102 12. F. Grupp, F. Grupp (2002) MATLAB 6 f¨r Ingenieure
. Oldenbourg, M¨n
hen u u 13. D. Hu man (1952) A Method for the Constru
tion of Mini
mum Redundan
y Codes. Pro
eedings of the IRE 40: 1098–1101 14. B. Jaehne (1995) Di
gital Image Pro
essing. Springer, Berlin Heidelberg Ne York 15. N.S. Jayant, P.
Noll (1984) Digital Coding of Waveforms. Prenti
e Hall, Upper Saddle River (NJ)
16. A. Jensen, A. laCour Harbo (2001) Ripples in Mathemati
s The Dis
rete Wav
elet Transform. Springer, Berlin Heidelberg Ne York 17. G. Kaiser (1993) A Frie
ndly Guide to Wavelets. Birkh¨user, Boston Basel Berlin a 18. M. Kobayashi (1996)
Listening for Defe
ts: Wavelet Based A
ousti
al Signal Pro
essing in Japan. SIAM
Ne s 29, No. 2
148
Referen
es
19. J.C. Lagarias, J. A. Reeds, M. H. Wright, P. E. Wright (1998) Convergen
e Pr
operties of the Nelder Mead Simplex Method in Lo Dimensions. SIAM Journal of Op
timization 9:112–147 20. A.K. Louis, P. Maaß, A. Rieder (1997) Wavelets, Theory and
Appli
ations. Wiley, Ne York 21. P. Maaß, H. G. Stark (1994) Wavelets and Digital
Image Pro
essing. Surveys on Mathemati
s for Industry 4: 195–235 22. S. Mallat (1
998) A Wavelet Tour of Signal Pro
essing. A
ademi
Press, Ne York (NY) 23. Y. M
eyer (1993) Wavelets, Algorithms and Appli
ations. SIAM, Philadelphia 24. M. Mis
iti Y. Misiti, G. Oppenheim, J. M. Poggi (2000) Wavelet Toolbox for Use ith MAT
LAB User’s Guide. The MATH WORKS In
. 25. A.H.H. Ngu, Q.Z. Sheng, D.Q. Huynh, R.
Lei (2000) Combining Multi visual Features for E
ient Indexing in a Large Image
Database. The VLDB Journal 9:1–15 26. Y. Nievergelt (2001) Wavelets Made Easy. Bir
kh¨user, Boston Basel Berlin a 27. S. Santini, R. Jain (1997) Similarity is a geom
eter. Multimedia Tools Appl. 5:277 306 28. G. Salton, C. Bu
kley (1990) Improvin
g Retrieval Performan
e by Relevan
e Feedba
k. Journal of the ASIS 41:288–297 29.
C.E. Shannon (1948) A Mathemati
al Theory of Communi
ation. Bell System Te
hni
a
l Journal 27 30. K. Sigmon (1994) MATLAB Primer. Chapman & Hall/CRC, Bo
a Raton
London Ne York Washington D.C. 31. H. G. Stark (1996) On Image Retrieval ith W
avelets. International Journal of Imaging Systems and Te
hnology 7:200–210 32. H.
G. Stark (1997) Wavelets und Bildar
hive. at – Automatisierungste
hnik 45:577–584 33
. S.D. Stearns, R.A. David (1996) Signal Pro
essing Algorithms in MATLAB. Prenti
e Hall, Upper Saddle River (NJ) 34. W. Strampp, E.V. Vorozhtsov (2004) Mathemat
is
he Methoden der Signalverarbeitung. Oldenbourg, M¨n
hen u 35. R.F. Streater, A.
S. Wightman (1980) PCT, Spin and Statisti
s and All That. Benjamin/Cummings, Rea
ding (MA), London, Amsterdam 36. T. Strutz (2000) Bilddatenkompression, Grundlag
en, Codierung, MPEG, JPEG. Vie eg, Brauns
h eig 37. J.T. Tou, R.C. Gonzalez, (19
74) Pattern Re
ognition Prin
iples. AddisonWesley Publishing Company, Reading (M
A) 38. J.S. Walker (1999) A Primer on Wavelets and Their S
ienti
Appli
ations. C
hapman & Hall/CRC, Bo
a Raton London Ne York Washington D.C. 39. G.K. Walla
e (
1991) The JPEG Still Pi
ture Compression Standard, Comm. of the ACM 34: 31–44 40.
D. Ziener (2002) FPGA Implementierung eines arithmetis
hen Codierers. Diploma Th
esis, As
ha enburg University of Applied S
ien
es
Index
Admissibility
ondition, 21 Approximation signal, 61 Arithmeti
oding, 104 Bits
per pixel, 111 bpp, 111 Cas
ade algorithm, 56 Co
hlea, 40 Coi ets, 85 Compression
bit per pixel (bpp), 111
ompression rate, 96 na¨ s
heme, 84 ıve peaked histograms,
84 transform
ompression s
heme, 95 CWT, 4, 21 fast algorithm, 23 re
onstru
tio
n, 24 transform, 21 visualization, 7 Daube
hies avelets, 53 Denoising, 88 Detai
l signal, 61 diagonal details, 66 horizontal details, 66 verti
al details, 66 DF
T, 128 Digital ltering, 130 Dis
rete Fourier transform, 128 Distortion measures n
orm distan
e, 96 PSNR, 96
DWT, 43 one dimensional signals, 61 real time properties, 88 t o dimensional sig
nals, 68 Energy
ompa
tion, 64, 81 Entropy, 100 Entropy
oding, 99 Fourier trans
form
ontinuous transform, 3, 125 dis
rete Fourier transform, 128 un
ertainty re
lation, 128 Gabor transform, 19 Hu man
oding, 102 Iterative optimization Nelder M
ead, 114 Quasi Ne ton, 114 JPEG, 98 JPEG 2000, 75, 98 progressive mode, 123 Lo
a
l analysis, 2 Lo
alization parameters, 127 MATLAB, 11 Optimization Toolbox, 114
STFT fun
tions myistft, 20 mystft, 19
150
Index spe
trum estimation, 129 Short time Fourier transform (STFT), 4, 14 fast a
lgorithm, 15 Signal
ontinuous time, 2, 125, 146 dis
rete (sampled), 13, 43, 128
, 146 nite energy, 125 Signal analysis, 1 Signal
lassi
ation, 36 Signal
ompressi
on, 1 Similarity retrieval, 116 Spe
trum estimation, 129 STFT, 4, 14 Transfer fu
n
tion amplitude response, 131 biorthogonal lters, 75 dbn lters, 60, 74 frequen
y
response, 131 phase response, 131 Un
ertainty relation, 128 Vanishing moments
o
ndition, 26 pra
ti
al impli
ations, 27 Varian
e, 116 Wavelet transform
ontinuou
s transform (CWT), 4, 21 dis
rete transform (DWT), 43 Wavelets admissibility
on
dition, 21 biorthogonal avelets, 73 Coi ets, 85 Daube
hies avelets, 53 dbn avel
ets, 55 generation by derivation, 26 Haar avelet, 26 mexi
an hat avelet, 27 Mo
rlet avelet, 28 vanishing moments, 26 WFT, 4, 14 Windo ed Fourier transform (WF
T), 4, 14
avelet fun
tions CompAudioToolbox, 87 CompImageToolbox, 87 bases1d, 81 bases2d,
81 my
t, 29 myi
t, 31 optshell, 113 Wavelet Toolbox, 11
t, 31 avede
, 70
avede
2, 71 avere
, 71 avere
2, 73 filters, 73 Multis
ale analysis, 60 Norm d
istan
e, 96 Pattern re
ognition, 38 Pattern ve
tor, 38, 118 Peak signal to noise
ratio, 96 Phase plane, 8, 127 lo
alization parameters, 127 Phase spa
e represen
tation, 8 PSNR, 96 Quantization non uniform, 111 optimal, 108 per
eptually lossl
ess, 85, 123 uniform, 99 Rate distortion fun
tion, 123 Re
onstru
tion from sampl
ed signals, 93 Relevan
e feedba
k, 121 Sampling
ir
ular frequen
y, 146 Sampling
distan
e, 13, 146 Sampling rate, 146 S
ale plane, 8 S
ale spa
e representation,
8 Shannon sampling
ondition, 128 re
onstru
tion from sampled signals, 93