TC-515 Advanced Digital Signal

Processing
Dr. Muhammad Imran Aslam
Spring 2015

Books
Discrete Time Signal Processing, 2nd ed. By
Alan V. Oppenhem and Ronald W. Schafer
Digital Signal Processing: principles algorithms
and applications 4th ed. By John G. Proakis and
Dimitris G. Manolakis
Digital Signal Processing: A computer based
approach, 2nd ed. by Sanjit K. Mitra
Schaums outline of Digital Signal Processing

DSP is Everywhere
Sound applications
Compression, enhancement, special effects, synthesis, recognition,
echo cancellation,
Cell Phones, MP3 Players, Movies, Dictation, Text-to-speech,
Communication
Modulation, coding, detection, equalization, echo cancellation,
Cell Phones, dial-up modem, DSL modem, Satellite Receiver,
Automotive
ABS, GPS, Active Noise Cancellation, Cruise Control, Parking,
Medical
Magnetic Resonance, Tomography, Electrocardiogram,
Military
Radar, Sonar, Space photographs, remote sensing,
Image and Video Applications
DVD, JPEG, Movie special effects, video conferencing,
Mechanical
Motor control, process control, oil and mineral prospecting,

Signal Processing
Humans are the most advanced signal processors
speech and pattern recognition, speech synthesis,
We encounter many types of signals in various applications
Electrical signals: voltage, current, magnetic and electric fields,
Mechanical signals: velocity, force, displacement,
Acoustic signals: sound, vibration,
Other signals: pressure, temperature,
Most real-world signals are analog
They are continuous in time and amplitude
Convert to voltage or currents using sensors and transducers
Analog circuits process these signals using
Resistors, Capacitors, Inductors, Amplifiers,
Analog signal processing examples
Audio processing in FM radios
Video processing in traditional TV sets

Limitations of Analog Signal Processing

Accuracy limitations due to
Component tolerances
Undesired nonlinearities
Limited repeatability due to
Tolerances
Changes in environmental conditions
Temperature
Vibration
Sensitivity to electrical noise
Limited dynamic range for voltage and currents
Inflexibility to changes
Difficulty of implementing certain operations
Nonlinear operations
Time-varying operations
Difficulty of storing information

Digital Signal Processing

Represent signals by a sequence of numbers

Sampling or analog-to-digital conversions
Perform processing on these numbers with a digital processor
Digital signal processing
Reconstruct analog signal from processed numbers
Reconstruction or digital-to-analog conversion
analog
signal

A/D

digital
signal

DSP

digital
signal

D/A

analog
signal

Analog input analog output

Digital recording of music

Analog input digital output

Touch tone phone dialing

Digital input analog output

Text to speech

Digital input digital output

Compression of a file on computer

Pros and Cons of Digital Signal Processing

Pros
Accuracy can be controlled by choosing word length
Repeatable
Sensitivity to electrical noise is minimal
Dynamic range can be controlled using floating point numbers
Flexibility can be achieved with software implementations
Non-linear and time-varying operations are easier to implement
Digital storage is cheap
Digital information can be encrypted for security
Price/performance and reduced time-to-market
Cons
Sampling causes loss of information
A/D and D/A requires mixed-signal hardware
Limited speed of processors
Quantization and round-off errors
7

Signal Types
Analog signals: continuous in time and amplitude
Example: voltage, current, temperature,
Digital signals: discrete both in time and amplitude
Example: attendance of this class, digitizes analog signals,
Discrete-time signal: discrete in time, continuous in amplitude
Example: hourly change of temperature in Austin
Theory for digital signals would be too complicated
Requires inclusion of nonlinearities into theory
Theory is based on discrete-time continuous-amplitude signals
Most convenient to develop theory
Good enough approximation to practice with some care
In practice we mostly process digital signals on processors
Need to take into account finite precision effects

Periodic (Uniform) Sampling

Sampling is a continuous to discrete-time conversion

-3 -2 -1 0 1 2 3 4

Most common sampling is periodic

x[n] = x c (nT ) < n <

T is the sampling period in second

fs = 1/T is the sampling frequency in Hz
Sampling frequency in radian-per-second s=2fs rad/sec
Use [.] for discrete-time and (.) for continuous time signals
This is the ideal case not the practical but close enough
In practice it is implement with an analog-to-digital converters
We get digital signals that are quantized in amplitude and time
9

Periodic Sampling
Sampling is, in general, not reversible
Given a sampled signal one could fit infinite continuous signals
through the samples
1

0.5
0
-0.5
-1
0

20

40

60

80

100

Fundamental issue in digital signal processing

If we loose information during sampling we cannot recover it

Under certain conditions an analog signal can be sampled without

loss so that it can be reconstructed perfectly

10

Representation of Sampling

Mathematically convenient to represent in two stages

Impulse train modulator
Conversion of impulse train to a sequence
s(t)
xc(t)

Convert impulse
train to discretetime sequence

xc(t)

x[n]=xc(nT)

x[n]

s(t)

-3T-2T-T 0 T 2T 3T 4T

-3 -2 -1 0 1 2 3 4

11

Continuous-Time Fourier Transform

Continuous-Time Fourier transform pair is defined as
X c (j ) =

x (t )e
c

jt

dt

1
jt
(
)
x c (t ) =
X
j

e
d
c

We write xc(t) as a weighted sum of complex exponentials

Remember some Fourier Transform properties
Time Convolution (frequency domain multiplication)
x(t) y(t) X( j)Y( j)

Frequency Convolution (time domain multiplication)

x(t)y(t) X( j) Y( j)

Modulation (Frequency shift)

x(t)e jot X(j( o ))

12

Frequency Domain Representation of Sampling

Modulate (multiply) continuous-time signal with pulse train:
x s (t ) = x c (t )s(t ) =

x (t )(t nT )

n =

s(t) =

(t nT )

n =

Lets take the Fourier Transform of xs(t) and s(t)

1
X c (j ) S(j )
X s (j ) =
2

2
S(j ) =
( k s )

T k =

Fourier transform of pulse train is again a pulse train

Note that multiplication in time is convolution in frequency
We represent frequency with = 2f hence s = 2fs
1
X s (j ) =
X c (j( k s ))

T k =

13

Frequency Domain Representation of Sampling

Convolution with pulse creates replicas at pulse location:

1
X c (j( k s ))
X s (j ) =

T k =
This tells us that the impulse train modulator
Creates images of the Fourier transform of the input signal
Images are periodic with sampling frequency

If s< N sampling maybe irreversible due to aliasing of images

X c (j )

-N

X s (j )

s>2N
3s

-2s

s -N

2s

3s

X s (j )

s<2N
3s

-2s

s -N

2s

3s

14

Nyquist Sampling Theorem

Let xc(t) be a bandlimited signal with

X c ( j) = 0

for N

Then xc(t) is uniquely determined by its samples x[n]= xc(nT) if

s =

2
= 2fs 2N
T

N is generally known as the Nyquist Frequency

The minimum sampling rate that must be exceeded is known as
the Nyquist Rate
Low pass filter

X s (j )

s>2N
3s

-2s

s -N

2s

3s

X s (j )

s<2N
3s

-2s

s -N

2s

3s
15

Discrete-Time Signals: Sequences

Discrete-time signals are represented by sequence of numbers

The nth number in the sequence is represented with x[n]
Often times sequences are obtained by sampling of continuous-time
signals
In this case x[n] is value of the analog signal at xc(nT)
Where T is the sampling period
10

-10
0
10

t (ms)

20

40

60

80

100

10

20

30

40

50 n (samples)

-10
0

16

Basic Sequences and Operations

Delaying (Shifting) a sequence
y[n] = x[n no ]

1.5
1

Unit sample (impulse) sequence

0 n 0
[n] =
1 n = 0
Unit step sequence
0 n < 0
u[n] =
1 n 0

0.5
0
-10

-5

10

-5

10

-5

10 17

1.5
1
0.5
0
-10
1

Exponential sequences
x[n] = An

0.5

0
-10

Sinusoidal Sequences
Important class of sequences
x[n] = cos(on + )

An exponential sequence with complex = e j and A = A e j

o

x[n] = A = A e e
n

jon

= A e j(on + )
n

x[n] = A cos(on + ) + j A sin(on + )

n

x[n] is a sum of weighted sinusoids

Different from continuous-time, discrete-time sinusoids
Have ambiguity of 2k in frequency
cos(on + ) = cos(on + oN + ) only if N =

2k
is an integer
o

Are not necessary periodic with 2/o

18

Discrete-Time Systems
Discrete-Time System is a mathematical operation that maps
a given input sequence x[n] into an output sequence y[n]
y[n] = T{x[n]}

x[n]

T{.}

y[n]

Example Discrete-Time Systems

Moving (Running) Average
y[n] = x[n] + x[n 1] + x[n 2] + x[n 3]

Maximum

y[n] = max{x[n], x[n 1], x[n 2]}

Ideal Delay System

y[n] = x[n no ]
19

Memoryless System
Memoryless System
A system is memoryless if the output y[n] at every value of n
depends only on the input x[n] at the same value of n
Example Memoryless Systems
Square
2
y[n] = (x[n])
Sign

y[n] = sign{x[n]}

Counter Example
Ideal Delay System
y[n] = x[n no ]

20

Linear Systems

Linear System: A system is linear if and only if

T {x1[n] + x2 [n]}= T { x1[n]} + T { x2 [n]} (additivity)
and
T {ax[n]} = aT { x[n]} (scaling/Homogeneity)

Examples

Ideal Delay System

y[n] = x[n no ]

= x1[n no ] + x2[n no ]
T{x1[n] + x2[n]}
T{x2[n]} + T{x1[n]} = x1[n no ] + x2[n no ]
=
T{ax[n]}
ax1[n no ]
=
aT{x[n]}
ax1[n no ]
21

Time-Invariant Systems
Time-Invariant (shift-invariant) Systems
A time shift at the input causes corresponding time-shift at
output
y[n] = T{x[n]} y[n no ] = T{x[n no ]}
Example
Square
2
y[n] = (x[n])

Delay the input the output is

Delay the output gives

Counter Example
Compressor System
y[n] = x[Mn]

Delay the input the output is

Delay the output gives

y1 [n] = (x[n no ])

y[n - no ] = (x[n no ])

y1 [n] = x[Mn no ]

y[n - no ] = x[M(n no )]

22

Causal System
Causality
A system is causal its output is a function of only
the current and previous samples

Examples
Backward Difference
y[n] = x[n] x[n 1]

Counter Example
Forward Difference
y[n]= x[n + 1] x[n]
23

Stable System
Stability (in the sense of bounded-input bounded-output BIBO)
A system is stable if and only if every bounded input produces a
bounded output

x[n] B x < y[n] B y <

Example

Square

y[n] = (x[n])

if input is bounded by x[n] B x <

output is bounded by y[n] B2x <

Counter Example
Log

y[n] = log10 ( x[n] )

even if input is bounded by x[n] B x <

output not bounded for x[n] = 0 y[0] = log10 ( x[n] ) =

24

25

Transformations of the Independent Variable

26

27

28

29

Example

30

Example

31

32

33

34

35

36

37

38

39

40

41

Linear-Time Invariant System

Special importance for their mathematical tractability

Most signal processing applications involve LTI systems
LTI system can be completely characterized by their impulse
response
T{.}

[n-k]

=
x [ n]

Represent any input

y [ n ] T x [ k ]=
[ n k ]
=
k =

y [ n ]=

hk[n]

x [ k ] [ n k ]

k =

k =

k =

{ [ n k ]} x [ k ] h [ n]
x [ k ]T =
k

x [ k ] h [ n k ]= x [ k ] h [ k ]

k =

From time invariance we arrive at convolution

Output of a LTI system is convolution of its impulse response
and the input.

42

LTI System Example

LTI

0.5
0
-5

0
-5

LTI
0

LTI
0

LTI
0

1
0
-5
4

1
0
-5

1
0
-5
2

1
0
-5
2

1
0
-5
2

0.5

2
0
-5

43

44

Properties of LTI Systems

Convolution is commutative
x[k ] h[k ] = x[k ]h[n k ] = h[k ]x[n k ] = h[k ] x[k ]
x[n]

k =

k =

h[n]

y[n]

h[n]

x[n]

y[n]

Convolution is distributive
x[k ] (h [k ] + h [k ]) = x[k ] h [k ] + x[k ] h [k ]
1

h1[n]
x[n]

y[n]

x[n]

h1[n]+ h2[n]

y[n]

h2[n]
45

Properties of LTI Systems

Cascade connection of LTI systems
x[n]

h1[n]

h2[n]

y[n]

x[n]

h2[n]

h1[n]

y[n]

x[n]

h1[n]h2[n]

y[n]

46

Properties of LTI Systems

47

48

Convolution Examples
A good JAVA applet is available at:
http://pages.jh.edu/~signals/discreteconv2/
Convolution can be calculated
Directly
Graphical method
Slide rule method

49

Stable and Causal LTI Systems

An LTI system is (BIBO) stable if and only if

Impulse response is absolute
summable

h[k ] <

k =

Lets write the output of the system as

y[n] =

k =

k =

h[k ]x[n k ] h[k ] x[n k ]

If the input is bounded

x[n] B x

Then the output is bounded by

y[n] B x

h[k ]

k =

The output is bounded if the absolute sum is finite

An LTI system is causal if and only if

h[k ] = 0 for k < 0

50

Linear Constant-Coefficient Difference Equations

LTI systems can be mathematically represented by linear constant
coefficient difference equation (LCCDE) of the form
N

k ] b x [ n k ] OR
a y [ n=
k
k 0=
k 0

=
y[n]

b x [n k ] a y [n k ]

k
=
k 0=
k 1

The coefficients ak and bk are constants that define the system.

The initial conditions are required to calculate output
Linearity, time invariance, and causality depend on the initial
conditions
If initial conditions are assumed to be zero system is linear, time
invariant, and causal

Example

Moving Average

y[n] = x[n] + x[n 1] + x[n 2] + x[n 3]

Difference Equation Representation
0

a y[n k ] = b x[n k ]

k =0

k =0

where ak = bk = 1
51

Linear Constant-Coefficient Difference Equations

=
y[n]

b x [n k ] a y [n k ]

k
=
k 0=
k 1

If the difference equation has one or more terms ak that are

nonzero, the difference equation is said to be recursive.
Following is an example of a first-order recursive difference
equation
On the other hand, if all of the coefficients ak are equal to
zero, the difference equation is said to be nonrecursive. E.g.

52

Linear Constant-Coefficient Difference Equations

53

Linear Constant-Coefficient Difference Equations

Characteristic Equation

54

Linear Constant-Coefficient Difference Equations

For the particular solution, it is necessary to find the sequence yp(n)

that satisfies the difference equation for the given x(n). In general,
this requires some creativity and insight. However, for many of the
typical inputs that we are interested in, the solution will have the
same form as the input.

55

Linear Constant-Coefficient Difference Equations

56

57

58