CHAPTER-4

4/19/2012 1 Prof.H.T.Patil,CCOEW,PUNE
WHAT IS THE POWER DENSITY
SPECTRUM AND THEIR ESTIMATION
The signal processing methods which characterises the
frequency content of signal is known as spectrum analysis.
The signals which are analysed in any communication system
are either purely random or will have noise component also.
If the signal is random ,then only an estimate of the signal can
be obtained.
This is possible only if the statistical attributes of the random
signals are known.
4/19/2012 2 Prof.H.T.Patil,CCOEW,PUNE
Attributes
Consider the signal x(t), which is deterministic,
complex and finite energy.
The signal energy is given by Parsevals relation-:
The density of the energy of x(t) w.r.t. frequency is
represented by |X(f)|
2
Sxx(f) = |X(f)|
2
Sxx(f)=Energy Spectral Density (ESD)
3 4/19/2012 Prof.H.T.Patil,CCOEW,PUNE
Let Rxx() be the autocorrelation function of the
signal x(t) where Rxx() is given by
Rxx() =
Energy Spectral Density (ESD), Sxx(f), of the
signal is calculated as the Fourier Transform of
autocorrelation function of the signal
Sxx(f) = F(Rxx()) =|X(f)|
2
4
Attributes
4/19/2012 Prof.H.T.Patil,CCOEW,PUNE
Consider the signal x(t), which is random
stationary , complex and infinite energy.
The infinite energy signal do not have
Fourier Transform.
The infinite energy signal have finite
average power.
For such signals, spectral analysis id
done with Power Spectral Function.
5
Attributes
4/19/2012 Prof.H.T.Patil,CCOEW,PUNE
Consider the signal x(t), which is random stationary ,
complex and infinite energy.
Let
xx
() be the statistical autocorrelation function
of the signal x(t) , given by

xx
() = E[x(t) x(t+ )]
Power Spectral Density (PSD), xx(f), of the signal is
calculated as the Fourier Transform of the statistical
autocorrelation function
xx
() of the signal x(t),
xx(f) = F(
xx
() )
6
Attributes
4/19/2012 Prof.H.T.Patil,CCOEW,PUNE
Statistical Autocorrelation function of a
random process is time average autocorrelation
that will use to characterizing random signal in
the time domain .
Fourier transform of that autocorrelation
function is called Power Density Spectrum.
Power Spectral Estimation method is to
obtain an approximate estimation of the power
spectral density of a given real random
process.
Power Spectral Estimation
4/19/2012 7 Prof.H.T.Patil,CCOEW,PUNE
WHY WE USE POWER SPECTRUM
ESTIMATION ?
To estimate the spectral characteristics of
signal characterized as random processes.
To estimation of spectra in frequency domain
when signals are random in nature.
4/19/2012 8 Prof.H.T.Patil,CCOEW,PUNE
Spectral Estimation
Parametric
Non Parametric
Ex: Periodogram
and Welch method
Subspace Based
(high-resolution)
AR, ARMA based
Ex: MUSIC
and ESPRIT
Model fitting based
Ex: Least Squares
AR: Autoregressive (all-pole IIR)
ARMA: Autoregressive Moving Average (IIR)
MUSIC: MUltiple SIgnal Classification
ESPRIT: Estimation of Signal Parameters using Rotational Invariance Techniques
Spectral Estimation Techniques
4/19/2012 9 Prof.H.T.Patil,CCOEW,PUNE
Estimators
Estimation error: difference between the
estimator and the parameter.
Bias: expected value of the error.
Mean Square Error: mean square value of
error.
=

) ( e
= = ] [ ] [ ) (

E e E bias
) ( ) var( ] [ ) (
2 2


bias e E MSE + = =
4/19/2012 10 Prof.H.T.Patil,CCOEW,PUNE
Estimator comparison
Bias: lower the bias the better.
Variance: smaller variance implies less
deviation hence better.
MSE: in most cases more convenient.
measure of efficiency.
Consistency: if bias and variance both tend
to zero as the number of observations
become larger.
4/19/2012 11 Prof.H.T.Patil,CCOEW,PUNE
NON-PARAMETRIC METHODS
Non-parametric: PSE does NOT assume any
data-generating process or model i.e no
assumption about how data were generated.
Methods that rely on the direct use of the given
finite duration signal to compute the
autocorrelation to the maximum allowable length
(beyond which it is assumed zero), are called
Non-parametric methods.
Non-Parametric Methods:: PSD is estimated
directly from the signal itself.
4/19/2012 12 Prof.H.T.Patil,CCOEW,PUNE
Periodogram
Bartlett method
Welch method
Blackman-Tukey method
Types of Nonparametric methods
4/19/2012 13 Prof.H.T.Patil,CCOEW,PUNE
The Periodogram

=
+ =
k N
n
x
n x k n x
N
k r
1
0
) ( * ) (
1
) (

=

=
k
jk
x
jk
per
e k r e P

) (

) (

Estimated autocorrelation:
Estimated power spectrum or periodogram:
4/19/2012 14 Prof.H.T.Patil,CCOEW,PUNE
Discrete-time Fourier transform of the samples
of the process and then the magnitude squared
of the result.
where,
L f
f X
f P
s
L
xx
2
) (
) ( =

=
1
0
/ 2
] [ ) (
L
n
f jfn
L L
s
e n x f X

L f
f X
f P
s
k L
xx
2
) (
) ( =

N
kf
f
s
k
=
1 ......, 2 , 1 , 0 = N k
The Periodogram
4/19/2012 15 Prof.H.T.Patil,CCOEW,PUNE
Performance of the Periodogram (Spectral
Leakage)
x
L
[n] can be interpreted as the result of
multiplying an infinite signal, x[n], by a finite-
length rectangular window, w
R
[n]
x
L
[n] = x[n].w
R
[n]
Multiplication in the time domain corresponds
to convolution in the frequency domain.
4/19/2012 16 Prof.H.T.Patil,CCOEW,PUNE
The side lobes account for
the effect known as
spectral leakage.
While the infinite-length
signal has its power
concentrated exactly at the
discrete frequency
points f
k
, the windowed
signal has a continuum of
power "leaked" around the
discrete frequency
points f
k
.
frequency response of a rectangular
window
Performance of the Periodogram (Spectral
Leakage)
4/19/2012 17 Prof.H.T.Patil,CCOEW,PUNE
Performance of the Periodogram
(resolution)
Ability to discriminate spectral features.
In order to resolve two sinusoids, the
difference between the two frequencies
should be greater than the width of the
mainlobe of the leaked spectra for either
one of these sinusoids.
Mainlobe width is defined as 3 dB width
and is approximately equal to f
s
/ L
L
f
f f
s
) (
2 1
4/19/2012 18 Prof.H.T.Patil,CCOEW,PUNE
Performance of the Periodogram
(bias and variance)
A biased estimator. Its expected value can be
shown to be
The estimates correspond to a leaky PSD rather
than the true PSD.
Inconsistent estimator, its variance is
4/19/2012 19 Prof.H.T.Patil,CCOEW,PUNE
Modified Periodogram
The sidelobes can be interpreted as spurious
frequencies introduced into the signal by the
abrupt truncation that occurs when a rectangular
window is used.
The time-domain signal is windowed prior to
computing the FFT in order to smooth the edges
of the signal.
A window function is a function that is zero-
valued outside of some chosen interval.
The height of the sidelobes or spectral leakage is
reduced.
4/19/2012 20 Prof.H.T.Patil,CCOEW,PUNE
For nonrectangular windows, the end points of the
truncated signal are attenuated smoothly, and
hence the spurious frequencies introduced are
much less severe.
But, nonrectangular windows also broaden the
mainlobe, which results in a net reduction of
resolution.
Nonrectangular windowing affects the average
power due to attenuation of the time samples
Modified Periodogram
4/19/2012 21 Prof.H.T.Patil,CCOEW,PUNE
To compensate for this, we normalize the window
to have an average power of unity.
The modified periodogram estimate of the PSD is
where U is the window normalization constant
Choice of window does not affect the average
power of the signal after introduction of U.
Modified Periodogram
4/19/2012 22 Prof.H.T.Patil,CCOEW,PUNE
For reducing the variance in the periodogram
involves three steps .
The N-point sequence is subdivided into K
nonoverlapping segments where each segment
has length L .This results in K data segments.
For each segment, Compute the periodogram .
Averaging the periodogram for the K segment
to obtain the Bartlett power spectrum estimate.
Bartletts method
4/19/2012 23 Prof.H.T.Patil,CCOEW,PUNE
The Bartlett Method: Averaging
Periodograms
It consists of Three steps
Step 1: The N-pt sequence is subdivided into K nonoverlapping
segments, where each segment has length M. This results in the K data
segments
( ) ( ); 0,1, 2,..., 1
0,1, 2,..., 1
i
x n x n iM i K
n M
= + =
=
Step 3: Finally, we average the periodograms for the K segments to obtain
the Bartlett PSD estimate.
2
1
( ) 2
0
1
( ) ( ) ; 0,1, 2,..., 1
M
i i fn
xx i
n
P f x n e i K
M

=
= =

Step 2: For each segment, we compute the periodogram

1
( )
0
1
( ) ( )
K
B i
xx xx
i
P f P f
K

=
=

4/19/2012 24 Prof.H.T.Patil,CCOEW,PUNE
Mean:
1
( ) ( )
0
1
( ) ( ) = ( )
K
B i i
xx xx xx
i
E P f E P f E P f
K

=
( ( (
=

[ ] [ ]
( )
( ) { ( )} { ( ) }
| |
{ ( ). ( )} 1 . ( )
( ) ( )
i
xx xx xx
B xx xx
B xx
E P f E DTFT r m DTFT E r m
m
DTFT w m m DTFT m
M
W f f

(
= =


| |
= =
`
|
\
)
=
2
1 sin( )
( )
sin( )
B
fM
W f
M f

| |
=
|
\
where
1
2
2
( )
1
2
1 sin( ( ) )
( ) ( ) ( ) ( )
sin( ( ))
i
xx B xx xx
f M
E P f W f f d
M f

| |
(
= =
|


\

The Bartlett Method: Averaging

Periodograms
4/19/2012 25 Prof.H.T.Patil,CCOEW,PUNE
Variance:
1
( ) ( )
2
0
2
2
1 1
( ) var ( ) = var ( )
1 sin(2 )
( ) 1
sin(2 )
K
B i i
xx xx xx
i
xx
var P f P f P f
K K
fM
f
K M f

=
( ( (
=

(
| |
= + (
|
( \

The effect of reducing the length of the data from N points to

M=N/K results in a window whose spectral width has been
increased by a factor of K. Consequently, the frequency resolution
has been reduced by a factor K.
In return for this reduction in resolution, we have reduced the
variance. The variance of the Bartlett estimate is
Therefore, the variance of the Bartlett PSD estimate has been
reduced by the factor K.
The Bartlett Method: Averaging Periodograms
4/19/2012 26 Prof.H.T.Patil,CCOEW,PUNE
Bias: Bias: Bias: Bias:
Variance Variance Variance Variance:
2
1 1
0 0
1

( ) ( )
k L
j jn
B
i n
P e x n iL e
N

= =
= +

) ( ) (
2
1
)} (

j
B
j
x
j
B
e W e P e P E =
) (
1
)} (

{
2 j
x
j
per
e P
k
e P Var
Bartletts method
4/19/2012 27 Prof.H.T.Patil,CCOEW,PUNE
Welch's Method
The method consists of dividing the time series data into
(possibly overlapping) segments, computing a modified
periodogram of each segment, and then averaging the PSD
estimates
The averaging of modified periodograms tends to decrease
the variance of the estimate relative to a single
periodogram estimate of the entire data record
Overlap between segments tends to introduce redundant
information, this effect is diminished by the use of a
nonrectangular window, which reduces the importance or
weight given to the end samples of segments (the samples
that overlap).
4/19/2012 28 Prof.H.T.Patil,CCOEW,PUNE
The combined use of short data records and
nonrectangular windows results in reduced
resolution of the estimator. ie, there is a tradeoff
between variance reduction and resolution.
One can manipulate the parameters in Welch's
method to obtain improved estimates relative to
the periodogram.
Welch's Method
4/19/2012 29 Prof.H.T.Patil,CCOEW,PUNE
The Welch Method: Averaging Modified
Periodograms
Welch made two modifications to the Bartlett method.
Step 1: The data segments are allowed to overlap. Thus the data segments
can be represented as
( ) ( ); 0,1, 2,..., 1
0,1, 2,..., 1
i
x n x n iD i L
n M
= + =
=
Step 2: Apply window to the data segments prior to computing the
periodogram. The result is a modified Periodogram
Where U is the normalization factor for the power in the window function
1
2
0
1
( )
N
n
U w n
M

=
=

2
1
( )
2
0
1
( ) ( ) ( ) ; 0,1, 2,..., 1
M
i
i fn
xx
i
n
P f x n w n e i L
MU

=
= =

Where iD is the starting point for the i

th
sequence. If D=M/2, there is 50%
overlap between successive data segments and L=2K.
4/19/2012 30 Prof.H.T.Patil,CCOEW,PUNE
The Welch PSD estimate is the average of these modified periodograms, that
is
{ }
1 1
( )
* 2 ( )
0 0
1 1
2 ( )
0 0
1
( ) ( ) ( ) ( ) ( )
1
( ) ( ) ( )
M M
i
j f n m
xx
i i
n m
M M
j f n m
xx
n m
E P f w n w m E x n x m e
MU
w n w m n m e
MU

= =


= =
(
=
(

=


Since
1/ 2
2
1/ 2
( ) ( )
j fn
xx xx
n e d

1
( )
0
1
( ) ( )
L
i
W
xx
xx
i
P f P f
L

=
=

Mean:
1
( ) ( )
0
1
( ) ( ) = ( )
L
i i
W
xx xx
xx
i
E P f E P f E P f
L

=
( (
(
=

( (

The Welch Method: Averaging Modified

Periodograms
4/19/2012 31 Prof.H.T.Patil,CCOEW,PUNE
1 1
1/ 2 ( )
2 ( )( )
1/ 2
0 0
1/ 2
1/ 2
1
( ) ( ) ( ) ( )
( ) ( )
M M
i
j f n m
xx
xx
n m
xx
E P f w n w m e d
MU
W f d

= =

(
(
=
(
(


=

where
2
1
2
0
1
( ) ( )
M
i fn
n
W f w n e
MU

=
=

Variance:
{ }
1 1
2
( ) ( )
2
0 0
1
( ) ( ) ( ) ( )
L L
i j
W W
xx xx
xx xx
i j
var P f E P f P f E P f
L

= =
(
( (
=

(

Case(1): No overlap (L=K)

( )
2
1 1
( ) var ( ) ( )
i
W
xx
xx xx
var P f P f f
L L
(
(
=

(

Case(2): 50% overlap (L=2K)
2
9
( ) ( )
8
W
xx xx
var P f f
L
(

The Welch Method: Averaging Modified Periodograms
4/19/2012 32 Prof.H.T.Patil,CCOEW,PUNE
A biased estimator of the PSD, the expected value
is
For a fixed length data record, the bias of Welch's
estimate is larger than that of the periodogram
because L
s
< L.
The variance of Welch's estimator is difficult to
compute because it depends on both the window
used and the amount of overlap between segments.
The variance is inversely proportional to the
number of segments whose modified
periodograms are being averaged.
Welch's Method
4/19/2012 33 Prof.H.T.Patil,CCOEW,PUNE
The Blackman-Tukey Method: Smoothing the
Periodogram
The sample autocorrelation sequence is windowed first and then Fourier
transformed to yield the estimate of PSD.
where
Where P
xx
(f) is Periodogram. The effect of windowing the autocorrelation is to
smooth the periodogram estimate, thus decreasing its variance.
Where the window function w(n) has length 2M-1 and zero for |m|>(M-1).
1
2
( 1)
( ) ( ) ( )
M
BT i fm
xx xx
m M
P f r m w m e

=
=

1/ 2
1/ 2
( ) ( ) ( )
BT
xx xx
P f P W f d

Mean:
[ ]
1/ 2
1/ 2
( ) ( ) ( )
BT
xx xx
E P f E P W f d

( =

[ ]
1/ 2
1/ 2
( ) ( ) ( )
xx xx B
E P W d

4/19/2012 34 Prof.H.T.Patil,CCOEW,PUNE
We should select the window length for w(n) such that M<<N, i.e., w(n) should
be narrower than w
B
(n) to provide additional smoothing of the periodogram.
Under this condition, we get
Variance:
1/ 2 1/ 2
1/ 2 1/ 2
( ) ( ) ( ) ( )
BT
xx xx B
E P f W W f d d

(
=

1/ 2
1/ 2
( ) ( ) ( )
BT
xx xx
E P f W f d

(


1/ 2 1/ 2
1/ 2 1/ 2
( ) ( ) ( ) ( ) ( )
B B
W W f d W W f d W f

=

{ }
{ }
2
2
var ( ) ( ) ( )
BT BT BT
xx xx xx
P f E P f E P f ( ( ( =

where
{ }
[ ]
1/ 2 1/ 2 2
1/ 2 1/ 2
( ) ( ) ( ) ( ) ( )
BT
xx xx xx
E P f E P P W f W f d d

( =

After simplification, we get
1
1/ 2
2 2 2 2
1/ 2
( 1)
1 1
var ( ) ( ) ( ) ( ) ( )
M
BT
xx xx xx
m M
P f f W d f w m
N N

=
(
(
(
(
(

The Blackman-Tukey Method: Smoothing the Periodogram

4/19/2012 35 Prof.H.T.Patil,CCOEW,PUNE
PARAMETRIC METHODS
Parametric methods:
PSD is estimated from a signal that is assumed to
be output of a linear system driven by white
noise.
Parametric methods:
ARMAModelling
Yule-Walker autoregressive (AR) method,
Burg method.
4/19/2012 36 Prof.H.T.Patil,CCOEW,PUNE