Ch2 Wiener Filters
Ch2 Wiener Filters
Ch2 Wiener Filters
1
Linear Optimum Filtering: Statement
How?
An optimization problem.
3
Linear Optimum Filtering: Statement
Optimization (minimization) criterion:
1. Expectation of the absolute value,
2. Expectation (mean) square value,
3. Expectation of higher powers of the absolute value
of the estimation error.
Minimization of the Mean Square value of the Error (MSE) is
mathematically tractable.
Problem becomes:
Design a linear discrete-time filter whose output y(n) provides an
estimate of a desired response d(n), given a set of input samples
u(0), u(1), u(2) ..., such that the mean-square value of the
estimation error e(n), defined as the difference between the
desired response d(n) and the actual response, is minimized.
4
Principle of Orthogonality
Filter output is the convolution of the filter IR and the input
5
6
Principle of Orthogonality
Error:
7
Derivative in complex variables
Let
Hence
or
the cost function J is
a scalar independent
of time n.
8
Principle of Orthogonality
Partial derivative of J is
Hence
b k
n 9
Principle of Orthogonality
Since , or
A good basis for testing whether the linear filter is operating in its
optimum condition.
10
Principle of Orthogonality
Corollary:
11
Minimum Mean-Square Error
Let the estimate of the desired response that is optimized in the MSE sense,
depending on the inputs which span the space
i.e. so
Then the error in optimal conditions is
or
Meaning
13
Wiener-Hopf Equations
We have (principle of orthogonality)
i i
i
i
Rearranging k 0, 1, 2, ...
Wiener-Hopf
Equations
(set of infinite
eqn.s)
where
14
Wiener-Hopf Equations
Solution of Wiener-Hopf Equations for Linear Transversal (FIR) Filter
Then
and
17
Wiener-Hopf Equations (Matrix Form)
Then the Wiener-Hopf equations can be written as
where
18
Error-Performance Surface
Substitute
Rewriting
19
Error-Performance Surface
Quadratic function of the filter coefficients convex function, then
k
or
Wiener-Hopf
Equations
20
Minimum value of Mean-Squared Error
We calculated that
Then w oH p p H w o
At wo.
(Jmin is independent of w)
21
Canonical Form of the Error-Performance Surface
Then, by substituting
In other words,
22
Canonical Form of the Error-Performance Surface
Observations:
J(w) is quadratic in w,
Minimum is attained at w=wo,
Jmin is bounded below, and is always a positive quantity,
Jmin>0
23
Canonical Form of the Error-Performance Surface
Transformations may significantly simplify the analysis,
Use Eigen-decomposition for R
Then
Let
a vector
Canonical form
Q
v1
Transformation
(1)
w1
25
Multiple Linear Regressor Model
Wiener Filter tries to match the filter coefficients to the model of the
desired response, d(n).
, R m E um (n )umH (n )
But we know that
u m (n )
u( n ) ,
u M m ( n )
uM-m(n) is an (M-m)-by-l vector made up of past data
samples immediately preceding the m-by-l vector um(n).
28
Numerical Example (Ch2:P11)
The desired response d(n) is modeled as an AR process of order 1;
that is, it may be produced by applying a white-noise process v(n) of
zero mean and variance 12=0.27 to the input of an all-pole filter of
order 1;
1
H1 ( z )
1 + 0.8458z -1
The process d(n) is applied to a communication channel modeled by
the all-pole transfer function
1
H 2 ( z)
1 - 0.9458z -1
The channel output x(n) is corrupted by an additive white-noise
process v2(n) of zero mean and variance 22 = 0.1, so a sample of the
received signal u(n) equals u(n) = x(n) + v2(n)
29
+
+
30
The requirement is to specify a Wiener filter consisting of a
transversal filter with two taps, which operates on the received
signal u(n) so as to produce an estimate of the desired response that
is optimum in the mean-square sense.
Statistical Characterization of the Desired Response d(n) and
the Received signal u(n)
d(n) + a1 d(n - 1) = v1(n)
where a1 = 0.8458. The variance of the process d(n) equals
12 0.27
d
2
0.9486
1 a1
2
1 0.8458
2
The process d(n) acts as input to the channel. Hence, from Fig. (b).
we find that the channel output x(n) is related to the channel input
d(n) by the first-order difference equation
x(n) + b1x(n - 1) = d(n)
31
where b1 = -0.9458 . We also observe from the two parts of Fig. that
the channel output x(n) may be generated by applying the white-
noise process v1(n) to a second-order all-pole filter whose transfer
function equals.
1
H (z ) H 1 (z )H 2 (z )
(1 + 0.8458z -1 )(1 - 0.9458z -1 )
X (z ) H (z )V (z )
32
Since the process x(n) and v2(n) are uncorrelated, it follows
that the correlation matrix R equals the correlation matrix of
x(n) plus the correlation matrix of v2(n). R=Rx+R2
rx (0) rx (1)
Rx
x
r (1) rx (0)
1 a2 1 2
a1 2
x rx (0)
2
. rx (1) x
1 a2
1 a2 (1 a2 ) a1
2 2
0.1
1 0.8 0.27 0.5
1 1 0.8
1 0.8 (1 0.8) (0.1)
2 2
0.1 0
R2
0 0.1
1.1 0.5
R R x R2
0.5 1.1
34
x(n) + b1x(n - 1) = d(n)
p(0)
p
u(n)=x(n)+v2(n)
Since these two processes
p(1)
are real valued
Then
v2
(2)
Jmin
v1
(1)
39
* Application Channel Equalization
41
The impulse response of the equalizer
where the sequence Wk is equal to the convolution of the sequences cn and hk, i.e.
42
Let the data sequence u(n) applied to the channel input consist of a white-noise
sequence of zero mean and unit variance.
1, l 0
r (l )
0, l0
For d(n) supplied to the equalizer, we assume the availability of a delayed
"replica" of the transmitted sequence. This d(n) may be generated by using
another feedback shifter of identical design to that used to supply the original
data sequence u(n). The two feedback shift registers are synchronized with each
other such that we may set
d(n) = u(n)
Thus the cross-correlation between the transmitted sequence u(n) and the desired
response d(n) is defined by
1, l0
p(l )
0, l 1, 2, ..., N 43
1, l 0
wl
0, l 1, 2, ..., N
N
1, l 0
hc k l k
l 1, 2, ..., N
k N 0,
44
Given the impulse response of the channel
characterized by the coefficients c-N , ,c-1, , c0, c1,
, cN we may use above Eq. to solve for the
unknown tap-weights h-N , ,h-1, , h0, h1, , hN of
the equalizer.
In the literature on digital communications, an
equalizer designed in accordance the above Eq. is
referred to as a zero-forcing equalizer. The equalizer
is so called because, with a single pulse transmitted
over the channel, it "forces" the receiver output to
be zero at all the sampling instances, except for the
time instant that corresponds to the transmitted
pulse.
45
Application Channel Equalization - MMSE
L v(n) M
x(n) y(n) z(n) (n)
Channel, h + Filter, w - +
x(n-)
Delay,
46
Application Channel Equalization
MMSE cost function is:
47
Application Channel Equalization
Combine last two equations Convolution
hL-1
x
Toeplitz matrix performs convolution -2 ???
Compact form of the filter output
48
Application Channel Equalization
Rewrite the MMSE cost function
49
Application Channel Equalization
Quadratic function gradient is zero at minimum
And Jmin is
50
Application Linearly Constrained
Minimum Variance (LCMV) Filter
Problem:
1. We want to design an FIR filter which suppresses all frequency
components of the filter input except o, with a gain of g at o.
51
51
Application Linearly Constrained
Minimum - Variance Filter
Problem:
2. We want to design a beamformer which can resolve an incident
wave coming from angle o (with a scaling factor g), while at the same
time suppress all other waves coming from other directions.
53
Application Linearly Constrained
Minimum - Variance Filter
Mathematical model:
Filter output | Beamformer output
Normalized angular freq. with respect
to the sampling rate
g is a complex valued gain
54
Application Linearly Constrained
Minimum - Variance Filter
Cost function: output power quadratic convex
Constraint : linear
Method of Lagrange multipliers can be utilized to solve the problem.
55
Application Linearly Constrained
Minimum - Variance Filter
Rewrite the equations in matrix form:
* j0 j0 ( M 1) T
Rw o s(0 ) where s(0 ) 1 e e
2
Hence *
to find
For o, wo is
the linearly Constrained Minimum-Variance (LCMV) beamformer
For o, wo is
the linearly Constrained Minimum-Variance (LCMV) filter
56
Minimum-Variance Distortionless Response
Beamformer/Filter
Distortionless set g=1, then
58
Summary
For stationary signals, the MSE is a quadratic function of
linear filter coefficients.
optimal linear filter in the MMSE sense is found by
setting gradients to zero
orthogonality principle
Wiener filter.
It depends on the second order statistics.
It can be used as an approximation, if the signals are
locally stationary.
A competing optimization criterion is to minimize the filter
output mean
power (variance) given constraints on desired outputs
optimization by the method of Lagrange multipliers.
59
Generalized Sidelobe Cancellers
Continuing with the discussion of the LCMV narrowband beamformer
defined by the linear constraint of Eq.(2.76), we note that this constraint
represents the inner product
in which w is the weight vector and s(0 ) is the steering vector pointing
along the electrical angle 0 . The steering vector is an M-by-1 vector,
where M is the number of antenna elements in the beamformer. We
may generalize the notion of a linear constraint by introducing multiple
linear constraints defined by
CHw=g (2.91)
60
Generalized Sidelobe Cancellers
0
the narrowband beamformer is constrained to preserve a signal of
interest impinging on the array along the electrical angle o and, at
the same time, to suppress an interference known to originate along
the electrical angle l.
61
Generalized Sidelobe Cancellers
Let the columns of an M-by-(M - L) matrix Ca be defined as a basis
for the orthogonal complement of the space spanned by the columns
of matrix C. Using the definition of an orthogonal complement, we
may thus write
CH Ca 0 (2.92)
or, just as well,
Ca H C 0 (2.93)
The null matrix 0 in Eq.(2.92) is L-by-(M - L), whereas in Eq.(2.93)
it is (M - L)-by-L; we naturally have M > L. We now define the M-
by-M partitioned matrix
U [C Ca ] (2.94)
whose columns span the entire M-dimensional signal space. The
inverse matrix U-1 exists by virtue of the fact that the determinant of
matrix U is nonzero.
62
Generalized Sidelobe Cancellers
Next, let the M-by-1 weight vector of the beamformer be written in
terms of the matrix U as w Uq (2.95)
Equivalently, the M-by-1 vector q is defined by
q U-1w (2.96)
Let q be partitioned in a manner compatible with that in Eq.(2.94),
as shown by v
q
(2.97)
w a
where v is an L-by-1 vector and the (M - L)-by-l vector wa is that
portion of the weight vector w that is not affected by the constraints.
We may then use the definitions of Eqs. (2.94) and (2.97) in
Eq.(2.95) to write v
w [C Ca ] Cv C w
a a (2.98)
w a 63
Generalized Sidelobe Cancellers
We may now apply the multiple linear constraints of Eq.(2.91),
(CHw=g) obtaining
C Cv C Ca w a g
H H
2.99
But, from Eq.(2.92), we know that CH Ca is zero; hence, Eq.(2.99)
reduces to
C Cv g
H
2.100
Solving for the vector v, we thus get
v C C g
H 1
(2.101)
which shows that the multiple linear constraints do not affect wa
64
Generalized Sidelobe Cancellers
wq=Cv=C(CHC)-1g (2.102)
which is orthogonal to the columns of matrix Ca by virtue of the
property described in Eq.(2.93); the rationale for using the subscript
q in wq will become apparent later. From this definition, we may use
Eq.(2.98) to express the overall weight vector of the beamformer as
w=wq-Cawa (2.103)
65
Generalized Sidelobe Cancellers
CHwq=g (2.104)
Equation (2.104) shows that weight vector wq is that part of the
weight vector w which satisfies the constraints. In contrast, the
vector wa is unaffected by the beamformer.
66
Generalized Sidelobe Cancellers
where
67
FIGURE 2.11 (a) Block diagram of generalized sidelobe
canceller. (b) Reformulation of the generalized sidelobe
cancelling problem as a standard optimum filtering problem.
68
Generalized Sidelobe Cancellers
69
Generalized Sidelobe Cancellers
If we now define
and
where d(n) plays the role of a desired response for the GSC and
x(n) plays the role of input vector, as depicted in Fig.2.11(b).
70
Generalized Sidelobe Cancellers
We thus see that the combined use of vector wq and matrix Ca has
converted the linearly constrained optimization problem into a
standard optimum filtering problem.
In particular, we now have an unconstrained optimization problem
involving the adjustable portion wa of the weight vector, which may
be formally written as
71
Generalized Sidelobe Cancellers
The cost function of Eq.(2.111) is quadratic in the unknown vector
wa , which, as previously stated, embodies the available degrees of
freedom in the GSC.
Most importantly, this cost function has exactly the same
mathematical form as that of the standard wiener filter defined in
Eq.(2.50).
Accordingly, we may readily use our previous results to obtain the
optimum value of wa as
The matrix Ca has full rank, and the correlation matrix R is positive
definite, since the incoming data always contain some form of
additive sensor noise, with the result that Rx is nonsingular.
Accordingly, we may rewrite the optimum solution of Eq.(2.114) as
73
Generalized Sidelobe Cancellers
Let Po denote the minimum output power of the GSC attained by using
the optimum solution wao. Then adapting the previous result derived in
Eq.(2.49) for the standard Wiener filter and proceeding in a manner
similar to that just described, we may express Po as
Now consider the special case of a quiet environment, for which the
received signal consists of white noise acting alone. Let the
corresponding value of the correlation matrix R be written as
74
Generalized Sidelobe Cancellers
where I is the M-by-M identity matrix and 2 is the noise variance.
Under this condition we readily find, from Eq.(2.117),that
75
Generalized Sidelobe Cancellers
Filtering Interpretations of Wq and Ca
The quiescent weight vector wq and matrix Ca play critical roles of their
own in the operation of the GSC. To develop physical interpretations of
them, consider an MVDR spectrum estimator (formulated in temporal
terms) for which we have
and
Hence, the use of these values in Eq. (2.102) yields the corresponding
value of the quiescent weight vector, viz.,
76
Generalized Sidelobe Cancellers
which represents an FIR filter of length M-Be frequency response of
this filter is given by
79
Generalized Sidelobe Cancellers
FlGURE 2.12(a) Interpretation of wHq s() as the response of an FIR filter. (b) Interpretation of
each column of matrix Ca as a band-rejection filter. In both parts of the figure It is assumed
that 0 =1