Practice Exercise - Chapter 2
Practice Exercise - Chapter 2
Practice Exercise - Chapter 2
Exercises
A = [ !l B = [! l and C = [! J
(a) (A')' = A
(b) (C')-1 = cc-1)'
Q = [
5
13
112]
13
is an orthogonal matrix.
2.6. Let
A= [ 9
-2
-2]
6
(a) Is A symmetric?
(b) Show that A is positive definite.
I 04 Chapter 2 Matrix Algebra and Random Vectors
A = [1 2]
2 -2
find the eigenvalues A1 and A2 and the associated normalized eigenvectors e 1 and e 2 •
Determine the spectral decomposition (2-16) of A.
2.9. Let A be as in Exercise 2.8.
(a) Find A- 1•
(b) Compute the eigenvalues and eigenvectors of A- 1.
(c) Write the spectral decomposition of A-1, and compare it with that of A from
Exercise 2.8.
2.10. Consider the matrices
A =
4
[ 4.001
4.001
4.002
J and B =
[4
4.001
4.001
4.002001
J
These matrices are identical except for a small difference in the (2, 2) position.
Moreover, the columns of A (and B) are nearly linearly dependent. Show that
A_, ( -3)8- 1. Consequently, small changes-perhaps caused by rounding--can give
substantially different inverses.
2.11. Show that the determinant of the p X p diagonal matrix A= {a;i} with a;i = 0, i j, *
is given by the product of the diagonal elements; thus, IA I = a 1 1a 22 · · · aPr
Hint: By Definition 2A.24, I A I = a 11 A 11 + 0 + · · · + 0. Repeat for the submatrix
A 1 1 obtained by deleting the first row and first column of A.
2.12. Show that the determinant of a square symmetric p X p matrix A can be expressed as
the product of its eigenvalues A1 , A2 , ... , AP; that is, IA I = ITf= 1 A;.
Hint: From (2-16) and (2-20), A =PAP' with P'P =I. From Result 2A.11(e),
/A/= /PAP'/= /P //AP' I= /P /I A //P'/ =/A 1/1/,since/1/ = /P'P/ = /P' 1/P/. Apply
Exercise 2.11.
2.13. Show that IQ I = + 1 or - 1 if Q is a p X p orthogonal matrix.
Hint: /QQ' I = /1/. Also, from Result 2A.ll, JQ 1/Q' I = /Q /2 . Thus, IQ /2 = /1/. Now
use Exercise 2.11.
2.14. Show that Q' A Q and A have the same eigenvalues if Q is orthogonal.
(p><p)(p><p)(p><p) (p><p)
Hint: Let A be an eigenvalue of A. Then 0 = IA - All. By Exercise 2.13 and Result
2A.ll(e),wecanwrite0 = JQ'I/A- AII/QJ = /Q'AQ- Al/,sinceQ'Q =I.
2.1 S. A quadratic form x 'Ax is said to be positive definite if the matrix A is positive definite.
Is the quadratic form 3xt + 3xi - 2x 1x 2 positive definite?
2.1 6. Consider an arbitrary n X p matrix A. Then A' A is a symmetric p X p matrix. Show
that A' A is necessarily nonnegative definite.
Hint: Sety = Axsothaty'y = x'A'Ax.
Exercises I 05
A=[::
(a) Calculate AA' and obtain its eigenvalues and eigenvectors.
(b) Calculate A' A and obtain its eigenvalues and eigenvectors. Check that the nonzero
eigenvalues are the same as those in part a.
(c) Obtain the decomposition of A.
2.23. Verify the relationships V 112pv 112 = I and p = (V 1f2)- 1I(V 112f\ where I is the
p X p population covariance matrix (2-32)], pis the p X p population cor-
relation matrix [Equation (2-34)], and V 12 is the population standard deviation matrix
[Equation (2-35)]. (Ignore this question)
2.24. Let X have covariance matrix
Find
(a) I- 1
(b) The eigenvalues and eigenvectors of I.
(c) The eigenvalues and eigenvectors of I- 1.
106 Chapter 2 Matrix Algebra and Random Vectors
I=
25
-2
-2
4 1
4]
[
4 1 9
(a) Determine p VI/2.
(b) Multiply your matrices to check the relation Vif2pVI/2 = I.
2.26. Use I as given in Exercise 2.25.
(a) Find p 13 •
(b) Find the correlation between XI and +
2.27. Derive expressions for the mean and variances of the following linear combinations in
terms of the means and covariances of the random variables XI, X 2 , and X 3 •
(a) XI- 2X 2
(b) -XI+ 3X2
(c) XI + Xz + X3
(e) XI + 2Xz - X 3
(f) 3XI - 4X2 if XI and X 2 are independent random variables.
2.28. Show that (Ignore this question)
where c) = [c 11 , ci 2 , ... , ci p] and ci = [c2 I, c22 , •.. , c2 p]· This verifies the off-diagonal
elements CixC' in (2-45) or diagonal elements if ci = c2 •
Hint: By (2-43),ZI- E(ZI) = cii(XI- 1-ti) + ··· + cip(Xp- 1-tp) and
Z 2 - E(Zz) = c21(XI- 1-ti) + ··· + Czp(Xp- /-tp).SoCov(ZI,Z2 ) =
E[(ZI - E(ZI))(Zz- E(Zz))] = E[(c11(XI- 1-ti) +
··· + Cip(Xp- 1-tp))(czi(XI - 1-ti) + Czz(Xz- 1-tz) + ··· + czp(Xp- 1-tp))].
The product
(c11(XI- 1-ti) + cdXz- 1-tz) + · ··
+ cip(Xp- l-tp))(c2l(XI- 1-ti) + czz(Xz- + ··· + Czp(Xp- 1-tp))
p p
= 2: 2: CifCzm(Xe- 1-tf)(Xm- 1-tm)
e=I m=l
Verify the last step by the definition of matrix multiplication. The same steps hold for all
elements.
Exercises I0 7
2.29. Consider the arbitrary random vector X' = X 2 , X 3 , X 4 , X 5 ] with mean vector
JL' = [JLJ. IL2· J.LJ, IL4• J.L 5 ]. Partition X into
X=
where
l
3 0 12 02]
0 1
Ix = 2 1 9 -2
2 0 -2 4
Partition X as
Let
A = [1 2] and B = [ =J
and consider the linear combinations AX(' l and BX(2). Find
(a) E(X(ll)
(b) E(AX(ll)
(c) Cov (X(!))
(d) Cov(AX(!l)
(e) E(X(2l)
(f) E(BX(2l)
(g) Cov(X(2l)
(h) Cov (BX< 2l)
(i) Cov(X< 1>, X(2l)
(j) Cov (AX(ll, BX(2))
2.31. Repeat Exercise 2.30, but with A and B replaced by
A = [1 -1] and B =[ - J
108 Chapter 2 Matrix Algebra and Random Vectors
2.32. You are given the random vector X' = [X1 , X2 , ..• , X 5 J with mean vector
P.x = [2, 4, -1, 3, OJ and variance-covariance matrix
I I
4 -1 2 -;z 0
-1 3 -1 0
I
Ix = 2 6 -1
-:zI -1 1 4 0
0 0 -1 0 2
Partition X as
Let
A = D-J and B= U _J
and consider the linear combinations AX(lJ and BX(2). Find
(a) E(X(Il)
(b) E(AX('l)
(c) Cov(X( 1))
(d) Cov (AX(Il)
(e) E(X(2l)
(f) E(BX( 2 ))
(g) Cov(X( 2))
(h) Cov(BX( 2l)
(i) Cov(X(l), X(2))
0) Cov(AX( 1l, BXI 2l)
2.33. Repeat Exercise 2.32, but with X partitioned as
A=U -1 OJ 1 3 and B =
[1 2]
1 -1
2.34. Considerthevectorsb' = [2, -1, 4, OJ and d' = [-1, 3, -2, 1J. Verify the Cauchy-Schwarz
2
inequality(b'd) (b'b)(d'd).
Exercises I09
2.35. Using the vectors b' = [-4, 3] and d' = [1, 1], verify the extended Cauchy-Schwarz
2
inequality (b'd) :s; (b'Bb)(d'B- 1d) if
B =[ 2 -2]
-2 5
2.36. Fmd the maximum and minimum values of the quadratic form 4_x1 + + 6x,xz for
all points x' = [x 1 , x 2 ] such that x'x = 1.
2.37. With A as given in Exercise 2.6, fmd the maximum value of x' Ax for x' x = 1.
2.38. Find the maximum and minimum values of the ratio x' A xjx'x for any nonzero vectors
x' = [x 1 ,x2 ,x3 ] if
13 -4
A= -4 13
[
2 -2 10
2.39. Show that
s r
A B C has (i, j)th entry 2: 2: a;ebekCkj
(rXs)(sXr)(tXv)
r
Hint: BC has ( e, j)th entry 2: bekCki = de;· So A(BC) has (i, j)th element
l30 ]
0
0 3 0
Ix = 0 0 3
0 0 0
Let
References
1. Bellman, R.lntroduction to Analysis (2nd ed.) Philadelphia: Soc for Industrial &
Applied Math (SIAM), 1997.
2. Eckart, C., and G. Young. "The Approximation of One Matrix by Another of Lower
Rank." Psychometrika,1 (1936), 211-218.
3. Graybill, F. A. Introduction to Matrices with Applications in Statistics. Belmont, CA:
Wadsworth, 1969.
4. Halmos, P.R. Finite-Dimensional Vector Spaces. New York: Springer-Verlag, 1993.
5. Johnson, R. A., and G. K. Bhattacharyya. Statistics: Principles and Methods (5th ed.) New
York: John Wiley, 2005.
6. Noble, B., and J. W. Daniel. Applied Linear Algebra (3rd ed.). Englewood Cliffs, NJ:
Prentice Hall, 1988.