Scribd
Upload
58 views

AI-ML Class-16

The document defines orthogonal projection of a vector v onto a unit vector u as Pu(v) = <u,v>u. It provides an example of finding the orthogonal projection of a vector y onto a vector u, and expressing y as the sum of two orthogonal vectors, one in the span of u and one orthogonal to u. It then proves that any orthogonal set in an inner product space is linearly independent. It defines an orthonormal basis as an orthogonal basis whose vectors have unit length. It demonstrates constructing an orthonormal basis for R2 using the Gram-Schmidt orthogonalization process.

Uploaded by

Mihir Patel
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
58 views

AI-ML Class-16

The document defines orthogonal projection of a vector v onto a unit vector u as Pu(v) = <u,v>u. It provides an example of finding the orthogonal projection of a vector y onto a vector u, and expressing y as the sum of two orthogonal vectors, one in the span of u and one orthogonal to u. It then proves that any orthogonal set in an inner product space is linearly independent. It defines an orthonormal basis as an orthogonal basis whose vectors have unit length. It demonstrates constructing an orthonormal basis for R2 using the Gram-Schmidt orthogonalization process.

Uploaded by

Mihir Patel
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 12

Definition

Let u be a unit vector and let v be any arbitrary vector. The orthogonal
projection of v along the vector u is defined as Pu (v ) = ⟨u, v ⟩u. If u is
u
any non-zero vector Pu (v ) = ⟨ ∥u∥ u
, v ⟩ ∥u∥ = ⟨u,v ⟩
⟨u,u⟩ u

Note: The vector v − Pu (v ) is called the component of vector v


orthogonal to u.
Observe that v = Pu (v ) + v − Pu (v )

Dr. Sabitha D’Souza September 22,2022 1 / 13


Example
Let y = [7 , 6] and u = [4 , 2]. Find the orthogonl projection of y onto u.
Also write y as the sum of 2 orthogonal vectors one in span{u} and one
orthogonal to u.

Dr. Sabitha D’Souza September 22,2022 2 / 13


Answer:
  y = P + (y − Pu (y ))
u (y ) 

7 8 −1
 = + 
6 4 2
What is the distance from vector y to span{u}
p √
Ans= ∥y − Pu (y )∥ = (−1)2 + 22 = 5

Dr. Sabitha D’Souza September 22,2022 3 / 13


Theorem
Any orthogonal set in an inner product space V is linearly independent

Proof.
Let {v1 , v2 , . . . , vk } be an orthogonal set of non-zero vectors. Assume that
c1 v1 + c2 v2 + . . . + ck vk = 0 , ci ∈ R.
Then ⟨(c1 v1 + c2 v2 + . . . + ck vk ), vj ⟩ = 0 for 1 ≤ j ≤ n and thus
Pk
i=1 ci ⟨vi , vj ⟩ = 0

Since ⟨vi , vj ⟩ = 0 for i ̸= j, the only term not vanishing is cj ⟨vj , vj ⟩. Since
vj ̸= 0, we have ⟨vj , vj ⟩ =
̸ 0. This implies cj = 0. As j is arbitrary, we see
that cj = 0 for all j.

Dr. Sabitha D’Souza September 22,2022 4 / 13


Definition
A basis {v1 , v2 , . . . , vn } of V is said to be orthonormal if
⟨vi , vj ⟩ = 0, for i ̸= j
= 1 for i = j.
This means that {v1 , v2 , . . . , vn } are pairwise orthogonal and the length of
each vector is unity.

Dr. Sabitha D’Souza September 22,2022 5 / 13


Example
• The standard basis of R n is an orthonormal basis.

• The basis { e1√+e2 , e1√−e2 } is an orthonormal basis of R 2


2 2

Dr. Sabitha D’Souza September 22,2022 6 / 13


Show that (R 2 , ⟨, ⟩) has an orthonormal basis.
Proof.
It is enough to produce an orthogonal basis for (R 2 , ⟨, ⟩). Let {v1 , v2 } be a
basis for (R 2 , ⟨, ⟩). Let u1 = v1 . We have to find a non zero vector u2
which is orthogonal to u1 .
Let u2 = v2 − Pu1 (v2 ). We shall show that u1 ⊥ u2 .
⟨u1 , u2 ⟩ = ⟨u1 , v2 − Pu1 (v2 )⟩ = ⟨u1 , v2 ⟩ − ⟨u1 , Pu1 (v2 )⟩
⟨u1 ,v2 ⟩ ⟨u1 ,v2 ⟩
= ⟨u1 , v2 ⟩ − ⟨u1 , ⟨u 1 ,u1 ⟩
.u1 ⟩ = ⟨u1 , v2 ⟩ − ⟨u1 ,u1 ⟩ ⟨u1 , u1 ⟩

= ⟨u1 , v2 ⟩ − ⟨u1 , v2 ⟩
=0

Dr. Sabitha D’Souza September 22,2022 7 / 13


Problem

⟨v1 ,v2 ⟩
Further u2 ̸= 0. Suppose u2 = 0 then we get v2 = Pu1 (v2 ) = ⟨v1 ,v1 ⟩ .v1 .

This implies v2 is a scalar multiple of v1 . A contradiction to {v1 , v2 } is a


basis of R 2 . This proves that {v1 , v2 } is an orthogonal basis of (R 2 , ⟨, ⟩).

Dr. Sabitha D’Souza September 22,2022 8 / 13


Gram-Scmidth Orthogonalization Process

Theorem
Let V be any inner product space. Then V has an orthonoral basis

Proof.
It is enough to produce an orthogonal basis of V . Let {v1 , v2 , . . . , vn } be a
basis of V . Let u1 = v1 . In case of R 2 we have already proved that
⟨v2 ,u1 ⟩
u2 = v2 − ⟨u1 ,u1 ⟩ .u1 and u2 ⊥ u1 . Thus ⟨u1 , u2 ⟩ = 0. Also u2 ̸= 0. For if
⟨v2 ,u1 ⟩
u2 = 0 then v2 = ⟨u 1 ,u1 ⟩
.v1 , will imply {v1 , v2 } is linearly dependent, a
⟨v3 ,u1 ⟩ ⟨v3 ,u2 ⟩
contradiction. Let u3 = v3 − ⟨u 1 ,u1 ⟩
.u1 − ⟨u2 ,u2 ⟩
.u2 . Then we see that
⟨u3 , u1 ⟩ = 0 = ⟨u3 , u2 ⟩ = 0 . Also u2 ̸= 0. For otherwise, v3 is a linear
combination of u1 and u2 , a contradiction to the independedent set.
Dr. Sabitha D’Souza September 22,2022 9 / 13
Pk−1 ⟨vk ,ui ⟩
Proceeding as above, by induction, we define uk = vk − i=1 ⟨ui ,ui ⟩ .ui .

Then ⟨uk , ui ⟩ = 0 for all 1 ≤ i ≤ k − 1 and as before uk ̸= 0. We have


ui
thus produced an orthogonal basis {u1 , u2 , . . . , un } of V . Then ∥ui ∥ is an
orthonormal basis of V .

Dr. Sabitha D’Souza September 22,2022 10 / 13


Gram-Scmidth Orthogonalization process

Given a basis {v1 , v2 , . . . vn }.


Set u1 = v1
⟨v2 ,u1 ⟩
u2 = v2 − ⟨u1 ,u1 ⟩ .u1
⟨v3 ,u1 ⟩
⟨v3 ,u2 ⟩
u3 = v3 − − ⟨u
⟨u1 ,u1 ⟩ .u1
2 ,u2 ⟩
.u2 .
Pk−1 ⟨vk ,ui ⟩
In general uk = vk − i=1 ⟨ui ,ui ⟩ .ui
Then {u1 , u2 , . . . , un } is an orthogonal basis.

Dr. Sabitha D’Souza September 22,2022 11 / 13


Problems

• Construct an orthogonal basis for R 3 from the set


{v1 = [3 , 0 , 4], v2 = [−1, 0, 7], v3 = [2, 9, 11]}

• Construct an orthogonal basis for R 3 from the set


S = {(1, −1, 0), (2, −1, −2), (1, −1, −2)}

Dr. Sabitha D’Souza September 22,2022 12 / 13

You might also like