Mat9004 Lecture08 Empty

Linear algebra: Introduction
Retap F CH
:÷:÷÷÷÷÷÷÷:: .i ÷
•
Fundamental Theorem of calculus (
FTC)
Sabfcxldx Fcb) Fca ) if Fox ) is

(
-
,
of far ) )
=
antideriv .
• Rules of integration
etc )
"
( linear
"
,
catch up session
Friday 28 August 11am :
10 -
MAT9004 - Week 4 - Lecture 8

Topics for today
Vectors in Euclidean space Rd

Addition and scalar multiplication
Geometric interpretation of vectors
The dot product
Introduction to matrices
What is a matrix?
Matrix addition and scalar multiplication
MAT9004 Lecture 8 Slide 2/28

Pie graph for ‘The mathematics of machine learning’
(Source: datascience.ibm.com)

The Euclidean vector space
Rd is the set of all d-tuples of real numbers
0 1
x1
B C
We write @ ... A for the d-tuple of numbers x1 , . . . , xd .
xd
80 1 9
>
< 1 x >
=
B . C
Hence R = @ .. A : x1 , . . . , xd 2 R .
d
>
: >
;
xd
Elements of Rd are called vectors
Vectors are sometimes indicated by an arrow, e.g. ~v
~ . We will use v and w instead.
and w
Example. The set R3 consists of tuples like

0 1 0 1 0 1
1 2.5 e
@0A , @ 0.4 A , @ 2⇡ A .
p
0 2 17
Addition and scalar multiplication
Two elements of Rd can be added:
0 1 0 1 0 1
x1 y1 x1 + y1
B .. C B .. C B .. C
@ . A+@ . A=@ . A
xd yd xd + yd
An element of Rd can be multiplied by a scalar c 2 R:

0 1 0 1
x1 cx1
B C B C
c @ ... A = @ ... A
xd cxd
Example.
0 1 0 1 0 1 0 1 0 1
1 2 3 1 2
@ 1A + @1A = @0A , 2 @ 1A = @ 2A .
0 3 3 0 0

Geometric interpretation of vectors
✓ ◆
3
u=
1
✓ ◆
a
A 2-dimensional vector can
b
be represented by an arrow from
(0, 0) to (a, b).
However, in general a vector has
no defined starting point, it
rather describes a displacement.
✓ ◆
1
v=
4

Geometric interpretation of addition
✓ ◆ ✓ ◆
3 1 ✓ ◆
Vectors u = and v = 2
1 4 u+v =w =
3

Minions!
A vector has direction and length but no position.

Geometric interpretation of scalar multiplication
✓ ◆ ✓ ◆
4 2
Vector u = Vectors 12 u = v =
2 1
✓ ◆
8
and 2u = w =
4

How to subtract a vector
What is a b? It’s just addition: a b = a + ( 1) · b.
Same for vectors:

Same for vectors:

Same for vectors:

Same for vectors:

A line segment represented by vectors
The black vector is v + 13 (u v)

A line segment represented by vectors
The black vector is v + 13 (u v)
1 1 1 2
=v+ u v = u + v.
3 3 3 3
The dashed line is the line
interval joining points u and v. It
contains the set of points of the
form v + ↵(u v) where
0 6 ↵ 6 1. This simplifies to:
The line interval joining points u and v contains exactly the

points corresponding to all the vectors in
{↵u + (1 ↵)v : ↵ 2 [0, 1]}.

Example: line interval
The set of points on the line interval joining (1, 4) and ( 1, 7)

correspond to the set of vectors of the form
✓ ◆ ✓ ◆ ✓ ◆
1 1 1 + 2↵
↵ + (1 ↵) = ,
4 7 7 3↵
so the line interval is {( 1 + 2↵, 7 3↵) : ↵ 2 [0, 1]}.
Higher dimensions: We have shown all this in two dimensions,

but the analogous things work for higher dimensions.

Medians with vectors
The median of a triangle joins a vertex to the

midpoint of the opposite side.


The lines along w and x and y are medians;
do they always meet in a single point?


The lines along w and x and y are medians;
do they always meet in a single point?
Note: w = 12 (u + v), and the point 2/3 of the
way along w’s median is 23 w = 13 u + 13 v.
Note: x = 12 v u, and the point 2/3 of the
way along x’s median is u + 23 x = 13 u + 13 v.
2
The points 3 of the way along any two medians are equal!
The three medians of a triangle meet in one point.

Linear combinations
A vector w is a linear combination of vectors v1 , . . . vn if we

can write
w = a1 v1 + · · · + an vn
for some real numbers a1 , . . . , an .

0 1 0 1 0 1 0 1
1 1 0 0
Example. @2A is a linear combination of @0A , @1A , @0A
3 0 0 1
since
0 1 0 1 0 1 0 1
1 1 0 0
@2A = 1 @0A + 2 @1A + 3 @0A .
3 0 0 1

Linear independence
v1 , . . . , vn are linearly dependent if one of the vj is a linear
combination of the other vectors v1 , . . . , vj 1 , vj+1 , . . . , vn .
If not linearly dependent, they are linearly independent.
Example.
0 1 0 1 0 1
1 0 1
@0A , @1A , @ 1A are linearly dependent since
0 0 0
0 1 0 1 0 1
1 1 0
@ 1A = 1 @0A + ( 1) @1A .
0 0 0
Think of linear dependence as redundancy in your data: you
can reconstruct one of the vectors by a linear combination of
the others.
Independence = no redundancy.
Exercise
Decide whether the following vectors are linearly independent:
✓ ◆ ✓ ◆
1 3
(a) v1 = , v2 =
5 15
0 1 0 1
1 2
(b) v1 = @ 1A , v2 = @ 1 A
0 0
0 1 0 1 0 1
1 3 0
(c) v1 = @1A , v2 = @0A , v3 = @ 3A
1 2 1

Exercise
Decide whether the following vectors are linearly independent:
✓ ◆ ✓ ◆
1 3
(a) v1 = , v2 =
5 15
0 1 0 1
1 2
(b) v1 = @ 1A , v2 = @ 1 A
0 0
0 1 0 1 0 1
1 3 0
(c) v1 = @1A , v2 = @0A , v3 = @ 3A
1 2 1
Answers:
(a) No, since v2 = 3v1 There is a general method to
(b) Yes determine linear independence:
It uses next week’s topic.
(c) No, since v3 = v2 + ( 3)v1

The dot product
Given two vectors

0 1 0 1
v1 w1
B C B C
v = @ ... A and w = @ ... A ,
vd wd
we define their dot product as the number
v · w = v1 w 1 + v2 w 2 + · · · + vd w d .
It’s also called scalar product or inner product.

Sometimes it’s written as hv, wi.
Why? Many reasons!
It can be used to define angles, lengths, and more complicated products!

The Euclidean norm
0 1
v1
B C
The (Euclidean) norm of v = @ ... A is defined as
vd
q
|vk = v12 + v22 + · · · + vd2 .
For vectors in dim 2 and 3, Pythagoras’ theorem tells us kvk

is in fact the length of v.
p
Note that kvk = v · v, so we’ve used the dot product to
define the norm.

Examples
01 0 1
1 2
If v = @ 1A and w = @2A , then
0 1
v · w = 1 ⇥ 2 + ( 1) ⇥ 2 + 0 ⇥ 1 = 0
q p
2 2
kvk = 1 + ( 1) + 0 = 2 2
p
kwk = 22 + 22 + 12 = 3.

Examples
01 0 1
1 2
If v = @ 1A and w = @2A , then
0 1
v · w = 1 ⇥ 2 + ( 1) ⇥ 2 + 0 ⇥ 1 = 0
q p
2 2
kvk = 1 + ( 1) + 0 = 2 2
p
kwk = 22 + 22 + 12 = 3.
Exercise
Compute v · w, kvk, kwk for
0 1 0 1
1 2 ✓ ◆ ✓ ◆
4 2
(a) v = @1A , w = @ 3 A , (b) v= ,w =
2 1
1 1

Exercise
Compute v · w, kvk, kwk for
0 1 0 1
1 2 ✓ ◆ ✓ ◆
4 2
(a) v = @1A , w = @ 3 A , (b) v= ,w =
2 1
1 1
(a) (b)
v·w =2 v·w = 10
p p
kvk = 3 kvk = 20
p p
kwk = 14 kwk = 5

Orthogonal vectors
Vectors v and w are called orthogonal if v · w = 0.
Two vectors in 2 or 3 dimensions are perpendicular if and

only if they are orthogonal, i.e. their dot product is 0.
Examples.
✓ ◆ ✓ ◆
1 1
and are orthogonal.
1 1
0 1 0 1 0 1
1 0 0
@0A, @1A, @0A are pairwise orthogonal
0 0 1

Orthogonality and linear independence
and
nonzero
If v1 , . . . , vn
v
are pairwise orthogonal, then they are linearly
independent.
Challenge: Can you show that this is true?

Note that the converse does not hold in general:
✓ ◆ ✓ ◆
1 1
and are linearly independent but not orthogonal.
0 1

Matrices
An m ⇥ n matrix is a rectangular array A with m rows and

n columns; the entry of a matrix A at row i and column j is
denoted by Aij . We also write A = (Aij )i,j .
Examples
✓ ◆
1 1
is a 2 ⇥ 2 matrix
0 2
0 1
1.5 0 2 1
@0 0.7 1.8 15A is a 3 ⇥ 4 matrix
0 0 0 0
The dimensions of an m ⇥ n matrix are m and n or m ⇥ n.

Matrix addition
The sum of two m ⇥ n matrices A and B is the component-

wise addition of the matrices, so A + B = (Aij + Bij )i,j , or:
0 1 0 1
a11 . . . a1n b11 . . . b1n
B .. .. C + B .. .. C
@ . . A @ . . A
am1 . . . amn bm1 . . . bmn
0 1
a11 + b11 . . . a1n + b1n
B .. .. C
=@ . . A
am1 + bm1 . . . amn + bmn
Example.
✓ ◆ ✓ ◆ ✓ ◆
1 1 3 2 4 1
+ =
2 0 1 0 3 0

Scalar multiplication
The product of c 2 R and an m ⇥ n matrix A is the
component-wise multiplication of A by c, so cA = (cAij )i,j ,
or:
0 1 0 1
a11 ... a1n ca11 ... ca1n
B .. C = B .. .. C
c @ ... . A @ . . A
am1 . . . amn cam1 . . . camn
Example.
✓ ◆ ✓ ◆
0 1 0 3
3 =
2 3 6 9
A 1 ⇥ 1 matrix looks like a number, and behaves like one too:
3(1) (7) = ( 4).

Vectors as matrices
✓ ◆
0
A vector is just an m ⇥ 1 matrix, e.g. A = = v.
2
Addition and scalar multiplication of vectors mean the same
thing, whether you think of them as a vector or a matrix.
But you cannot compute the dot product of two matrices (unless
they are vectors of the same dimension).
What about matrix multiplication? We could multiply two
matrices of the same dimension as follows:
✓ ◆✓ ◆ ✓ ◆
0 1 4 5 ??? 0 · 4 1 · 5
=
2 3 6 7 2·6 3·7

Vectors as matrices
✓ ◆
0
A vector is just an m ⇥ 1 matrix, e.g. A = = v.
2
Addition and scalar multiplication of vectors mean the same
thing, whether you think of them as a vector or a matrix.
But you cannot compute the dot product of two matrices (unless
they are vectors of the same dimension).
What about matrix multiplication? We could multiply two
matrices of the same dimension as follows:
✓ ◆✓ ◆ ✓ ◆
0 1 4 5 ??? 0 · 4 1 · 5
=
2 3 6 7 2·6 3·7
While this is one way to define a matrix product (called

Hadamard product), we will not use this here. Next week
we see a more useful definition of matrix multiplication.

Mat9004 Lecture08 Empty

Uploaded by

Copyright:

Available Formats

Mat9004 Lecture08 Empty

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mat9004 Lecture08 Empty

Uploaded by

Copyright:

Available Formats

Linear algebra: Introduction

Sabfcxldx Fcb) Fca ) if Fox ) is

MAT9004 - Week 4 - Lecture 8

Vectors in Euclidean space Rd

MAT9004 Lecture 8 Slide 2/28

MAT9004 Lecture 8 Slide 3/28

Example. The set R3 consists of tuples like

An element of Rd can be multiplied by a scalar c 2 R:

MAT9004 Lecture 8 Slide 5/28

MAT9004 Lecture 8 Slide 6/28

MAT9004 Lecture 8 Slide 7/28

A vector has direction and length but no position.

MAT9004 Lecture 8 Slide 8/28

MAT9004 Lecture 8 Slide 9/28

MAT9004 Lecture 8 Slide 10/28

MAT9004 Lecture 8 Slide 10/28

MAT9004 Lecture 8 Slide 10/28

MAT9004 Lecture 8 Slide 10/28

The black vector is v + 13 (u v)

MAT9004 Lecture 8 Slide 11/28

The black vector is v + 13 (u v)

The line interval joining points u and v contains exactly the

{↵u + (1 ↵)v : ↵ 2 [0, 1]}.

MAT9004 Lecture 8 Slide 12/28

The set of points on the line interval joining (1, 4) and ( 1, 7)

so the line interval is {( 1 + 2↵, 7 3↵) : ↵ 2 [0, 1]}.

Higher dimensions: We have shown all this in two dimensions,

MAT9004 Lecture 8 Slide 13/28

The median of a triangle joins a vertex to the

MAT9004 Lecture 8 Slide 14/28

The median of a triangle joins a vertex to the

MAT9004 Lecture 8 Slide 14/28

The median of a triangle joins a vertex to the

The three medians of a triangle meet in one point.

MAT9004 Lecture 8 Slide 14/28

A vector w is a linear combination of vectors v1 , . . . vn if we

for some real numbers a1 , . . . , an .

MAT9004 Lecture 8 Slide 15/28

MAT9004 Lecture 8 Slide 17/28

MAT9004 Lecture 8 Slide 17/28

Given two vectors

we define their dot product as the number

It’s also called scalar product or inner product.

MAT9004 Lecture 8 Slide 18/28

For vectors in dim 2 and 3, Pythagoras’ theorem tells us kvk

MAT9004 Lecture 8 Slide 19/28

MAT9004 Lecture 8 Slide 20/28

MAT9004 Lecture 8 Slide 20/28

MAT9004 Lecture 8 Slide 21/28

Vectors v and w are called orthogonal if v · w = 0.

Two vectors in 2 or 3 dimensions are perpendicular if and

MAT9004 Lecture 8 Slide 22/28

Challenge: Can you show that this is true?

MAT9004 Lecture 8 Slide 23/28

An m ⇥ n matrix is a rectangular array A with m rows and

The dimensions of an m ⇥ n matrix are m and n or m ⇥ n.

MAT9004 Lecture 8 Slide 24/28

The sum of two m ⇥ n matrices A and B is the component-

MAT9004 Lecture 8 Slide 25/28