Introductory Multi-linear Algebra: From Dual

Spaces to Tensor Products

Hayden Borg
26 April 2020

This work is licensed under a Creative Commons ”Atribution 4.0 Interna-

tional (CC BY 4.0)” license.

1
1 Introduction
In the same way that vector spaces embody linearity, Tensor Products embody
multi-linearity. As such tensors (the elements of tensor products) are a widely
applicable generalization of vectors and Vector Spaces.
In this paper, I seek to give an introductory, abstract understanding of ten-
sors and Tensor Products. This paper assumes knowledge commensurate with
an introductory undergraduate course in Linear Algebra and an undergraduate
course in Abstract Algebra.

2 Dual Spaces
To understand Tensor Products it is easiest to begin with Dual Spaces and
linearity.
Given finite dimensional vector spaces V and W over a field F we define the
set of all linear transformations Hom(V, W ). [1]
Theorem 1. Given two finite dimension vector spaces V and W over a field F ,
Hom(V, W ) forms a vector space over F where. for S, T ∈ Hom(V, W ), α ∈ F,
and ~v ∈ V , vector addition is defined: (S + T )(~v = S(~v ) + T (~v ) and scalar
multiplication is defined: (αS)(~v ) = αS(~v ). [1]
Proof. Let ~v , ~u ∈ V, α, β ∈ F , and w
~ ∈ W.
Grab S, T ∈ Hom(V, W ). Then,
(S + T )(~u + ~v ) = S(~u + ~v ) + T (~u + ~v )
= S(~u) + S(~v ) + T (~u) + T (~v )
= S(~u) + T (~u) + S(~v ) + T (~v ) Vector addition is commutative in W
= (S + T )(~u) + (S + T )(~v )
So, (S + T ) ∈ Hom(V, W ) and addition is closed on Hom(V, W ).
And,
(S + T )(~v ) = S(~v ) + T (~v )
= T (~v ) + S(~v ) Vector addition is commutative in W
= (T + S)(~v )
So, addition in Hom(V, W ) is commutative.
Grab S, T, R ∈ Hom(V, W ). Then,
((S + T ) + R)(~v ) = (S + T )(~v ) + R(~v )
= S(~v ) + T (~v ) + R(~v )
= S(~v ) + (T + R)(~v )
= (S + (T + R))(~v )
So, addition in Hom(V, W ) is associative.

2
 Define O : V → W by O(~v ) = ~0. O ∈ Hom(V, W ). Grab S ∈ Hom(V, W ).
Then, (S +O)(~v ) = S(~v )+O(~v ) = S(~v )+~0 = S(~v ). Addition is commutative, so
(O +S)~v = S(~v ) So, Hom(V, W ) is non-empty and contains an identity element.
~ Define Ŝ : V → W ∈ Hom(V, W ) by Sˆ(~v ) = −w.
Let S(~v ) = w. ~ Then,

(S + Ŝ)(~v ) = S(~v ) + Sˆ(~v )

=w
~ + (−w)
~
~
=0
= O(~v )

So, Ŝ = S −1 and for each S ∈ Hom(V, W ) there exists a S −1 ∈ Hom(V, W ).

Grab S ∈ Hom(V, W ). Then,

(αS)(~u + ~v ) = α(S(~u + ~v )
= α(S(~u + S(~v )
= (αS)(~u) + (αS)(~v )) Scalar multiplication is distributive in W

So, α(S) ∈ Hom(V, W ) and scalar multiplication is closed.

Then consider α(βS),
(α(βS(~v )) = αS(β~v ) = S(αβ~v ) = αβS(~v ) = (αβS)(~v ).
So, scalar multiplication is associative in Hom(V, W ).
Then consider α(S + T ). (S + T )(~v ) = S(~v ) + T (~v ). Note that S(~v ) and
T (~v ) are elements of W . So, (α(S + T )(~v ) = α(S(~v ) + T (~v )) = α(S(~v ) + α(T (~v ).
So, scalar multiplication distributes across vector addition in Hom(V, W ).
Then consider (α + β)S,

((α + β)S)(~v ) = (α + β)(S~v )

= αS(~v ) + βS(~v )
= (αS)(~v ) + (βS)(~v )

So, scalar multiplication is distributive across scalar addition in Hom(V, W ).

Finally consider 1S.

(1S)(~v ) = 1S(~v )
= S(~v ) Definition of 1 in W

So, Hom(V, W ) has multiplicative identity.

Therefore, Hom(V, W ) fulfills the axioms which describe a vector space and
Hom(V, W ) is a vector space for our defined operations.

...

3
 Given a F -vector space V , we define the dual space of V to be V ∗ =
Hom(V, F ). The elements of V ∗ are linear forms, linear functional, or linear
maps. Then, V ∗ is a vector space over F .

Theorem 2. Let B = v~1 , . . . , v~n be a basis of V . For ~v ∈ V where ~v =

a1 v~1 + · · · + an v~n define φi : V → F ∈ Hom(V, F ) by φi (~v ) = ai . Then,
B ∗ = {φ1 , . . . , φn } forms a basis of V ∗ . B ∗ is a dual basis.
Pn
Proof. Let i=1 ρi (φi ) = id and v~k ∈ B.
Then,
n
X
ρ(φi (v~k )) = id(v~k )
i=1

ρ1 (φ1 (v~k )) + · · · + ρk (φk (~k)) + · · · + ρn (φn (v~k )) = ~0

ρ1 (~0) + · · · + ρk (~1) + · · · + ρn (~0) = ~0
ρk = 0

So, for each basis vector v~k , ρk = 0. B spans V . V is the domain for V ∗ , so B ∗
is linearly independent.
Grab S ∈ V ∗ and ~v ∈ V . ~v = a1 v~1 + · · · + an v~n .
First note, because S is linear, S(a~v ) = aS(~v ).
Consider,

S(~v ) = S(a1 v~1 + · · · + an v~n )

= S(a1 v~1 ) + · · · + S(an v~n )
= a1 S(v~1 ) + · · · + an S(v~n )
= φ1 (v~1 )S(v~1 ) + · · · + φn (v~n )S(v~n )
= φ1 (v~1 )f1 + · · · + φn (v~n )fn vi ) = fi and fi ∈ F
where S(~
= φ1 (~v )f1 + · · · + φn (~v )fn Note that φi (~
vi ) = φi (~v )

So, B ∗ spans V ∗ and B ∗ forms a basis of V ∗ .

The following corollary is then obvious but important:
Corollary 1 (Theorem 2). Given a finite vector space V , V ' V ∗ .
Proof. dim(V ) = |B| = n = |B ∗ | = dim(V ∗ ). So, dim(V ) = dim(V ∗ ). Vector
spaces of the same dimension are isomorphic.

We can also consider the more general case, Hom(V, W ), where V and W
are vector spaces over field F .
Corollary 2 (Theorem 2). Given finite vector spaces V and W over field F
with dim(V ) = n and dim(W ) = m, dim(Hom(V, W ) = nm

4
Proof. Let ai , bi , ci be scalars in F , {v~1 , . . . , v~n } be a basis of V and {w~1 , . . . , w~m }
be a basis of W .
Consider the set B = {φkl ∈ Hom(V, W ) : φkl (a1 v~1 + · · · + ak v~k + · · · +
an v~n ) =P~l } P
w
n m
Let k=1 l=1 akl φkl = id. Then,
n X
X m
( akl φkl )(v~k ) = O(v~k )
k=1 l=1
n X
X m
(akl (φkl (v~k ))) = O(v~k )
k=1 l=1
m
X
~ l = ~0
akl w
l=1
akl = 0 w
~i are linearly independent

So, for each, vk ∈ {v~1 , . . . , v~n }, akj = 0 for 1 ≤ j ≤ m.

We can then conclude that B is linearly independent.
Without loss of generality, consider S ∈ Hom(V, W ) where S(~v ) = w.
~ Then,

S(~v ) = S(a1 v~1 + · · · + an v~n )

= a1 S(v~1 ) + · · · + an S(v~n )
= a1 (b1 w~1 + . . . bm w~m ) + . . . an (c1 w~1 + . . . cm w~m )
= a1 (b1 φ11 (~v ) + . . . bm φ1m (~v ) + · · · + an (c1 φn1 (~v ) + . . . cm φnm (~v )
= d11 φ11 (~v ) + · · · + dnm φ(~v )

So, any S ∈ Hom(V, W ) can be expressed as a linear combination of φi ∈ B.

So, B spans Hom(V, W ).
Then, B forms a basis of Hom(V, W ). So dim(Hom(V, W )) = |B| = nm.

Proposition 1. First, for all ~v ∈ V, ~v 6= ~0, ∃ϕ ∈ V ∗ such that ϕ(~v ) = 0.

Second, ~v = ~0 if and only if φ(~v ) = 0 for all φ ∈ V ∗ . [4]
Proof. The zero transformation is an element of V ∗ and for all ~v ∈ V where
~v 6= ~0, id(~v ) 6= 0.
First, grab ~0 ∈ V and S ∈ V ∗ . Then, S(~0) = 0S(~v ) = ~0. Then, let ~v ∈ V
and assume φ(~v ) = 0∀φ ∈ V ∗ . Consider, id ∈ V ∗ .
If id(~v ) = 0. Obviously, ~v = ~0.

...

We can also consider the dual space of the dual space. That is the double
(algebraic) dual space (V ∗ )∗ = V ∗∗ which consists of all linear forms ω : V ∗ →
F . So, the elements of V ∗∗ are maps which assign scalar values to each linear
form in V ∗ . [2]

5
Remark 1. V ∗∗ is a vector space over F .
It is then natural to consider the map ω̄~v : V ∗ → F defined by ω~v (S) = S(~v ).
This map is obviously an evaluation homomorphism.
Proposition 2. ω~v ∈ V ∗∗
Proof. let S, T ∈ V ∗ and a, b ∈ F . Then,

ω~v (aS + bT ) = (aS + bT )(~v )

= aS(~v ) + bT (~v ) = aω~v (S) + bω~v (T )

So, ω~v is a linear map from V ∗ to F

Then we can define the canonical map π : V → V ∗∗ by π(~v ) = ω~c
Theorem 3. Given finite dimensional vector space V over field F the canonical
map π : V → V ∗∗ is an isomorphism.
Proof. Let a, b ∈ F , ~u, ~v ∈ V , and S ∈ V ∗ .
Consider.

π(a~u) + π(b~v )) = ωa~u + ωb~v

= S(a~u) + S(b~v ) ∀S ∈ V ∗
= S(a~u + b~b)
= ωa~u+b~v
= π(a~u + b~v )

So, π is linear (operation preserving), so π is a homomorphism.

Finally consider ker(π) = {~v ∈ V |π(~v ) = 0} π(~v ) = 0 implies ω~v = 0. That
is, for all φ ∈ V ∗ such that φ(~v ) = 0. By P roposition2, ker(π) = {0}. Then,
ker(φ) = {0} implies that π is injective
By Proposition 1, dim(V ∗∗ = dimV ∗ = dimV and π is injective, so π is
surjective.
While we expect V ∗∗ ' V , it is notable that the canonical map is an iso-
morphism. That makes V ∗∗ and V reflexive algebras. [2]

3 Multi-Linear Forms
We can further extend the ideas embodied in the Dual Space to include multi-
linearity. So, as with Dual Spaces, we begin with (multi-) linear forms.
Given vector spaces V1 , V2 , . . . , Vn , and W over field F we can define a map
φ : V1 × · · · × Vn → W . If φ is linear on each term. φ is n−linear.
For a bi-linear map, T : V1 × V2 → W is linear if, for a, b ∈ F, ~u, ~v ∈ V1 , and
~x, ~y ∈ V2 , T (a(~u + b~v , ~x) = aT (~u, ~x) + bT (~v , ~x) and T (~u, a~x + b~y ) = aT (~u, ~x) +
bT (~u, ~y ).

6
 In the same way that V ∗ is a vector space if V is a vector space, the set
of all n−linear maps from V1 × · · · × Vn to W , which is denoted L(U, V ; W ).
If V1 , . . . , Vn are vector spaces over field F and φ : V1 × · · · × Vn → F , φ is a
n−linear form.
It is important to note that V1 × · · · × Vn is a Cartesian product (not a direct
product). The notation Ln (V1 × · · · × Vn ; W ) denotes the set of linear maps
from the direct product V1 × · · · × Vn to W . [2]
Lemma 4. Given finite vector spaces V and W for all T ∈ L(V, W ):
• There exists a ϕ ∈ L(V, W ) such that ϕ(~v ) = ~0 for all ~v ∈ V .
• ~v = ~0 if and only if, for all φi ∈ L(V, W ), φi (~v ) = ~0
Proof. Let O ∈ L(V, W ) be the zero transformation. Then, for all ~v ∈ V ,
O(~v ) = ~0,
Then consider ~0 ∈ V and φ ∈ L(V, W ).
φ(~0) = 0φ(~v ) = ~0
Finally, O ∈ L(V, W ). O(~v ) = ~0 implies ~v = ~0.
Theorem 5. Given vector spaces V, U, and W over field F, L2 (V, U ; W ) is a
vector space.
Proof. First consider Hom(U, Hom(V, W )). By Theorem 1, Hom(V, W ) is a
vector space. Similarly Hom(U, Hom(V, W )) is a vector space.
As an aside, T : Hom(U, Hom(V, W )) is L(U, L(V, W ).
Let a, b ∈ F, u~i ∈ U, v~i ∈ V, and φi ∈ L(U, L(V, W ))
Define map T : L(U, L(V, W ) → L2 (U, V ; W ) by T (φ) = φ̂(~u, ~v ) = (φ(~u))(~v ),
for all ~u ∈ U and ~v ∈ V.
First, we check the bilinearity of φ̂
Consider,

φ̂(au~1 + bu~2 , ~v ) = (φ(au~1 + bu~2 ))(~v )

= (aφ(u~1 ))(~v ) + (bφ(u~2 ))(~v ) By the linearity of φ

And,

φ̂(~u, av~1 + bv~2 ) = (φ(~u))(av~1 + bv~2 )

= a(φ(~u))(~v ) + b(φ(~u))(v~1 ) By the linearity of φ(~u)

So, φ̂ is bilinear.
Consider,

aT (φ1 ) + bT (φ2 ) = a(φ1 (~u))(~v ) + b(φ2 (~u))(~v ) for all ~u ∈ U, ~v ∈ V

= (aφ1 (~u) + bφ2 (~u))(~v ) By the linearity of φi
= T (aφ1 + bφ2 )

So, T is linear.

7
Consider, ker(T ) = {φ ∈ L(U, L(V, W ) : T (φ) = ~0 = O}.
So, O(~u, ~v ) = φ̂(~u, ~v ) = (φ(~u))(~v ) ( for all ~u ∈ U, ~v ∈ V.)
Therefore, φ(~u) = ~0 for all ~u ∈ U .
Then, by Lemma 4, φ = ~0. So, ker(T ) = {~0}.
Then, T is injective.
Note dim(ker(T )) = dim({~0) = 0
Consider, Im(T ) = {T (φ) ∈ L(U, V ; W ) : φ ∈ L(U, L(V, W ))}.
Using dim(L2 (U, V ; W )) = dim(Im(T )) − dim(ker(T )), we conclude that
dim(Im(T )) = dim(L(U, V ; W )).
Therefore, T is surjective.
Therefore, T is an isomorphism and L(U, L(V, W )) ' L(U, V ; W ).
Therefore, L2 (U, V ; W ) is a vector space.

Corollary 3. Given vector spaces U, V, and W over field F,

dim(L2 (U, V ; W )) = dim(U ) dim(V ) dim(W ).
Proof. By Corollary 2, dim(L(V, W )) = dim(V ) dim(W ).
Then, because L(V, W ) is a vector space in its own right, we can apply
Corollary2 again.
So, dim(L(U, L(V, W ))) =
dim(U )(dim(V ) dim(W )) = dim(U ) dim(V ) dim(W )
The, because L(U, L(V, W )) ' L2 (U, V ; W ),
dim(L(U, V ; W )) = dim(L(U, L(V, W ))) = dim(U ) dim(V ) dim(W ).
We can similarly prove that Ln (V1 , . . . , Vn ; W ) is a vector space.
Theorem 6. Given finite vector spaces V1 , . . . , Vn , and W over field F ,
Ln (V1 , . . . , Vn ; W ) is a vector space.
Proof. We will prove Theorem 5 by induction on n for n ≥ 2.
Let v~i ∈ Vi for 1 ≤ i ≤ n
For n = 2, T : L(V1 , L(V2 , W )) → L(V1 , V2 ; W ) defined by T (φ) = φ̂ for
φ̂(v~1 , v~2 ) = (φ(v~1 ))(v~2 ) is an isomorphism.
So, L(V1 , L(V2 , W )) ' L(V1 , V2 ; W )

Assume the induction hypothesis.

That is, for n = k:
L(V1 , Lk−1 (V2 , . . . , Vk ; W )) ' L(V1 , . . . , Vk ; W )
Therefore,
L(V1 , Lk−1 (V2 , . . . , Vk ; W )) ' · · · ' L(V1 , L(V2 , . . . , L(Vk , W )) . . . ))
Additionally,
Tk : L(V1 , Lk−1 (V2 , . . . , Vk ; W )) → L(V1 , . . . , Vk ; W ) defined by
Tk (φ(v~1 , v~2 , . . . vk−1 ~ )) =
φ̂(v~1 , v~2 , . . . , v~k ) = (φ(v~1 , v~2 , . . . vk−1
~ ))(v~k ) is an isomorphism.
Note that Tk is linear. Therefore,
L(V1 , Lk−1 (V2 , . . . , Vk ; W )) ' L(V1 , . . . , Vk ; W )

8
 Consider the n = k + 1 case: Lk+1 (V1 , . . . , Vk+1 ; W ).
Define map Tk+1 : L(V1 , Lk (V2 , . . . , vk+1 ; W )) → Lk+1 (V1 , . . . , Vk ; W ) by
Tk+1 (φ(v~1 , v~2 , . . . v~k )) = φ̂(v~1 , v~2 , . . . , vk+1
~ ) = (φ(v~1 , v~2 , . . . v~k ))(vk+1
~ ).
The proof that Tk+1 is almost identical as the proof in Theorem 5.
By the induction hypothesis φ(v~1 , v~2 , . . . , v~k ) = (ϕ(v~1 , v~2 , . . . vk−1 ~ ))(v~k ) for
ϕ ∈ Lk−1 (V1 , . . . , Vk−1 ).
That is, Tk (ϕ) = φ. So, Tk+1 (φ) = (Tk (ϕ))(vk+1 ~ )
Consider ker(Tk1 ) = {φ ∈ L(V1 , Lk (V2 , . . . , vk ; W )) : Tk+1 (φ) = ~0}.
For all v~1 ∈ V1 , . . . vk+1 ~ ∈ Vk+1 and for all φ ∈ L(V1 , Lk (V2 , . . . , vk ; W )),

~0 = φ̂(v~1 , . . . , vk+1
~ )
= (φ(v~1 , . . . , v~k ))(vk+1
~ )
= (Tk (ϕ(v~1 , . . . , v~k ))(vk+1
~ )

So, Tk (ϕ(v~1 , . . . , v~k ) = ~0

By the induction hypothesis, Tk is an isomorphism. So, ker(Tk ) = {~0}.
Therefore, ker(Tk+1 ) = {~0}. So, Tk+1 is injective.
Then, because dim(L(V1 , Lk (V2 , . . . , Vk+1 ; W )))
= dim(Im(Tk+1 )) − dim(ker(Tk+1 )), we conclude that
dim(Im(Tk+1 )) = dim(L(U, V ; W )).
Therefore, Tk+1 is surjective.
Therefore, Tk+1 is an isomorphism and L(V1 , Lk (V2 , . . . , vk+1 ; W ))
' Lk+1 (V1 , . . . , Vk ; W ).

4 Tensor Products
Now, we can begin to understand tensors and tensor products. We start with
the simplest case: two finite vector spaces.
Given finite vector spaces U , V , and W over a field F , we can construct
another vector space U ⊗ V with bilinear map π : U × V → U ⊗ V . If for any W
and every T ∈ L2 (U, V ; W ), there exists a unique linear map, T̂ : U ⊗ V → W
such that T = π ◦ T̂ , U ⊗ V is a tensor product. [2]
First we must prove that tensor products exist. As we will later see, the
tensor product of two vector spaces is unique up to isomorphism. For this
reason, our choice of ”W ” is unimportant. [4]

Theorem 7. Given finite vector spaces U, V, and W , the tensor product of U

and V exists.
Proof. The proof of this theorem goes well beyond the scope of this paper.
[5]
Tensor products are unique up to isomorphism.

9
Theorem 8. Given two finite vector spaces U and V , U ⊗ V is unique up to
isomorphism.
Proof. Let (U ⊗ V )1 , φ1 and (U ⊗ V )2 , φ2 be two tensor products of U, V .
First note that (U ⊗ V )1 and (U ⊗ V )2 are Vector Spaces. So, we can define
a tensor product ”to” them.
By definition, we can obtain a unique linear map T : (U ⊗ V )1 → (U ⊗ V )2
such that φ2 = T ◦ φ1 .
Similarly, we can obtain a unique linear map S : (U ⊗ V )2 → (U ⊗ V )1 such
that φ1 = S ◦ φ2 .
Therefore, (S ◦ T ) ◦ φ1 = S ◦ φ2 = φ1 .
Then, by the uniqueness of T and S, S ◦ T = id. Therefore, S = T −1 and T
is an isomorphism. So, (U ⊗ V )1 , φ1 ' (U ⊗ V )2 , φ2 .
Because a Tensor Product of two vectors spaces is unique up to isomorphism,
we refer to the Tensor Product.
It should then be obvious that L(U ⊗ V ; W ) ' L2 (U, V ; W )
This allows us to ”set” W to the field of scalars F .
It then becomes more convenient to use V, V ∗ , and V ∗∗ .
For the next theorem, it is necessary to understand ~v ⊗ w.
~ For the purposes
of this paper, it is adequate to say that ~v ⊗ w
~ is the outer product of ~v and w.
~
 
[1  
3 [1
As a concrete example, consider 2 ∈ R and
  ∈ R2 .
2]
     3]
1   1  1 2
2 ⊗ 1 = 2 1 2 = 2 4

2
3 3 3 6

Theorem 9. Given {u~1 , . . . , u~n } a basis of finite vector space U and {v~1 , . . . , v~m }
a basis of finite vector space V , the set {u~i ⊗ v~j } for 1 ≤ i ≤ n and 1 ≤ j ≤ m
forms a basis of U ⊗ V .
Proof. Again, the proof of this theorem goes well beyond the scope of this paper.
[5]
Corollary 4. Given finite dimensional Vector Spaces U and V , dim(U ⊗ V ) =
dim(U ) dim(V )
Proof. If {u~i ⊗ v~j } for 1 ≤ i ≤ n and 1 ≤ j ≤ m forms a basis f U ⊗ V ,
dim(U ⊗ V ) = |{u~i ⊗ v~j }| = dim(U ) dim(V ).
We can then begin to explore the behavior of ⊗. [4]
Proposition 3. Given finite vector spaces U and V where ~u ∈ U and ~v ∈ V ,
if ~u ⊗ ~v = ~0, then, ~u = ~0 or ~v = ~0.
Proposition 4. Given finite dimensional vector spaces U and V , U ⊗ V '
V ⊗ U.

10
Proposition 5. Given finite vector spaces U, V , and W then, U ⊗ (V ⊗ W ) '
(U ⊗ V ) ⊗ W .

5 Conclusion
At its core, the tensor product generalizes the concept of multi-linearity to form
a ”new” finite vector space from a given set of finite vector spaces with the
operation of multi-linear composition.
The elements of a tensor product are called tensors. In general, tensors are
generalizations of vectors.
In the same way that many systems (physical or otherwise) require more
than a simple scalar to be described, some systems require more than a vector
to be described.
Tensors, as a generalization of vectors, allow for these systems to be de-
scribed.
For example, the stress in a 3−Dimensional object requires 2 ”components”
per direction. So, stress lends itself to being described by a tensor with 2
components per basis, making it a (2, 0)−Tensor.
As a resultant, Tensors are widely applicable across a variety of fields.

6 Sources
[1] Judson,T. Abstract Algebra: Theory and Applications, Orthogonal Publishing
L3C, 2020. Chp 18.
[2] Roman, S. Advanced Linear Algebra: Second Edition, New York, NY:
Springer, 2005. Chp 14
[3] Herstein, N. Topics in Algebra: Second Edition, New York, NY: John
Wiley & Sons, 1975. Chp. 4.
[4] Erdman, J. ”Elements of Linear and Multilinear Algebra.” (Lecture
Notes, Portland State University, 2014).
[5] Lang, S. Algebra: Third Edition, New York, NY: Springer, 2002. Chp 16

11