Matt Carrick - Foundations of Digital Signal Processing - Complex Numbers
Matt Carrick - Foundations of Digital Signal Processing - Complex Numbers
Matt Carrick - Foundations of Digital Signal Processing - Complex Numbers
Copyright © 2022 Matt Carrick. All rights reserved. No portion of this book
may be reproduced in any form without permission from the publisher,
except as permitted by U.S. copyright law.
How?
Why?
This book is the first in a series in which I will answer Why? for digital
signal processing. In this first edition I’ve done my best to answer questions
such as why sometimes Cartesian form is used instead of Polar form, why
arctangent has a phase ambiguity and why sine and cosine are used so
frequently in DSP among others.
This book summarizes all of my experience to help you answer your own
“Why?”. Please email me at [email protected] or leave a
comment at https://wavewalkerdsp.com with any of your own why?
questions and I will do my best to answer them.
The author has gathered 20 years of DSP experience across academia and
industry. He has performed research both academically and in industry for
cyclostationary and software radio systems. He received a PhD in Electrical
Engineering from Virginia Tech on the topic of interference-mitigating
multicarrier waveforms using cyclostationarity 1 which resulted in a patent.
He also has his MSEE and BSEE in Electrical Engineering.
See, I am doing a new thing! Now it springs up; do you not perceive it?
Isaiah 43:19
The heart of the discerning acquires knowledge, for the ears of the wise
seek it out. Proverbs 18:15
Contents
1 Complex Numbers
1.1 Real, Imaginary and Complex Numbers
1.2 RE(⋅) and IM(⋅) Operators
1.3 Cartesian Form
1.4 Adding Complex Numbers In Cartesian Form
1.5 Multiplying Complex Numbers in Cartesian Form
1.6 Complex Number as a Triangle
1.7 Hypotenuse of a Triangle
1.8 Angle of a Triangle
1.9 Arctangent
1.10 Radians and Degrees
1.11 Positive and Negative Angles
1.12 Arctangent2
1.13 Conjugation
1.14 Euler’s Formula
1.15 The Unit Circle
1.16 Polar Form
1.17 Multiplying Complex Numbers in Polar Form
1.18 Dividing Complex Numbers
2 Derivations
2.1 Simplifying jn
2.2 Deriving Euler’s Formula
2.3 Sine as Complex Exponentials
2.4 Cosine as Complex Exponentials
2.5 Phase Relationship Between Sine and Cosine
2.6 Derivative of ejx
2.7 Derivative of Sine
2.8 Derivative of Cosine
2.9 Deriving sin2x + cos2x = 1
3 References
Chapter 1
Complex Numbers
This chapter introduces complex numbers, the mathematical language for all of
DSP. Geometry and trigonometry is applied to the complex plane in order to derive
mathematical tools for complex numbers such as calculating the magnitude and
phase of a complex number. Euler’s formula is presented as a way for a complex
number to be represented by its amplitude and phase. Later sections assume
knowledge and expertise with the foundational tools developed in this section.
One way to think of real numbers is they are the numbers you use everyday: the
balance in your bank account, the temperature and the price of milk. Real numbers
can be whole numbers, often called integers. Real numbers have multiple decimal
places. Real numbers can be positive and negative. The following are examples of
real numbers:
2.39392
62
−15.3
1⁄3
2π
53.6j
−j
−7.519j
j
j⁄7
Complex numbers are the superset of real and imaginary numbers. A complex
number c is typically represented using the form
(1.3)
where a is the real component and jb is the imaginary component.
75.4 − 7j
−2.8 − 0.8j
0 + 5.4j
0.28 + 0j
−4 − j
6.02 + πj
The RE(⋅) and IM(⋅) are the mathematical operations which extract the real and
imaginary components of a complex number. The real and imaginary portions of a
complex number c = a + jb can be obtained by the operators
(1.4)
(1.5)
Note that the IM operator in (1.5) does not include the j term. For example, when
(1.6)
(1.8)
Being able to treat the real and imaginary components of c independently can be
useful in situations such as measuring the angle of a complex number (Section 1.8).
are referred to as Cartesian form. Complex numbers in the Cartesian form may also
be referred to as Cartesian coordinates because they can be interpreted as a two-
dimensional vector in the complex plane. The complex plane has two axes, the real
dimension and the imaginary dimension. Figure 1.1 is an example of the complex
number 2 + 3j being plotted in the complex plane as a complex vector.
Notice from Figure 1.1 that the number a describes how far and what direction the
complex vector protrudes into the real dimension. Similarly the number b describes
how far and what direction the complex vector protrudes into the imaginary
dimension.
The Cartesian form is the default representation for complex numbers used in
computing systems, software radios and field-programmable gate arrays (FPGA)
because they are easily represented as a two-dimensional array. However, the polar
form (Section 1.16) tends to be easier to manipulate when working with equations
by hand.
the real component a and imaginary component jb are like oil and water: they do
not mix and cannot mix. The mathematical way to say this is that they are
orthogonal. Adding a real number m to a complex number a + jb only effects the
real component,
(1.11)
Figure 1.2 shows how only the real dimension of the complex number is effected by
the addition of a real number in (1.12).
Figure 1.2: Adding the real number 2 to 2+3j only effects the real portion of
the complex vector.
Figure 1.3 shows how only the imaginary dimension of the complex number is
effected by the addition of a imaginary number in (1.14).
Figure 1.3: Adding the imaginary number −5j to 2 + 3j only effects the
imaginary portion of the complex vector.
Combining (1.11) and (1.13), the addition of two complex numbers is defined as
(1.15)
Both the real and imaginary components are multiplied by m. For example,
multiplying complex number 1 + 2j by real number 1.5 results in
(1.18)
as shown in Figure 1.5.
Figure 1.5: Multiplying a complex number by a real number scales both real
and imaginary components of the complex number.
(1.19)
such that
(1.21)
Therefore
(1.22)
(1.23)
(1.24)
(1.25)
(1.26)
(1.27)
The multiplication of two complex numbers can be written using (1.17) and (1.26),
(1.28)
(1.29)
(1.30)
(1.32)
(1.33)
Computing systems multiply complex numbers using the Cartesian form in (1.30)
however it can be cumbersome when working with pen and paper. Multiplication of
two complex numbers using the polar form (Section 1.16) is much easier and less
prone to error when working with equations by hand.
Figure 1.8: The real and imaginary components of the complex number c in
the complex plane.
The “open” end of the triangle can be enclosed by the imaginary component as
shown in Figure 1.9. The number 2j is not moved, it is only drawn to make
visualizing the triangle easier so the angle can be displayed.
(1.35)
The magnitude of a complex number is it’s length in the complex plane. The sides
of the triangle represented by c can be negative, however the magnitude cannot be a
negative number. The numbers a and b are real numbers and the square of a real
number must be positive. Therefore (1.34) must be a positive number.
(1.36)
(1.37)
where |c| is the hypotenuse of the triangle or equivalently the magnitude of the
complex number c and a = RE and b = IM . The angle can be calculated by
taking the arccosine of (1.36),
(1.38)
(1.39)
(1.41)
(1.42)
where arcsine is the inverse of sine such that
(1.43)
(1.44)
(1.45)
(1.46)
Figure 1.11: The angle can be calculated using trigonometric identities which
relate the sides of the triangle to the hypotenuse.
Section 1.9 demonstrates how the angle of a triangle can be calculated more
efficiently using the arctangent of the two sides since it does not require the
additional calculation of the magnitude.
1.9 Arctangent
Tangent and arctangent are useful because they calculate the angle of a triangle
based on the sides a and b and without having to also calculate the magnitude.
Calculating the angle of a complex number is common in DSP and saving
computations results in more efficient systems.
(1.47)
Substituting in (1.36) and (1.37) for cos and sin in (1.49), tangent can then be
written in terms of a and b,
(1.48)
(1.49)
The inverse of tangent, or arctangent, is used to calculate the angle. Taking the
arctangent of both sides of (1.49),
(1.50)
(1.51)
(1.53)
which is the same angle as shown in Figure 1.11.
Warning! The arctangent has a angle ambiguity which is discussed in Section 1.12
and demonstrates why arctan2 should be used instead of arctan.
The angles in Sections 1.8 and 1.9 are given in terms of π because their unit is in
radians, however degrees are more commonly used in every day language. In 1999
Tony Hawk completed a “900” 1 in which he spun 900∘ on a skateboard. Using
“900” to represent the number of revolutions appears to be a more elegant method
than using 5π radians or even 15.7 radians. So why are π and radians used in DSP?
Radians are the unit used to represent angles in terms of π. The number π represents
the ratio of the circumference of a circle C to it’s diameter d,
(1.54)
The diameter d of a circle is twice the length of the radius r, d = 2r, such that the
circumference C (1.54) can be written as
(1.55)
Figure 1.12 shows a circle of radius r. The distance between the two vectors shows
an example arc length L. The arc length is the distance along the circumference of
the circle corresponding to the angle
Figure 1.12: The arc length L is the distance along the circumference
corresponding to the angle. The arc length at multiple points along the
circumference are labeled outside the circle.
Figure 1.13: The circumference of the circle is laid out on a number line from
0 to 2πr. The arc length L is the distance between the two points along the
circumference of the circle.
Figure 1.13 lays out the circumference of the circle on a number line from 0 to 2πr.
A longer radius r results in a larger arc length for the same angle. Therefore
dividing the arc length by the radius (Figure 1.14) produces the range of angles in
the circle from 0 radians to 2π radians 2.
Figure 1.14: The angle is laid out on a number line from 0 to 2π.
(1.56)
which scales the angle in radians by the ratio of degrees in a circle by the number of
radians. Degrees can be converted into radians by rearranging (1.56),
(1.57)
Radians are used because of the geometric relationships that cause angles to be in
terms of π. The polar form (Section 1.16) for complex numbers is a compact way to
represent sine and cosine of an angle, therefore the polar form will also use angles
in radians.
Angles in the complex plane are measured from the point 1 + 0j. Positive angles are
measured in the counter-clockwise direction from 1 + 0j and are represented from 0
to 2π as in Figure 1.15.
Negative angles are measured in the clockwise direction from 1 + 0j and are
represented from −2π to 0 as in Figure 1.16.
Figure 1.16: Negative angles are measured from 1 + 0j in the clockwise
direction and are given within the range of −2π to 0.
However, the most common way of representing angles is on the range from −π to π
as in Figure 1.17.
Figure 1.17: Angles are most commonly measured between −π to π.
1.12 Arctangent2
(1.59)
(1.61)
The angles of c0 and c1 are calculated to be the same value, arctan , due to the
division of b⁄a. However, their actual angles are π⁄4 and −3π⁄4 as shown in Figure
1.18.
Figure 1.18: The complex numbers c0 and c1 with their correct angles
displayed.
(1.63)
also have a problem with angle ambiguity when using arctangent. The angles of c2
and c3 are calculated to be the same when using arctangent,
(1.64)
(1.65)
However, the correct angles of c2 and c3 are 3π⁄4 and −π⁄4 as shown in Figure 1.19.
Figure 1.19: The complex numbers c2 and c3 with their correct angles
displayed.
The arctangent2 takes into account the polarity of the real and imaginary portions to
correct for the angle ambiguity in arctangent. The angle of a complex number c = a
+ jb is denoted by the symbol ∢.
(1.66)
The angles for c0, c1, c2 and c3 are calculated using arctangent2,
(1.67)
(1.68)
(1.69)
(1.70)
1.13 Conjugation
Conjugation is the mirroring of c across the imaginary axis which also negates the
angle of the complex number,
(1.72)
Euler’s formula 5,
(1.73)
Figure 1.21: The cos and j sin terms represent the adjacent and opposite
sides of a triangle in the complex plane.
Where the geometry in Section 1.6 is true for any size triangle, Euler’s formula
refers to a specific subset of triangles whose hypotenuse is 1. Taking the magnitude
of (1.73) with the aid of (2.66),
(1.74)
(1.75)
(1.76)
(1.77)
(1.78)
is
derived according to
(1.79)
(1.80)
(1.81)
which can also been seen from Figure 1.21. Euler’s formula therefore represents a
complex vector with magnitude 1 and phase
(1.82)
Euler’s formula is widely applied in DSP because it is the foundation for the polar
form (Section 1.16) and in representing the unit circle (Section 1.15). Section 2.2
describes how Euler’s formula is derived and why a complex exponential is written
as sine and cosine, forming the basis for complex sinusoids.
The unit circle is a circle of radius 1 centered at 0 + 0j in the complex plane. The
unit circle can be represented by a complex exponential
The unit circle is useful because it provides a reference point in the complex plane.
Signal and filter parameters can be measured based on their relationship to the unit
circle. For example, the unit circle is used as a reference when measuring and
visualizing the phase, frequency and amplitude of a complex signal, displaying
constellation diagrams in the complex plane and for displaying pole-zero plots to
describe the response of digital filters.
The polar form of a complex number can be converted into the Cartesian form
through Euler’s formula (1.73),
(1.84)
(1.85)
(1.86)
(1.88)
Polar form is most commonly used mathematically (pen and paper) due to its
simplified multiplication (Section 1.17). Almost all useful DSP functions require
complex multiplication and therefore the polar form is the most common when
working through the mathematics manually.
and
For example, two complex exponentials 1.5ejπ and 2ej3π⁄4 are multiplied resulting in
(1.91)
Figure 1.24 shows how the magnitude of the multiplication result is the scaled
version of the two magnitudes and the phase of the result is the addition of the two
phases.
The division of complex numbers is most easily understood using the polar form.
Any division can be written as the multiplication of one quantity with the inverse of
another. As an example the division of real numbers x and y can be written as a
multiplication by
(1.92)
and
(1.93)
(1.94)
(1.95)
(1.97)
(1.98)
By substituting (1.98) into (1.93) the division of the two complex numbers is
written as
(1.99)
(1.100)
(1.101)
(1.102)
which is the division of the two magnitudes and the subtraction of the two phases.
Compare (1.102) against the multiplication of two complex numbers (Section 1.17)
which is the multiplication of their magnitudes and the addition of their phases.
For example, the division of two complex numbers 3ejπ and 2ejπ⁄3 using (1.102) is
given by
(1.103)
(1.104)
Figure 1.24 shows result of the division of the two complex exponentials.
Figure 1.25: The division of two complex numbers results in the ratio of the
two magnitudes and the subtraction of the two phases.
Chapter 2
Derivations
Much of the undergraduate DSP education has to be taken on faith due to the large
amount of material to be covered in a limited time. This section answers why? some
of the foundational formulas and equations came to be and how they are derived.
The derivation of Euler’s formula is described as well as host of trigonometric
identities which use Euler’s formula.
2.1 Simplifying jn
(2.2)
(2.5)
(2.7)
(2.8)
(2.10)
(2.12)
(2.13)
(2.14)
(2.15)
The even and odd terms are gathered into their respective series,
(2.16)
(2.17)
which is the Taylor series expansion for cosine 3,
(2.18)
(2.19)
(2.20)
The Taylor series expansion of ejx can therefore be written as a combination of the
even and odd series,
(2.21)
(2.23)
Start with Euler’s formula (1.73),
(2.24)
Subtract
(2.26)
(2.27)
(2.28)
(2.29)
Add
to both sides of Euler’s formula,
(2.30)
(2.31)
(2.32)
Sine and cosine are related to one another through a delay of π⁄2,
(2.33)
Figure 2.1: Sine and cosine are related through a delay of π⁄2.
Using the variable ϕ, cosine can be written as the summation of two complex
exponentials (2.28),
(2.34)
Defining
(2.35)
(2.37)
(2.38)
(2.39)
(2.40)
(2.41)
therefore
(2.42)
Sine can be written as the summation of two complex exponentials (2.23),
(2.43)
(2.44)
as stated in (2.33).
(2.45)
(2.46)
(2.48)
(2.49)
(2.50)
from (2.46),
(2.51)
(2.52)
(2.53)
(2.54)
(2.55)
(2.56)
(2.57)
(2.58)
(2.59)
Substituting for the derivative
from (2.46),
(2.60)
(2.61)
(2.62)
(2.63)
(2.64)
(2.65)
(2.66)
The terms sin2x and cos2x are considered separately and then summed.
(2.67)
therefore sine-squared is
(2.68)
(2.69)
(2.70)
(2.71)
therefore cosine-squared is
(2.72)
(2.73)
(2.74)
Adding (2.70) and (2.74),
(2.75)
(2.76)
(2.77)
(2.78)
https://www.wavewalkerdsp.com/2022/02/02/new-gnu-radio-beginner-tutorials/
1Dose Skateboarding, Remembering Tony Hawk’s 900. June 29, 2020,
https://doseskateboarding.com/articles/remembering-tony-hawk-900
2Wikipedia, Radian. March 2, 2022, https://en.wikipedia.org/wiki/Radian
3The Mathworks, Inc., atan2. March 4, 2022,
https://www.mathworks.com/help/matlab/ref/atan2.html
4NumPy Developers, numpy.arctan2. March 4, 2022,
https://numpy.org/doc/stable/reference/generated/numpy.arctan2.html
5R. Lyons, Understanding Digital Signal Processing, Third Edition. 2011.
13Brown1Blue, What is Euler’s formula actually saying? — Ep. 4 Lockdown live math. April 28,
2020, https://www.youtube.com/watch?v=ZxYOEwM6Wbk
2Wikipedia, Taylor series. March 3, 2022, https://en.wikipedia.org/wiki/Taylor_series
3Wikipedia, Taylor series. March 3, 2022, https://en.wikipedia.org/wiki/Taylor_series
4Wikipedia, Taylor series. March 3, 2022, https://en.wikipedia.org/wiki/Taylor_series
5Wikipedia, Differentiation rules. March 4, 2022, https://en.wikipedia.org/wiki/Differentiation_rules
6Wikipedia, Differentiation of trigonometric functions. March 2, 2022,
https://en.wikipedia.org/wiki/Differentiation_of_trigonometric_functions
7Wikipedia, Differentiation of trigonometric functions. March 2, 2022,
https://en.wikipedia.org/wiki/Differentiation_of_trigonometric_functions