Western University Faculty of Engineering

Department of Electrical and Computer Engineering

ECE 9039/9309: Machine Learning: From Theory to Applications

Winter 2021

Description: The objective of this course is to introduce students to Machine Learning techniques
based on a unified, probabilistic approach. Regression, classification, clustering, neural networks,
mixture models, ensemble methods, and structure prediction will be covered in this course.
Students will get hands- on experience with machine learning from a series of practical
engineering case-studies.

Instructor: Dr. Abdallah Shami, P.Eng

ACEB 4455, [email protected]
Office Hours: Check OWL course website
Contact policy: Please contact the instructor through OWL Messages. If there is
a problem with OWL, use [email protected]

Prerequisite(s): Knowledge of probability and statistics, calculus, and linear algebra are
required as well as strong programming skills in Python.

Recommended References:
o NNDL: C. Aggarwal. Neural Networks and Deep Learning, Springer 2018. [Free through
Western]
o HTF: Hastie, Tibshirani, Friedman. The Elements of Statistical Learning. New York:
Springer. [Free: https://web.stanford.edu/~hastie/ElemStatLearn/]
o JWHT: James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to
statistical learning with applications in R. New York: Springer. [Free through Western]
o MLPP: Murphy, K. P. (2012). Machine Learning: a Probabilistic Perspective. MIT press.
[Free: https://www.cs.ubc.ca/~murphyk/MLbook/ ]
o BSH: Legler and Roback. Broadening Your Statistical Horizons. [Free:
https://bookdown.org/roback/bookdown-bysh/ ]
o Class Notes

Other Required References:

Students must check OWL (http://owl.uwo.ca) on a regular basis for news and updates. This is
the primary method by which information will be disseminated to all students in the class.
Students are responsible for checking OWL on a regular basis.

1
Major Topics:
• Introduction to Machine Learning
• Probability Theory Review
• Linear Algebra Review
• Optimization Overview
• Supervised Learning
o Regression
o Classification
• Characterization of Time Series Data
o Seasonality
o Trends
• Regularized regression
o Lasso Regression
o Ridge Regression
• Unsupervised Learning
o Clustering
▪ k-Nearest Neighbor Algorithm
▪ Mixture Models
▪ Expectation-Maximization (EM) Algorithm
• Principle Component Analysis
• Anomaly Detection
• Introduction to Neural Networks
• Introduction to Reinforcement Learning
• Machine Learning Workflow:
o Feature selection
o Algorithms Evaluation
▪ Variance: Test set, cross-validation, bootstrap
▪ Bias: Confounding, causal inference
o Performance Measures

SPECIFIC LEARNING OBJECTIVES:

At the end of the course, the students will be able to:
• Have a good understanding of the fundamental issues and challenges of machine
learning: data, features selection, model selection, and model complexity.
• Have an understanding of the strengths and weaknesses of many popular machine
learning approaches.
• Understand the underlying mathematical relationships within and across machine
learning algorithms and the paradigms of supervised and unsupervised learning.
• Be able to design and implement various machine learning algorithms in a range of real-
world engineering applications.

2
 EVALUATION:

Course Component Weight

Exam 50%
Programming Assignments 0%
Attendance & Participation 5%
Progress report 5%
Project Group presentation 25% 45%
Final project report and code submission 70%

Project & Homework Assignments Code: The students’ project software will be released under
Apache License Version 2.0. Apache License Version 2.0 details are available at
https://www.apache.org/licenses/. Project information and deadlines will be released before the
end of January 2021.

Exam: The Exam will cover concepts up to and including those covered up to this point. The
exam will include a practical component; each student will need a laptop/computer to complete
the Exam. Students will be given a data set and a set of practical data analytic problems to solve.
Python programming language and various Python-based machine learning packages will be used
in this course. The exam is “open book & open web”, meaning that students can access any notes
or any documents on the web. Electronic communication with other people inside or outside the
classroom is prohibited. The Exam will be conducted online. The date and location will be
shared before the end of January.

Attendance: Any student who, in the opinion of the instructor, is absent too frequently from
class, laboratory, or tutorial periods will be reported to the Dean (after due warning has been
given). On the recommendation of the department, and with the permission of the Dean, the
student will be debarred from taking the regular final examination in the course.

Late Submission Policy: Assignments should be submitted by 5:00 pm on the specified due
date. Late assignment submissions will be penalized 20% per day.

Recording Online Activities: Remote learning sessions for this course may be recorded. The
data captured during these recordings may include your image, voice recordings, chat logs and
personal identifiers (name displayed on the screen). The recordings will be used for educational
purposes related to this course, including evaluations. The recordings may be disclosed to other
individuals participating in the course for their private or group study purposes. Please contact
the instructor if you have any concerns related to session recordings. Participants in this course
are NOT permitted to record the sessions, except where recording is an approved accommodation,
or the participant has the prior written permission of the instructor.

COURSE CONTENT
The lecture notes and online lecture videos are copyrighted to the instructor and legally protected.
Do not post these videos and lecture notes on any other website or online forums. The recording

3
of the live/synchronous sessions of the course without the permission from the instructor is
prohibited. The illegal posting and sharing of the copyrighted course content could be subjected
to legal actions.

CHEATING, PLAGIARISM/ACADEMIC OFFENCES

Academic integrity is an essential component of learning activities. Students must have a clear
understanding of the course activities in which they are expected to work alone (and what working
alone implies) and the activities in which they can collaborate or seek help; see information above
under “Assessments” and ask instructor for clarification if needed. Any unauthorized forms of
help-seeking or collaboration will be considered an academic offense. University policy states that
cheating is an academic offence. If you are caught cheating, there will be no second warning.
Students must write their essays and assignments in their own words. Whenever students take an
idea or a passage of text from another author, they must acknowledge their debt both by using
quotation marks where appropriate and by proper referencing such as footnotes or
citations. Plagiarism is a major academic offence. Academic offences are taken seriously and
attended by academic penalties which may include expulsion from the program. Students are
directed to read the appropriate policy, specifically, the definition of what constitutes a Scholastic
Offence (see Western's scholastic discipline regulations for graduate students).

SYNCHRONOUS LEARNING ACTIVITIES

Students are expected to participate in synchronous learning activities as outlined in the course
syllabus and/or described by the instructor. If you have issues that will impede your ability to
participate in synchronous activities, please discuss with the course instructor at the beginning of
the course.

CONDUCT
Students are expected to follow proper etiquette during synchronous and asynchronous activities
to maintain an appropriate and respectful academic environment. Any student who, in the opinion
of the instructor, is not appropriately participating in the synchronous and asynchronous learning
activities and/or is not following the rules and responsibilities associated with the online learning
activities, will be reported to the Associate Dean (Graduate) (after due warning has been given).
On the recommendation of the Department concerned, and with the permission of the Associate
Dean (Graduate), the student could be debarred from completing the assessment activities in the
course as appropriate.

HEALTH/WELLNESS
As part of a successful graduate student experience at Western, we encourage students to make
their health and wellness a priority. Western provides several health and wellness related services
(remotely accessible) to help you achieve optimum health and engage in healthy living while
pursuing your graduate degree. Information regarding health- and wellness-related services
available to students may be found at http://www.health.uwo.ca/.
Students seeking help regarding mental health concerns are advised to speak to someone they feel
comfortable confiding in, such as their faculty supervisor, their program director (graduate chair),
or other relevant administrators in their unit. Campus mental health resources may be found at
http://www.health.uwo.ca/mental_health/resources.html
https://www.uwo.ca/health/psych/index.html

4
SICKNESS
Students should immediately consult with the Instructor (for a particular course) or Associate Chair
(Graduate) (for a range of courses) if they have problems that could affect their performance. The
student should seek advice from the Instructor or Associate Chair (Graduate) regarding how best
to deal with the problem. Failure to notify the Instructor or the Associate Chair (Graduate)
immediately (or as soon as possible thereafter) will have a negative effect on any appeal.
Obtaining appropriate documentation (e.g., a note from the doctor) is valuable when asking for
accommodation due to illness.

ACCESSIBLITY
Please contact the course instructor if you require material in an alternate format or if any other
arrangements can make this course more accessible to you. You may also wish to contact
Accessible Education at 661-2111 x 82147 or
http://academicsupport.uwo.ca/accessible_education/index.html, for any specific question
regarding an accommodation.