Readings
Students are expected to read the corresponding sections about a class’s topic from notes before class as each class will discuss each topic in more detail and address questions about the material.
All readings are from the (in progress) machine learning notes. These are designed to be short, so that you can read every chapter. I recommend avoiding printing these notes, since later parts of the notes are likely to be modified (even if only a little bit).
Schedule
This schedule is tentative, and is likely to change throughout the semester. Most of the lectures will be whiteboard lectures.
The course will build on CMPUT 267: Basics of ML. We will revisit previous concepts in the first part of the course, but now in more advanced settings. This serves as a refresher for the Basics of ML, and is also core content for Intermediate ML. Then we will focus on something only lightly touched on in the basics of ML: data re-representation. Central to many methods is (nonlinear) transformations that improve modelling capabilities and trainability.
Links will be broken, until I add the pdf file for that lecture. So if you click the slides for a lecture in the future, you will not find them! But if you’d like to see last years lectures, you can find the link to last years schedule below.
| Week | Date | Topic | Readings |
|---|---|---|---|
| 1 | September 1, 2022 | Introduction to the Course with a brief refresher of Basics of ML. Cover background on matrices. | Assignment #1 released with associated code (Tex files) Thought questions #1: - read Chapters 1-5 from the notes.pdf (pp 8 - 56, up to and including GLMs) |
| 2 | September 6, 2022 | Intermediate Probability Concepts (multivariate distributions, mixture models, KL divergences), Probability slides | |
| 2 | September 8, 2022 | Revisiting Linear Regression, with matrices | |
| 3 | September 13, 2022 | Revisiting l2 regularization and bias and variance | |
| 3 | September 15, 2022 | Intermediate Optimization Concepts (Hessians, stepsize algorithms) | |
| 4 | September 20, 2022 | Optimization continued | Thought questions #1 due on Sept 20 Thought questions #2: - read Chapters 6-9 from the notes.pdf (pg 56 - 88, up to and including data representations) |
| 4 | September 22, 2022 | Generalized Linear Models and multi-class classification | |
| 5 | September 27, 2022 | Constrained optimization and start Estimating Generalization Error with Cross Validation | |
| 5 | September 29, 2022 | Finish GE and using CV for hyperparameter selection | Assignment #1 due on Saturday, October 1 Assignment #2 released with associated code (Tex files) |
| 6 | October 4, 2022 | Review Chapters 1-5, with slides | |
| 6 | October 6, 2022 | Quiz | |
| 7 | October 11, 2022 | The Goal of Data representations and Separability, with examples using fixed representations | Thought questions #2 due Tuesday Oct. 11 Thought questions #3: - read Chapters 10-12 from the notes.pdf(pg 90 - 107, up to and including Generalization Theory) |
| 7 | October 13, 2022 | Learning data representations: latent variable methods and (probabilistic) PCA | Assignment #2 due on Sunday, Oct. 16 Assignment #3 released with associated code (Tex files) |
| 8 | October 18, 2022 | Learning data representations: Neural networks and backpropagation | |
| 8 | October 20, 2022 | Autoencoders and the connection to PCA | |
| 9 | October 25, 2022 | Generative Models: Mixture models and EM | Thought questions #3 due Tuesday, Oct. 25 Thought questions #4: - read Chapters 13 - 15 from the notes.pdf |
| 9 | October 27, 2022 | Generative Models and Data Representations | |
| 10 | November 1, 2022 | Variational Autoencoders (continued) | |
| 10 | November 3, 2022 | Generalization Theory Basics | Assignment #3 due on Monday, Nov. 7 Assignment #4 released with associated code (Tex file) |
| 11 | November 8, 2022 | No classes, Reading Week | |
| 11 | November 10, 2022 | No classes, Reading Week | |
| 12 | November 15, 2022 | Midterm Review | |
| 12 | November 17, 2022 | Midterm (Chapters 1 - 9) | |
| 13 | November 22, 2022 | Convergence Rates | |
| 13 | November 24, 2022 | Missing data and generative models | |
| 14 | November 29, 2022 | Missing data (cont.), start Bayesian methods | Thought questions #4 due Tuesday, November 29 |
| 14 | December 1, 2022 | Bayesian methods and the kernel trick (Gaussian processes) | |
| 15 | December 5, 2022 | Course review, with Final Review slides highlighting which topics are tested. | |
| 15 | December 7, 2022 | Practice Final session | Assignment #4 due on Friday, Dec. 8 |
| Final | Monday, December 12, 2022, 9:00 a.m. | Final Exam, Chapters 1 - 15 | 2 hours. |