View on GitHub

rlcourse

An Introduction to Reinforcement Learning

Link back to the Syllabus

All readings are from the textbook. These readings are designed to be short, so that it should be easy to keep up with the readings.

This schedule is tentative, and is likely to change throughout the semester.

Week Date Topic Deadlines
1 September 4, 6 Introduction to AI and Organizing discussions

Worksheet in class about random variables and expectations
In preparation for Week 2, watch videos in Course 1, Module 2, starting with the K-Armed Bandit

Complete Quiz 1 by midnight on Tuesday, September 10 link

Submit discussion question, through Google form

Notebook for Week 1 is due on Friday, September 13, link
2 September 9 Background on Statistics and Probability

Finish Worksheet
Complete Practice Quiz 1 by midnight on Tuesday, September 10 link

Submit discussion question through Google form by midnight on Tuesday, September 10 link. You must be logged into your UAlberta account to submit
2 September 11 In-class Discussion on Course 1, Week 1: The K-Armed Bandit Problem

Overview slides for the discussion
 
2 September 13 Review with Question-and-Answer session

In-class Worksheet
Complete Graded Notebook for Week 1 by midnight on Friday, September 13, link
3 September 16 Review, Q&A session and Worksheet Question 4, for content Course 1, Week 2: Markov Decision Processes Sept. 16 last day to drop courses without fees.

Complete Practice Quiz by midnight on Sunday, September 15 link

Submit discussion question through Google form by midnight on Sunday, September 15 link. You must be logged into your UAlberta account to submit
3 September 18 In-class Discussion on MDPs Complete Peer-graded Assignment by midnight on Thursday, September 19 link
3 September 20 In-class Worksheet (the slides and the worksheet),
Extra: this diagram for a Bandit as an MDP
Compete Peer-review by Sunday night: Link
4 September 23 Review, Q&A session, and worksheet questions Course 1, Week 3: Value Function and Bellman Equations,
Extra: Derivation of Bellman Equation
Complete Practice Quiz 3 by midnight on Sunday, September 22 link

Submit discussion question through Google form by midnight on Sunday, September 22 link. You must be logged into your UAlberta account to submit
4 September 25 In-class Discussion on Value Functions and Bellman equations  
4 September 27 Worksheet and practice questions on Value Functions and Bellman Equations Complete Graded Quiz by Midnight, Friday, September 27, link
5 September 30 Dynamic Programming review Oct. 3 last day to drop course (50% fees). Complete Practice Quiz 4 by midnight on Sunday, September 29 link

Submit discussion question through Google form by midnight on Sunday, September 29 link. You must be logged into your UAlberta account to submit
5 October 2 Dynamic Programming clarifications and discussion, with Worksheet 5
For Q1, see these formulas for the value function
 
5 October 4 Slides to discuss Dynamic Programming worksheet Complete Graded Notebook for Course 1, Week 4 by midnight on Friday, October 4 link
6 October 7 Review of Monte Carlo Methods In preparation for Week 6, watch videos in Course 2, Week 2. Complete Blackjack notebook by midnight on Sunday October 6.

OPTIONAL: Submit questions about the course content here.
6 October 9 Clarifications and discussion on Monte Carlo Submit discussion question through Google form by midnight on Tuesday, October 8 link. You must be logged into your UAlberta account to submit.
6 October 11 Monte Carlo Methods worksheet slides and the worksheet itself Complete Graded Quiz by midnight on Thursday, October 10, link
7 October 14 Holiday Complete Practice Quiz by midnight on Sunday, October 13, link
7 Ocobter 16 Temporal Difference Learning Methods for Prediction Submit discussion question through Google form by midnight on Tuesday, October 15 link. You must be logged into your UAlberta account to submit

OPTIONAL: Submit questions about the course content here.
7 October 18 TD for Prediction worksheet slides with corresponding worksheet Complete Graded Notebook by midnight on Friday, October 18, link

Worksheet challenge question solution link
8 October 21 Temporal Difference Learning Methods for Control with additional note from in-class about difference between Sarsa, Expected Sarsa and Q-learning Complete Practice Quiz by midnight on Sunday, October 20, link
8 October 23 Temporal Difference Learning Methods for Control Q and A with additional note from in-class about worksheet Q1. Submit discussion question through Google form by midnight on Tuesday, October 22 link. You must be logged into your UAlberta account to submit
8 October 25 Temporal Difference Learning Methods for Control Worksheet slides with the corresponding worksheet. Complete Graded Notebook by midnight on Friday, October 25, link
9 October 28 Planning Learning and Acting review Complete Practice Quiz by midnight on Sunday, October 27, link

OPTIONAL: Submit questions about the course content here.
9 October 30 Planning Learning and Acting Demo with worksheet questions, with the corresponding worksheet here. Submit discussion question through Google form by midnight on Tuesday, October 29 link. You must be logged into your UAlberta account to submit
9 November 1 Planning Learning and Acting worksheet Complete Graded Notebook by midnight on Friday, November 1, link
10 November 4 Prediction with Approximation review Complete Practice Quiz by midnight on Sunday, November 3, link

OPTIONAL: Submit questions about the course content here
10 November 6 Prediction with Approximation (more) Submit discussion question through Google form by midnight on Tuesday, November 5 link. You must be logged into your UAlberta account to submit
10 November 8 Prediction with Approximation questions and worksheet with worksheet Complete Graded Notebook by midnight on Friday, November 8, link
11 November 11, 13, 15 No classes: Reading week Start Capstone Project. Here is the project overview.
12 November 18 Midterm Review OPTIONAL: Ask live questions in class here
12 November 20 Midterm Review Midterm on November 22.
12 November 22 Midterm on November 22  
13 November 25 Constructing Features for Prediction Complete Practice Quiz by midnight on Sunday, November 24, link
13 November 27 Constructing Features for Prediction Submit discussion question through Google form by midnight on Tuesday, November 26 link. You must be logged into your UAlberta account to submit
13 November 29 Constructing Features for Prediction Complete Graded Notebook by midnight on Friday, November 29, link
14 December 2 Control with Approximation and Review Project progress Complete Practice Quiz by midnight on Sunday, December 1, link
14 December 4 Control with Approximation Submit discussion question through Google form by midnight on Tuesday, December 3 link. You must be logged into your UAlberta account to submit
14 December 6 Course Review Complete Graded Notebook by midnight on Friday, December 6, link

Submit Capstone Project zip file by midnight on Friday, December 6.
Final Thursday, December 19, 2017, 2:00 p.m. Final Exam You can bring a two page cheat-sheet.