DS551/CS525 - Reinforcement Learning - Fall 2024

Version: June 24th 2024

Class Information:

Instructor and Teaching Assistant:

Prof. Yanhua Li
Email: yli15 at wpi.edu
Website: http://wpi.edu/~yli15/

Weekly Office hours:

Tues 12:00pm-1:00pm;
In person (Location: UH 384)

Others by appointments

Teaching Assistant (TA):
Mingzhi Hu,
Email: mhu3@wpi.edu

Weekly Office hours:
Mondays 10-11am on Zoom (Zoom link is available on Canvas);
Wednesdays 12-1pm in-persion (Location: UH 341)
Others by appointments

Course Description:

[Topics.] Reinforcement Learning (RL) is an area of machine learning concerned with how agents take actions in an environment with a goal of maximizing some notion of “cumulative reward”. The problem, due to its generality, is studied in many disciplines, and applied in many domains, including robotics and industrial automation, marketing, education and training, health and medicine, text, speech, dialog systems, finance, among many others. In this course, we will cover topics including: Markov decision processes, reinforcement learning algorithms value function based, actor-critics, policy gradient methods, representations for reinforcement learning (including deep learning), and inverse reinforcement learning. The course project(s) will require the implementation and application of many of the algorithms discussed in class.

[Recommended background.] This is an *advanced* graduate course which is primarily targeted for second (or higher) year Ph.D/MS graduate students. Suggested Prerequisites: Machine Learning (CS 539), and programming experience. Statistics at the undergraduate level, or permission of the instructor..

Textbook:

The topic is evolving. Thus no one comprehensive text book exists that would contain the material we will study in this course. Instead we will be utilizing a variety of sources, including publications from the primary literature and book chapters. These manuscripts will be provided to the class and/or linked into our schedule.
In particular, a number of the supporting readings will come from:
Reinforcement Learning: An Introduction, Sutton and Barto, 2nd Edition. This is available for free here and references will refer to the final pdf version available here.
Some other additional references that may be useful are listed below:
Reinforcement Learning: State-of-the-Art, Marco Wiering and Martijn van Otterlo, Eds. [link]
Artificial Intelligence: A Modern Approach, Stuart J. Russell and Peter Norvig. [link]
Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville. [link]
David Silver's course on Reiforcement Learning [link]

Coursework and Evaluation:

The grading system for this course is A,B,C,D,F (without +/-).
Oral Work: 10%.
Quizzes/Exams: 30%.
Class projects: 60% (Project 1 for 5%, Project 2 for 10%, Project 3 for 15%, Project 4 for 30%)
Note:Please see more details of the breakdowns for each part in the grading page,and projects in the projects page.