Reinforcement Learning and Bandit Algorithms Joint Reading Group
Jan-July 2024
In this reading group we will primarily read the lecture notes Foundations of Reinforcement Learning and Interactive Decision Making by Dylan J. Foster and Alexander Rakhlin as the learning material. We refer to the book Bandit Algorithms by Tor Lattimore and Csaba Szepesvari for more detailed background knowledge.
This joint reading group is hybrid –- we meet biweekly on Friday via Zooms at:
- 17:00-19:00 (UK time)
- 09:00-11:00 (California time)
- 10:00-12:00 (Colorado time)
- 11:00-13:00 (Chicago time)
- 12:00-14:00 (Michigan time)
Please check this website regularly for the most up-to-date arrangement.
Materials
- Week 1. Introduction and basic technical tools (Notes: Chapter 1 and Appendix A)
- Week 2. Multi-Armed Bandits (Notes: Chapter 2, Book: Chapter 6-7)
- Week 3. Multi-Armed Bandits (Notes: Chapter 2; Book: Chapter 6-7) / Contextual Bandits (Book: Chapter 18; Notes: Chapter 3)
- Week 4. Contextual and Linear Stochastic Bandits (Book: Chapter 19-20; Notes: Chapter 3)/Stochastic Linear Bandits with Sparsity (Book: Chapter 23)
- Week 5. Stochastic Linear Bandits with Sparsity (Book: Chapter 23) and Online Decision Making with High-Dimensional Covariates (Paper by Bastani and Bayati, 2020)
- Slides for Chapter 23 by Siyu
- Slides for BB2020 paper by Siyu
- Week 6. Dynamic Batch Learning in High-Dimensional Sparse Linear Contextual Bandits (Paper by Ren and Zhou, 2024)
- Week 7. Structured Bandits (Notes: Chapter 4.1-4.2)
- Week 8. Structured Bandits (Notes: Chapter 4.3-4.6)
- Week 9. Reinforcement Learning: Basics (Notes: Chapter 5)
- Week 10. General Decision Marking (Notes: Chapter 6)
- Week 11. Online Policy Learning and Inference by Matrix Completion
- Week 12. Reinforcement Learning: Function Approximation and Large State Spaces (Notes: Chapter 6)
Group Members
Southampton
Chicago/San Diego
Michigan
Colorado
Other Supplementary References
…
Roles of Presenter and Discussant
If it is your first time attend a reading group, you might find the reading group tips by Lester Mackey and Percy Liang helpful.
Every time we will have one people (presenter) present the main contents and another people (discussant) raise questions and lead the discussion.
-
As a presenter: you should have an in-depth reading and develop a solid understanding of all the details in the assigned topic. You should prepare well, and make sure you deliver a logically clear and technically accessible presentation. In short words, it is your job to have everyone in the meeting understand the main ideas of the reading.
-
As a discussant: you should be more familiar with the content than if you were simply in the group. You don’t need to know everything. You can pause the presentation, ask questions (to the presenter or to the audience), and facilitate discussions. It is your job to help the presenter to have everyone (yourself included!) in the meeting understand the main ideas of the reading and having learned something.
Before each session, although not compulsory I would recommend following amount of time spent on reading:
- Presenter: > 10 hours;
- Discussant: 5 hours;
- General audience: 2 hours.
Webpage maintained by Chao Zheng. Last updated on 06/01/2024
Reinforcement Learning and Bandit Algorithms Joint Reading Group
Jan-July 2024
In this reading group we will primarily read the lecture notes Foundations of Reinforcement Learning and Interactive Decision Making by Dylan J. Foster and Alexander Rakhlin as the learning material. We refer to the book Bandit Algorithms by Tor Lattimore and Csaba Szepesvari for more detailed background knowledge.
This joint reading group is hybrid –- we meet biweekly on Friday via Zooms at:
Timetable (provisional)
Please check this website regularly for the most up-to-date arrangement.
Materials
Group Members
Southampton
Chicago/San Diego
Michigan
Colorado
Other Supplementary References
…
Roles of Presenter and Discussant
If it is your first time attend a reading group, you might find the reading group tips by Lester Mackey and Percy Liang helpful.
Every time we will have one people (presenter) present the main contents and another people (discussant) raise questions and lead the discussion.
As a presenter: you should have an in-depth reading and develop a solid understanding of all the details in the assigned topic. You should prepare well, and make sure you deliver a logically clear and technically accessible presentation. In short words, it is your job to have everyone in the meeting understand the main ideas of the reading.
As a discussant: you should be more familiar with the content than if you were simply in the group. You don’t need to know everything. You can pause the presentation, ask questions (to the presenter or to the audience), and facilitate discussions. It is your job to help the presenter to have everyone (yourself included!) in the meeting understand the main ideas of the reading and having learned something.
Before each session, although not compulsory I would recommend following amount of time spent on reading:
Webpage maintained by Chao Zheng. Last updated on 06/01/2024