Empirical Processes Reading Group

2022 Spring-Summer, University of Southampton

In this reading group we are interested in the theory of empirical processes (EP) and its applications in time series and statistical machine learning.

We will mainly go through the lecture notes by Bodhi Sen from Columbia: A Gentle Introduction to Empirical Process Theory and Applications.

We will also have a read on relevant research papers at some point. You are more than welcome to suggest papers related to your research topics.

Coordinator: Chao Zheng
Join our Microsoft Teams Group

This reading group is hybrid — we will meet weekly on Friday 4-5 pm, both at 5001/B54 (in-person) and via MS Teams (online). Feel free to choose your preferred method to join in.

Link to join the meetings via MS Teams

Timetable

Below is a provisional timetable . Some chapters are quite challenging to be finished in just one session, we will make changes accordingly. Please check this website regularly for the most up-to-date arrangement.

	Date	Topic	Presenter	Discussant	Videos
1	11 Mar	Chapter 1. Introduction to EP	Chao	Fangsheng	link
2	18 Mar	Chapter 2. Complexity of function class	Lulu	Chao	link
3	25 Mar	Chapter 3. Glivenko-Cantelli (GC) classes I	Shubin	Chao	link
4	01 Apr	Chapter 3. Glivenko-Cantelli (GC) classes II	Faridah	Chao	link
	Easter Break
5	22 Apr	Chapter 3. Glivenko-Cantelli (GC) classes III	Baiyu	Chao	link
6	29 Apr	Chapter 4. Chaining and uniform entropy	Qiangqiang	Chao	link
7	06 May	Chapter 5. Rates of convergence of M-estimators	Fangsheng	Chao	link
8	13 May	Chapter 6. Infinite dimensional parameters	Chao	Zudi	link
9	20 May	Chapter 7. Vapnik-Chervonenkis (VC) classes I	Lulu	Chao	link
10	27 May	Chapter 7. Vapnik-Chervonenkis (VC) classes II	Shubin	Chao	link
	Platinum Bank Holiday
11	10 Jun	Chapter 8. Talagrand’s inequality I	Baiyu	Chao	link
12	17 Jun	Chapter 8. Talagrand’s inequality II	Qiangqiang	Chao	link
	Break
13	1 July	Chapter 11. Donsker classes	Shubin	Zudi	link
14	8 July	Chapter 12. Limiting distribution of M-estimators	Chao	Zudi	link
15	15 July	Chapter 13. Concentration inequalities	Yan	Chao	link

Materials

If you have created any slides, handouts, notes, minutes, or found anything that will be helpful for the reading/presentation, please send them over to be posted here.

Week 1. Introduction to EP
- Slides by Chao Zheng
- Note on Basic Asymptotic Statistics Terminologies by Han Hong
- Note on Kolmogrov-Simirnov and Mann–Whitney-Wilcoxon Tests by Richard Melrose
Week 2. Complexity of function classes
- Handout by Lulu Wang
- Metric Entropy of Some Classes of Sets with Differentiable Boundaries by Ricard M.Dudley
- Slides on Metric Entropy by Peter Bartlett
- $\epsilon$ -Entropy of Convex Sets and Functions by ‪Efim Bronshtein
Week 3. Chapter 3. Glivenko-Cantelli (GC) classes I
- Proof of Lemma 3.8 and Lemma 3.16 by Shubin Wu
- Note on Empirical Processes: Glivenko–Cantelli Theorems by Moulinath Banerjee
- Note on Sub Gaussian Processes, Metric Entropy and Chaining by Alessandro Rinaldo
Week 4. Chapter 3. Glivenko-Cantelli (GC) classes II
- Slides by Faridah Alruwaili
- Note on Glivenko–Cantelli Theorems proof by Jian Zhang
- Note on Glivenko–Cantelli Theorems and its Generalizations by Thomas Kahle
Week 5. Chapter 3. Glivenko-Cantelli (GC) classes III
- Slides by Baiyu Wang
- Proof of Theorem 3.26 by Baiyu Wang
- Note on Bounded Differences Inequality and Concentration Inequalities by Alessandro Rinaldo
- Note on Basic Tail and Concentration Bounds by Martin Wainwright
Week 6. Chapter 4. Chaining and uniform entropy
- Slides by Qiangqiang Zhu
- A Chaining Algorithm for Online Nonparametric Regression by Pierre Gaillard
Week 7. Chapter 5. Rates of convergence of M-estimators
- Handout by Fangsheng Ge
- Persistence in High-Dimensional Linear Predictor Selection and the Virtue of Overparametrization by Eitan Greenshtein and Ya’acov Ritov
Week 8. Chapter 6. Infinite dimensional parameters
- Handout by Chao Zheng
- Convergence Rate of Sieve Estimates by Xiaotong Shen and Wing W. Wong
- A Short Note on Local Gaussian complexity by Alessandro Rinaldo
Week 9. Chapter 7. Vapnik-Chervonenkis (VC) classes I
- Handout by Lulu Wang
Week 10. Chapter 7. Vapnik-Chervonenkis (VC) classes II
- Handout for Lemma 7.12 by Shubin Wu
Week 11. Chapter 8. Talagrand’s inequality I
- Handout by Baiyu Wang
Week 12. Chapter 8. Talagrand’s inequality II
- Handout by Qiangqiang Zhu
Week 13&15. Chapter 11. Donsker Classes
- Handout and proofs of Donsker Theorem by Shubin Wu
Week 14. Chapter 12. Limiting distribution of M-estimator
- Handout by Chao Zheng
Week 15. Chapter 13. Concentration Inequalties
- Handout by Yan Zhang
- Concentration Inequalities: A Nonasymptotic Theory of Independence by Stéphane Boucheron, Gábor Lugosi, and Pascal Massart (Section 4 & 5)

Erratum

Corrections on the lecture notes

Group Members

Prof. Zudi Lu (z.lu@soton.ac.uk)
Prof. Hou-Duo Qi (hdqi@soton.ac.uk)
Dr. Chao Zheng (chao.zheng@soton.ac.uk)
Dr. Fangsheng Ge (f.ge@soton.ac.uk)
Dr. Rong Peng (rong.peng@ed.ac.uk)
Dr. Huan Yu (Huan.Yu@soton.ac.uk)
Dr. Libo Li (libo.li@soton.ac.uk)
Dr. Xiaohang Ren (domrxh@outlook.com)
Lulu Wang (lulu.wang@soton.ac.uk)
Shubin Wu (sw6y19@soton.ac.uk)
Faridah Alruwaili （f.alruwaili@soton.ac.uk）
Baiyu Wang (bw1g21@soton.ac.uk)
Yan Zhang (yz4u18@soton.ac.uk)
Qiangqiang Zhu (qz6u21@soton.ac.uk)

Anyone interested in the topics of EP theory and applications are welcome (even if you are not based at Southampton). If you want to join the reading group, please do not hesitate to contact Chao Zheng.

Roles of Presenter and Discussant

If it is your first time attend a reading group, you might find the reading group tips by Lester Mackey and Percy Liang helpful.

Every time we will have one people (presenter) present the main contents and another people (discussant) raise questions and lead the discussion.

As a presenter: you should have an in-depth reading and develop a solid understanding of all the details in the assigned topic. You should prepare well, and make sure you deliver a logically clear and technically accessible presentation. In short words, it is your job to have everyone in the meeting understand the main ideas of the reading. Beside above lecture notes, you may also find the following supplementary references helpful for preparing your presentation.
As a discussant: you should be more familiar with the content than if you were simply in the group. You don’t need to know everything. You can pause the presentation, ask questions (to the presenter or to the audience), and facilitate discussions. It is your job to help the presenter to have everyone (yourself included!) in the meeting understand the main ideas of the reading and having learned something.

Before each session, although not compulsory I would recommend following amount of time spent on reading:

Presenter: > 10 hours;
Discussant: 5 hours;
General audience: 2 hours.

If you encounter any problem during your reading，feel free to discuss with me or other staff members.

Supplementary References

Weak Convergence and Empirical Processes. Aad van der Vaart and Jon Wellner, Springer, 1996.
Empirical Processes: Theory and Application. Jon Wellner, lecture notes, 2005
Empirical Processes in M-Estimation. Sara van de Geer, Cambridge Press, 2000.
Empirical Processes: Theory and Application. David Pollard, 1990
Introduction to Empirical Processes and Semiparametric Inference. Michael Kosorok, Springer, 2008.
Lecture Notes on Empirical Process Theory. Kengo Kato, lecture notes, 2019.
Empirical Processes with Applications in Statistics. Galen Shorack and Jon Wellner, Wiley, 1986
Empirical Process Techniques for Dependent Data. Herold Dehling, Thomas Mikosch and Michael Sørensen, Springer, 2002.
Mathematical Foundations of Infinite-Dimensional Statistical Models. Evarist Gine and Richard Nickl, Cambridge Press, 2016.