Stastistical Learning and Reinforcement Learning
Under development.
Course objective
The first part of this course will present modern statistical methods for data analysis, encompassing both their theoretical properties and practical aspects. In addition to the lectures, students will gain hands-on experience with the methods through exercises (to be defined in R or Python).
Course content
Reminder
- Basic probability theory: events, random variables, cumulative distribu- tion function, probability density function, law of large numbers, central limit theorem, etc.
- Statistics: statistical models, estimation methods, consistency, hypothesis testing, confidence interval, etc.
Multivariate linear regression
- Hypothesis of linear regressions
- Least-squares estimate
- Representation of categorical variables
- Statistical tests: t-test, F-test, etc.
Beyond linearity
- Non-parametric regression: spline, kernel, additive models
Model selection and evaluation
- Bias-variance trade-off
- Regularization: Ridge, LASSO, elastic-net
- Generalization error, cross-validation
- Selection of hyper-parameters
Bootstrap
- Jackknife
- Bootstrap confidence interval: non-parametric, model-based (parametric), and double bootstrap
- Bootstrap estimate of the generalization error
Statistical tests
- Permutation test
- Multiple testing: false discovery rate, family-wise error rate, Bonferroni correction, Benjamini–Hochberg procedure
