Laurea Magistrale (M.Sc.) in Stochastics and Data Science

Bayesian statistics

Bayesian statistics

Academic year 2021/2022

Sommario del corso

• Course objectives
• Results of learning outcomes
• Course delivery
• Learning assessment methods
• Program
• Suggested textbooks and readings
• Teaching tools

Course objectives

Bayesian statistics established itself as the main alternative approach to "classical" statistical inference, which is based on frequentist principles and largely relies on the often too simplistic hypothesis of independence and identity in distribution of the data. Thanks to its appealing mathematical properties and its wide interpretability, Bayesian statistics has nowadays found widespread appreciation by the most diverse scientific communities, as witnessed by the countless registered applications in virtually any applied discipline.

The course aims at providing a modern introduction to Bayesian statistical methods, covering the fundamentals of both the parametric and the nonparametric approach. The course will focus on the key probabilistic and foundational concepts, together with well established stochastic modelling tools and the most widely used computational strategies for its implementation.

A short module of the course, included in the overall courseload, will be taught by Visiting Professor Bas Kleijn (University of Amsterdam) on Frequentist limits from Bayesian Statistics (see International visiting professors).

Results of learning outcomes

Course delivery

The course is composed of 48 hours of lectures held in presence with synchronous online streaming. Please check this page for the teaching modalities foreseen for the a.y. 2021/22.

Teaching materials and updates will be delivered via Moodle (see link below).

Learning assessment methods

The exam consists in the written verification of the student's knowledge and mastering of the material covered in class, by exposition of the main results and constructions and through exercises. The exam will be divided into two parts, one for each haldf course covering the parametric and the nonparametric frameowkrs. The final mark will be the simple average of the two evaluations given by the two lecturers on the respective modules.

From the 2022 winter session the exam is foreseen to be held in presence. 

Program

Bayesian parametric statistics:

• The Bayesian parametric framework
  • Motivation and formal setting for the parametric approach
  • Exchengeability and de Finetti's Theorem 
• One-parameter models
  • Binomial/Poisson/Exponential models
  • Point and interval estimation
  • Exponential families and conjugate priors
  • More on priors: discrete, mixture, default, Jeffreys, non-conjugate, improper 
• (Quick review of) Monte Carlo approximation and Markov chain Monte Carlo methods for parametric inference
• The Normal model: natural conjugate priors, (independent) Jeffreys' prior, predictive distributions
• Other multi-parameters models and hierarchical modelling
• Introduction to Bayesian linear regression models

Bayesian nonparametric statistics:

• The Bayesian nonparametric framework
  • Motivation and formal setting
  • de Finetti's Theorem for general sequences
  • Generic approach to nonparametric posterior inference 
• The Dirichlet Process
  • Properties of the Dirichlet distribution
  • Definition and properties of the Dirichlet process
  • Induced clustering structure and sampling schemes
• Constructions of the Dirichlet process
  • Via finite-dimensional distributions
  • Via gamma subordinators
  • Via stick-breaking
  • Via generalised Polya urn schemes
• Beyond the Dirichlet process
  • Mixtures of Dirichlet processes and with respect to the Dirichlet process
  • Hierarchical Dirichlet process
  • Pitman-Yor processes and species sampling models
  • Normalised random measures with independent increments

Lecture notes will be made available. Additional suggested reading are:

Bayesian parametric statistics

• HOFF, P.D. (2009). A First course in Bayesian statistical methods. Springer. (Recommended)
• GELMAN, A., CARLIN, J. B., STERN, H. S., DUNSON, D. B., VETHARI, A., and RUBIN, D. B. (2014). Bayesian data analysis, Third Edition. CRC Press. (Optional)

Bayesian nonparametric statistics

• GHOSAL, S. and VAN DER VAART, A. (2017). Theory of nonparametric Bayesian inference. Cambridge University Press.
• HJORT, N., HOLMES, C., MUELLER, P. and WALKER, S.G. (eds.) (2010). Bayesian Nonparametrics. Cambridge University Press.
• GHOSH, J.K. and RAMAMOORTHI, R.V. (2003). Bayesian Nonparametrics. Springer.
  
  
  

Class schedule

• Teaching materials
• Exams and finals
• Go to Moodle
• Visit the forums