Probabilistic Inference

Module aims

In this module you will have the opportunity to:

  • learn about and implement probabilistic inference methods. Probabilistic inference is used to learn posterior distributions over unknown parameters and quantities in probabilistic machine learning models
  • look at Gaussian processes as one specific model in more detail
  • find out about classical and modern approximate inference techniques. These are being applied in global production systems.

The recommended pre-requisite of this course is a strong mathematical understanding, including probability, statistics, linear algebra and vector calculus.   

Learning outcomes

Upon successful completion of this module you will be able to:

  • implement probabilistic models from scratch
  • develop and apply probabilistic inference techniques
  • analyse the quality of probabilistic models
  • critically evaluate inference models and improve them       

Module syllabus

  • Probabilistic methods for modeling data and making inferences
  • Graphical models
  • Gaussian processes
  • Bayesian optimisation
  • Sampling techniques
  • Variational inference
  • Modern topics, e.g., implicit models, normalising flows, amortised inference, stochastic gradient estimators

Teaching methods

The approximate inference techniques used are developed from probabilistic principles, so you will obtain a deep understanding of the application of the foundations of probability and statistics as well as the application of probabilistic inference to datasets, thereby linking the theory with practice. The material is taught through traditional lectures, backed up by assessed exercises (parts of which will be programming exercises) and tutorials, designed to reinforce the material as it is taught. Tutorials are run by Graduate Teaching Assistants (GTAs) and are designed to reinforce your understanding of the key topics taught.

The Piazza Q&A web service will be used as an open online discussion forum for the module.

Assessments

There will be one coursework that contributes 20% of the mark for the module. There will be a final written exam, which counts for the remaining 80% of the marks.

Feedback will be provided for the coursework.

Reading list

Module leaders

Dr Marc Deisenroth