Machine Learning - AIMS CDT - MT 2021, Week 4

AIMS CDT, 1 – 4 Nov 2021 (Microsoft Teams)
Lectures: 10:30 – 12:30, Practicals: 15:00 – 17:00

Machine learning is a collection of techniques that allow creation or tuning of models using example data or past experience. This course provides an introduction to machine learning with an emphasis on the recent research and application landscape. The course introduces fundamental techniques including differentiable programming and generative modeling. The course also covers probabilistic programming and simulation-based inference, including techniques at the intersection of probabilistic modeling and deep learning.

This module is a part of the Autonomous Intelligent Machines and Systems (AIMS) Centre for Doctoral Training (CDT) programme.

Objectives

• Understand the overall landscape of modern machine learning research and applications
• Get acquainted with differentiable programming (automatic differentiation, or autodiff) and the machinery underlying machine learning frameworks such as PyTorch and TensorFlow
• Learn about generative approaches, Bayesian deep learning, and Bayesian modeling
• Learn about probabilistic programming and simulation-based inference

Content

• Introduction to machine learning [L1 slides] [L2 slides]
• Differentiable programming [L3 slides] [L4 slides]
• Generative modeling [L5 slides] [L6 slides]
• Probabilistic programming and simulation-based inference [L7/L8 slides]

Lecture slides will be linked from here before the beginning of each lecture.

Practicals and Assessment

The demonstrators for the practicals are Benjamin Moseley and A. Tuan Nguyen. The repository for the practicals and the assessed assignment can be found here.

Assignment deadline: 23:59 UK time, 8 Nov 2021

Prerequisites

Basic linear algebra, calculus, probability theory, and programming (ideally in Python)

Reading material

Textbooks

• Christopher M. Bishop, “Pattern Recognition and Machine Learning”, Springer, 2006
• Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, MIT Press, 2016 (Freely available online)
• Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, “Mathematics for Machine Learning”, Cambridge University Press, 2020 (Freely available online) Highly recommended for refreshing your knowledge of the prerequisites.

Extra reading material

• Ghahramani, Z. “Probabilistic machine learning and artificial intelligence.” Nature 521, 452–459 (2015). (Preprint) (Nature)
• Cranmer, K., Brehmer, J. and Louppe, G., 2020. “The frontier of simulation-based inference.” Proceedings of the National Academy of Sciences. (arXiv)
• Baydin, A.G., Pearlmutter, B.A., Radul, A., Siskind, J.M. 2018. “Automatic Differentiation in Machine Learning: a Survey.” Journal of Machine Learning Research (JMLR) 18 (153): 1–43. (arXiv)
• van de Meent, J.W., Paige, B., Yang, H. and Wood, F., 2018. “An introduction to probabilistic programming.” (arXiv)