Machine Learning - AIMS CDT - MT 2020, Week 4

AIMS CDT, 2 – 5 Nov 2020 (Microsoft Teams)
Lectures: 10:30 – 12:30, Practicals: 15:00 – 17:00

Machine learning is a collection of techniques that allow creation or tuning of models using example data or past experience. This course provides an introduction to machine learning with an emphasis on the recent research and application landscape. The course covers fundamental concepts including supervised and unsupervised learning, generative and discriminative modeling, differentiable programming, and a taxonomy of model architectures that are frequently used in practice. The course also covers probabilistic programming and simulation-based inference, including techniques at the intersection of probabilistic modeling and deep learning such as flow models.

This module is a part of the Autonomous Intelligent Machines and Systems (AIMS) Centre for Doctoral Training (CDT) programme.

Objectives

• Understand the overall landscape of modern machine learning research and applications
• Get acquainted with differentiable programming (automatic differentiation, or autodiff) and the machinery behind machine learning frameworks such as PyTorch and TensorFlow
• Learn about architectures that work with simple, sequential, and graph-structured data and the recent advances in flow models and transformers
• Learn about probabilistic programming and simulation-based inference

Content

• Introduction to machine learning: [L1 slides] [L2 slides]
• Differentiable programming: [L3 slides] [L4 slides]
• Recent deep learning [L5/L6 slides]
• Probabilistic programming and simulation-based inference [L7/L8 slides]

Lecture slides will be linked from here before the beginning of each lecture.

Practicals and Assessment

The demonstrators for the practicals are Benjamin Moseley and Lewis Smith. The repository for the practicals and the assessed assignment can be found here.

Assignment deadline: 23:59 UK time, 9 Nov 2020

Prerequisites

Basic linear algebra, calculus, probability theory, and programming (ideally in Python)

Reading material

Textbooks

• Christopher M. Bishop, “Pattern Recognition and Machine Learning”, Springer, 2006
• Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, MIT Press, 2016 (Freely available online)
• Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, “Mathematics for Machine Learning”, Cambridge University Press, 2020 (Freely available online) Highly recommended for refreshing your knowledge of the prerequisites.

Extra reading material

• Ghahramani, Z. “Probabilistic machine learning and artificial intelligence.” Nature 521, 452–459 (2015). (Preprint) (Nature)
• Cranmer, K., Brehmer, J. and Louppe, G., 2020. “The frontier of simulation-based inference.” Proceedings of the National Academy of Sciences. (arXiv)
• Baydin, A.G., Pearlmutter, B.A., Radul, A., Siskind, J.M. 2018. “Automatic Differentiation in Machine Learning: a Survey.” Journal of Machine Learning Research (JMLR) 18 (153): 1–43. (arXiv)
• van de Meent, J.W., Paige, B., Yang, H. and Wood, F., 2018. “An introduction to probabilistic programming.” (arXiv)