Machine Learning and Deep Learning - Intelligent Earth CDT - MT 2024, Week 5

Intelligent Earth CDT, 11–15 Nov 2024
Lectures: 10:00–12:00 (Mon-Tue and Thu-Fri), DTC Seminar Room 2A
Practicals: 15:00–17:00 (Mon-Tue and Thu-Fri), DTC student offices 6,7

Machine learning is a collection of techniques that allow creation or tuning of models using example data or past experience. This course provides an introduction to machine learning with an emphasis on the recent research and application landscape. We aim at building a foundational understanding of key concepts and practical techniques. Over four days, participants will dive into essential topics, starting with an overview of machine learning and its relationship to AI and statistics, exploring both the theoretical landscape and real-world applications. Day 2 introduces differentiable programming, emphasizing the role of derivatives in model training and covering automatic differentiation techniques. Day 3 focuses on training machine learning models, covering topics like neural network architectures, activation functions, optimizers, and regularization strategies. Finally, foundational machine learning architectures such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and transformers are introduced, providing participants with a practical understanding of these powerful tools used in modern AI.

This module is a part of the UKRI AI Centre for Doctoral Training in AI for the Environment (Intelligent Earth) programme.

Lecturers

• Atilim Gunes Baydin, Department of Computer Science, University of Oxford
• Annie Qurat Ul Ain, Big Data Institute, University of Oxford

Content

• Introduction to machine learning [L1 slides] [L2 slides]
• Differentiable programming [L3 slides] [L4 slides]
• Training machine learning models
• Foundational architectures

Lecture slides will be provided here before the beginning of each lecture.

Practicals and Assessment

The demonstrators for the practicals are Sixtine Dromigny and Zeynep Duygu Tekler. The repository for the practicals and the assessed assignment can be found in this GitHub repository.

Assignment deadline: 23:59 UK time, 22 November 2024

Prerequisites

Basic linear algebra, calculus, probability theory, and programming (ideally in Python)

Reading material

Textbooks

• Christopher M. Bishop, “Pattern Recognition and Machine Learning”, Springer, 2006
• Ian Goodfellow, Yoshua Bengio and Aaron Courville, “Deep Learning”, MIT Press, 2016 (Freely available online)
• Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong, “Mathematics for Machine Learning”, Cambridge University Press, 2020 (Freely available online) Highly recommended for refreshing your knowledge of the prerequisites.

Extra reading material

• Ghahramani, Z. “Probabilistic machine learning and artificial intelligence.” Nature 521, 452–459 (2015). (Preprint) (Nature)
• Cranmer, K., Brehmer, J. and Louppe, G., 2020. “The frontier of simulation-based inference.” Proceedings of the National Academy of Sciences. (arXiv)
• Baydin, A.G., Pearlmutter, B.A., Radul, A., Siskind, J.M. 2018. “Automatic Differentiation in Machine Learning: a Survey.” Journal of Machine Learning Research (JMLR) 18 (153): 1–43. (arXiv)
• van de Meent, J.W., Paige, B., Yang, H. and Wood, F., 2018. “An introduction to probabilistic programming.” (arXiv)