Undergraduate Course: Machine Learning Theory (INFR11202)
This course will be closed from 31 July 2025
Course Outline
| School | School of Informatics |
College | College of Science and Engineering |
| Credit level (Normal year taken) | SCQF Level 11 (Year 4 Undergraduate) |
Availability | Available to all students |
| SCQF Credits | 10 |
ECTS Credits | 5 |
| Summary | Following the closure of this course, a suggested replacement for students to consider is: Advanced Topics in Machine Learning INFR11286.
This course is an introduction to the theory of learning algorithms and their properties that are relevant to the widespread use of machine learning. The course starts with the standard mathematical concepts in theoretical ML. It then covers classical analytic results about accuracy, confidence, sample complexity and model complexity. Standard learning/optimisation algorithms are described in this context. In modern research areas such as trustworthy machine learning, several properties including privacy, fairness and interpretability are considered vital for widespread reliable use of machine learning. These topics are discussed in a mathematical perspective.
The course aims to provide a firm foundation in reading and understanding mathematical ideas so that students are equipped to follow the latest developments and research, and to interpret relevant properties and trade-offs. Throughout, the course will take the approach of precise mathematical definition and analysis, coupled with easy examples and intuition to aid understanding. |
| Course description |
The following is an indicative list of topics in the course:
1. Notations, terminology and formal models.
2. Learning theory: Empirical risk minimisation and sampling complexity. Probably approximately correct (PAC) guarantees.
3. Complexity of learning models (e.g. VC dimension) and bias-complexity tradeoff.
4. Optimization algorithms. Regression, SVM, Stochastic gradient descent and its variants.
5. Regularization, convexity, stability, Lipschitzness and other properties
6. Statistical Privacy
7. Mechanisms for privacy preserving machine learning. Differentially private stochastic gradient descent.
8. Interpretable machine learning. (E.g. Feature importance)
9. Fairness.
The topics will be discussed with reference to standard machine learning techniques, and examples of realistic problems. Our approach will include precise definitions and analysis as well as examples and intuitive explanations. The relevance and domain of applicability of the various concepts will be discussed.
Tutorials and problem sets will be available to help understanding and exploration of the subject.
|
Information for Visiting Students
| Pre-requisites | As above. |
| High Demand Course? |
Yes |
Course Delivery Information
| Not being delivered |
Learning Outcomes
On completion of this course, the student will be able to:
- interpret and explain rigorous statements about properties of machine learning methods
- evaluate properties of learning models through proofs and examples
- telate, compare, and contrast the implications of various qualities of machine learning models covered in the course
- formulate precise mathematical requirements corresponding to desired properties in real learning problems, and explain their decisions
|
Reading List
| 'Understanding Machine Learning: From Theory to Algorithms', by Shai Ben-David and Shai Shalev-Schwartz |
Additional Information
| Course URL |
https://opencourse.inf.ed.ac.uk/mlt |
| Graduate Attributes and Skills |
Problem solving
Critical / analytical thinking
Independent learning
Written communication |
| Keywords | Machine Learning,Data Science,Algorithms,Theory |
Contacts
| Course organiser | Dr Rik Sarkar
Tel: (0131 6)50 4444
Email: Rik.Sarkar@ed.ac.uk |
Course secretary | Miss Kerry Fernie
Tel: (0131 6)50 5194
Email: kerry.fernie@ed.ac.uk |
|
|
University Homepage