THE UNIVERSITY of EDINBURGH

DEGREE REGULATIONS & PROGRAMMES OF STUDY 2022/2023

Timetable information in the Course Catalogue may be subject to change.

University Homepage
DRPS Homepage
DRPS Search
DRPS Contact
DRPS : Course Catalogue : School of Informatics : Informatics

Undergraduate Course: Machine Learning Theory (INFR11202)

Course Outline
SchoolSchool of Informatics CollegeCollege of Science and Engineering
Credit level (Normal year taken)SCQF Level 11 (Year 4 Undergraduate) AvailabilityAvailable to all students
SCQF Credits10 ECTS Credits5
SummaryThis course is an introduction to the theory of learning algorithms and their properties that are relevant to the widespread use of machine learning. The course starts with the standard mathematical concepts in theoretical ML. It then covers classical analytic results about accuracy, confidence, sample complexity and model complexity. Standard learning/optimisation algorithms are described in this context. In modern research areas such as trustworthy machine learning, several properties including privacy, fairness and interpretablity are considered vital for widespread reliable use of machine learning. These topics are discussed in a mathematical perspective.

The course aims to provide a firm foundation in reading and understanding mathematical ideas so that students are equipped to follow the latest developments and research, and to interpret relevant properties and tradeoffs. Throughout, the course will take the approach of precise mathematical definition and analysis, coupled with easy examples and intuition to aid understanding.
Course description The following is an indicative list of topics in the course:

1. Notations, terminology and formal models.
2. Learning theory: Empirical risk minimisation and sampling complexity. Probably approximately correct (PAC) guarantees.
3. Complexity of learning models (e.g. VC dimension) and bias-complexity tradeoff.
4. Optimization algorithms. Regression, SVM, Stochastic gradient descent and its variants.
5. Regularization, convexity, stability, Lipschitzness and other properties
6. Statistical Privacy
7. Mechanisms for privacy preserving machine learning. Differentially private stochastic gradient descent.
8. Interpretable machine learning. (E.g. Feature importance)
9. Fairness.

The topics will be discussed with reference to standard machine learning techniques, and examples of realistic problems. Our approach will include precise definitions and analysis as well as examples and intuitive explanations. The relevance and domain of applicability of the various concepts will be discussed.

Tutorials and problem sets will be available to help understanding and exploration of the subject.
Entry Requirements (not applicable to Visiting Students)
Pre-requisites It is RECOMMENDED that students have passed Introductory Applied Machine Learning (INFR10069) OR Applied Machine Learning (INFR11211) OR Machine Learning and Pattern Recognition (INFR11130) OR Machine Learning (INFR10086) OR Data Analysis and Machine Learning 4 (ELEE10031)
Co-requisites
Prohibited Combinations Students MUST NOT also be taking Machine Learning Theory (UG) (INFR11224)
Other requirements MSc students must register for this course, while Undergraduate students must register for INFR11224 instead.

Students should be confident in standard machine learning ideas: training and test sets, classification, regression, clustering; standard machine learning methods: support vector machines, linear regression, k-means etc. Good understanding of probability and probabilistic arguments is necessary.

Experience of hands on data analysis and use of machine learning can be beneficial, but is not necessary.
Information for Visiting Students
Pre-requisitesAs above.
High Demand Course? Yes
Course Delivery Information
Academic year 2022/23, Available to all students (SV1) Quota:  None
Course Start Semester 2
Timetable Timetable
Learning and Teaching activities (Further Info) Total Hours: 100 ( Lecture Hours 18, Seminar/Tutorial Hours 3, Feedback/Feedforward Hours 1, Summative Assessment Hours 2, Revision Session Hours 1, Programme Level Learning and Teaching Hours 2, Directed Learning and Independent Learning Hours 73 )
Assessment (Further Info) Written Exam 70 %, Coursework 30 %, Practical Exam 0 %
Additional Information (Assessment) Written Exam 70%
Coursework 30%
Feedback Students will receive feedback in the forms of assessment of coursework, tutorials, and problem sets.
Exam Information
Exam Diet Paper Name Hours & Minutes
Main Exam Diet S2 (April/May)Machine Learning Theory (INFR11202)2:00
Learning Outcomes
On completion of this course, the student will be able to:
  1. Interpret and explain rigorous statements about properties of machine learning methods.
  2. Evaluate properties of learning models through proofs and examples.
  3. Relate, compare, and contrast the implications of various qualities of machine learning models covered in the course.
  4. Formulate precise mathematical requirements corresponding to desired properties in real learning problems, and explain their decisions.
Reading List
'Understanding Machine Learning: From Theory to Algorithms', by Shai Ben-David and Shai Shalev-Schwartz
Additional Information
Graduate Attributes and Skills Problem solving
Critical / analytical thinking
Independent learning
Written communication
KeywordsMachine Learning,Data Science,Algorithms,Theory
Contacts
Course organiserDr Rik Sarkar
Tel: (0131 6)50 4444
Email: Rik.Sarkar@ed.ac.uk
Course secretaryMrs Helen Tweedale
Tel: (0131 6)50 3827
Email: Helen.Tweedale@ed.ac.uk
Navigation
Help & Information
Home
Introduction
Glossary
Search DPTs and Courses
Regulations
Regulations
Degree Programmes
Introduction
Browse DPTs
Courses
Introduction
Humanities and Social Science
Science and Engineering
Medicine and Veterinary Medicine
Other Information
Combined Course Timetable
Prospectuses
Important Information