THE UNIVERSITY of EDINBURGH

DEGREE REGULATIONS & PROGRAMMES OF STUDY 2021/2022

Information in the Degree Programme Tables may still be subject to change in response to Covid-19

University Homepage
DRPS Homepage
DRPS Search
DRPS Contact
DRPS : Course Catalogue : School of Informatics : Informatics - Distance Learning

Postgraduate Course: Explainable Machine Learning: A Practical Introduction (INFD11019)

Course Outline
SchoolSchool of Informatics CollegeCollege of Science and Engineering
Credit level (Normal year taken)SCQF Level 11 (Postgraduate)
Course typeOnline Distance Learning AvailabilityNot available to visiting students
SCQF Credits10 ECTS Credits5
SummaryArtificial intelligence (AI) provides many opportunities to improve private and public life. Discovering patterns and structures in large troves of data in an automated manner is a core component of data science, and currently drives applications in diverse areas such as computational biology, law and finance. However, such a highly positive impact is coupled with significant challenges: how do we understand the decisions suggested by these systems in order that we can trust them? In this course, we focus specifically on data driven methods - machine learning (ML) and pattern recognition models in particular - so as to survey and distill the results and observations from the literature. The purpose of this course is to provide and explore the principles and practice of enabling explainability in machine learning models. The course builds a narrative around a putative data scientist, and discusses how she might go about explaining her models by asking the right questions.
Course description Machine learning models are increasingly deployed in a wide range of businesses. However, with the increasing prevalence and complexity of methods, business stakeholders may have concerns about the drawbacks of models, data-specific biases, and so on. Analogously, data science practitioners are often not aware about approaches emerging from the academic literature, or may struggle to appreciate the differences between different methods, so end up using industry standards such as SHAP. Here, we aim to help industry practitioners understand the field of explainable machine learning better and apply the right tools.
From an organization viewpoint, after motivating the area broadly, we turn to technical insights, which includes three frameworks: a taxonomic framework provides an overview of explainable ML, and the other two frameworks further study so-called transparent models vs opaque models in that taxonomy. The latter then requires model specific or model agnostic post-hoc explainability approaches, which have their individual limitations and strengths that are also discussed. We also briefly reflect on deep learning models, and conclude with a discussion about future research directions. In terms of exercises, we will focus on 6 popular techniques and try to understand how to use these explainability techniques with coding problems for publicly available datasets.

Intermediate assignments will be evaluated against workbooks that make use of the post hoc explainability methods discussed in the course. For final assessment, participants can work on a project of their choice, including their own data and demonstrate the ability to mix and match different methods towards a problem statement.

As we expect to use the publicly available Google colab books or similar, no specific hardware is required. Most tools will run in a web-browser.

Individuals will work on their own projects, alone or in groups, and will go through all the stages of the exploration of posthoc explainability, in alignment with the learning outcomes; explore data and make initial findings, critique the tools and list shortcomings and possible future features and present an analysis.

The course has 11 lectures, each targeting a set of principles and/or technique, and which are organized as follows. Topics are provided as pre-recorded lectures, and following a flipped classroom model, during the interaction/live sessions there will be a discussion about the content and insights from the pre-recorded lectures. Participants will have an opportunity to debate conceptual aspects during these live sessions. In addition, there will be tutorial sessions that will give participants an opportunity to engage in coding exercises and raise practical issues about the corresponding topics.

Lectures and topics:
1. Preface to ML: 1 hour
2. Preface to XAI: concepts, ideas, discussions drawn
from the literature 1 hour
3. Frameworks: 1 hour
4. 6 XAI techniques: 6 hours (currently: SHAP, InTrees,
Anchors, PDP, Counterfactuals, Deletion diagnostics)
5. Advanced techniques and concepts: 30 mins
6. Deep learning strategies, future research directions: 30
mins

**This is an introductory Masters-level course. It provides foundational skills in terms of an overview of the subject of explainability in machine learning thus, we expect a basic understanding of machine learning. It is also assumed that participants of the course have some prior programming experience in Python. For more details, see the "Other requirements" box below.
Entry Requirements (not applicable to Visiting Students)
Pre-requisites Co-requisites
Prohibited Combinations Other requirements *Only open to students enrolled on the Data Skills Workforce Development programme*.

This standalone course expects users to have some prior experience in machine learning and programming in python, but does not require them to have taken full courses. Prior experience in machine learning implies some knowledge of the underlying concepts for a few standard prediction/classification models (e.g., random forests, regression), training, feature selection, and evaluation. Mathematical knowledge of these concepts is not assumed but it is helpful to be able to read machine learning-related mathematical notation.
Course Delivery Information
Academic year 2021/22, Not available to visiting students (SS1) Quota:  None
Course Start Flexible
Course Start Date 01/10/2021
Timetable Timetable
Learning and Teaching activities (Further Info) Total Hours: 100 ( Lecture Hours 10, Seminar/Tutorial Hours 8, Feedback/Feedforward Hours 8, Programme Level Learning and Teaching Hours 2, Directed Learning and Independent Learning Hours 72 )
Assessment (Further Info) Written Exam 0 %, Coursework 100 %, Practical Exam 0 %
Additional Information (Assessment) A1: Assignments X 2: Participants analyse problems provided in workbooks by the course.

A2: Final project: For a dataset of their choice, participants describe a problem requiring explainability, and consequently motivate a one or more posthoc explainability methods that apply and finally, discuss results and implications.

A Portfolio (100%) containing 2 assignments and one final project. For the assignments, students analyse problem provided in workbooks. For the final project, students describe a problem requiring explainability for a dataset of their choice, and consequently motivate a one or more posthoc explainability methods that apply and finally, discuss results and implications.

Each item will have a separate due date, written feedback and a pass/fail mark. Students must pass the assignments and the final project to pass the course.
Feedback Feedback will be given to participants as
- oral feedback during tutorials
- Individual written feedback per assignment handed in.
- individual feedback during online forum discussions
- feedforward instructions for final project
No Exam Information
Learning Outcomes
On completion of this course, the student will be able to:
  1. Analyze: Describe the context of the machine learning application and why explainability would help, but also scrutinise which kind of explainability technique is necessary.
  2. Design: Define the implementation pipeline for the project: provide a means to clean the data, install and set up one or more post hoc explain ability techniques through a self-chosen set of programming platforms.
  3. Evaluation: Critically reflect on the results from such techniques and suggest how it helps the problem context.
  4. Apply: Competently apply a wide range of techniques and tools, also knowing their particular features and drawbacks. Have the foundations to understand new and upcoming methods and techniques.
Reading List
- Molnar, C., 2020. Interpretable Machine Learning: A Guide For Making Black Box Models Explainable. https://christophm.github.io/interpretable-ml-book/
- Belle, Vaishak & Papantonis, Ioannis. (2020). Principles and Practice of Explainable Machine Learning.
- Ribeiro, Marco Tulio et al. Anchors: High-Precision Model-Agnostic Explanations. AAAI (2018).
- Ginart, Antonio & Guan, Melody & Valiant, Gregory & Zou, James. (2019). Making AI Forget You: Data Deletion in Machine Learning.
- Brophy, J., 2020. Exit Through the Training Data: A Look into Instance-Attribution Explanations and Efficient Data Deletion in Machine Learning
Additional Information
Graduate Attributes and Skills Problem analysis: analyze the problem related to exploring and communicating data in a specific context
Critical thinking: thinking critically about the effectiveness of post hoc explainability for a given challenge, in a given context.
Creativity: searching for the right set of explainability solutions to a specific challenge
Communication: sensitivity about how to use explainability to explain, improve & debug ML models
KeywordsMachine learning,data analysis,explainable artificial intelligence,interpretable data models,XAI
Contacts
Course organiserMr Vaishak Belle
Tel: (0131 6)50 5150
Email: Vaishak@ed.ac.uk
Course secretaryMs Lindsay Seal
Tel: (0131 6)50 2701
Email: lindsay.seal@ed.ac.uk
Navigation
Help & Information
Home
Introduction
Glossary
Search DPTs and Courses
Regulations
Regulations
Degree Programmes
Introduction
Browse DPTs
Courses
Introduction
Humanities and Social Science
Science and Engineering
Medicine and Veterinary Medicine
Other Information
Combined Course Timetable
Prospectuses
Important Information