Undergraduate Course: Privacy and Security with Machine Learning (INFR11240)
Course Outline
| School | School of Informatics |
College | College of Science and Engineering |
| Credit level (Normal year taken) | SCQF Level 11 (Year 4 Undergraduate) |
Availability | Not available to visiting students |
| SCQF Credits | 10 |
ECTS Credits | 5 |
| Summary | The increasing popularity of machine learning (ML) and its new applications has raised security and privacy concerns. ML techniques provide adversaries with new capabilities to undermine the security and privacy of ICT systems. Moreover, there is a growing recognition of the security and privacy issues of ML algorithms. These issues can have tremendous consequences in a society that increasingly relies on ML, so they call for more robust methods that allow harnessing the benefits of ML in adversarial settings.
This is an introductory course on the topic. The course will provide an overview of adversarial applications of ML techniques, including those that undermine the privacy and security of ML-based systems. The course will also cover existing countermeasures and mitigation strategies. |
| Course description |
The course is divided into two blocks:
1. Emerging applications of ML techniques in the security and privacy domain
2. Threats to the privacy and security of ML models and risk mitigation strategies
In Block 1, we will cover novel applications of ML techniques to a range of problems in the security domain: traffic analysis, (de)anonymization of documents, hardware side channels, etc. The focus will be on how ML has augmented the attacker's capabilities.
In Block 2, we will turn to threats to the security and privacy of ML algorithms. We will review some of the following attacks: membership inference attacks, poisoning attacks, adversarial examples, etc. We will also discuss countermeasures that have been proposed to mitigate these attacks.
The lectures will combine presentations of specific research papers and the background in ML and Cyber Security required to follow the papers.
Most of the lectures will have a traditional format, where the lecturer will guide the discussion. The students will prepare for the lectures by reading the papers and reviewing the theory.
|
Entry Requirements (not applicable to Visiting Students)
| Pre-requisites |
It is RECOMMENDED that students have passed
Machine Learning (INFR10086) OR
Applied Machine Learning (INFR11211) OR
Machine Learning and Pattern Recognition (INFR11130) OR
Computer Security (INFR10067)
|
Co-requisites | |
| Prohibited Combinations | Students MUST NOT also be taking
Privacy and Security with Machine Learning (UG) (INFR11252)
|
Other requirements | Students may only register for this course if it is explicitly listed in their DPT.
MSc students must register for this course, while Undergraduate students must register for INFR11252 instead.
It is expected that students have most of the following security and machine learning skills. These would be satisfied by taking the required security prerequisite course, and one of the recommended machine learning prerequisite courses or a similar alternative machine learning course.
Security and privacy
Skills:
- Basic threat modelling skills: be able to identify the threat and the security/privacy property at stake, be aware of the need to define the adversary's knowledge and capabilities, etc.
- Be aware of the principles underlying popular security and privacy solutions.
- Be able to indicate potential strategies to mitigate a threat or the use of tools/protocols that are designed to protect against a privacy/security threat.
Concepts: network protocols (TCP/IP, DNS, HTTPS); basic privacy and security properties (confidentiality, integrity, availability, etc.); privacy enhancing technology; anonymity sets; entropy; personally identifiable information; basic cryptographic protocols; access control; authentication
Machine learning
Skills:
- Previous experience implementing and evaluating a supervised learning algorithm at a high level (e.g., with the help of a third-party ML library)
- Be familiar with some of the applications of ML: object recognition, natural language processing, optimization, recommendation systems, etc.
- Be aware of some of the strengths and weaknesses of learning algorithms (e.g., lack of interpretability, complexity and tendency to overfit, etc.)
Concepts: supervised learning pipeline (training, validation, testing); some of the basic ML algorithms: decision trees, neural networks, SVM, Naïve Bayes, k-nearest neighbours, etc.; bias-variance tradeoff (under-/over-fitting); generative vs discriminative; classification/regression; cross-validation; decision boundary; ROC curve; regularization; feature analysis and representation. |
Course Delivery Information
|
| Academic year 2024/25, Not available to visiting students (SS1)
|
Quota: None |
| Course Start |
Semester 2 |
Timetable |
Timetable |
| Learning and Teaching activities (Further Info) |
Total Hours:
100
(
Lecture Hours 15,
Supervised Practical/Workshop/Studio Hours 3,
Summative Assessment Hours 2,
Programme Level Learning and Teaching Hours 2,
Directed Learning and Independent Learning Hours
78 )
|
| Assessment (Further Info) |
Written Exam
75 %,
Coursework
25 %,
Practical Exam
0 %
|
| Additional Information (Assessment) |
Written Exam 75%
Coursework 25%
The coursework will not require an implementation; it will be a report (10 pages max) that the students will have to write in teams about the privacy and security risks of an ML application of their choice. The students will receive feedback on a draft of a report that will not be marked and will submit the final report (15%) in the last week of the semester.
There will be a quiz at the end of the first part of the course (10%).
There will be two lab sessions that require implementing one of the attacks. The lab sessions are not marked but the students are strongly encouraged to attend the sessions in preparation for the exam.
There will be a final exam (75%) to assess the students' progress in achieving the course's learning outcomes. |
| Feedback |
The students will receive detailed feedback from the draft of the report indicating what aspects they can improve. They will also receive feedback from the lab demonstrators and obtain the correct answers to the quiz. |
| Exam Information |
| Exam Diet |
Paper Name |
Hours & Minutes |
|
| Main Exam Diet S2 (April/May) | Privacy and Security with Machine Learning | 2:00 | |
Learning Outcomes
On completion of this course, the student will be able to:
- classify the attacks covered by the course according to their adversary model (e.g., objective, background knowledge, and capabilities)
- define the underlying privacy and security properties undermined by the attacks
- identify properties and assumptions of each setting that are necessary for the effectiveness of some of the attacks (e.g., in attacks against a supervised model: overfitting of the model, i.i.d. assumption, etc)
- discuss the challenges in mitigating specific attacks (e.g., trade-offs between accuracy and privacy)
- identify the privacy and security concerns in real-world applications of ML. For example, if you train an ML model on healthcare data, what could go wrong for your future users¿ privacy?
|
Reading List
Mark Stamp. Introduction to Machine Learning with Applications in Information Security. 2022.
Nelson, Rubinstein, Joseph, Tygar. Adversarial Machine Learning. Cambridge University Press, 2019.
Papernot et al. SoK: Security and Privacy in Machine Learning. IEEE Euro S&P 2018
MITRE's ATLAS: https://atlas.mitre.org |
Additional Information
| Graduate Attributes and Skills |
Critical thinking, adversarial thinking, and knowledge integration. Creativity. Teamwork skills, verbal, and cross-disciplinary communication. |
| Keywords | Privacy,Cyber Security,Machine Learning |
Contacts
| Course organiser | Dr Marc Juarez Miro
Tel: (0131 6)50 2707
Email: marc.juarez@ed.ac.uk |
Course secretary | Miss Yesica Marco Azorin
Tel: (0131 6)505113
Email: ymarcoa@ed.ac.uk |
|
|
University Homepage