Postgraduate Course: Accelerated Natural Language Processing (INFR11125)
Course Outline
| School | School of Informatics |
College | College of Science and Engineering |
| Credit level (Normal year taken) | SCQF Level 11 (Postgraduate) |
Availability | Available to all students |
| SCQF Credits | 20 |
ECTS Credits | 10 |
| Summary | This course provides students with a fast-paced introduction to the field of natural language processing: what makes modelling linguistic data different from other areas of machine learning, along with current NLP methods and applications. We cover a range of foundational architectures from deep learning and relevant linguistic phenomena. We focus on what makes automatic processing of language unique and challenging: its statistical properties, complex structure, and pervasive ambiguity. We use English as the primary exemplar throughout but also discuss similarities and differences to other languages, and the implications for NLP models.
The course starts with simple models for text classification and generation. We will then discuss neural models to represent the meaning of words and model language, such as Recurrent Neural Networks and Transformers. We will cover the training pipeline for current Large Language Models, including pre-training, supervised fine-tuning, and alignment with human feedback. As part of the course, we will also introduce methodological and ethical considerations (e.g., evaluation, data collection, algorithmic bias) that are important for research or working in the field |
| Course description |
The course will first introduce simple and interpretable models, such as n-gram and bag-of-words models and logistic regression, to illustrate a range of NLP tasks (language modelling, classification, and generation), as well as the basic framework for NLP experiments (training, evaluation, baselines).
Next, we will cover neural architectures for NLP (such as Multi-Layer Perceptrons, Recurrent Neural Networks, and Transformers), which are more opaque and data-hungry, but also achieve better performance on NLP tasks.
Finally, we will discuss the framework of transfer learning, which leverages large amounts of unsupervised data, and the training pipeline for Large Language Models. This spans from pre-training to supervised fine-tuning and alignment with human feedback.
Throughout the course, we will introduce concepts and findings from linguistics as a way to understand the challenges of this type of data. We will discuss the strengths and weaknesses of different approaches, including both technical and ethical challenges (such as bias and interpretability). We will illustrate how NLP models can be used for specific applications (e.g., translation and summarisation) as well as for general-purpose execution of natural language instructions.
|
Entry Requirements (not applicable to Visiting Students)
| Pre-requisites |
|
Co-requisites | |
| Prohibited Combinations | Students MUST NOT also be taking
Foundations of Natural Language Processing (INFR09028) OR
Foundations of Natural Language Processing (INFR10078)
|
Other requirements | Maths requirements:
Linear algebra: Vectors: scalar (dot) product, transpose, unit vectors, vector length and orthogonality. Matrices: addition, matrix multiplication. Tensors with more than 2 axes.
Special functions: properties and combination rules for logarithm and exponential.
Calculus: Rules for differentiating standard functions (including chain rule), partial derivative.
Probability theory: Discrete univariate and multivariate random variables. Expectation and variance. Joint and conditional distributions.
Students with little or no previous programming experience must also register for LASC11096 Computer Programming for Speech and Language Processing.
Labs and assignments require programming in Python at a level designed for students who are learning Python simultaneously with this course. |
Information for Visiting Students
| Pre-requisites | Maths requirements:
Linear algebra: Vectors: scalar (dot) product, transpose, unit vectors, vector length and orthogonality. Matrices: addition, matrix multiplication. Tensors with more than 2 axes.
Special functions: properties and combination rules for logarithm and exponential.
Calculus: Rules for differentiating standard functions (including chain rule), partial derivative.
Probability theory: Discrete univariate and multivariate random variables. Expectation and variance. Joint and conditional distributions.
Students with little or no previous programming experience must also register for LASC11096 Computer Programming for Speech and Language Processing.
Labs and assignments require programming in Python at a level designed for students who are learning Python simultaneously with this course. |
| High Demand Course? |
Yes |
Course Delivery Information
|
| Academic year 2025/26, Available to all students (SV1)
|
Quota: None |
| Course Start |
Semester 1 |
Timetable |
Timetable |
| Learning and Teaching activities (Further Info) |
Total Hours:
200
(
Lecture Hours 28,
Seminar/Tutorial Hours 4,
Supervised Practical/Workshop/Studio Hours 6,
Feedback/Feedforward Hours 1,
Summative Assessment Hours 2,
Revision Session Hours 1,
Programme Level Learning and Teaching Hours 4,
Directed Learning and Independent Learning Hours
154 )
|
| Assessment (Further Info) |
Written Exam
70 %,
Coursework
30 %,
Practical Exam
0 %
|
| Additional Information (Assessment) |
Written exam 70%
Coursework 30%
The coursework will include a single practical assignment with written reports, in which parts of an NLP system will be implemented and the results analysed. |
| Feedback |
Not entered |
| Exam Information |
| Exam Diet |
Paper Name |
Minutes |
|
| Main Exam Diet S1 (December) | Accelerated Natural Language Processing (INFR11125) | 120 | |
Learning Outcomes
On completion of this course, the student will be able to:
- explain and provide examples illustrating some of the main challenges facing machine learning approaches to natural language data, including issues arising from properties of language (e.g., long sequence modelling; variation across languages, domains, and genres) and from social and ethical concerns (e.g., algorithmic bias and discrimination, interpretability)
- describe NLP models for classification and generation tasks; the experimental setup for training and testing; and how these models address some of the technical challenges described above
- for a range of NLP applications, outline possible approaches, including standard data sets, models, and evaluation methods. Discuss potential strengths and weaknesses of the suggested approaches (including both technical and ethical issues, where appropriate), and provide examples to illustrate
- implement parts of an NLP system with the help of appropriate support code and/or tools. Evaluate and interpret the results of implemented methods on natural language data sets
|
Reading List
| Jurafsky and Martin, Speech and Language Processing, 3rd ed. (draft). |
Contacts
| Course organiser | Dr Edoardo Ponti
Tel: (0131 6)51 1336
Email: eponti@ed.ac.uk |
Course secretary | Ms Lindsay Seal
Tel: (0131 6)50 5194
Email: lindsay.seal@ed.ac.uk |
|
|
University Homepage