Postgraduate Course: Accelerated Natural Language Processing (INFR11125)
Course Outline
| School | School of Informatics |
College | College of Science and Engineering |
| Credit level (Normal year taken) | SCQF Level 11 (Postgraduate) |
Availability | Available to all students |
| SCQF Credits | 20 |
ECTS Credits | 10 |
| Summary | This course synthesizes ideas from linguistics and computer science to provide students with a fast-paced introduction to the field of natural language processing. We cover a range of foundational concepts, theoretical and computational models of language, and linguistic phenomena. We focus on what makes automatic processing of language unique and challenging: its statistical properties, complex structure, and pervasive ambiguity. We use English as the primary exemplar throughout, but also discuss similarities and differences to other languages, and the implications for computational models.
As we progress from lower levels of linguistic structure (words and morphemes) through syntax to semantics and discourse, we cover formal models and algorithms for representing and analysing these different types of structure (e.g., n-gram models, HMMs, dependency parsing, dynamic programming, and word embeddings). These methods form the conceptual foundation for understanding state-of-the-art approaches, which are covered in depth in the Semester 2 follow-on course NLU+. We also introduce methodological and ethical considerations (e.g., evaluation, data collection, algorithmic bias) that are important for research in the field. |
| Course description |
The course follows a roughly 3-part structure, following the progression of linguistic structure. Each part introduces linguistic phenomena and associated NLP models and algorithms. Interspersed throughout we discuss potential applications (e.g., sentiment analysis, text classification, QA systems) and methodological topics (e.g., evaluation, annotation, ethical concerns).
Part I: Words
* Inflectional and derivational morphology
* Finite state methods and Regular expressions
* Bag-of-Words models and their applications
* Word Classes and Parts of speech
* Sequence Models (n-gram and Hidden Markov models, smoothing)
* The Viterbi algorithm, Forward Backward, EM
Part II: Syntax
* Syntactic Concepts (e.g., constituency, subcategorisation, bounded and unbounded dependencies, feature representations)
* Analysis in CFG - Greedy algorithms---Shift-reduce parsing
* Divide-and-conquer algorithms---CKY
* Lexicalised grammar formalisms (e.g., CCG, dependency grammar)
* Statistical parsing (PCFGs, dependency parsing)
Part III: Semantics and Discourse
* Logical semantics and compositionality
* Semantic derivations in grammar
* Lexical Semantics (e.g., word embeddings, word senses, semantic roles)
* Discourse (e.g., anaphora)
|
Entry Requirements (not applicable to Visiting Students)
| Pre-requisites |
|
Co-requisites | |
| Prohibited Combinations | Students MUST NOT also be taking
Foundations of Natural Language Processing (INFR09028) OR
Foundations of Natural Language Processing (INFR10078)
|
Other requirements | Students with little or no previous programming experience must also register for Computer Programming for Speech and Language Processing.
Labs and assignments require programming in Python at a level designed for students who are learning Python simultaneously with this course. |
Information for Visiting Students
| Pre-requisites | None |
| High Demand Course? |
Yes |
Course Delivery Information
|
| Academic year 2024/25, Available to all students (SV1)
|
Quota: None |
| Course Start |
Semester 1 |
Timetable |
Timetable |
| Learning and Teaching activities (Further Info) |
Total Hours:
200
(
Lecture Hours 30,
Seminar/Tutorial Hours 5,
Supervised Practical/Workshop/Studio Hours 8,
Summative Assessment Hours 2,
Programme Level Learning and Teaching Hours 4,
Directed Learning and Independent Learning Hours
151 )
|
| Assessment (Further Info) |
Written Exam
70 %,
Coursework
30 %,
Practical Exam
0 %
|
| Additional Information (Assessment) |
Coursework will include two practical assignments with written reports, in which parts of an NLP system will be implemented and the results analyzed. Students' knowledge of ethical and contextual issues will be assessed via a third short written assignment and/or on the exam.
|
| Feedback |
Not entered |
| Exam Information |
| Exam Diet |
Paper Name |
Hours & Minutes |
|
| Main Exam Diet S1 (December) | Accelerated Natural Language Processing (INFR11125) | 2:120 | |
Learning Outcomes
On completion of this course, the student will be able to:
- identify, construct, and analyse examples of different kinds of ambiguity in natural language (e.g., ambiguity in part-of-speech, word sense, syntactic attachment). Explain how ambiguity presents a problem for computational analysis, and some of the ways it can be addressed.
- describe and apply standard sequence and classification models; describe parsing and search algorithms for different levels of analysis (e.g. morphology, syntax, and semantics) and simulate each algorithm step-by-step with pen and paper.
- for a range of NLP tasks, outline a processing pipeline for that task, including standard data sets, models, algorithms, and evaluation methods. Given a particular pipeline or part of the pipeline, identify potential strengths and weaknesses of the suggested dataset/method (including both technical and ethical issues, where appropriate), and provide examples to illustrate.
- implement parts of the NLP pipeline with the help of appropriate support code and/or tools. Evaluate and interpret the results of implemented methods on natural language data sets.
|
Reading List
| Jurafsky and Martin, Speech and Language Processing, 3rd ed. (draft). |
Contacts
| Course organiser | Dr Edoardo Ponti
Tel: (0131 6)51 1336
Email: eponti@ed.ac.uk |
Course secretary | Ms Lindsay Seal
Tel: (0131 6)50 5194
Email: lindsay.seal@ed.ac.uk |
|
|
University Homepage