Postgraduate Course: Reinforcement Learning (INFR11010)
|School||School of Informatics
||College||College of Science and Engineering
|Credit level (Normal year taken)||SCQF Level 11 (Postgraduate)
||Availability||Available to all students
|Summary||Reinforcement learning (RL) refers to a collection of machine learning techniques which solve sequential decision making problems using a process of trial-and-error. It is a core area of research in artificial intelligence and machine learning, and today provides one of the most powerful approaches to solving decision problems. This course covers foundational models and algorithms used in RL, as well as advanced topics such as scalable function approximation using neural network representations and concurrent interactive learning of multiple RL agents.
Main topics to be covered include the following (see course website for more details):
* Reinforcement learning framework
* Bandit problems and action selection
* Dynamic programming
* Monte Carlo methods
* Temporal difference learning
* Planning in RL
* Function approximation for generalisation
* Actor-critic and gradient-based optimisation
* Multi-agent reinforcement learning
* Training agents and evaluating performance
Relevant QAA Computing Curriculum Sections: Artificial Intelligence, Data Structures and Algorithms, Intelligent Information Systems Technologies, Simulation and Modelling
Entry Requirements (not applicable to Visiting Students)
||Other requirements|| Mathematical background, at the level of undergraduate informatics, particularly linear algebra, multivariate calculus, probability theory, and statistics.
Good understanding of Python programming, familiarity with the scientific package NumPy. Familiarity with PyTorch is helpful.
It is recommended that students also take other machine learning courses such as MLPR, IAML, PMR. Experience has shown that students who struggled in these courses also struggle in the RL course.
This course is open to all Informatics students including those on joint degrees. For external students where this course is not listed in your DPT, please seek special permission from the course organiser.
Information for Visiting Students
|High Demand Course?
Course Delivery Information
|Academic year 2020/21, Available to all students (SV1)
|Learning and Teaching activities (Further Info)
Lecture Hours 20,
Seminar/Tutorial Hours 8,
Summative Assessment Hours 2,
Programme Level Learning and Teaching Hours 2,
Directed Learning and Independent Learning Hours
|Assessment (Further Info)
|Additional Information (Assessment)
||Written Exam 50%
One assignment worth 50%, one exam worth 50%. The assignment will consist of a large programming exercise (Python) in which several RL algorithms will be implemented and evaluated. The exam will test factual knowledge and understanding of modelling/algorithmic concepts.
||Hours & Minutes
|Main Exam Diet S2 (April/May)||2:00|
On completion of this course, the student will be able to:
- Knowledge of basic and advanced reinforcement learning techniques.
- Identification of suitable learning tasks to which these learning techniques can be applied.
- Appreciation of some of the current limitations of reinforcement learning techniques.
- Formulation of decision problems, set up and run computational experiments, evaluation of results from experiments.
|Reinforcement Learning: An Introduction (second edition). R. Sutton and A. Barto. MIT Press, 2018|
Algorithms for Reinforcement Learning. C. Szepesvari. Morgan and Claypool Publishers, 2010
Reinforcement Learning: State-of-the-Art. M. Wiering and M. van Otterlo. Springer, 2012
|Graduate Attributes and Skills
|Keywords||Artificial Intelligence,Machine Learning,Reinforcement Learning
|Course organiser||Dr Stefano Albrecht
Tel: (0131 6)51 3218
|Course secretary||Ms Lindsay Seal
Tel: (0131 6)50 2701