THE UNIVERSITY of EDINBURGH

DEGREE REGULATIONS & PROGRAMMES OF STUDY 2018/2019

University Homepage
DRPS Homepage
DRPS Search
DRPS Contact
DRPS : Course Catalogue : School of Informatics : Informatics

Postgraduate Course: Reinforcement Learning (INFR11010)

Course Outline
SchoolSchool of Informatics CollegeCollege of Science and Engineering
Credit level (Normal year taken)SCQF Level 11 (Postgraduate) AvailabilityAvailable to all students
SCQF Credits10 ECTS Credits5
SummaryReinforcement learning (RL) refers to a collection of machine learning techniques which solve sequential decision making problems using a process of trial-and-error. It is a core area of research in artificial intelligence and machine learning, and today provides one of the most powerful approaches to solving decision problems. This course covers foundational models and algorithms used in RL, as well as advanced topics such as concurrent interactive learning of multiple RL agents and scalable function approximation using neural network representations.
Course description Main topics to be covered include the following (see course website for more details):

* Reinforcement learning framework
* Dynamic programming methods
* Bandit problems and action selection
* Temporal difference methods
* Q-learning and eligibility traces
* Function approximation for generalisation
* Actor-critic method
* Planning in the RL context
* Training agents and evaluating performance
* Multi-agent reinforcement learning
* Environments with partial observability

Relevant QAA Computing Curriculum Sections: Artificial Intelligence, Data Structures and Algorithms, Intelligent Information Systems Technologies, Simulation and Modelling
Entry Requirements (not applicable to Visiting Students)
Pre-requisites Co-requisites
Prohibited Combinations Other requirements This course is open to all Informatics students including those on joint degrees. For external students where this course is not listed in your DPT, please seek special permission from the course organiser (lecturer).

Mathematical background, at the level of undergraduate informatics, particularly linear algebra, multivariate calculus, probability theory and statistics. The coursework involves substantial programming work.
Information for Visiting Students
Pre-requisitesNone
High Demand Course? Yes
Course Delivery Information
Academic year 2018/19, Available to all students (SV1) Quota:  None
Course Start Semester 2
Timetable Timetable
Learning and Teaching activities (Further Info) Total Hours: 100 ( Lecture Hours 20, Seminar/Tutorial Hours 8, Summative Assessment Hours 2, Programme Level Learning and Teaching Hours 2, Directed Learning and Independent Learning Hours 68 )
Assessment (Further Info) Written Exam 75 %, Coursework 25 %, Practical Exam 0 %
Additional Information (Assessment) One assignment worth 25%, one exam worth 75%. The assignment will consist of a large programming exercise in which several of the discussed RL algorithms will be implemented and evaluated. The exam will test factual knowledge and understanding of modelling/algorithmic concepts.
Feedback Not entered
Exam Information
Exam Diet Paper Name Hours & Minutes
Main Exam Diet S2 (April/May)2:00
Learning Outcomes
On completion of this course, the student will be able to:
  1. Knowledge of basic and advanced reinforcement learning techniques.
  2. Identification of suitable learning tasks to which these learning techniques can be applied.
  3. Appreciation of some of the current limitations of reinforcement learning techniques.
  4. Formulation of decision problems, set up and run computational experiments, evaluation of results from experiments.
Reading List
Reinforcement Learning: An Introduction. R. Sutton and A. Barto. MIT Press, 1998

Algorithms for Reinforcement Learning. C. Szepesvari. Morgan and Claypool Publishers, 2010

Reinforcement Learning: State-of-the-Art. M. Wiering and M. van Otterlo. Springer, 2012
Additional Information
Course URL http://course.inf.ed.ac.uk/rl
Graduate Attributes and Skills Not entered
KeywordsArtificial Intelligence,Machine Learning,Reinforcement Learning
Contacts
Course organiserDr Stefano Albrecht
Tel: (0131 6)51 3218
Email: s.albrecht@ed.ac.uk
Course secretaryMrs Sam Stewart
Tel: (0131 6)51 3266
Email: Sam.Stewart@ed.ac.uk
Navigation
Help & Information
Home
Introduction
Glossary
Search DPTs and Courses
Regulations
Regulations
Degree Programmes
Introduction
Browse DPTs
Courses
Introduction
Humanities and Social Science
Science and Engineering
Medicine and Veterinary Medicine
Other Information
Combined Course Timetable
Prospectuses
Important Information