Postgraduate Course: Reinforcement Learning (INFR11010)
Course Outline
| School | School of Informatics |
College | College of Science and Engineering |
| Credit level (Normal year taken) | SCQF Level 11 (Postgraduate) |
Availability | Available to all students |
| SCQF Credits | 10 |
ECTS Credits | 5 |
| Summary | Reinforcement learning (RL) refers to a collection of machine learning techniques which solve sequential decision making problems using a process of trial-and-error. It is a core area of research in artificial intelligence and machine learning, and today provides one of the most powerful approaches to solving decision problems. This course covers foundational models and algorithms used in RL, as well as advanced topics such as concurrent interactive learning of multiple RL agents and scalable function approximation using neural network representations. |
| Course description |
The main topics to be covered are some or all of the following (there are some changes from year to year)
* Reinforcement learning framework
* Bandit problems and action selection
* Dynamic programming methods
* Monte-Carlo methods
* Temporal difference methods
* Q-learning and eligibility traces
* Environment modelling
* Function approximation for generalisation
* Actor-critic, applications
* Planning in the RL context
* Unsupervised, self-organising networks and RL
* Constructive methods - nets that grow
* Evaluating performance
Relevant QAA Computing Curriculum Sections: Artificial Intelligence, Data Structures and Algorithms, Intelligent Information Systems Technologies, Simulation and Modelling
|
Entry Requirements (not applicable to Visiting Students)
| Pre-requisites |
|
Co-requisites | |
| Prohibited Combinations | |
Other requirements | This course is open to all Informatics students including those on joint degrees. For external students where this course is not listed in your DPT, please seek special permission from the course organiser (lecturer).
Mathematical background, at the level of undergraduate informatics, particularly linear algebra, multivariate calculus and statistics. Some programming (e.g. in Matlab) will be required. |
Information for Visiting Students
| Pre-requisites | None |
| High Demand Course? |
Yes |
Course Delivery Information
|
| Academic year 2018/19, Available to all students (SV1)
|
Quota: None |
| Course Start |
Semester 2 |
Timetable |
Timetable |
| Learning and Teaching activities (Further Info) |
Total Hours:
100
(
Lecture Hours 20,
Seminar/Tutorial Hours 8,
Summative Assessment Hours 2,
Programme Level Learning and Teaching Hours 2,
Directed Learning and Independent Learning Hours
68 )
|
| Assessment (Further Info) |
Written Exam
80 %,
Coursework
20 %,
Practical Exam
0 %
|
| Additional Information (Assessment) |
Written exam 80%, Assessed Course work 20%, Oral Presentations 0%.
One assignment worth 20%, one exam worth 80%. The assignment will consist of a large programming exercise in which several of the discussed RL algorithms will be implemented and evaluated. The exam will test factual knowledge and understanding of modelling/algorithmic concepts.
If delivered in semester 1, this course will have an option for semester 1 only visiting undergraduate students, providing assessment prior to the end of the calendar year. |
| Feedback |
Not entered |
| Exam Information |
| Exam Diet |
Paper Name |
Hours & Minutes |
|
| Main Exam Diet S2 (April/May) | | 2:00 | |
Learning Outcomes
On completion of this course, the student will be able to:
- Knowledge of basic and advanced reinforcement learning techniques.
- Identification of suitable learning tasks to which these learning techniques can be applied.
- Appreciation of some of the current limitations of reinforcement learning techniques.
- Formulation of decision problems, ability to set up and run computational experiments, evaluation of results from the students own experiments.
|
Reading List
Reinforcement Learning: An Introduction. R. Sutton and A. Barto. MIT Press, 1998
Algorithms for Reinforcement Learning. C. Szepesvari. Morgan and Claypool Publishers, 2010
Reinforcement Learning: State-of-the-Art. M. Wiering and M. van Otterlo. Springer, 2012 |
Additional Information
| Course URL |
http://course.inf.ed.ac.uk/rl |
| Graduate Attributes and Skills |
Not entered |
| Keywords | Artificial Intelligence,Machine Learning |
Contacts
| Course organiser | Dr Stefano Albrecht
Tel: (0131 6)51 3218
Email: s.albrecht@ed.ac.uk |
Course secretary | Mrs Sam Stewart
Tel: (0131 6)51 3266
Email: Sam.Stewart@ed.ac.uk |
|
|
University Homepage