THE UNIVERSITY of EDINBURGH

DEGREE REGULATIONS & PROGRAMMES OF STUDY 2012/2013
- ARCHIVE for reference only
THIS PAGE IS OUT OF DATE

University Homepage
DRPS Homepage
DRPS Search
DRPS Contact
DRPS : Course Catalogue : School of Informatics : Informatics

Postgraduate Course: Reinforcement Learning (INFR11010)

Course Outline
SchoolSchool of Informatics CollegeCollege of Science and Engineering
Course typeStandard AvailabilityAvailable to all students
Credit level (Normal year taken)SCQF Level 11 (Postgraduate) Credits10
Home subject areaInformatics Other subject areaNone
Course website http://www.inf.ed.ac.uk/teaching/courses/rl Taught in Gaelic?No
Course descriptionThis module covers a range of adaptive learning systems, in particular reinforcement learning and unsupervised methods, particularly as used in RL systems. By the end of the module the student should have a grasp of modern learning techniques and the issues involved in dealing with real-world data. The main techniques covered in the course are basic reinforcement learning, dynamic programming, Monte Carlo methods, Q-learning, function approximation, unsupervised and constructive methods, radial basis and other local functions, classifier systems as compared to RL systems.
Entry Requirements (not applicable to Visiting Students)
Pre-requisites Co-requisites It is RECOMMENDED that students also take Natural Computing (INFR09038)
Prohibited Combinations Other requirements For Informatics PG and final year MInf students only, or by special permission of the School. Students should be familiar with the mathematical concepts therein, particularly vectors and matrices, partial differentiation, and some probability.
Additional Costs None
Information for Visiting Students
Pre-requisitesNone
Displayed in Visiting Students Prospectus?Yes
Course Delivery Information
Delivery period: 2012/13 Semester 2, Available to all students (SV1) Learn enabled:  No Quota:  None
Location Activity Description Weeks Monday Tuesday Wednesday Thursday Friday
CentralLecture1-11 12:10 - 13:00
CentralLecture1-11 12:10 - 13:00
First Class Week 1, Tuesday, 12:10 - 13:00, Zone: Central. Lecture Theatre 4, 7 Bristo Square
No Exam Information
Summary of Intended Learning Outcomes
1 - Knowledge of basic and advanced reinforcement learning techniques.
2 - Insight into the problems involved in applying these techniques to deal with real world data, and how to overcome those problems.
3 - Appreciation and identification of suitable learning tasks to which these learning techniques can be applied
4 - Ability to evaluate how effective a particular learning procedure has been -- internal indicators of learning success vs. actual behaviour of the learner.
5 - Use and writing of Matlab programs, ability to set up and run computational experiments to produce statistically sound results
6 - Formulation of problems, evaluation of results from the student's own experiments and those presented in some cases in the research literature.
Assessment Information
Written Examination 80
Assessed Assignments 20
Oral Presentations 0

Assessment
Two assignments are set, each accounting for 10% of the overall mark. Typically they require programming a learning system or experimenting with an existing system, using MATLAB.

If delivered in semester 1, this course will have an option for semester 1 only visiting undergraduate students, providing assessment prior to the end of the calendar year.
Special Arrangements
None
Additional Information
Academic description Not entered
Syllabus The main topics to be covered are some or all of the following (there are some changes from year to year)
* Reinforcement learning framework
* Bandit problems and action selection
* Dynamic programming methods
* Monte-Carlo methods
* Temporal difference methods
* Q-learning and eligibility traces
* Environment modelling
* Function approximation for generalisation
* Actor-critic, applications
* Planning in the RL context
* Unsupervised, self-organising networks and RL
* Constructive methods - nets that grow
* Evaluating performance

Relevant QAA Computing Curriculum Sections: Artificial Intelligence, Data Structures and Algorithms, Intelligent Information Systems Technologies, Simulation and Modelling
Transferable skills Not entered
Reading list # Reinforcement Learning. An Introduction. Richard S. Sutton and Andrew G. Barto. MIT Press, Cambridge MA, 1998.
# Other material as handouts or on web page
Study Abroad Not entered
Study Pattern Lectures 20
Tutorials 0
Timetabled Laboratories 0
Non-timetabled assessed assignments 25
Private Study/Other 55
Total 100
KeywordsNot entered
Contacts
Course organiserDr Iain Murray
Tel: (0131 6)51 9078
Email: I.Murray@ed.ac.uk
Course secretaryMiss Kate Weston
Tel: (0131 6)50 2701
Email: Kate.Weston@ed.ac.uk
Navigation
Help & Information
Home
Introduction
Glossary
Search DPTs and Courses
Regulations
Regulations
Degree Programmes
Introduction
Browse DPTs
Courses
Introduction
Humanities and Social Science
Science and Engineering
Medicine and Veterinary Medicine
Other Information
Timetab
Prospectuses
Important Information
 
© Copyright 2012 The University of Edinburgh - 14 January 2013 4:09 am