Postgraduate Course: Stochastic Optimization (MATH11010)
Course Outline
School  School of Mathematics 
College  College of Science and Engineering 
Credit level (Normal year taken)  SCQF Level 11 (Postgraduate) 
Availability  Not available to visiting students 
SCQF Credits  5 
ECTS Credits  2.5 
Summary  Stochastic Optimization provides an introduction to stateoftheart quantitative modelling and solution methods for problems of decisionmaking under uncertainty. 
Course description 
Stochastic Optimization is structured into four twohour lectures and a twohour presentation session during which students will present to the class the outcome of a group assignment.
Lecture 1: this lecture covers Decision Analysis and Decision Trees; these are simple and yet effective tools for analysing problems of decision making under uncertainty.
Reference reading: Ch 15, Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001
Learning outcomes: A1, A2, B1, B2, C1, D3
Lecture 2: this lecture provides an introduction to Stochastic Dynamic Programming, a modelling and solution framework originally introduced in Bellman¿s seminal work
Bellman, Richard (1957), Dynamic Programming, Princeton University Press. Dover paperback edition (2003), ISBN 0486428095.
It also introduces specific classes of Stochastic Dynamic Programs, such as sequential sampling problems and bandit problems, the aim of which is to provide an effective framework for assessing expected value of information.
Reference reading: Ch 18 & 19, W. L. Winston, Operations Research: Applications and Algorithms (7th Edition), Duxbury Press, 2003
Learning outcomes: A3, B2, C2, D3
Lecture 3: this lecture discusses foundations and properties of Markov Chains, a modelling tools for modelling stochastic systems featuring the socalled Markov property, i.e. the property that event probabilities at a given time only depend on the state of the system under scrutiny at that point in time.
Reference readings: Ch 16, Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001; Ch 4, Gallager, Stochastic processes: theory for applications, book working draft, http://www.rle.mit.edu/rgallager/documents/6.262vbo4.pdf
Learning outcomes: A4, B2
Lecture 4: this lecture discusses foundations and applications of Markov Decision Problems, a modelling and solution framework for problems of decision making under uncertainty featuring the Markov property.
Reference readings: Ch 21, Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001; Ch 4, Gallager, Stochastic processes: theory for applications, book working draft,
http://www.rle.mit.edu/rgallager/documents/6.262vbo4.pdf
Learning outcomes: A4, B2, C3, D3
Group presentations: this twohour session is led by students. Students will engage into a group activity during the first four weeks of the course. The activity consists in reading a research article from the academic literature on decision making under uncertainty and preparing, (i) a set of PowerPoint slides summarizing the message and the content of the paper in a form that is accessible to persons unfamiliar with the topic; (ii) a group presentation (10 to 15 minutes) that will be delivered to the class.
Learning outcomes: B3, D1, D2

Entry Requirements (not applicable to Visiting Students)
Prerequisites 

Corequisites  
Prohibited Combinations  
Other requirements  None 
Course Delivery Information

Academic year 2015/16, Not available to visiting students (SS1)

Quota: None 
Course Start 
Block 4 (Sem 2) 
Timetable 
Timetable 
Learning and Teaching activities (Further Info) 
Total Hours:
50
(
Lecture Hours 10,
Summative Assessment Hours 1.5,
Programme Level Learning and Teaching Hours 1,
Directed Learning and Independent Learning Hours
38 )

Assessment (Further Info) 
Written Exam
80 %,
Coursework
20 %,
Practical Exam
0 %

Additional Information (Assessment) 
The summative assessment includes:
A final written exam (1.5 hours) in which the student must answer TWO questions out of THREE. Each question covers one the topics discussed in the course: Decision Analysis, Stochastic Dynamic Programming and Markov Decision Problems. (80% of the final mark)
Learning outcomes: A14, B12, C13, D3
A group assignment consists in reading a research article from the academic literature on decision making under uncertainty and preparing, (i) a set of PowerPoint slides summarizing the message and the content of the paper in a form that is accessible to persons unfamiliar with the topic; (ii) a group presentation (10 to 15 minutes) that will be delivered to the class. (20% of the final mark)
Learning outcomes: B3, D13,

Feedback 
Not entered 
Exam Information 
Exam Diet 
Paper Name 
Hours & Minutes 

Main Exam Diet S2 (April/May)  Stochastic Optimization (MATH11010  1:30  
Learning Outcomes
On completion of this course, the student will be able to:
 *A. Academic knowledge* 1. Describe the structure of a Decision Table and of a Decision Tree. 2. Define a number of probabilityindependent and probabilitydependent decision criteria. 3. Describe the constituent elements of a Stochastic Dynamic Program. 4. Describe the structure of a Markov Chain and of a Markov Decision Problem and the underpinning assumptions of these modelling frameworks
 *B. Intellectual skills* 1. Discuss advantages and limitations of Decision Tables/Trees, Stochastic Dynamic Programs and Markov Decision Problems. 2. Identify the most appropriate tool for modelling a specific problem of decision making under uncertainty. 3. Demonstrate the ability to read, understand and summarise the content of an article in the academic literature on decision making under uncertainty
 *C. Professional/subject specific/practical skills* 1. Demonstrate the ability to apply Decision Analysis tools (i.e. decision tables and decision trees) to problems of decision making under uncertainty. 2. Demonstrate the ability to model and solve problems of decision making under uncertainty using Stochastic Dynamic Programming. 3. Demonstrate the ability to model and solve Markov Decision Problems
 *D. Transferable skills* 1. Demonstrate the ability to summarise the content of a document in presentation format, e.g. PowerPoint slides. 2. Demonstrate presentation skills. 3. Demonstrate problem analysis and problem solving skills

Reading List
Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001
W. L. Winston, Operations Research: Applications and Algorithms (7th Edition), Duxbury Press, 2003
Gallager, Stochastic processes: theory for applications, book working draft, Ch 4
http://www.rle.mit.edu/rgallager/documents/6.262vbo4.pdf

Contacts
Course organiser  Dr Julian Hall
Tel: (0131 6)50 5075
Email: J.A.J.Hall@ed.ac.uk 
Course secretary  Mrs Frances Reid
Tel: (0131 6)50 4883
Email: f.c.reid@ed.ac.uk 

