Postgraduate Course: Stochastic Optimization (MATH11010)
Course Outline
School  School of Mathematics 
College  College of Science and Engineering 
Credit level (Normal year taken)  SCQF Level 11 (Postgraduate) 
Availability  Not available to visiting students 
SCQF Credits  5 
ECTS Credits  2.5 
Summary  Stochastic Optimization provides an introduction to stateoftheart quantitative modelling and solution methods for problems of decisionmaking under uncertainty. 
Course description 
Stochastic Optimization is structured into four twohour lectures and a twohour presentation session during which students will present to the class the outcome of a group assignment.
Lecture 1: this lecture covers Decision Analysis and Decision Trees; these are simple and yet effective tools for analysing problems of decision making under uncertainty.
Reference reading: Ch 15, Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001
Learning outcomes: A1, A2, B1, B2, C1, D3
Lecture 2: this lecture provides an introduction to Stochastic Dynamic Programming, a modelling and solution framework originally introduced in Bellman¿s seminal work
Bellman, Richard (1957), Dynamic Programming, Princeton University Press. Dover paperback edition (2003), ISBN 0486428095.
It also introduces specific classes of Stochastic Dynamic Programs, such as sequential sampling problems and bandit problems, the aim of which is to provide an effective framework for assessing expected value of information.
Reference reading: Ch 18 & 19, W. L. Winston, Operations Research: Applications and Algorithms (7th Edition), Duxbury Press, 2003
Learning outcomes: A3, B2, C2, D3
Lecture 3: this lecture discusses foundations and properties of Markov Chains, a modelling tools for modelling stochastic systems featuring the socalled Markov property, i.e. the property that event probabilities at a given time only depend on the state of the system under scrutiny at that point in time.
Reference readings: Ch 16, Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001; Ch 4, Gallager, Stochastic processes: theory for applications, book working draft, http://www.rle.mit.edu/rgallager/documents/6.262vbo4.pdf
Learning outcomes: A4, B2
Lecture 4: this lecture discusses foundations and applications of Markov Decision Problems, a modelling and solution framework for problems of decision making under uncertainty featuring the Markov property.
Reference readings: Ch 21, Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001; Ch 4, Gallager, Stochastic processes: theory for applications, book working draft,
http://www.rle.mit.edu/rgallager/documents/6.262vbo4.pdf
Learning outcomes: A4, B2, C3, D3
Group presentations: this twohour session is led by students. Students will engage into a group activity during the first four weeks of the course. The activity consists in reading a research article from the academic literature on decision making under uncertainty and preparing, (i) a set of PowerPoint slides summarizing the message and the content of the paper in a form that is accessible to persons unfamiliar with the topic; (ii) a group presentation (10 to 15 minutes) that will be delivered to the class.
Learning outcomes: B3, D1, D2

Entry Requirements (not applicable to Visiting Students)
Prerequisites 

Corequisites  
Prohibited Combinations  
Other requirements  None 
Course Delivery Information
Not being delivered 
Learning Outcomes
On completion of this course, the student will be able to:
 *A. Academic knowledge* 1. Describe the structure of a Decision Table and of a Decision Tree. 2. Define a number of probabilityindependent and probabilitydependent decision criteria. 3. Describe the constituent elements of a Stochastic Dynamic Program. 4. Describe the structure of a Markov Chain and of a Markov Decision Problem and the underpinning assumptions of these modelling frameworks
 *B. Intellectual skills* 1. Discuss advantages and limitations of Decision Tables/Trees, Stochastic Dynamic Programs and Markov Decision Problems. 2. Identify the most appropriate tool for modelling a specific problem of decision making under uncertainty. 3. Demonstrate the ability to read, understand and summarise the content of an article in the academic literature on decision making under uncertainty
 *C. Professional/subject specific/practical skills* 1. Demonstrate the ability to apply Decision Analysis tools (i.e. decision tables and decision trees) to problems of decision making under uncertainty. 2. Demonstrate the ability to model and solve problems of decision making under uncertainty using Stochastic Dynamic Programming. 3. Demonstrate the ability to model and solve Markov Decision Problems
 *D. Transferable skills* 1. Demonstrate the ability to summarise the content of a document in presentation format, e.g. PowerPoint slides. 2. Demonstrate presentation skills. 3. Demonstrate problem analysis and problem solving skills

Reading List
Hillier & Lieberman, Introduction to Operations Research (7th Edition), McGrawHill, 2001
W. L. Winston, Operations Research: Applications and Algorithms (7th Edition), Duxbury Press, 2003
Gallager, Stochastic processes: theory for applications, book working draft, Ch 4
http://www.rle.mit.edu/rgallager/documents/6.262vbo4.pdf

Contacts
Course organiser  Dr Roberto Rossi
Tel: (0131 6)51 5239
Email: Roberto.Rossi@ed.ac.uk 
Course secretary  Mrs Frances Reid
Tel: (0131 6)50 4883
Email: f.c.reid@ed.ac.uk 

