Undergraduate Course: Stochastic Modelling (MATH10007)
Course Outline
School  School of Mathematics 
College  College of Science and Engineering 
Credit level (Normal year taken)  SCQF Level 10 (Year 3 Undergraduate) 
Availability  Available to all students 
SCQF Credits  10 
ECTS Credits  5 
Summary  This is an advanced probability course dealing with discrete and continuous time Markov chains. The course covers the fundamental theory, and provides many examples. Markov chains has countless applications in many fields raging from finance, operation research and optimization to biology, chemistry and physics. 
Course description 
Markov Chains in discrete time: classification of states, first passage and recurrence times, absorption problems, stationary and limiting distributions.
Markov Processes in continuous time: Poisson processes, birthdeath processes.
The Q matrix, forward and backward differential equations, imbedded Markov Chain, stationary distribution.
Syllabus summary: Probability review: Conditional probability, basic definition of stochastic processes. Discretetime Markov chains: Modelling of real life systems as Markov chains, transient behaviour, limiting behaviour and classification of states, first passage and recurrence times, absorption problems, ergodic theorems, Markov chains with costs and rewards, reversibility. Poisson processes: Exponential distribution, counting processes, alternative definitions of Poisson processes, splitting, superposition and uniform order statistics properties, nonhomogeneous Poisson processes. Continuoustime Markov chains: transient behaviour, limiting behaviour and classification of states in continuous time, ergodicity, basic queueing models.

Information for Visiting Students
Prerequisites  Visiting students are advised to check that they have studied the material covered in the syllabus of each prerequisite course before enrolling. 
High Demand Course? 
Yes 
Course Delivery Information

Academic year 2022/23, Available to all students (SV1)

Quota: None 
Course Start 
Semester 1 
Timetable 
Timetable 
Learning and Teaching activities (Further Info) 
Total Hours:
100
(
Lecture Hours 22,
Seminar/Tutorial Hours 5,
Summative Assessment Hours 2,
Programme Level Learning and Teaching Hours 2,
Directed Learning and Independent Learning Hours
69 )

Assessment (Further Info) 
Written Exam
95 %,
Coursework
5 %,
Practical Exam
0 %

Additional Information (Assessment) 
Coursework 5%, Examination 95% 
Feedback 
Not entered 
Exam Information 
Exam Diet 
Paper Name 
Hours & Minutes 

Main Exam Diet S1 (December)   2:00  
Learning Outcomes
On completion of this course, the student will be able to:
 Formulate mathematically a range of reallife scenario of a stochastic process described in words
 Demonstrate an understanding of discrete and continuous time stochastic processes by being able to calculate finite dimensional distributions.
 Analyse the transient behaviour of Markov chains, and classify their states.
 Demonstrate an understanding of stationary and limiting behaviour by deriving corresponding probability distributions, and first passage properties.
 Calculate the finite dimensional distributions of Poisson processes.

Reading List
1. R. Durrett. Essentials of Stochastic Processes, Springer, 2012.
2. V. Kulkarni. Modeling and Analysis of Stochastic Systems, CRC Press, 2010. 
Contacts
Course organiser  Dr Theo Assiotis
Tel:
Email: theo.assiotis@ed.ac.uk 
Course secretary  Miss Greta Mazelyte
Tel:
Email: greta.mazelyte@ed.ac.uk 

