Postgraduate Course: Statistical Signal Processing (SSP) (MSc) (PGEE11122)
Course Outline
School  School of Engineering 
College  College of Science and Engineering 
Credit level (Normal year taken)  SCQF Level 11 (Postgraduate) 
Availability  Not available to visiting students 
SCQF Credits  10 
ECTS Credits  5 
Summary  The Statistical Signal Processing course considers representing realworld signals by stochastic or random processes. The tools for analysing these random signals are developed in the Probability, Random Variables, and Estimation Theory course, and this course extends them to deal with time series. The notion of statistical quantities such as autocorrelation and autocovariance are extended from random vectors to random processes, and a frequencydomain analysis framework is developed. This course also investigates the affect of systems and transformations on timeseries, and how they can be used to help design powerful signal processing algorithms to achieve a particular task.
The course introduces the notion of representing signals using parametric models; it extends the broad topic of statistical estimation theory covered in the Probability, Random Variables, and Estimation Theory course for determining optimal model parameters. In particular, the Bayesian paradigm for statistical parameter estimation is introduced. Emphasis is placed on relating these concepts to stateoftheart applications and signals.
This course, in combination with the Probability, Random Variables, and Estimation Theory course, provides the fundamental knowledge required for the advanced signal, image, and communication courses in the MSc course. 
Course description 
Any minor modifications to the latest syllabus and lectures are always contained in the lecture handout.
Introduction and Review of Discretetime Systems (3 lectures).
1. Introduction and course overview. Role of deterministic and random signals, and the various interpretations of random processes in the different physical sciences.
2. Brief review of Fourier transform theorem:
a. Transforms for continuoustime, discretetime, periodic or aperiodic, signals.
b. Parseval's Theorem.
c. Properties of the discrete Fourier transform (DFT).
d. The DFT as a linear transformation.
e. Summary of frequently used transform pairs.
3. Review of discretetime systems.
a. Basic discretetime signals.
b. The ztransform and basic properties.
c. Summary of frequently used transform pairs.
d. Definitions of linear timeinvariant (LTI) and linear timevarying (LTV) systems.
e. Rational transfer functions; polezero models.
f. Frequency response of LTI systems.
g. Example of inverse bilateral ztransforms, and different approaches to get the same answer; partial fraction expansions using the coverup rule.
Stochastic Processes (6 lectures).
1. Introduction to stochastic processes, and their definition as an ensemble of deterministic realisations resulting from the outcome of a sample space; also covers the various interpretations of the samples of a random process.
2. Covers predictable processes with an example of harmonic processes; description of stochastic processes using probability density functions (pdfs).
3. Notion of stationary and nonstationary processes.
4. Statistical description of random processes; examples of some predictable processes through a MATLAB demonstration; secondorder statistics including mean and autocorrelation sequences, with an example of calculating autocorrelation for a harmonic process.
5. Types of random processes, including independent, independent and identically distributed (i. i. d.) random processes, and uncorrelated and orthogonal processes.
6. Introduction to stationary processes, both orderN stationary, strictsense stationary, and widesense stationary; example of testing whether a Wiener process is stationary or not; also covers wide sense periodic and widesense cyclostationary processes.
7. Notion of ergodicity, and the notion of timeaverages being equal to ensemble averages in the meansquare sense.
8. Secondorder statistical descriptions, including autocorrelation and covariances; jointsignal statistics; types of joint stochastic processes; correlation matrices.
9. Basic introduction to Markov processes.
FrequencyDomain Description of Stationary Processes (3 lectures).
1. Introduction to random processes in the frequency domain, including the stochastic decomposition interpretation, the transform of averages interpretation, and the connections between these interpretations.
2. Formal definition of the power spectral density (PSD) and its properties; general form of the PSD including autocorrelation sequences (ACSs) with periodic components; the PSD of a harmonic signal (as a linear summation of sinusoids).
3. The PSD of common stationary processes: introducing white noise, harmonic processes, complexexponentials.
4. Definition of the crosspower spectral density (CPSD), a physical overview, and the properties of the CPSD; an overview of complex spectral density functions, their relationships with PSDs, and how to find their inverses; properties of complex spectral densities.
Linear systems with stationary random inputs (3 lectures).
1. Considers the effect of linear systems on random processes, and the resulting output processes; discusses the linearity of the expectation operator.
2. Develops the basic relationships between the input and output for stationary random processes, including inputoutput crosscorrelation, output autocorrelation, and output power. Discusses the case of LTI systems, and the fact that most real world applications will be a LTV system.
3. System identification using crosscorrelation.
4. Frequencydomain analysis of LTI systems, including inputoutput CPSD and output PSD.
5. Equivalence of timedomain and frequencydomain methods.
6. LTV systems with nonstationary inputs.
Linear signal models (2 lectures).
1. Introduction to the notion of parametric modelling.
2. Nonparametric vs parametric signal models.
3. Types of polezero models.
4. Allpole models: impulse response, autocorrelation functions, poles, minimumphase conditions.
5. Linear prediction, autoregressive (AR) processes, YuleWalker equations.
6. Allzero models: impulse response, autocorrelation functions, zeros, and moving average (MA) processes.
7. PoleZero models: autocorrelation functions, autoregressive moving average (ARMA) processes.
8. Overview of extension to timevarying processes.
9. Applications and examples.
Estimation Theory for Random Processes (3 lectures).
1. Sample autocorrelation and autocovariance functions.
2. Leastsquares for AR modelling.
3. Estimating signals in noise, using parametric signal models.
4. Bayesian estimation of sinusoids in noise, and other applications of Bayesian estimation methods to timeseries analysis.
2 x 2 hour lectures, and 1 x 2 hour tutorial, per week from Week 6 to Week 11 (one lecture in Week 6 only).

Course Delivery Information

Academic year 2020/21, Not available to visiting students (SS1)

Quota: 0 
Course Start 
Block 2 (Sem 1) 
Timetable 
Timetable 
Learning and Teaching activities (Further Info) 
Total Hours:
100
(
Lecture Hours 44,
Seminar/Tutorial Hours 22,
Feedback/Feedforward Hours 11,
Programme Level Learning and Teaching Hours 2,
Directed Learning and Independent Learning Hours
21 )

Assessment (Further Info) 
Written Exam
100 %,
Coursework
0 %,
Practical Exam
0 %

Additional Information (Assessment) 
100% Examination 
Feedback 
Not entered 
No Exam Information 
Learning Outcomes
On completion of this course, the student will be able to:
 explain, describe, and understand the notion of a random process and statistical time series. Characterise random processes in terms of its statistical properties, including the notion of stationarity and ergodicity;
 define, describe, and understand the notion of the power spectral density of stationary random processes; analyse and manipulate power spectral densities;
 analyse in both time and frequency the affect of transformations and linear systems on random processes, both in terms of the density functions, and statistical moments;
 explain the notion of parametric signal models, and describe common regressionbased signal models in terms of its statistical characteristics, and in terms of its affect on random signals;
 apply least squares, maximumlikelihood, and Bayesian estimators to model based signal processing problems.

Reading List
1. Recommended course text book: Therrien C. W. and M. Tummala, Probability and Random Processes for Electrical and Computer Engineers, Second edition, CRC Press, 2011. IDENTIFIERS  Hardback, ISBN10: 1439826986, ISBN13: 9781439826980
2. Manolakis D. G., V. K. Ingle, and S. M. Kogon, Statistical and Adaptive Signal Processing: Spectral Estimation, Signal Modeling, Adaptive Filtering and Array Processing, McGraw Hill, Inc., 2000.
3. Kay S. M., Fundamentals of Statistical Signal Processing: Estimation Theory, PrenticeHall, Inc., 1993.
4. Papoulis A. and S. Pillai, Probability, Random Variables, and Stochastic Processes, Fourth edition, McGraw Hill, Inc., 2002. 
Additional Information
Graduate Attributes and Skills 
Not entered 
Keywords  Statistical Signal Processing,Discretetime Random Signals,Linear Systems 
Contacts
Course organiser  Dr James Hopgood
Tel: (0131 6)50 5571
Email: James.Hopgood@ed.ac.uk 
Course secretary  Mrs Megan InchKellingray
Tel: (0131 6)51 7079
Email: M.Inch@ed.ac.uk 

