THE UNIVERSITY of EDINBURGH

DEGREE REGULATIONS & PROGRAMMES OF STUDY 2012/2013
- ARCHIVE for reference only
THIS PAGE IS OUT OF DATE

University Homepage
DRPS Homepage
DRPS Search
DRPS Contact
DRPS : Course Catalogue : School of Informatics : Informatics

Postgraduate Course: Information Theory (INFR11087)

Course Outline
SchoolSchool of Informatics CollegeCollege of Science and Engineering
Course typeStandard AvailabilityAvailable to all students
Credit level (Normal year taken)SCQF Level 11 (Postgraduate) Credits10
Home subject areaInformatics Other subject areaNone
Course website http://www.inf.ed.ac.uk/teaching/courses/it/ Taught in Gaelic?No
Course descriptionInformation theory describes the fundamental limits on our ability to store, process and communicate data, whether in natural or artificial systems. Understanding and approaching these limits is important in a wide variety of topics in informatics.

This course covers the theory introduced by Shannon in 1948, which revolutionized how we think about information and communication, and some of the practical techniques for compression and reliable communication that have been developed since.
Entry Requirements (not applicable to Visiting Students)
Pre-requisites Co-requisites
Prohibited Combinations Other requirements - A solid mathematical background is required.
- Some general mathematical ability (at MfI 1 level) is essential: Special functions log, exp are fundamental; mathematical notation (such as sums) used throughout; some calculus.
- Probability theory is used extensively: Random variables, expectation, Bernoulli trials, Binomial distribution, joint and conditional probabilities (MfI 1&4).
Additional Costs None
Information for Visiting Students
Pre-requisitesNone
Displayed in Visiting Students Prospectus?No
Course Delivery Information
Delivery period: 2012/13 Semester 1, Available to all students (SV1) Learn enabled:  No Quota:  None
Location Activity Description Weeks Monday Tuesday Wednesday Thursday Friday
CentralLecture1-11 14:10 - 15:00
CentralLecture1-11 14:10 - 15:00
First Class Week 1, Tuesday, 14:00 - 14:50, Zone: Central. LT4 7 Bristo Square
Exam Information
Exam Diet Paper Name Hours:Minutes
Main Exam Diet S1 (December)2:00
Delivery period: 2012/13 Semester 1, Part-year visiting students only (VV1) Learn enabled:  No Quota:  None
Location Activity Description Weeks Monday Tuesday Wednesday Thursday Friday
CentralLecture1-11 14:00 - 14:50
CentralLecture1-11 14:00 - 14:50
First Class Week 1, Tuesday, 14:00 - 14:50, Zone: Central. LT4 7 Bristo Square
Exam Information
Exam Diet Paper Name Hours:Minutes
Main Exam Diet S1 (December)2:00
Summary of Intended Learning Outcomes
By taking this course, students should be able to:

- Explain the source coding and noisy channel theorems and describe their implications for applications covered in lectures.
- Compute information theoretic quantities, construct bounds and describe+implement algorithms involving high-dimensional probability distributions.
- Describe the techniques covered in the course: identify their limitations, discuss their practical merits and design and describe alternatives.
- For a novel data source, communication channel or application, identify relevant information theoretic aspects to provide insight or suggest useful methods.
Assessment Information
Written Examination: 80%
Assessed Assignments: 20%
Oral Presentations: 0%

Assessment:

The assessed assignments will test that students are keeping up with the material and test an ability to experiment and implement codes while solving problems.
Special Arrangements
None
Additional Information
Academic description Not entered
Syllabus -Differential Entropy and information content
-Source coding theorem
-Symbol codes, Kraft-McMillan inequality, Huffman codes
-Stream codes, adaptive models, arithmetic coding
-Compression in practice
-Relative Entropy, mutual information, related inequalities
-Noisy channel coding theorem, channel capacity
-Error correcting codes
-Codes robust to erasures
-Lossy compression
-Hash codes
Transferable skills Not entered
Reading list "Information Theory, Inference and Learning Algorithms", David MacKay, CUP, 2003. http://www.inference.phy.cam.ac.uk/mackay/itila/book.html
Elements of Information Theory, 2nd Edition, Cover and Thomas, Wiley 2006
Study Abroad Not entered
Study Pattern Lectures 20
Tutorials 6
Timetabled Laboratories 0
Non-timetabled Assessed Assignments 20
Private Study/Other 54
Total 100
KeywordsNot entered
Contacts
Course organiserDr Iain Murray
Tel: (0131 6)51 9078
Email: I.Murray@ed.ac.uk
Course secretaryMiss Kate Weston
Tel: (0131 6)50 2701
Email: Kate.Weston@ed.ac.uk
Navigation
Help & Information
Home
Introduction
Glossary
Search DPTs and Courses
Regulations
Regulations
Degree Programmes
Introduction
Browse DPTs
Courses
Introduction
Humanities and Social Science
Science and Engineering
Medicine and Veterinary Medicine
Other Information
Timetab
Prospectuses
Important Information
 
© Copyright 2012 The University of Edinburgh - 14 January 2013 4:10 am