Postgraduate Course: Modern Optimization Methods for Big Data Problems (MATH11146)
|School||School of Mathematics
||College||College of Science and Engineering
|Credit level (Normal year taken)||SCQF Level 11 (Postgraduate)
||Availability||Available to all students
|Summary||The course covers modern optimization algorithms and theory developed in recent years, suitable for big data applications; that is applications with millions or billions of design parameters and data points.
Problems of these sizes are ever more common as we live in a digital age in which it is increasingly easier to collect and store data in digital form (e.g. transaction records, YouTube clicks, internet activity, Wikipedia, Twitter, customer behaviour databases, government records, image collections).
New methods and tools are needed to analyze such vast datasets and optimization algorithms are at the heart of such efforts, underpinning much of data science, including machine learning, operations research and statistical analysis. Optimization is one of three pillars of big data analysis, with the other two being computer science and statistics.
The material is designed for students wishing to continue with PhD studies or those wishing to enter big data industry and is suitable for MSc students in quantitative disciplines (e.g. optimization, informatics, data science, mathematics, operations research, machine learning, engineering), PhD students and researchers interested in some recent developments in the area.
Applications of the methods covered in the course can be found virtually in all fields of data science including text analysis, page ranking, speech recognition, image classification, finance and decision sciences.
1) Optimization models and structure of big data, including:
1.1) Regularized, stochastic and linear conic optimization
1.2) Convexity and duality
1.3) The role of dimension, data quality, data size, solution accuracy, separability, sparsity and randomization in the design of algorithms
2) Algorithms for big data problems, including:
2.1) Stochastic coordinate descent (parallel, distributed, accelerated)
2.2) Semi-stochastic gradient descent
2.3) Nesterov's subgradient descent
3) Applications in data science, including:
3.1) Machine learning (e.g. support vector machine classification)
3.2) Internet (e.g. ranking)
3.3) Least squares and logistic regression (e.g. object recognition)
Entry Requirements (not applicable to Visiting Students)
||Other requirements|| It is recommended that students have familiarity with fundamental concepts in linear algebra, multivariate calculus, probability theory and the design of algorithms.
Also desirable: prior exposure to convex analysis, optimization theory, parallel computing, analysis of iterative algorithms
Information for Visiting Students
|High Demand Course?
Course Delivery Information
|Not being delivered|
| Ability to:
- Implement prototype optimization code applicable to big data problems
- Analyze the theoretical performance (complexity) of selected modern optimization algorithms suitable for big data applications
- Explain how different problem structures (e.g. separability, sparsity) and data size (e.g. fits into the memory of a single computer or not) calls for differences in algorithmic design
- Apply the algorithms to a range of selected applications
|Graduate Attributes and Skills
|Course organiser||Dr Peter Richtarik
Tel: (0131 6)50 5049
|Course secretary||Mrs Frances Reid
Tel: (0131 6)50 4883