1. Convexity
2. Gradient and subgradient methods
3. Duality and conjugate functions
4. Proximal algorithms
5. Limits of performance
6. Accelerated methods
7. Coordinate descent
8. Conditional gradient
9. Monotone operators
10. Operator splitting methods
11. Stochastic gradient descent
12. Variance reduction techniques and limits of performance
13. Newton and quasi-Newton methods
14. Nonsmooth and stochastic second-order methods
15. Conjugate gradients
16. Sequential convex programming
17. Architectures and algorithms for parallel optimisation
18. Decomposition and parallelization
19. Asynchrony I – time-varying update rates and information delays
20. Asynchronous computations II – random effects and communication efficiency
After the course the student will be able to:
- know basic terminology and concepts in convex optimization.
- design and analyze optimization algorithms for convex optimization problems.
- characterize the limits of performance for first-order methods
- analyze and use modern methods for scalable convex optimization, including: gradient and subgradient methods, proximal algorithms, coordinate descent, conditional gradient, conjugate gradient, operator splitting and quasi-Newton methods.
- handle stochastic effects in optimization problems using stochastic gradient descent and variance reduction techniques.
- describe modern architectures for parallel numerical computating
- use duality and decomposition can be for parallelization of optimization algorithms
- assess the impact of asynchrony and information delay on iterative algorithms
- apply techniques for reducing information exchange between computing nodes.
