Bigmath Advanced Course 4: Large scale and distributed optimization

January 16, 2020 Natasa Krejic

The goal of the training course, realized within the H2020 Marie Skłodowska-Curie project Big Data Challenges for Mathematics, Grant Agreement N bigmath_logo2 o 812912, is to provide an overview of tools and algorithms in the area of large scale and distributed optimization. An illustrative examples which help in understanding how optimization-based modelling can be applied on machine learning problems will be part of the course. The topics covered by the course are the following: Machine learning and optimization; Optimality conditions for unconstrained problems; Convexity; Line search methods; Gradient methods; Second order methods; Optimality conditions for constrained problems; Augmented Lagrangian methods; Parallel methods: duality theory and dual subgradient method; primal decomposition; dual decomposition; augmented Lagrangian; alternating direction method of multipliers. Distributed methods: distributed gradient descent; stochastic distributed methods. Part of the course will be devoted to computer labs for software/implementation tutorial. The course is carried out in 4 days, with 6 hours of lectures each day.