Accelerated Forward-Backward Optimization using Deep Learning

DOI10.1137/22M1532548MaRDI QIDQ6367408zbMATH OpenOpenAlexFDO

Authors Sebastian Banert, Jevgenija Rudzusika, Ozan Öktem, Jonas Adler

Publication date 11 May 2021

Full work available at URL https://doi.org/10.1137/22m1532548

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Convex programming (90C25) Artificial neural networks and deep learning (68T07) Large-scale problems in mathematical programming (90C06) Numerical methods based on nonlinear programming (49M37)

Abstract: We propose several deep-learning accelerated optimization solvers with convergence guarantees. We use ideas from the analysis of accelerated forward-backward schemes like FISTA, but instead of the classical approach of proving convergence for a choice of parameters, such as a step-size, we show convergence whenever the update is chosen in a specific set. Rather than picking a point in this set using some predefined method, we train a deep neural network to pick the best update. Finally, we show that the method is applicable to several cases of smooth and non-smooth optimization and show superior results to established accelerated solvers.

Recommendations

• Data-driven nonsmooth optimization
• Regularized nonlinear acceleration
• Deep inverse optimization
• Objective acceleration for unconstrained optimization
• A variational perspective on accelerated methods in optimization

Cited in

(7)

• Tutorial on Amortized Optimization
• New Douglas-Rashford splitting algorithms for generalized DC programming with applications in machine learning
• A step function based recursion method for 0/1 deep neural networks
• Data-Driven Mirror Descent with Input-Convex Neural Networks
• A zeroing neural dynamics based acceleration optimization approach for optimizers in deep neural networks
• Accelerated Douglas-Rachford splitting algorithm using neural network
• Direct nonlinear acceleration

This page was built for publication: Accelerated Forward-Backward Optimization using Deep Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6367408)