The Proximal Augmented Lagrangian Method for Nonsmooth Composite Optimization

DOI10.1109/TAC.2018.2867589zbMATH Open1482.90168arXiv1610.04514OpenAlexW3104429825WikidataQ129318494 ScholiaQ129318494MaRDI QIDQ5223796FDOQ5223796

Authors: Neil K. Dhingra, Sei Zhen Khong, Mihailo R. Jovanović 

Publication date: 18 July 2019

Published in: IEEE Transactions on Automatic Control (Search for Journal in Brave)

Abstract: We study a class of optimization problems in which the objective function is given by the sum of a differentiable but possibly nonconvex component and a nondifferentiable convex regularization term. We introduce an auxiliary variable to separate the objective function components and utilize the Moreau envelope of the regularization term to derive the proximal augmented Lagrangian $-$ a continuously differentiable function obtained by constraining the augmented Lagrangian to the manifold that corresponds to the explicit minimization over the variable in the nonsmooth term. The continuous differentiability of this function with respect to both primal and dual variables allows us to leverage the method of multipliers (MM) to compute optimal primal-dual pairs by solving a sequence of differentiable problems. The MM algorithm is applicable to a broader class of problems than proximal gradient methods and it has stronger convergence guarantees and a more refined step-size update rules than the alternating direction method of multipliers. These features make it an attractive option for solving structured optimal control problems. We also develop an algorithm based on the primal-descent dual-ascent gradient method and prove global (exponential) asymptotic stability when the differentiable component of the objective function is (strongly) convex and the regularization term is convex. Finally, we identify classes of problems for which the primal-dual gradient flow dynamics are convenient for distributed implementation and compare/contrast our framework to the existing approaches.

Full work available at URL: https://arxiv.org/abs/1610.04514

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Nonsmooth analysis (49J52) Derivative-free methods and methods using generalized derivatives (90C56)

Cited In (21)

• Local R-linear convergence of ADMM-based algorithm for \(\ell_1\)-norm minimization with linear and box constraints
• Semi-global exponential stability of augmented primal-dual gradient dynamics for constrained convex optimization
• Fast and stable nonconvex constrained distributed optimization: the ELLADA algorithm
• Local properties and augmented Lagrangians in fully nonconvex composite optimization
• Distributed optimization of high-order nonlinear multi-agent systems with disturbance under switching topologies
• Exponential stability of partial primal-dual gradient dynamics with nonsmooth objective functions
• Proximal gradient flow and Douglas-Rachford splitting dynamics: global exponential stability via integral quadratic constraints
• Linear convergence of primal-dual gradient methods and their performance in distributed optimization
• Distributed coordination for nonsmooth convex optimization via saddle-point dynamics
• An accelerated proximal augmented Lagrangian method and its application in compressive sensing
• Analytical convergence regions of accelerated gradient descent in nonconvex optimization under regularity condition
• Dynamical systems coupled with monotone set-valued operators: formalisms, applications, well-posedness, and stability
• A proximal augmented method for semidefinite programming problems
• An inexact proximal augmented Lagrangian framework with arbitrary linearly convergent inner solver for composite convex optimization
• Augmented Lagrangian duality for composite optimization problems
• Solving a class of nonsmooth resource allocation problems with directed graphs through distributed Lipschitz continuous multi-proximal algorithms
• Image multiplicative denoising using adaptive Euler's elastica as the regularization
• Convergence rate bounds for the mirror descent method: IQCs, Popov criterion and Bregman divergence
• Tracking control by the Newton–Raphson method with output prediction and controller speedup
• On a primal-dual Newton proximal method for convex quadratic programs
• Constrained composite optimization and augmented Lagrangian methods

This page was built for publication: The Proximal Augmented Lagrangian Method for Nonsmooth Composite Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5223796)