A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems

From MaRDI portal
Publication:6141533



Abstract: In this paper, we consider a class of structured nonconvex nonsmooth optimization problems, in which the objective function is formed by the sum of a possibly nonsmooth nonconvex function and a differentiable function whose gradient is Lipschitz continuous, subtracted by a weakly convex function. This type of structured problems has many practical applications in machine learning and statistics such as compressed sensing, signal recovery, sparse dictionary learning, clustering, matrix factorization, and others. We develop a flexible extrapolated proximal subgradient algorithm for solving these problems with guaranteed subsequential convergence to a stationary point. The global convergence of the whole sequence generated by our algorithm is also established under the Kurdyka-Lojasiewicz property. To illustrate the promising numerical performance of the proposed algorithm, we conduct numerical experiments on two important nonconvex models. This includes a least squares problem with a nonconvex regularization and an optimal power flow problem with distributed energy resources.


In the paper under review, the authors study a following broad optimization problem which has many important applications in diverse areas, including power control problems, compressed sensing, portfolio optimization, supply chain problems, image segmentation and many others. The objective function is formed by the sum of a possibly nonsmooth nonconvex function and a differentiable function with Lipschitz continuous gradient, subtracted by a weakly convex function. This general framework allows for problems involving nonconvex loss functions and problems with specific nonconvex constraints. The problem is: \(\min_{x\in C} F(x)\) where \(F(x) := f (x) + h(Ax)-g(x)\) and where \(C\) is a nonempty closed subset of a finite-dimensional real Hilbert space \(H\), \(A\) is a linear mapping from \(H\) to another finite-dimensional real Hilbert space, \(f : H\to (-\infty,\infty]\) is a proper lower semicontinuous (possibly nonsmooth and nonconvex) function, \(h\) is a real valued differentiable (possibly nonconvex) function whose gradient is Lipschitz continuous and \(g:H\to (-\infty,\infty]\) is a continuous weakly convex function with modulus on an open convex set containing \(C\). Two examples are: (1) From statistical learning, \(\min_{x\in \mathbb R^d}(\phi(x) + \gamma r(x))\) where \(\phi\) is called a loss function which measures the data fitting, \(r\) is a regularization which promotes specific structure in the solution such as sparsity, and \(\gamma>0\) is a weighting parameter. (2) \(\min_{x\in \mathbb R^d} (f (x) + h(x)- g(x))\). The paper is well written with a good set of references.



Cites work









This page was built for publication: A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6141533)