Proximal gradient method with extrapolation and line search for a class of non-convex and non-smooth problems
From MaRDI portal
Publication:6182324
Abstract: In this paper, we consider a class of possibly nonconvex, nonsmooth and non-Lipschitz optimization problems arising in many contemporary applications such as machine learning, variable selection and image processing. To solve this class of problems, we propose a proximal gradient method with extrapolation and line search (PGels). This method is developed based on a special potential function and successfully incorporates both extrapolation and non-monotone line search, which are two simple and efficient accelerating techniques for the proximal gradient method. Thanks to the line search, this method allows more flexibilities in choosing the extrapolation parameters and updates them adaptively at each iteration if a certain line search criterion is not satisfied. Moreover, with proper choices of parameters, our PGels reduces to many existing algorithms. We also show that, under some mild conditions, our line search criterion is well defined and any cluster point of the sequence generated by PGels is a stationary point of our problem. In addition, by assuming the Kurdyka-ojasiewicz exponent of the objective in our problem, we further analyze the local convergence rate of two special cases of PGels, including the widely used non-monotone proximal gradient method as one case. Finally, we conduct some numerical experiments for solving the regularized logistic regression problem and the regularized least squares problem. Our numerical results illustrate the efficiency of PGels and show the potential advantage of combining two accelerating techniques.
Recommendations
- An inexact proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth optimization problems
- Linear convergence of proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimization problems
- A class of modified accelerated proximal gradient methods for nonsmooth and nonconvex minimization problems
- Some accelerated alternating proximal gradient algorithms for a class of nonconvex nonsmooth problems
- scientific article; zbMATH DE number 7404502
Cites work
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A Nonmonotone Line Search Technique and Its Application to Unconstrained Optimization
- A Nonmonotone Line Search Technique for Newton’s Method
- A block coordinate descent method for regularized multiconvex optimization with applications to nonnegative tensor factorization and completion
- A generalized proximal point algorithm for certain non-convex minimization problems
- A globally convergent algorithm for nonconvex optimization based on block coordinate update
- A nonmonotone alternating updating method for a class of matrix factorization problems
- A proximal difference-of-convex algorithm with extrapolation
- A proximal method for composite minimization
- A successive difference-of-convex approximation method for a class of nonconvex nonsmooth optimization problems
- Accelerated gradient methods for nonconvex nonlinear and stochastic programming
- Adaptive restart for accelerated gradient schemes
- An introduction to continuous optimization for imaging
- Approximation accuracy, gradient methods, and error bound for structured convex optimization
- Asymptotic properties of bridge estimators in sparse high-dimensional regression models
- Asymptotics for Lasso-type estimators.
- Block stochastic gradient iteration for convex and nonconvex optimization
- Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
- Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods
- Difference-of-convex learning: directional stationarity, optimality, and sparsity
- Efficient Reconstruction of Piecewise Constant Images Using Nonsmooth Nonconvex Minimization
- Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
- Further properties of the forward-backward envelope with applications to difference-of-convex programming
- Gradient methods for minimizing composite functions
- Group sparse optimization via \(\ell_{p,q}\) regularization
- Introductory lectures on convex optimization. A basic course.
- Linear convergence of proximal gradient algorithm with extrapolation for a class of nonconvex nonsmooth minimization problems
- Minimization of \(\ell_{1-2}\) for compressed sensing
- Nearly unbiased variable selection under minimax concave penalty
- Nonmonotone Spectral Projected Gradient Methods on Convex Sets
- On the convergence of the proximal algorithm for nonsmooth functions involving analytic features
- On the nonmonotone line search
- Penalty methods for a class of non-Lipschitz optimization problems
- Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
- Proximal alternating linearized minimization for nonconvex and nonsmooth problems
- Proximal splitting methods in signal processing
- Smoothing methods for nonsmooth, nonconvex minimization
- Sparse Reconstruction by Separable Approximation
- Sparse Recovery via Partial Regularization: Models, Theory, and Algorithms
- Splitting Algorithms for the Sum of Two Nonlinear Operators
- Splitting methods with variable metric for Kurdyka-Łojasiewicz functions and general convergence rates
- Templates for convex cone problems with applications to sparse signal recovery
- The Łojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Variational Analysis
- iPiano: inertial proximal algorithm for nonconvex optimization
Cited in
(2)
This page was built for publication: Proximal gradient method with extrapolation and line search for a class of non-convex and non-smooth problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6182324)
