Newton-type methods with the proximal gradient step for sparse estimation
From MaRDI portal
(Redirected from Publication:6130698)
Abstract: In sparse estimation, in which the sum of the loss function and the regularization term is minimized, methods such as the proximal gradient method and the proximal Newton method are applied. The former is slow to converge to a solution, while the latter converges quickly but is inefficient for problems such as group lasso problems. In this paper, we examine how to efficiently find a solution by finding the convergence destination of the proximal gradient method. However, the case in which the Lipschitz constant of the derivative of the loss function is unknown has not been studied theoretically, and only the Newton method has been proposed for the case in which the Lipschitz constant is known. We show that the Newton method converges when the Lipschitz constant is unknown and extend the theory. Furthermore, we propose a new quasi-Newton method that avoids Hessian calculations and improves efficiency, and we prove that it converges quickly, providing a theoretical guarantee. Finally, numerical experiments show that the proposed method can significantly improve the efficiency.
Recommendations
- A proximal-gradient homotopy method for the sparse least-squares problem
- On the linear convergence of a proximal gradient method for a class of nonsmooth convex minimization problems
- Subspace Newton method for sparse group \(\ell_0\) optimization problem
- A semismooth Newton method with multidimensional filter globalization for l₁-optimization
- A multilevel framework for sparse optimization with application to inverse covariance estimation and logistic regression
Cites work
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A nonsmooth version of Newton's method
- A regularized semi-smooth Newton method with projection steps for composite convex programs
- A SemiSmooth Newton Method for Semidefinite Programs and its Applications in Electronic Structure Calculations
- A stochastic semismooth Newton method for nonsmooth nonconvex optimization
- An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems
- Convex analysis and monotone operator theory in Hilbert spaces
- Finite-Dimensional Variational Inequalities and Complementarity Problems
- Forward-backward quasi-Newton methods for nonsmooth optimization problems
- scientific article; zbMATH DE number 1552023 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 5937962 (Why is no real title available?)
- Iterative Solution of Nonlinear Equations in Several Variables
- Model Selection and Estimation in Regression with Grouped Variables
- Newton and Quasi-Newton Methods for a Class of Nonsmooth Equations and Related Problems
This page was built for publication: Newton-type methods with the proximal gradient step for sparse estimation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6130698)
