Efficient learning with a family of nonconvex regularizers by redistributing nonconvexity
From MaRDI portal
(Redirected from Publication:4558504)
Abstract: The use of convex regularizers allows for easy optimization, though they often produce biased estimation and inferior prediction performance. Recently, nonconvex regularizers have attracted a lot of attention and outperformed convex ones. However, the resultant optimization problem is much harder. In this paper, for a large class of nonconvex regularizers, we propose to move the nonconvexity from the regularizer to the loss. The nonconvex regularizer is then transformed to a familiar convex regularizer, while the resultant loss function can still be guaranteed to be smooth. Learning with the convexified regularizer can be performed by existing efficient algorithms originally designed for convex regularizers (such as the proximal algorithm, Frank-Wolfe algorithm, alternating direction method of multipliers and stochastic gradient descent). Extensions are made when the convexified regularizer does not have closed-form proximal step, and when the loss function is nonconvex, nonsmooth. Extensive experiments on a variety of machine learning application scenarios show that optimizing the transformed problem is much faster than running the state-of-the-art on the original problem.
Recommendations
- scientific article; zbMATH DE number 6276223
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
- Global convergence analysis of sparse regular nonconvex optimization problems
- Support recovery without incoherence: a case for nonconvex regularization
Cites work
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A generalized conditional gradient method and its connection to an iterative shrinkage method
- A proximal stochastic gradient method with progressive variance reduction
- A variational approach to remove outliers and impulse noise
- Accelerated gradient methods for nonconvex nonlinear and stochastic programming
- An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions
- Analysis of multi-stage convex relaxation for sparse regularization
- Angewandte Mathematik: Body and Soul
- Characterization of the subdifferential of some matrix norms
- Compressed sensing
- Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Enhancing sparsity by reweighted \(\ell _{1}\) minimization
- Exact matrix completion via convex optimization
- Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
- Global convergence of splitting methods for nonconvex composite optimization
- Gradient methods for minimizing composite functions
- scientific article; zbMATH DE number 1818892 (Why is no real title available?)
- scientific article; zbMATH DE number 3950216 (Why is no real title available?)
- scientific article; zbMATH DE number 51132 (Why is no real title available?)
- scientific article; zbMATH DE number 3574917 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 6159604 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- iPiano: inertial proximal algorithm for nonconvex optimization
- Low-rank optimization with trace norm penalty
- Nearly unbiased variable selection under minimax concave penalty
- Nonsmooth analysis of singular values. II: Applications
- On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method
- Online learning for matrix factorization and sparse coding
- Proximal methods for hierarchical sparse coding
- Regularized \(M\)-estimators with nonconvexity: statistical and algorithmic theory for local optima
- Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting
- Robust principal component analysis?
- Smoothing methods for nonsmooth, nonconvex minimization
- Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm
- Sparsity and Smoothness Via the Fused Lasso
- Spectral regularization algorithms for learning large incomplete matrices
- The Proximal Average: Basic Theory
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(8)- A FISTA-type accelerated gradient algorithm for solving smooth nonconvex composite optimization problems
- Accelerated inexact composite gradient methods for nonconvex spectral optimization problems
- An efficient adaptive accelerated inexact proximal point method for solving linearly constrained nonconvex composite problems
- An average curvature accelerated composite gradient method for nonconvex smooth composite optimization problems
- An aggressive reduction on the complexity of optimization for non-strongly convex objectives
- An adaptive superfast inexact proximal augmented Lagrangian method for smooth nonconvex composite optimization problems
- Average curvature FISTA for nonconvex smooth composite optimization problems
- Optimal regularization for a data source
This page was built for publication: Efficient learning with a family of nonconvex regularizers by redistributing nonconvexity
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4558504)
