Greedy sparsity-constrained optimization
From MaRDI portal
Parametric inference under constraints (62F30) Generalized linear models (logistic models) (62J12) Learning and adaptive systems in artificial intelligence (68T05) Large-scale problems in mathematical programming (90C06) Approximation methods and heuristics in mathematical programming (90C59) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)
Abstract: Sparsity-constrained optimization has wide applicability in machine learning, statistics, and signal processing problems such as feature selection and compressive Sensing. A vast body of work has studied the sparsity-constrained optimization from theoretical, algorithmic, and application aspects in the context of sparse estimation in linear models where the fidelity of the estimate is measured by the squared error. In contrast, relatively less effort has been made in the study of sparsity-constrained optimization in cases where nonlinear models are involved or the cost function is not quadratic. In this paper we propose a greedy algorithm, Gradient Support Pursuit (GraSP), to approximate sparse minima of cost functions of arbitrary form. Should a cost function have a Stable Restricted Hessian (SRH) or a Stable Restricted Linearization (SRL), both of which are introduced in this paper, our algorithm is guaranteed to produce a sparse vector within a bounded distance from the true sparse optimum. Our approach generalizes known results for quadratic cost functions that arise in sparse linear regression and Compressive Sensing. We also evaluate the performance of GraSP through numerical simulations on synthetic data, where the algorithm is employed for sparse logistic regression with and without -regularization.
Recommendations
- Algorithms for sparsity-constrained optimization
- Sparsity constrained nonlinear optimization: optimality conditions and algorithms
- A gradient projection algorithm with a new stepsize for nonnegative sparsity-constrained optimization
- scientific article; zbMATH DE number 6982922
- Sparse Optimization with Least-Squares Constraints
Cited in
(53)- Efficient projected gradient methods for cardinality constrained optimization
- Error bounds for rank constrained optimization problems and applications
- Restricted strong convexity implies weak submodularity
- A gradient projection algorithm with a new stepsize for nonnegative sparsity-constrained optimization
- An extended Newton-type algorithm for \(\ell_2\)-regularized sparse logistic regression and its efficiency for classifying large-scale datasets
- Double fused Lasso regularized regression with both matrix and vector valued predictors
- Subspace quadratic regularization method for group sparse multinomial logistic regression
- Weighted thresholding homotopy method for sparsity constrained optimization
- Gradient projection Newton algorithm for sparse collaborative learning using synthetic and real datasets of applications
- A data-driven line search rule for support recovery in high-dimensional data analysis
- Gradient projection Newton pursuit for sparsity constrained optimization
- Generalized greedy alternatives
- Nonsmooth sparsity constrained optimization problems: optimality conditions
- Optimality conditions for rank-constrained matrix optimization
- Solving equations of random convex functions via anchored regression
- Greedy approximation in convex optimization
- On solutions of sparsity constrained optimization
- The first-order necessary conditions for sparsity constrained optimization
- Newton method for \(\ell_0\)-regularized optimization
- A greedy Newton-type method for multiple sparse constraint problem
- On the minimization over sparse symmetric sets: projections, optimality conditions, and algorithms
- Algorithms for sparsity-constrained optimization
- Sparsity constrained nonlinear optimization: optimality conditions and algorithms
- Trading accuracy for sparsity in optimization problems with sparsity constraints
- Sparse Optimization with Least-Squares Constraints
- scientific article; zbMATH DE number 6982922 (Why is no real title available?)
- A tight bound of hard thresholding
- Generalized conditional gradient for sparse estimation
- Iterative hard thresholding methods for \(l_0\) regularized convex cone programming
- Conditional gradient algorithms for rank-one matrix approximations with a sparsity constraint
- Global and quadratic convergence of Newton hard-thresholding pursuit
- Sparse convex optimization via adaptively regularized hard thresholding
- Nonlinear Variable Selection via Deep Neural Networks
- Dual iterative hard thresholding
- Learning sparse classifiers: continuous and mixed integer optimization perspectives
- A survey on compressive sensing: classical results and recent advancements
- A unifying framework for sparsity-constrained optimization
- The greedy simplex algorithm for double sparsity constrained optimization problems
- A local MM subspace method for solving constrained variational problems in image recovery
- Stochastic privacy-preserving methods for nonconvex sparse learning
- Cardinality minimization, constraints, and regularization: a survey
- Statistical computational learning
- \texttt{skscope}: fast sparsity-constrained optimization in Python
- Relaxation quadratic approximation greedy pursuit method based on sparse learning
- Sparse Wasserstein barycenters and application to reduced order modeling
- Deep tobit model: an integrated framework for high-dimensional censored regression with variable selection
- Nonconvex regularizer and homotopy-based sparse optimization: convergent algorithms and applications
- Brief introduction in greedy approximation
- Sparse optimization for Poisson regression based on GPGN algorithm
- Iterative mix thresholding algorithm with continuation technique for mix sparse optimization and application
- Heavy-ball-based relaxed optimal \(\mathrm{s}\)-thresholding algorithms for solving compressed sensing problem
- Minimization over the _p ball using a hybrid first-order method
- A bilinear formulation for vector sparsity optimization
This page was built for publication: Greedy sparsity-constrained optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5405269)
