A general theory of concave regularization for high-dimensional sparse estimation problems
From MaRDI portal
Abstract: Concave regularization methods provide natural procedures for sparse recovery. However, they are difficult to analyze in the high dimensional setting. Only recently a few sparse recovery results have been established for some specific local solutions obtained via specialized numerical procedures. Still, the fundamental relationship between these solutions such as whether they are identical or their relationship to the global minimizer of the underlying nonconvex formulation is unknown. The current paper fills this conceptual gap by presenting a general theoretical framework showing that under appropriate conditions, the global solution of nonconvex regularization leads to desirable recovery performance; moreover, under suitable conditions, the global solution corresponds to the unique sparse local solution, which can be obtained via different numerical procedures. Under this unified framework, we present an overview of existing results and discuss their connections. The unified view of this work leads to a more satisfactory treatment of concave high dimensional sparse estimation procedures, and serves as guideline for developing further numerical procedures for concave regularization.
Recommendations
- A unified approach to model selection and sparse recovery using regularized least squares
- Asymptotic properties for combined \(L_{1}\) and concave regularization
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Support recovery without incoherence: a case for nonconvex regularization
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
Cites work
- \(\ell_{1}\)-penalization for mixture regression models
- A new approach to variable selection in least squares problems
- A Statistical View of Some Chemometrics Regression Tools
- Adaptive Forward-Backward Greedy Algorithm for Learning Sparse Representations
- Adaptive Lasso for sparse high-dimensional regression models
- Analysis of multi-stage convex relaxation for sparse regularization
- Asymptotics for Lasso-type estimators.
- Atomic decomposition by basis pursuit
- Comments on: \(\ell _{1}\)-penalization for mixture regression models
- Comments on: \(\ell_{1}\)-penalization for mixture regression models
- Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
- Decoding by Linear Programming
- Estimating the dimension of a model
- High-dimensional generalized linear models and the lasso
- High-dimensional graphs and variable selection with the Lasso
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 3444596 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Just relax: convex programming methods for identifying sparse signals in noise
- Lasso-type recovery of sparse representations for high-dimensional data
- Least angle regression. (With discussion)
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over \ell_q-Balls
- Near-ideal model selection by \(\ell _{1}\) minimization
- Nearly unbiased variable selection under minimax concave penalty
- Nonconcave penalized likelihood with a diverging number of parameters.
- On the conditions used to prove oracle results for the Lasso
- One-step sparse estimates in nonconcave penalized likelihood models
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
- Scaled sparse linear regression
- Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
- Shifting Inequality and Recovery of Sparse Signals
- Simultaneous analysis of Lasso and Dantzig selector
- Smoothly clipped absolute deviation on high dimensions
- Some Comments on C P
- Some sharp performance bounds for least squares regression with L₁ regularization
- SparseNet: coordinate descent with nonconvex penalties
- Sparsity oracle inequalities for the Lasso
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- Statistics for high-dimensional data. Methods, theory and applications.
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector and sparsity oracle inequalities
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Variable selection using MM algorithms
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- A unified approach to model selection and sparse recovery using regularized least squares
- Bayesian Bootstrap Spike-and-Slab LASSO
- Asymptotic normality and optimalities in estimation of large Gaussian graphical models
- Distributed testing and estimation under sparse high dimensional models
- Variable selection and parameter estimation with the Atan regularization method
- Homogeneity detection for the high-dimensional generalized linear model
- Principal components adjusted variable screening
- The use of random-effect models for high-dimensional variable selection problems
- Balanced estimation for high-dimensional measurement error models
- Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression
- A doubly sparse approach for group variable selection
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- Quantile regression for additive coefficient models in high dimensions
- A two-stage regularization method for variable selection and forecasting in high-order interaction model
- High-dimensional grouped folded concave penalized estimation via the LLA algorithm
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
- Tractable ADMM schemes for computing KKT points and local minimizers for \(\ell_0\)-minimization problems
- A unified primal dual active set algorithm for nonconvex sparse recovery
- Iteratively reweighted _1-penalized robust regression
- Second-order Stein: SURE for SURE and other applications in high-dimensional inference
- Dynamic variable selection with spike-and-slab process priors
- Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models
- Smoothing Newton method for \(\ell^0\)-\(\ell^2\) regularized linear inverse problem
- Weighted thresholding homotopy method for sparsity constrained optimization
- Robust moderately clipped LASSO for simultaneous outlier detection and variable selection
- High-dimensional linear regression with hard thresholding regularization: theory and algorithm
- Fundamental barriers to high-dimensional regression with convex penalties
- \(\ell_0\)-regularized high-dimensional accelerated failure time model
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- GSDAR: a fast Newton algorithm for \(\ell_0\) regularized generalized linear models with statistical guarantee
- De-biasing the Lasso with degrees-of-freedom adjustment
- Bias versus non-convexity in compressed sensing
- Almost sure uniqueness of a global minimum without convexity
- Sparse signal reconstruction via the approximations of \(\ell_0\) quasinorm
- Subspace learning by \(\ell^0\)-induced sparsity
- Matrix completion with nonconvex regularization: spectral operators and scalable algorithms
- Estimation and inference for precision matrices of nonstationary time series
- Robust low-rank multiple kernel learning with compound regularization
- On a monotone scheme for nonconvex nonsmooth optimization with applications to fracture mechanics
- Sorted concave penalized regression
- Variance prior forms for high-dimensional Bayesian variable selection
- Majorized proximal alternating imputation for regularized rank constrained matrix completion
- Joint feature screening for ultra-high-dimensional sparse additive hazards model by the sparsity-restricted pseudo-score estimator
- Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming
- Weaker regularity conditions and sparse recovery in high-dimensional regression
- Conditional sure independence screening by conditional marginal empirical likelihood
- A superlinearly convergent \(R\)-regularized Newton scheme for variational models with concave sparsity-promoting priors
- Calibrating nonconvex penalized regression in ultra-high dimension
- Estimation and variable selection with exponential weights
- Strong oracle optimality of folded concave penalized estimation
- Endogeneity in high dimensions
- Sparse estimation via lower-order penalty optimization methods in high-dimensional linear regression
- OR forum: An algorithmic approach to linear regression
- Global solutions to folded concave penalized nonconvex learning
- Best subset selection via a modern optimization lens
- Random subspace method for high-dimensional regression with the \texttt{R} package \texttt{regRSM}
- Simultaneous Variable and Covariance Selection With the Multivariate Spike-and-Slab LASSO
- Oracle inequalities for the lasso in the Cox model
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
- scientific article; zbMATH DE number 6982301 (Why is no real title available?)
- Goodness-of-Fit Tests for High Dimensional Linear Models
- The spike-and-slab LASSO
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
- A simple method for estimating interactions between a treatment and a large number of covariates
- Nonbifurcating Phylogenetic Tree Inference via the Adaptive LASSO
- The basins of attraction of the global minimizers of the non-convex sparse spike estimation problem
- REMI: REGRESSION WITH MARGINAL INFORMATION AND ITS APPLICATION IN GENOME-WIDE ASSOCIATION STUDIES
- Regularized projection score estimation of treatment effects in high-dimensional quantile regression
- Bayesian Estimation of Gaussian Conditional Random Fields
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
- Hard thresholding regularised logistic regression: theory and algorithms
- Nonlinear Variable Selection via Deep Neural Networks
- In defense of LASSO
- An unbiased approach to compressed sensing
- Time-varying hazards model for incorporating irregularly measured high-dimensional biomarkers
- Fast best subset selection: coordinate descent and local combinatorial optimization algorithms
- A tuning-free robust and efficient approach to high-dimensional regression
- Truncated \(L_1\) regularized linear regression: theory and algorithm
- High-dimensional linear model selection motivated by multiple testing
- Accelerated Stochastic Algorithms for Nonconvex Finite-Sum and Multiblock Optimization
- Penalized least squares estimation with weakly dependent data
- Tuning parameter selection for the adaptive LASSO in the autoregressive model
- Asymptotic properties for combined \(L_{1}\) and concave regularization
- Confidence intervals for low dimensional parameters in high dimensional linear models
- A review of distributed statistical inference
- On the finite-sample analysis of -estimators
- On the finite-sample analysis of \(\Theta\)-estimators
- Introduction to the special issue on sparsity and regularization methods
- Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- On high-dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization
- Simultaneous feature selection and outlier detection with optimality guarantees
- L 0 -regularization for high-dimensional regression with corrupted data
- Sparse and robust estimation with ridge minimax concave penalty
- Adaptive bridge estimator for Cox model with a diverging number of parameters
- A convex-Nonconvex strategy for grouped variable selection
- Retire: robust expectile regression in high dimensions
- Robust High-Dimensional Regression with Coefficient Thresholding and Its Application to Imaging Data Analysis
- Communication-efficient distributed estimation for high-dimensional large-scale linear regression
- High-dimensional composite quantile regression: optimal statistical guarantees and fast algorithms
This page was built for publication: A general theory of concave regularization for high-dimensional sparse estimation problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5965310)
