A general theory of concave regularization for high-dimensional sparse estimation problems
From MaRDI portal
Abstract: Concave regularization methods provide natural procedures for sparse recovery. However, they are difficult to analyze in the high dimensional setting. Only recently a few sparse recovery results have been established for some specific local solutions obtained via specialized numerical procedures. Still, the fundamental relationship between these solutions such as whether they are identical or their relationship to the global minimizer of the underlying nonconvex formulation is unknown. The current paper fills this conceptual gap by presenting a general theoretical framework showing that under appropriate conditions, the global solution of nonconvex regularization leads to desirable recovery performance; moreover, under suitable conditions, the global solution corresponds to the unique sparse local solution, which can be obtained via different numerical procedures. Under this unified framework, we present an overview of existing results and discuss their connections. The unified view of this work leads to a more satisfactory treatment of concave high dimensional sparse estimation procedures, and serves as guideline for developing further numerical procedures for concave regularization.
Recommendations
- A unified approach to model selection and sparse recovery using regularized least squares
- Asymptotic properties for combined \(L_{1}\) and concave regularization
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Support recovery without incoherence: a case for nonconvex regularization
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
Cites work
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 3444596 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- A Statistical View of Some Chemometrics Regression Tools
- A new approach to variable selection in least squares problems
- Adaptive Forward-Backward Greedy Algorithm for Learning Sparse Representations
- Adaptive Lasso for sparse high-dimensional regression models
- Analysis of multi-stage convex relaxation for sparse regularization
- Asymptotics for Lasso-type estimators.
- Atomic decomposition by basis pursuit
- Comments on: \(\ell _{1}\)-penalization for mixture regression models
- Comments on: \(\ell_{1}\)-penalization for mixture regression models
- Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
- Decoding by Linear Programming
- Estimating the dimension of a model
- High-dimensional generalized linear models and the lasso
- High-dimensional graphs and variable selection with the Lasso
- Just relax: convex programming methods for identifying sparse signals in noise
- Lasso-type recovery of sparse representations for high-dimensional data
- Least angle regression. (With discussion)
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over \ell_q-Balls
- Near-ideal model selection by \(\ell _{1}\) minimization
- Nearly unbiased variable selection under minimax concave penalty
- Nonconcave penalized likelihood with a diverging number of parameters.
- On the conditions used to prove oracle results for the Lasso
- One-step sparse estimates in nonconcave penalized likelihood models
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Rate minimaxity of the Lasso and Dantzig selector for the \(l_{q}\) loss in \(l_{r}\) balls
- Scaled sparse linear regression
- Sharp Thresholds for High-Dimensional and Noisy Sparsity Recovery Using $\ell _{1}$-Constrained Quadratic Programming (Lasso)
- Shifting Inequality and Recovery of Sparse Signals
- Simultaneous analysis of Lasso and Dantzig selector
- Smoothly clipped absolute deviation on high dimensions
- Some Comments on C P
- Some sharp performance bounds for least squares regression with L₁ regularization
- SparseNet: coordinate descent with nonconvex penalties
- Sparsity oracle inequalities for the Lasso
- Square-root lasso: pivotal recovery of sparse signals via conic programming
- Statistics for high-dimensional data. Methods, theory and applications.
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector and sparsity oracle inequalities
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Variable selection using MM algorithms
- \(\ell_{1}\)-penalization for mixture regression models
Cited in
(only showing first 100 items - show all)- Smoothing Newton method for \(\ell^0\)-\(\ell^2\) regularized linear inverse problem
- Matrix completion with nonconvex regularization: spectral operators and scalable algorithms
- Sparse signal reconstruction via the approximations of \(\ell_0\) quasinorm
- Robust decoding from binary measurements with cardinality constraint least squares
- Bayesian Bootstrap Spike-and-Slab LASSO
- Robust low-rank multiple kernel learning with compound regularization
- Nonlinear Variable Selection via Deep Neural Networks
- The basins of attraction of the global minimizers of the non-convex sparse spike estimation problem
- Adaptive bridge estimator for Cox model with a diverging number of parameters
- Weighted thresholding homotopy method for sparsity constrained optimization
- Level constrained first order methods for function constrained optimization
- Fully polynomial-time randomized approximation schemes for global optimization of high-dimensional minimax concave penalized generalized linear models
- De-biasing the Lasso with degrees-of-freedom adjustment
- Oracle inequalities for the lasso in the Cox model
- scientific article; zbMATH DE number 6982301 (Why is no real title available?)
- Subspace learning by \(\ell^0\)-induced sparsity
- A review of distributed statistical inference
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
- Sparse and robust estimation with ridge minimax concave penalty
- Truncated \(L_1\) regularized linear regression: theory and algorithm
- Going beyond oracle property: selection consistency and uniqueness of local solution of the generalized linear model
- Preconditioning for classical relationships: a note relating ridge regression and OLS \(p\)-values to preconditioned \textit{sparse} penalized regression
- MuSP: a multistep screening procedure for sparse recovery
- High-dimensional linear model selection motivated by multiple testing
- GSDAR: a fast Newton algorithm for \(\ell_0\) regularized generalized linear models with statistical guarantee
- Feature screening for case-cohort studies in the presence of interval censoring
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
- Learning from limited temporal data: dynamically sparse historical functional linear models with applications to earth science
- Penalized Sparse Covariance Regression with High Dimensional Covariates
- Robust moderately clipped LASSO for simultaneous outlier detection and variable selection
- In defense of LASSO
- Fundamental barriers to high-dimensional regression with convex penalties
- Penalized least squares estimation with weakly dependent data
- Tuning parameter selection for the adaptive LASSO in the autoregressive model
- Accelerated Stochastic Algorithms for Nonconvex Finite-Sum and Multiblock Optimization
- Fitting sparse linear models under the sufficient and necessary condition for model identification
- Instability, computational efficiency and statistical accuracy
- High-dimensional linear regression with hard thresholding regularization: theory and algorithm
- Fast best subset selection: coordinate descent and local combinatorial optimization algorithms
- Introduction to the special issue on sparsity and regularization methods
- Support recovery without incoherence: a case for nonconvex regularization
- A two-stage regularization method for variable selection and forecasting in high-order interaction model
- High-dimensional grouped folded concave penalized estimation via the LLA algorithm
- A simple method for estimating interactions between a treatment and a large number of covariates
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
- Estimation and inference for precision matrices of nonstationary time series
- Variance prior forms for high-dimensional Bayesian variable selection
- An unbiased approach to compressed sensing
- Bias versus non-convexity in compressed sensing
- Sorted concave penalized regression
- Confidence intervals for low dimensional parameters in high dimensional linear models
- Communication-efficient distributed estimation for high-dimensional large-scale linear regression
- Simultaneous feature selection and outlier detection with optimality guarantees
- The use of random-effect models for high-dimensional variable selection problems
- A communication-efficient method for generalized linear regression with ℓ 0 regularization
- Robust sparse portfolios for index tracking based on M-estimation
- Non-negative least squares for high-dimensional linear models: consistency and sparse recovery without regularization
- Asymptotic properties for combined \(L_{1}\) and concave regularization
- Understanding best subset selection: a tale of two c(omplex)ities
- Optimal forecast reconciliation with time series selection
- Rank-one convexification for sparse regression
- Balanced estimation for high-dimensional measurement error models
- Retire: robust expectile regression in high dimensions
- Robust High-Dimensional Regression with Coefficient Thresholding and Its Application to Imaging Data Analysis
- Global solutions to folded concave penalized nonconvex learning
- On convergence of iterative thresholding algorithms to approximate sparse solution for composite nonconvex optimization
- Sample average approximation with sparsity-inducing penalty for high-dimensional stochastic programming
- Hard thresholding regularised logistic regression: theory and algorithms
- Variable-selection consistency of linear quantile regression by validation set approach
- Learning latent variable Gaussian graphical model for biomolecular network with low sample complexity
- The spike-and-slab LASSO
- High-order block RIP for nonconvex block-sparse compressed sensing
- Cardinality minimization, constraints, and regularization: a survey
- Distributed testing and estimation under sparse high dimensional models
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- Sparse recovery based on the generalized error function
- Endogeneity in high dimensions
- Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression
- On the strong oracle property of concave penalized estimators with infinite penalty derivative at the origin
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- PDAS: a Newton-type method for L₀ regularized accelerated failure time model
- Estimation and variable selection with exponential weights
- A tuning-free robust and efficient approach to high-dimensional regression
- On high-dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization
- Nonbifurcating Phylogenetic Tree Inference via the Adaptive LASSO
- Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models
- \(\ell_0\)-regularized high-dimensional accelerated failure time model
- Variable selection using \(L_q\) penalties
- Tractable ADMM schemes for computing KKT points and local minimizers for \(\ell_0\)-minimization problems
- Random subspace method for high-dimensional regression with the \texttt{R} package \texttt{regRSM}
- Bayesian Estimation of Gaussian Conditional Random Fields
- A unified primal dual active set algorithm for nonconvex sparse recovery
- Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- Fast variable selection under _0 regularization in high-dimensions
- REMI: REGRESSION WITH MARGINAL INFORMATION AND ITS APPLICATION IN GENOME-WIDE ASSOCIATION STUDIES
- A convex-Nonconvex strategy for grouped variable selection
- Homogeneity detection for the high-dimensional generalized linear model
- Principal components adjusted variable screening
- Variable selection and parameter estimation with the Atan regularization method
- Iteratively reweighted _1-penalized robust regression
This page was built for publication: A general theory of concave regularization for high-dimensional sparse estimation problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5965310)
