Support recovery without incoherence: a case for nonconvex regularization
From MaRDI portal
Abstract: We demonstrate that the primal-dual witness proof method may be used to establish variable selection consistency and -bounds for sparse regression problems, even when the loss function and/or regularizer are nonconvex. Using this method, we derive two theorems concerning support recovery and -guarantees for the regression estimator in a general setting. Our results provide rigorous theoretical justification for the use of nonconvex regularization: For certain nonconvex regularizers with vanishing derivative away from the origin, support recovery consistency may be guaranteed without requiring the typical incoherence conditions present in -based methods. We then derive several corollaries that illustrate the wide applicability of our method to analyzing composite objective functions involving losses such as least squares, nonconvex modified least squares for errors-in variables linear regression, the negative log likelihood for generalized linear models, and the graphical Lasso. We conclude with empirical studies to corroborate our theoretical predictions.
A new primal-dual witness proof framework is given that may be used to establish variable selection consistency and \(\ell_{+\infty}\)-bounds for sparse regression problems, even when the loss function and regularizes are nonconvex. The analysis in this paper applies to regularized \(M\)-estimators. From a statistical perspective, the purpose of solving the problem above is to estimate the vector that minimizes the expected loss. The estimator must be unique and independent of the sample size. Conditions are developed under which a minimizer of \(M\)-estimators is consistent with the estimator which minimizes the expected loss. In this paper, it is proved that for certain nonconvex regularizers with vanishing derivative away from the origin, any stationary point can be used to recover the support without requiring the typical incoherence conditions present in \(\ell_1\)-based methods. Numerical examples are given to justify the statement above.
Recommendations
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
- Regularized \(M\)-estimators with nonconvexity: statistical and algorithmic theory for local optima
- A class of null space conditions for sparse recovery via nonconvex, non-separable minimizations
- A general theory of concave regularization for high-dimensional sparse estimation problems
- Optimal computational and statistical rates of convergence for sparse nonconvex learning problems
Cited in
(63)- On the sign consistency of the Lasso for the high-dimensional Cox model
- Second-order optimality conditions and improved convergence results for regularization methods for cardinality-constrained optimization problems
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Consistency bounds and support recovery of d-stationary solutions of sparse sample average approximations
- Inference in high dimensional linear measurement error models
- Iteratively reweighted _1-penalized robust regression
- Sparse classification: a scalable discrete optimization perspective
- The finite sample properties of sparse M-estimators with pseudo-observations
- High-dimensional robust approximated M-estimators for mean regression with asymmetric data
- High dimensional generalized linear models for temporal dependent data
- Bias versus non-convexity in compressed sensing
- Almost sure uniqueness of a global minimum without convexity
- Statistical analysis of sparse approximate factor models
- A class of null space conditions for sparse recovery via nonconvex, non-separable minimizations
- Which bridge estimator is the best for variable selection?
- Finite-sample analysis of \(M\)-estimators using self-concordance
- Sparse regression: scalable algorithms and empirical performance
- A discussion on practical considerations with sparse regression methodologies
- Structure learning of sparse directed acyclic graphs incorporating the scale-free property
- Oracle inequalities for local and global empirical risk minimizers
- Efficient learning with a family of nonconvex regularizers by redistributing nonconvexity
- Approximate message passing for nonconvex sparse regularization with stability and asymptotic analysis
- On uniqueness guarantees of solution in convex regularized linear inverse problems
- GemBag: group estimation of multiple Bayesian graphical models
- Nonbifurcating Phylogenetic Tree Inference via the Adaptive LASSO
- Bayesian Estimation of Gaussian Conditional Random Fields
- An unbiased approach to compressed sensing
- Bayesian regularization for graphical models with unequal shrinkage
- Difference-of-convex learning: directional stationarity, optimality, and sparsity
- Regularized \(M\)-estimators with nonconvexity: statistical and algorithmic theory for local optima
- A general theory of concave regularization for high-dimensional sparse estimation problems
- On high-dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization
- Byzantine-robust distributed sparse learning for M-estimation
- Penalized wavelet nonparametric univariate logistic regression for irregular spaced data
- An ensemble EM algorithm for Bayesian variable selection
- High‐dimensional sparse multivariate stochastic volatility models
- Low-Rank Regression Models for Multiple Binary Responses and their Applications to Cancer Cell-Line Encyclopedia Data
- Robust High-Dimensional Regression with Coefficient Thresholding and Its Application to Imaging Data Analysis
- Numerical characterization of support recovery in sparse regression with correlated design
- Sparse M-estimators in semi-parametric copula models
- Adaptive Huber trace regression with low-rank matrix parameter via nonconvex regularization
- Inference for high-dimensional linear expectile regression with de-biasing method
- Markov neighborhood regression for statistical inference of high-dimensional generalized linear models
- Comment: Feature Screening and Variable Selection via Iterative Ridge Regression
- Fully polynomial-time randomized approximation schemes for global optimization of high-dimensional minimax concave penalized generalized linear models
- High-dimensional Hilbert-Schmidt linear regression with Hilbert manifold variables
- Bounded support in linear random coefficient models: identification and variable selection
- Optimal distributed subsampling for high-dimensional linear measurement error models via doubly bias-corrected score
- Robust multitask feature learning with adaptive Huber regressions
- A novel framework for online supervised learning with feature selection
- Advances in Bayesian model selection consistency for high-dimensional generalized linear models
- Heterogeneous Functional Regression for Subgroup Analysis
- High-dimensional partially linear additive models on Riemannian manifolds
- High-dimensional generalized linear models for Hilbert manifold covariates
- Stochastic feature selection with annealing and its applications to streaming data
- Support estimation and sign recovery in high-dimensional heteroscedastic mean regression
- Robust Personalized Federated Learning with Sparse Penalization
- _1-penalized multinomial regression: estimation, inference, and prediction, with an application to risk factor identification for different dementia subtypes
- High-dimensional convolution-smoothed quantile linear models for Hilbert manifold covariates
- Learning extremal graphical structures in high dimensions
- Sparse minimum redundancy maximum relevance for feature selection
- Bayesian group regularization in generalized linear models with a continuous spike-and-slab prior
- Sparse recovery via nonconvex regularized \(M\)-estimators over \(\ell_q\)-balls
This page was built for publication: Support recovery without incoherence: a case for nonconvex regularization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q682289)
