Nearly unbiased variable selection under minimax concave penalty
From MaRDI portal
Abstract: We propose MC+, a fast, continuous, nearly unbiased and accurate method of penalized variable selection in high-dimensional linear regression. The LASSO is fast and continuous, but biased. The bias of the LASSO may prevent consistent variable selection. Subset selection is unbiased but computationally costly. The MC+ has two elements: a minimax concave penalty (MCP) and a penalized linear unbiased selection (PLUS) algorithm. The MCP provides the convexity of the penalized loss in sparse regions to the greatest extent given certain thresholds for variable selection and unbiasedness. The PLUS computes multiple exact local minimizers of a possibly nonconvex penalized loss function in a certain main branch of the graph of critical points of the penalized loss. Its output is a continuous piecewise linear path encompassing from the origin for infinite penalty to a least squares solution for zero penalty. We prove that at a universal penalty level, the MC+ has high probability of matching the signs of the unknowns, and thus correct selection, without assuming the strong irrepresentable condition required by the LASSO. This selection consistency applies to the case of , and is proved to hold for exactly the MC+ solution among possibly many local minimizers. We prove that the MC+ attains certain minimax convergence rates in probability for the estimation of regression coefficients in balls. We use the SURE method to derive degrees of freedom and -type risk estimates for general penalized LSE, including the LASSO and MC+ estimators, and prove their unbiasedness. Based on the estimated degrees of freedom, we propose an estimator of the noise level for proper choice of the penalty level.
Recommendations
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- The Adaptive Lasso and Its Oracle Properties
- Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
Cites work
- A new approach to variable selection in least squares problems
- Adaptive Lasso for sparse high-dimensional regression models
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- Decoding by Linear Programming
- Discussion: One-step sparse estimates in nonconcave penalized likelihood models
- Estimating the dimension of a model
- Estimation of the mean of a multivariate normal distribution
- High-dimensional generalized linear models and the lasso
- High-dimensional graphs and variable selection with the Lasso
- How Biased is the Apparent Error Rate of a Prediction Rule?
- scientific article; zbMATH DE number 5957408 (Why is no real title available?)
- scientific article; zbMATH DE number 5957506 (Why is no real title available?)
- scientific article; zbMATH DE number 409717 (Why is no real title available?)
- scientific article; zbMATH DE number 1089159 (Why is no real title available?)
- scientific article; zbMATH DE number 3444596 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Ideal spatial adaptation by wavelet shrinkage
- Just relax: convex programming methods for identifying sparse signals in noise
- Lasso-type recovery of sparse representations for high-dimensional data
- Least angle regression. (With discussion)
- Local operator theory, random matrices and Banach spaces.
- Minimax risk over \(l_ p\)-balls for \(l_ q\)-error
- Nonconcave penalized likelihood with a diverging number of parameters.
- On the degrees of freedom in shape-restricted regression.
- On the Non-Negative Garrotte Estimator
- One-step sparse estimates in nonconcave penalized likelihood models
- Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
- Piecewise linear regularized solution paths
- Regularization of Wavelet Approximations
- Relaxed Lasso
- Some Comments on C P
- Sparsity oracle inequalities for the Lasso
- The Adaptive Lasso and Its Oracle Properties
- The boosting approach to machine learning: an overview
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The risk inflation criterion for multiple regression
- The sparsity and bias of the LASSO selection in high-dimensional linear regression
- Variable selection using MM algorithms
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- Integrative sparse principal component analysis
- The group exponential Lasso for bi-level variable selection
- Regularized estimation in sparse high-dimensional time series models
- A modified local quadratic approximation algorithm for penalized optimization problems
- Factor-Adjusted Regularized Model Selection
- Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients
- D-learning to estimate optimal individual treatment rules
- Tuning parameter selection in sparse regression modeling
- Estimation and variable selection for partially functional linear models
- Hierarchically penalized quantile regression with multiple responses
- Solving norm constrained portfolio optimization via coordinate-wise descent algorithms
- Estimation of an oblique structure via penalized likelihood factor analysis
- Partially linear structure identification in generalized additive models with NP-dimensionality
- Model selection consistency of Lasso for empirical data
- An alternating direction method of multipliers for MCP-penalized regression with high-dimensional data
- Variable selection via generalized SELO-penalized linear regression models
- Censored cumulative residual independent screening for ultrahigh-dimensional survival data
- Conditional quantile correlation screening procedure for ultrahigh-dimensional varying coefficient models
- Are discoveries spurious? Distributions of maximum spurious correlations and their applications
- Distributed testing and estimation under sparse high dimensional models
- Variable selection and parameter estimation with the Atan regularization method
- Robust group identification and variable selection in regression
- Homogeneity detection for the high-dimensional generalized linear model
- Ultrahigh dimensional feature screening via projection
- Correlation rank screening for ultrahigh-dimensional survival data
- The use of random-effect models for high-dimensional variable selection problems
- Feature screening for generalized varying coefficient models with application to dichotomous responses
- Regularized estimation for the least absolute relative error models with a diverging number of covariates
- Bayesian variable selection for finite mixture model of linear regressions
- Robust variable selection of joint frailty model for panel count data
- A new nonparametric screening method for ultrahigh-dimensional survival data
- Variable selection for high dimensional Gaussian copula regression model: an adaptive hypothesis testing procedure
- Identification of local sparsity and variable selection for varying coefficient additive hazards models
- Balanced estimation for high-dimensional measurement error models
- Sparse pathway-based prediction models for high-throughput molecular data
- Estimating large covariance matrix with network topology for high-dimensional biomedical data
- A principal varying-coefficient model for quantile regression: joint variable selection and dimension reduction
- Moderately clipped Lasso
- A random-effect model approach for group variable selection
- Robust subspace segmentation via nonconvex low rank representation
- Model averaging with high-dimensional dependent data
- Regularized estimation in GINAR(\(p\)) process
- A globally convergent algorithm for nonconvex optimization based on block coordinate update
- Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression
- Estimation and variable selection for quantile partially linear single-index models
- Exponentially tilted likelihood inference on growing dimensional unconditional moment models
- A doubly sparse approach for group variable selection
- Restricted Robinson constraint qualification and optimality for cardinality-constrained cone programming
- Folded concave penalized sparse linear regression: sparsity, statistical performance, and algorithmic theory for local solutions
- Quantile regression for additive coefficient models in high dimensions
- \(\ell _p\) regularized low-rank approximation via iterative reweighted singular value minimization
- A penalized likelihood method for structural equation modeling
- Variable selection and estimation using a continuous approximation to the \(L_0\) penalty
- Cluster analysis of longitudinal profiles with subgroups
- Homotopy continuation approaches for robust SV classification and regression
- A constrained \(\ell1\) minimization approach for estimating multiple sparse Gaussian or nonparanormal graphical models
- A coordinate descent algorithm for computing penalized smooth quantile regression
- Extended differential geometric LARS for high-dimensional GLMs with general dispersion parameter
- A group adaptive elastic-net approach for variable selection in high-dimensional linear regression
- A new nonconvex approach to low-rank matrix completion with application to image inpainting
- Simultaneous variable selection and smoothing for high-dimensional function-on-scalar regression
- Optimization methods for regularization-based ill-posed problems: a survey and a multi-objective framework
- A two-stage regularization method for variable selection and forecasting in high-order interaction model
- A systematic review on model selection in high-dimensional regression
- High-dimensional grouped folded concave penalized estimation via the LLA algorithm
- A novel variational Bayesian method for variable selection in logistic regression models
- Simultaneous variable selection and class fusion with penalized distance criterion based classifiers
- Feature screening in ultrahigh-dimensional partially linear models with missing responses at random
- Improving the prediction performance of the Lasso by subtracting the additive structural noises
- A new piecewise quadratic approximation approach for \(L_0\) norm minimization problem
- Multiple-prespecified-dictionary sparse representation for compressive sensing image reconstruction with nonconvex regularization
- Nonconvex penalized ridge estimations for partially linear additive models in ultrahigh dimension
- Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis
- Variable selection via generalized SELO-penalized Cox regression models
- Portal nodes screening for large scale social networks
- Review: Reversed low-rank ANOVA model for transforming high dimensional genetic data into low dimension
- Hypothesis testing via a penalized-likelihood approach
- Regularization parameter selection for penalized empirical likelihood estimator
- Variable screening for ultrahigh dimensional heterogeneous data via conditional quantile correlations
- A proximal difference-of-convex algorithm with extrapolation
- Weighted-average least squares estimation of generalized linear models
- Convex and non-convex regularization methods for spatial point processes intensity estimation
- Pathwise coordinate optimization for sparse learning: algorithm and theory
- High dimensional censored quantile regression
- DC formulations and algorithms for sparse optimization problems
- Minimization of transformed L₁ penalty: theory, difference of convex function algorithm, and robust application in compressed sensing
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- Robust variable selection for finite mixture regression models
- Equivalent Lipschitz surrogates for zero-norm and rank optimization problems
- Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
- Capped \(\ell_p\) approximations for the composite \(\ell_0\) regularization problem
- High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
- Robust network-based analysis of the associations between (epi)genetic measurements
- Time-varying correlation structure estimation and local-feature detection for spatio-temporal data
- Broken adaptive ridge regression and its asymptotic properties
- Variable selection with Hamming loss
- On the exponentially weighted aggregate with the Laplace prior
- An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors
- Model selection via standard error adjusted adaptive Lasso
- Variable selection in linear mixed effects models
This page was built for publication: Nearly unbiased variable selection under minimax concave penalty
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q117379)
