Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
From MaRDI portal
Publication:58075
DOI10.1214/10-AOAS388zbMATH Open1220.62095arXiv1104.2748OpenAlexW3106108064WikidataQ39020286 ScholiaQ39020286MaRDI QIDQ58075FDOQ58075
Authors: Patrick Breheny, Jian Huang, Patrick Breheny, Jian Huang 
Publication date: 1 March 2011
Published in: The Annals of Applied Statistics (Search for Journal in Brave)
Abstract: A number of variable selection methods have been proposed involving nonconvex penalty functions. These methods, which include the smoothly clipped absolute deviation (SCAD) penalty and the minimax concave penalty (MCP), have been demonstrated to have attractive theoretical properties, but model fitting is not a straightforward task, and the resulting solutions may be unstable. Here, we demonstrate the potential of coordinate descent algorithms for fitting these models, establishing theoretical convergence properties and demonstrating that they are significantly faster than competing approaches. In addition, we demonstrate the utility of convexity diagnostics to determine regions of the parameter space in which the objective function is locally convex, even though the penalty is not. Our simulation study and data examples indicate that nonconvex penalties like MCP and SCAD are worthwhile alternatives to the lasso in many applications. In particular, our numerical results suggest that MCP is the preferred approach among the three methods.
Full work available at URL: https://arxiv.org/abs/1104.2748
Recommendations
- Group coordinate descent algorithms for nonconvex penalized regression
- SparseNet: coordinate descent with nonconvex penalties
- Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
- Coordinate descent algorithms for lasso penalized regression
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cites Work
- Nearly unbiased variable selection under minimax concave penalty
- Heuristics of instability and stabilization in model selection
- Least angle regression. (With discussion)
- Pathwise coordinate optimization
- Coordinate descent algorithms for lasso penalized regression
- Title not available (Why is that?)
- Title not available (Why is that?)
- Ideal spatial adaptation by wavelet shrinkage
- Understanding WaveShrink: variance and bias estimation
- Title not available (Why is that?)
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Title not available (Why is that?)
- Convergence of a block coordinate descent method for nondifferentiable minimization
- One-step sparse estimates in nonconcave penalized likelihood models
Cited In (only showing first 100 items - show all)
- Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors
- Adaptive and reversed penalty for analysis of high-dimensional correlated data
- Simultaneous estimation of quantile regression functions using B-splines and total variation penalty
- An unbiased approach to compressed sensing
- Sparse regression: scalable algorithms and empirical performance
- Coordinate majorization descent algorithm for nonconvex penalized regression
- Nonconvex penalized ridge estimations for partially linear additive models in ultrahigh dimension
- Portal nodes screening for large scale social networks
- A forward and backward stagewise algorithm for nonconvex loss functions with adaptive Lasso
- Estimation of an oblique structure via penalized likelihood factor analysis
- A penalized likelihood method for structural equation modeling
- Difference-of-convex learning: directional stationarity, optimality, and sparsity
- Screening active factors in supersaturated designs
- Convex and non-convex regularization methods for spatial point processes intensity estimation
- Partial penalized empirical likelihood ratio test under sparse case
- A new double-regularized regression using Liu and Lasso regularization
- Variable selection and estimation for semi-parametric multiple-index models
- Broken adaptive ridge regression and its asymptotic properties
- I-LAMM for sparse learning: simultaneous control of algorithmic complexity and statistical error
- On greedy randomized block Gauss-Seidel method with averaging for sparse linear least-squares problems
- A model-averaging approach for high-dimensional regression
- Stable prediction in high-dimensional linear models
- An alternating direction method of multipliers for MCP-penalized regression with high-dimensional data
- Model averaging for generalized linear models with missing at random covariates
- On complexity and convergence of high-order coordinate descent algorithms for smooth nonconvex box-constrained minimization
- A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
- Outlier detection under a covariate-adjusted exponential regression model with censored data
- Structural identification and variable selection in high-dimensional varying-coefficient models
- Variable selection in large margin classifier-based probability estimation with high-dimensional predictors
- A unified primal dual active set algorithm for nonconvex sparse recovery
- Variable selection for zero‐inflated and overdispersed data with application to health care demand in Germany
- High-dimensional sparse portfolio selection with nonnegative constraint
- Cross validation in sparse linear regression with piecewise continuous nonconvex penalties and its acceleration
- Efficiency for Regularization Parameter Selection in Penalized Likelihood Estimation of Misspecified Models
- The horseshoe-like regularization for feature subset selection
- Variable selection and parameter estimation with the Atan regularization method
- Identification of proportionality structure with two-part models using penalization
- High-dimensional variable selection via low-dimensional adaptive learning
- Natural coordinate descent algorithm for \(\ell_1\)-penalised regression in generalised linear models
- A two-step randomized Gauss-Seidel method for solving large-scale linear least squares problems
- Selection of fixed effects in high-dimensional generalized linear mixed models
- An outer-inner linearization method for non-convex and nondifferentiable composite regularization problems
- Robust network-based analysis of the associations between (epi)genetic measurements
- Pruning variable selection ensembles
- Fully Bayesian logistic regression with hyper-LASSO priors for high-dimensional feature selection
- Gauss-Seidel method with oblique direction
- Sparse vector Markov switching autoregressive models. Application to multivariate time series of temperature
- Linear hypothesis testing for high dimensional generalized linear models
- An ADMM with continuation algorithm for non-convex SICA-penalized regression in high dimensions
- Variable selection and estimation using a continuous approximation to the \(L_0\) penalty
- Model selection for Cox models with time-varying coefficients
- Title not available (Why is that?)
- A coordinate descent algorithm for computing penalized smooth quantile regression
- Truncated \(L_1\) regularized linear regression: theory and algorithm
- Variable Selection for Multiple Function-on-Function Linear Regression
- Variable selection for high-dimensional generalized linear models with the weighted elastic-net procedure
- GSDAR: a fast Newton algorithm for \(\ell_0\) regularized generalized linear models with statistical guarantee
- Adjusted regularized estimation in the accelerated failure time model with high dimensional covariates
- Inference for low‐ and high‐dimensional inhomogeneous Gibbs point processes
- Ranking-based variable selection for high-dimensional data
- A study on tuning parameter selection for the high-dimensional lasso
- Fast best subset selection: coordinate descent and local combinatorial optimization algorithms
- Multiple choice from competing regression models under multicollinearity based on standardized update
- Sparse estimation via nonconcave penalized likelihood in factor analysis model
- Model Selection via Bayesian Information Criterion for Quantile Regression Models
- A generic coordinate descent solver for non-smooth convex optimisation
- Multi-stage convex relaxation for feature selection
- Variable selection via generalized SELO-penalized linear regression models
- Variable selection via generalized SELO-penalized Cox regression models
- Integrative analysis of cancer diagnosis studies with composite penalization
- Pathwise coordinate optimization
- Coordinate descent algorithms for lasso penalized regression
- Penalized estimation in additive varying coefficient models using grouped regularization
- Global solutions to folded concave penalized nonconvex learning
- The spike-and-slab LASSO
- Skinny Gibbs: a consistent and scalable Gibbs sampler for model selection
- Group coordinate descent algorithms for nonconvex penalized regression
- Endogeneity in high dimensions
- Sparse classification: a scalable discrete optimization perspective
- Visualization and assessment of model selection uncertainty
- Relaxed sparse eigenvalue conditions for sparse estimation via non-convex regularized regression
- Coordinate descent algorithm for covariance graphical Lasso
- High-dimensional regression in practice: an empirical study of finite-sample prediction, variable selection and ranking
- Non-convex penalized estimation in high-dimensional models with single-index structure
- Sparse factor regression via penalized maximum likelihood estimation
- Asymptotic properties of lasso in high-dimensional partially linear models
- A new model selection procedure for finite mixture regression models
- Ultrahigh dimensional variable selection through the penalized maximum trimmed likelihood estimator
- Nonnegative estimation and variable selection under minimax concave penalty for sparse high-dimensional linear regression models
- Designing penalty functions in high dimensional problems: the role of tuning parameters
- Penalized empirical likelihood for the sparse Cox regression model
- Likelihood adaptively modified penalties
- \(\ell_0\)-regularized high-dimensional accelerated failure time model
- A discussion on practical considerations with sparse regression methodologies
- An iterative algorithm for fitting nonconvex penalized generalized linear models with grouped predictors
- Group structure detection for a high‐dimensional panel data model
- Penalized estimation in finite mixture of ultra-high dimensional regression models
- ncvreg
- Hierarchical Bayes, maximum a posteriori estimators, and minimax concave penalized likelihood estimation
- The group exponential Lasso for bi-level variable selection
Uses Software
This page was built for publication: Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q58075)
