High-dimensional variable selection

DOI10.1214/08-AOS646zbMATH Open1173.62054arXiv0704.1139OpenAlexW3103643510WikidataQ37362959 ScholiaQ37362959MaRDI QIDQ834336FDOQ834336

Authors: Kathryn Roeder, Larry Wasserman 

Publication date: 19 August 2009

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This paper explores the following question: what kind of statistical guarantees can be given when doing variable selection in high-dimensional models? In particular, we look at the error rates and power of some multi-stage regression methods. In the first stage we fit a set of candidate models. In the second stage we select one model by cross-validation. In the third stage we use hypothesis testing to eliminate some variables. We refer to the first two stages as "screening" and the last stage as "cleaning." We consider three screening methods: the lasso, marginal regression, and forward stepwise regression. Our method gives consistent variable selection under certain conditions.

Full work available at URL: https://arxiv.org/abs/0704.1139

Recommendations

• Variable selection in high dimensional data analysis with applications
• Variable selection for high dimensional multivariate outcomes
• Variable selection in high-dimensional partially linear models
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• Selection of variables and dimension reduction in high-dimensional non-parametric regression
• Variable selection methods in high-dimensional regression -- a simulation study
• Simultaneous dimension reduction and variable selection in modeling high dimensional data
• Variable selection and estimation in high-dimensional partially linear models
• High Dimensional Variable Selection via Tilting
• Feature selection for high-dimensional data

zbMATH Keywords

lassosimulationssparsitystepwise regressionosteoporotic fractures

Mathematics Subject Classification ID

Point estimation (62F10) Asymptotic properties of parametric estimators (62F12) Linear regression; mixed models (62J05) Applications of statistics to biology and medical sciences; meta analysis (62P10) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• The Adaptive Lasso and Its Oracle Properties
• Least angle regression. (With discussion)
• Title not available (Why is that?)
• Lasso-type recovery of sparse representations for high-dimensional data
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• High-dimensional graphs and variable selection with the Lasso
• Title not available (Why is that?)
• Persistene in high-dimensional linear predictor-selection and the virtue of overparametrization
• Boosting for high-dimensional linear models
• Uniform consistency in causal inference
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• Relaxed Lasso
• Just relax: convex programming methods for identifying sparse signals in noise
• Approximation and learning by greedy algorithms
• For most large underdetermined systems of linear equations the minimal 𝓁₁‐norm solution is also the sparsest solution
• Greed is Good: Algorithmic Results for Sparse Approximation
• Causation, prediction, and search. With additional material by David Heckerman, Christopher Meek, Gregory F. Cooper and Thomas Richardson.

Cited In (only showing first 100 items - show all)

• SLOPE-adaptive variable selection via convex optimization
• A regularization-based adaptive test for high-dimensional GLMs
• High-dimensional inference: confidence intervals, \(p\)-values and R-software \texttt{hdi}
• Hierarchical inference for genome-wide association studies: a view on methodology with software
• Minimal conditions for consistent variable selection in high dimension
• Robust Variable and Interaction Selection for Logistic Regression and General Index Models
• UPS delivers optimal phase diagram in high-dimensional variable selection
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Discussion: ``A significance test for the lasso
• Covariate assisted screening and estimation
• Testing covariates in high dimension linear regression with latent factors
• Classifier variability: accounting for training and testing
• Selective inference with a randomized response
• Analysis of testing-based forward model selection
• A significance test for the lasso
• Bootstrapping and sample splitting for high-dimensional, assumption-lean inference
• A knockoff filter for high-dimensional selective inference
• Variable screening in predicting clinical outcome with high-dimensional microarrays
• The geometry of least squares in the 21st century
• The adaptive and the thresholded Lasso for potentially misspecified models (and a lower bound for the Lasso)
• Sharp support recovery from noisy random measurements by \(\ell_1\)-minimization
• Endogeneity in high dimensions
• Exact tests via multiple data splitting
• Discussion: ``A significance test for the lasso
• Optimal screening and discovery of sparse signals with applications to multistage high throughput studies
• LOL selection in high dimension
• High-dimensional variable screening and bias in subsequent inference, with an empirical comparison
• ``Preconditioning for feature selection and regression in high-dimensional problems
• Screening-based Bregman divergence estimation with NP-dimensionality
• Thresholding least-squares inference in high-dimensional regression models
• Estimation for high-dimensional linear mixed-effects models using \(\ell_1\)-penalization
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• Honest variable selection in linear and logistic regression models via \(\ell _{1}\) and \(\ell _{1}+\ell _{2}\) penalization
• Statistical learning and selective inference
• Estimating and testing conditional sums of means in high dimensional multivariate binary data
• Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors
• The predictive power of the business and bank sentiment of firms: a high-dimensional Granger causality approach
• Bayesian high-dimensional screening via MCMC
• Estimation and Inference of Heterogeneous Treatment Effects using Random Forests
• High-dimensional regression and variable selection using CAR scores
• Projection-based high-dimensional sign test
• Projection tests for high-dimensional spiked covariance matrices
• Optimality of Graphlet Screening in High Dimensional Variable Selection
• Rejoinder: ``A significance test for the lasso
• Confidence intervals for high-dimensional inverse covariance estimation
• Goodness-of-Fit Tests for High Dimensional Linear Models
• Variable selection after screening: with or without data splitting?
• On asymptotically optimal confidence regions and tests for high-dimensional models
• High-Dimensional Variable Selection for Survival Data
• The benefit of group sparsity in group inference with de-biased scaled group Lasso
• High Dimensional Variable Selection via Tilting
• Two-directional simultaneous inference for high-dimensional models
• High-dimensional inference in misspecified linear models
• Selecting massive variables using an iterated conditional modes/medians algorithm
• Two-stage procedures for high-dimensional data
• Statistical significance in high-dimensional linear models
• Discussion: ``A significance test for the lasso
• False Discovery Rate Control Under General Dependence By Symmetrized Data Aggregation
• Accelerating a Gibbs sampler for variable selection on genomics data with summarization and variable pre-selection combining an array DBMS and R
• Faithful variable screening for high-dimensional convex regression
• Data-driven selection of the number of change-points via error rate control
• High-dimensional simultaneous inference with the bootstrap
• Optimal two-step prediction in regression
• Gaussian Bayesian network comparisons with graph ordering unknown
• Beyond support in two-stage variable selection
• A Dirichlet-Tree Multinomial Regression Model for Associating Dietary Nutrients with Gut Microorganisms
• Selective inference via marginal screening for high dimensional classification
• Covariate-assisted ranking and screening for large-scale two-sample inference
• Principled sure independence screening for Cox models with ultra-high-dimensional covariates
• Derandomizing Knockoffs
• Tests for high-dimensional single-index models
• Controlling the false-discovery rate by procedures adapted to the length bias of RNA-seq
• A global homogeneity test for high-dimensional linear regression
• A new data adaptive elastic net predictive model using hybridized smoothed covariance estimators with information complexity
• Variable selection techniques after multiple imputation in high-dimensional data
• A unified theory of confidence regions and testing for high-dimensional estimating equations
• Projection-based Inference for High-dimensional Linear Models
• Feature screening for network autoregression model
• Integrative analysis and variable selection with multiple high-dimensional data sets
• Convex and non-convex regularization methods for spatial point processes intensity estimation
• Inference under Fine-Gray competing risks model with high-dimensional covariates
• Simultaneous dimension reduction and variable selection in modeling high dimensional data
• Debiasing the Lasso: optimal sample size for Gaussian designs
• Efficient test-based variable selection for high-dimensional linear models
• Detection of gene-gene interactions using multistage sparse and low-rank regression
• Asymptotics for high dimensional regression \(M\)-estimates: fixed design results
• Self-semi-supervised clustering for large scale data with massive null group
• An \(L_1\)-regularized logistic model for detecting short-term neuronal interactions
• Cross-validation with confidence
• Post-model-selection inference in linear regression models: an integrated review
• A Critical Review of LASSO and Its Derivatives for Variable Selection Under Dependence Among Covariates
• Thresholding tests based on affine Lasso to achieve non-asymptotic nominal level and high power under sparse and dense alternatives in high dimension
• Spectral analysis of high-dimensional time series
• Fundamental limits of exact support recovery in high dimensions
• Which bridge estimator is the best for variable selection?
• Tight conditions for consistency of variable selection in the context of high dimensionality
• The Holdout Randomization Test for Feature Selection in Black Box Models
• Robust stability best subset selection for autocorrelated data based on robust location and dispersion estimator

Uses Software

• TETRAD

This page was built for publication: High-dimensional variable selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q834336)