Spike and slab variable selection: frequentist and Bayesian strategies

DOI10.1214/009053604000001147MaRDI QIDQ2388355zbMATH OpenOpenAlexFDO

Authors Hemant Ishwaran, J. Sunil Rao

Publication date 12 September 2005

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/math/0505633

zbMATH Keywords

diabetesgeneralized ridge regressionhypervariancemodel averagingmodel uncertaintyordinary least squarespenalizationrescalingshrinkagestochastic variable selectionZcut

Mathematics Subject Classification ID

Bayesian inference (62F15) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07) Applications of statistics to biology and medical sciences; meta analysis (62P10)

Abstract: Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. In this paper we introduce a variable selection method referred to as a rescaled spike and slab model. We study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefulness of continuous bimodal priors to model hypervariance parameters, and the effect scaling has on the posterior mean through its relationship to penalization. Several model selection strategies, some frequentist and some Bayesian in nature, are developed and studied theoretically. We demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification, and show this is achieved using the posterior from a rescaled spike and slab model. We also show how to verify a procedure's ability to reduce model uncertainty in finite samples using a specialized forward selection strategy. Using this tool, we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty.

Recommendations

• Consistency of spike and slab regression
• Bayesian Variable Selection in Linear Regression
• A variational Bayes approach to variable selection
• Bayesian variable selection with shrinking and diffusing priors
• Fast Bayesian variable selection for high dimensional linear models: marginal solo spike and slab priors

Cites work

• A strongly consistent procedure for model selection in a regression problem
• Asymptotics for Lasso-type estimators.
• Asymptotics in statistics: some basic concepts
• Bayesian Variable Selection in Linear Regression
• Bayesian variable selection with related predictors
• Bootstrap Model Selection
• Consistent Variable Selection in Linear Models
• Detecting Differentially Expressed Genes in Microarrays Using Bayesian Model Selection
• scientific article; zbMATH DE number 1271147 (Why is no real title available?)
• scientific article; zbMATH DE number 1034037 (Why is no real title available?)
• scientific article; zbMATH DE number 1034040 (Why is no real title available?)
• scientific article; zbMATH DE number 3390111 (Why is no real title available?)
• Least angle regression. (With discussion)
• Linear Model Selection by Cross-Validation
• Minimax multiple shrinkage estimation
• Model selection via multifold cross validation
• Multiple shrinkage and subset selection in wavelets
• On the Distributional Properties of Model Selection Criteria
• Optimal predictive model selection.
• Prediction Via Orthogonalized Model Mixing
• Ridge Regression: Biased Estimation for Nonorthogonal Problems
• Spike and Slab Gene Selection for Multigroup Microarray Data
• THE FINITE-SAMPLE DISTRIBUTION OF POST-MODEL-SELECTION ESTIMATORS AND UNIFORM VERSUS NONUNIFORM APPROXIMATIONS
• The GIC for model selection: A hypothesis testing approach
• The Little Bootstrap and Other Methods for Dimensionality Selection in Regression: X-Fixed Prediction Error
• Variable selection for regression models
• Variable Selection in Nonparametric Regression with Categorical Covariates

Cited in

(only showing first 100 items - show all)

• Bayesian variable selection with shrinking and diffusing priors
• P-splines with an \(\ell_1\) penalty for repeated measures
• Nonparametric Bayesian learning of heterogeneous dynamic transcription factor networks
• Penalized likelihood and Bayesian function selection in regression models
• Bayesian accelerated failure time models based on penalized mixtures of Gaussians: regularization and variable selection
• Prior distributions for objective Bayesian analysis
• Variable selection via penalized credible regions with Dirichlet-Laplace global-local shrinkage priors
• High-dimensional multivariate posterior consistency under global-local shrinkage priors
• Robust and parallel Bayesian model selection
• Fast Bayesian variable selection for high dimensional linear models: marginal solo spike and slab priors
• Model selection using mass-nonlocal prior
• Bayesian model selection for generalized linear models using non-local priors
• Bayesian effect fusion for categorical predictors
• Bayesian curve fitting and clustering with Dirichlet process mixture models for microarray data
• Bayesian estimation of sparse signals with a continuous spike-and-slab prior
• Penalising model component complexity: a principled, practical approach to constructing priors
• Variable selection for partially linear models via Bayesian subset modeling with diffusing prior
• Using favorite data to analyze asymmetric competition: machine learning models
• High-dimensional index volatility models via Stein's identity
• Bayesian effect selection in structured additive distributional regression models
• Theory of optimal Bayesian feature filtering
• Dynamic variable selection with spike-and-slab process priors
• Joint Bayesian estimation of voxel activation and inter-regional connectivity in fMRI experiments
• Mapping unobserved item-respondent interactions: a latent space item response model with interaction map
• The illusion of the illusion of sparsity: an exercise in prior sensitivity
• RADIOHEAD: radiogenomic analysis incorporating tumor heterogeneity in imaging through densities
• A Bayesian time-varying effect model for behavioral mHealth data
• Sparse linear mixed model selection via streamlined variational Bayes
• Sticky PDMP samplers for sparse and local inference problems
• How to go viral: a COVID-19 model with endogenously time-varying parameters
• On minimax optimality of sparse Bayes predictive density estimates
• Bayesian empirical likelihood inference and order shrinkage for autoregressive models
• Bayes factor asymptotics for variable selection in the Gaussian process framework
• Bayesian hierarchical modeling: application towards production results in the Eagle Ford Shale of South Texas
• Stochastic zeroth-order discretizations of Langevin diffusions for Bayesian inference
• Composite mixture of log-linear models with application to psychiatric studies
• Horseshoe shrinkage methods for Bayesian fusion estimation
• Bayesian-EUCLID: discovering hyperelastic material laws with uncertainties
• Spatial generalized linear models with non-Gaussian translation processes
• Projective inference in high-dimensional problems: prediction and feature selection
• Relevant parameter changes in structural break models
• Bayesian fusion estimation via \(t\) shrinkage
• Latent theme dictionary model for finding co-occurrent patterns in process data
• Joint modeling of longitudinal relational data and exogenous variables
• Consistent group selection with Bayesian high dimensional modeling
• Asymptotic theory of dependent Bayesian multiple testing procedures under possible model misspecification
• Discovering interactions using covariate informed random partition models
• BayICE: a Bayesian hierarchical model for semireference-based deconvolution of bulk transcriptomic data
• Bayesian matrix completion approach to causal inference with panel data
• Bayesian model selection for high-dimensional Ising models, with applications to educational data
• Shrinkage priors for single-spiked covariance models
• What belongs where? Variable selection for zero-inflated count models with an application to the demand for health care
• A robust estimation for the extended \(t\)-process regression model
• Variance prior forms for high-dimensional Bayesian variable selection
• Lasso meets horseshoe: a survey
• High-dimensional posterior consistency for hierarchical non-local priors in regression
• Variable selection with spatially autoregressive errors: a generalized moments Lasso estimator
• Contraction properties of shrinkage priors in logistic regression
• Bayesian discriminant analysis using a high dimensional predictor
• A Bayesian nonparametric multiple testing procedure for comparing several treatments against a control
• High-dimensional confounding adjustment using continuous Spike and Slab priors
• Comment: ``Bayes, oracle Bayes and empirical Bayes
• Estimating the cognitive diagnosis \(Q\) matrix with expert knowledge: application to the fraction-subtraction dataset
• Shrinkage priors for Bayesian penalized regression
• Bayesian variable selection with sparse and correlation priors for high-dimensional data analysis
• A graph Laplacian prior for Bayesian variable selection and grouping
• Negotiating multicollinearity with spike-and-slab priors
• Nearly optimal Bayesian shrinkage for high-dimensional regression
• Estimating promotion effects in email marketing using a large-scale cross-classified Bayesian joint model for nested imbalanced data
• Combining a relaxed EM algorithm with Occam's razor for Bayesian variable selection in high-dimensional regression
• Statistical inference via conditional Bayesian posteriors in high-dimensional linear regression
• Modeling time-varying parameters using artificial neural networks: a GARCH illustration
• Sparse estimation: an MMSE approach
• Bayesian hyper-Lassos with non-convex penalization
• Bayesian spectral analysis models for quantile regression with Dirichlet process mixtures
• Bayesian treatment effects models with variable selection for panel outcomes with an application to earnings effects of maternity leave
• Iterative selection using orthogonal regression techniques
• Geoadditive latent variable modeling of count data on multiple sexual partnering in Nigeria
• Particle EM for variable selection
• Wavelet-based functional linear mixed models: an application to measurement error-corrected distributed lag models
• A Bayesian Approach to Multistate Hidden Markov Models: Application to Dementia Progression
• Two-Stage Bayesian Approach for GWAS With Known Genealogy
• Simultaneous Credible Regions for Multiple Changepoint Locations
• AN IN-DEPTH LOOK AT HIGHEST POSTERIOR MODEL SELECTION
• A tutorial on Bayesian nonparametric models
• Bayesian model selection for logistic regression models with random intercept
• Sparsity constrained estimation in image processing and computer vision
• Bayesian nonparametric modeling for functional analysis of variance
• Learning from a lot: empirical Bayes for high-dimensional model-based prediction
• Correlated random measures
• The variational Garrote
• Variable selection for BART: an application to gene regulation
• Spike-and-slab priors for function selection in structured additive regression models
• Selecting massive variables using an iterated conditional modes/medians algorithm
• Expectation propagation in linear regression models with spike-and-slab priors
• Cross‐validation and peeling strategies for survival bump hunting using recursive peeling methods
• Sparse Bayesian modelling of underreported count data
• EMVS: the EM approach to Bayesian variable selection
• Fully Bayesian inference under ignorable missingness in the presence of auxiliary covariates
• From mixed effects modeling to spike and slab variable selection: A Bayesian regression model for group testing data

This page was built for publication: Spike and slab variable selection: frequentist and Bayesian strategies

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2388355)