Bayesian variable selection with shrinking and diffusing priors
From MaRDI portal
Abstract: We consider a Bayesian approach to variable selection in the presence of high dimensional covariates based on a hierarchical model that places prior distributions on the regression coefficients as well as on the model space. We adopt the well-known spike and slab Gaussian priors with a distinct feature, that is, the prior variances depend on the sample size through which appropriate shrinkage can be achieved. We show the strong selection consistency of the proposed method in the sense that the posterior probability of the true model converges to one even when the number of covariates grows nearly exponentially with the sample size. This is arguably the strongest selection consistency result that has been available in the Bayesian variable selection literature; yet the proposed method can be carried out through posterior sampling with a simple Gibbs sampler. Furthermore, we argue that the proposed method is asymptotically similar to model selection with the penalty. We also demonstrate through empirical work the fine performance of the proposed approach relative to some state of the art alternatives.
The authors are using the linear regression setup with high dimensional covariates where the number of covariates can be large relative to the sample size. Only a small number of covariates, called active covariates, have nonzero coefficients. The aim of this paper is to develop a new Bayesian methodology for selecting the active covariates that is asymptotically consistent and computationally convenient. A common notion of consistency for Bayesian variable selection is defined in terms of pairwise Bayes factors. Another notion of consistency for both frequentist and Bayesian methods is that the selected model equals the true model with probability converging to one. It is shown that the strong selection consistency of the proposed method in the sense that the posterior probability of the true model converges to one even when the number of covariates grows nearly exponentially with the sample size. We find important contributions to variable selection in this article. The conditions on the prior parameters and motivation for these conditions are given. The last part of the paper provides a discussion on the conditions assumed for proving the previous results. Some computational aspects of the proposed method, as well as simulation studies regarding the comparing of the proposed method with some existing methods are presented. At the end of this important paper we find also 35 references.
Recommendations
- Fast Bayesian variable selection for high dimensional linear models: marginal solo spike and slab priors
- Bayesian high-dimensional screening via MCMC
- Variable selection using shrinkage priors
- On Model Selection Consistency of Bayesian Method for Normal Linear Models
- A variational Bayes approach to variable selection
Cites work
- A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
- A tail inequality for quadratic forms of subgaussian random vectors
- AN IN-DEPTH LOOK AT HIGHEST POSTERIOR MODEL SELECTION
- Asymptotic oracle properties of SCAD-penalized least squares estimators
- Bayesian empirical likelihood for quantile regression
- Bayesian model selection in high-dimensional settings
- Bayesian Subset Modeling for High-Dimensional Generalized Linear Models
- Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
- Bayesian Variable Selection in Linear Regression
- Bayesian variable selection with shrinking and diffusing priors
- Calibration and empirical Bayes variable selection
- Consistency of objective Bayes factors as the model dimension grows
- Consistency of spike and slab regression
- Consistent high-dimensional Bayesian variable selection via penalized credible regions
- Consistent model selection criteria on high dimensions
- DASSO: Connections Between the Dantzig Selector and Lasso
- Efficient Empirical Bayes Variable Selection and Estimation in Linear Models
- Estimating the dimension of a model
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 6162361 (Why is no real title available?)
- Likelihood-based selection and sharp parameter estimation
- Nonconcave penalized likelihood with a diverging number of parameters.
- Optimal predictive model selection.
- Penalized orthogonal-components regression for large \(p\) small \(n\) data
- Simultaneous Regression Shrinkage, Variable Selection, and Supervised Clustering of Predictors with OSCAR
- Spike and slab variable selection: frequentist and Bayesian strategies
- The Adaptive Lasso and Its Oracle Properties
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(only showing first 100 items - show all)- Bayesian variable selection with shrinking and diffusing priors
- basad
- Bayesian variable selection for correlated covariates via colored cliques
- Variable selection via penalized credible regions with Dirichlet-Laplace global-local shrinkage priors
- Variable selection for high-dimensional genomic data with censored outcomes using group Lasso prior
- Bayesian variable selection with strong heredity constraints
- Variable selection using shrinkage priors
- High-dimensional multivariate posterior consistency under global-local shrinkage priors
- Fast Bayesian variable selection for high dimensional linear models: marginal solo spike and slab priors
- Model selection using mass-nonlocal prior
- Posterior graph selection and estimation consistency for high-dimensional Bayesian DAG models
- Bayesian estimation of sparse signals with a continuous spike-and-slab prior
- Variable selection for partially linear models via Bayesian subset modeling with diffusing prior
- Unified Bayesian theory of sparse linear regression with nuisance parameters
- Variable selection consistency of Gaussian process regression
- Dynamic variable selection with spike-and-slab process priors
- Adaptive random neighbourhood informed Markov chain Monte Carlo for high-dimensional Bayesian variable selection
- Bayesian empirical likelihood inference and order shrinkage for autoregressive models
- Bayesian high-dimensional semi-parametric inference beyond sub-Gaussian errors
- Bayesian penalized Buckley-James method for high dimensional bivariate censored regression models
- Bayesian group selection with non-local priors
- Objective Bayesian edge screening and structure selection for Ising networks
- A generalized likelihood-based Bayesian approach for scalable joint regression and covariance selection in high dimensions
- Random weighting in LASSO regression
- Bayesian factor-adjusted sparse regression
- Projective inference in high-dimensional problems: prediction and feature selection
- Consistent Bayesian sparsity selection for high-dimensional Gaussian DAG models with multiplicative and beta-mixture priors
- Bayesian fusion estimation via \(t\) shrinkage
- Consistent group selection with Bayesian high dimensional modeling
- Asymptotic theory of dependent Bayesian multiple testing procedures under possible model misspecification
- Maximum pairwise Bayes factors for covariance structure testing
- Quasi-Bayesian estimation of large Gaussian graphical models
- Minimax posterior convergence rates and model selection consistency in high-dimensional DAG models based on sparse Cholesky factors
- Variance prior forms for high-dimensional Bayesian variable selection
- A novel Bayesian approach for variable selection in linear regression models
- High-dimensional posterior consistency for hierarchical non-local priors in regression
- Bayesian estimation of large precision matrix based on Cholesky decomposition
- Subjective Bayesian testing using calibrated prior probabilities
- Comparison of Bayesian predictive methods for model selection
- Spike and slab variable selection: frequentist and Bayesian strategies
- Bayesian projection approaches to variable selection in generalized linear models
- Posterior model consistency in variable selection as the model dimension grows
- Nearly optimal Bayesian shrinkage for high-dimensional regression
- Additive Bayesian variable selection under censoring and misspecification
- Combining a relaxed EM algorithm with Occam's razor for Bayesian variable selection in high-dimensional regression
- Bayesian empirical likelihood of quantile regression with missing observations
- High-dimensional posterior consistency of the Bayesian Lasso
- Robust Bayesian variable selection in linear models with spherically symmetric errors
- Laplace approximation in high-dimensional Bayesian regression
- Particle EM for variable selection
- Tractable Bayesian variable selection: beyond normality
- Variable selection methods for penalized credible regions based on the Horseshoe+ prior
- On the Consistency of Bayesian Variable Selection for High Dimensional Binary Regression and Classification
- Bayesian constrained variable selection
- Bayesian regression based on principal components for high-dimensional data
- Prior influence in linear regression when the number of covariates increases to infinity
- Scalable Bayesian variable selection using nonlocal prior densities in ultrahigh-dimensional settings
- Consistent high-dimensional Bayesian variable selection via penalized credible regions
- Bayesian model selection in high-dimensional settings
- Bayesian model selection with graph structured sparsity
- Pruning variable selection ensembles
- Bayesian variable selection with spherically symmetric priors
- Bayes variable selection in semiparametric linear models
- Bayesian variable selection for multioutcome models through shared shrinkage
- On the beta prime prior for scale parameters in high-dimensional Bayesian regression models
- Bayesian inference in high-dimensional linear models using an empirical correlation-adaptive prior
- Bayesian Estimation of Gaussian Conditional Random Fields
- Bayesian Approaches to Shrinkage and Sparse Estimation
- On the computational complexity of high-dimensional Bayesian variable selection
- Bayesian model selection for high-dimensional data
- Targeted random projection for prediction from high-dimensional features
- Approximate spectral gaps for Markov chain mixing times in high dimensions
- Joint Bayesian Variable and DAG Selection Consistency for High-dimensional Regression Models with Network-structured Covariates
- Probabilistic Community Detection With Unknown Number of Communities
- Scalable Importance Tempering and Bayesian Variable Selection
- Skinny Gibbs: a consistent and scalable Gibbs sampler for model selection
- Bayesian regularization for graphical models with unequal shrinkage
- Bayesian variable selection in structured high-dimensional covariate spaces with applications in genomics
- Multivariate Bayesian variable selection exploiting dependence structure among outcomes: application to air pollution effects on DNA methylation
- Decoupling shrinkage and selection in Bayesian linear models: a posterior summary perspective
- A split-and-merge Bayesian variable selection approach for ultrahigh dimensional regression
- Bayesian inference for high-dimensional linear regression under mnet priors
- On the non-local priors for sparsity selection in high-dimensional Gaussian DAG models
- Bayesian Edge Regression in Undirected Graphical Models to Characterize Interpatient Heterogeneity in Cancer
- Bayesian Regression Using a Prior on the Model Fit: The R2-D2 Shrinkage Prior
- Bayesian variable selection for linear regression with the κ-G priors
- Bayesian group selection in logistic regression with application to MRI data analysis
- Identifying regions of inhomogeneities in spatial processes via an M‐RA and mixture priors
- The EAS approach for graphical selection consistency in vector autoregression models
- Weighted Bayesian bootstrap for scalable posterior distributions
- Discussion
- A Bayesian Subset Specific Approach to Joint Selection of Multiple Graphical Models
- On Joint Estimation of Gaussian Graphical Models for Spatial and Temporal Data
- BAYESIAN DYNAMIC VARIABLE SELECTION IN HIGH DIMENSIONS
- Bayesian analysis in single-index quantile regression with missing observation
- Variable Selection Via Thompson Sampling
- Neuronized Priors for Bayesian Sparse Linear Regression
- A latent slice sampling algorithm
- An ensemble EM algorithm for Bayesian variable selection
- A robust Bayesian analysis of variable selection under prior ignorance
This page was built for publication: Bayesian variable selection with shrinking and diffusing priors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q118687)
