Bayesian high-dimensional screening via MCMC

DOI10.1016/J.JSPI.2014.07.002MaRDI QIDQ466528zbMATHOpenAlexFDO

Authors Zuofeng Shang, Ping Li

Publication date 27 October 2014

Published in Journal of Statistical Planning and Inference (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1302.1154

zbMATH Keywords

model selectionhigh-dimensionalityposterior consistencyconstrained blockwise Gibbs samplerfully Bayesian methodgeneralized hyper-\(g\) priorgeneralized Zellner-Siow priorsize-control prior on model space

Mathematics Subject Classification ID

Bayesian inference (62F15) Linear regression; mixed models (62J05)

Abstract: We explore the theoretical and numerical property of a fully Bayesian model selection method in sparse ultrahigh-dimensional settings, i.e., $pggn$, where $p$ is the number of covariates and $n$ is the sample size. Our method consists of (1) a hierarchical Bayesian model with a novel prior placed over the model space which includes a hyperparameter $t_n$ controlling the model size, and (2) an efficient MCMC algorithm for automatic and stochastic search of the models. Our theory shows that, when specifying $t_n$ correctly, the proposed method yields selection consistency, i.e., the posterior probability of the true model asymptotically approaches one; when $t_n$ is misspecified, the selected model is still asymptotically nested in the true model. The theory also reveals insensitivity of the selection result with respect to the choice of $t_n$. In implementations, a reasonable prior is further assumed on $t_n$ which allows us to draw its samples stochastically. Our approach conducts selection, estimation and even inference in a unified framework. No additional prescreening or dimension reduction step is needed. Two novel $g$-priors are proposed to make our approach more flexible. A simulation study is given to display the numerical advantage of our method.

Recommendations

• Bayesian model selection in high-dimensional settings
• Bayesian estimation in a high dimensional parameter framework
• Model Based Screening Embedded Bayesian Variable Selection for Ultra-high Dimensional Settings
• Bayesian model selection for high-dimensional data
• High-dimensional Bayesian inference via the unadjusted Langevin algorithm
• BAYESIAN DYNAMIC VARIABLE SELECTION IN HIGH DIMENSIONS
• Selective inference via marginal screening for high dimensional classification
• On numerical aspects of Bayesian model selection in high and ultrahigh-dimensional settings
• High-Dimensional Filtering Using Nested Sequential Monte Carlo
• On the computational complexity of high-dimensional Bayesian variable selection

Cites work

• scientific article; zbMATH DE number 5819433 (Why is no real title available?)
• scientific article; zbMATH DE number 5957408 (Why is no real title available?)
• scientific article; zbMATH DE number 3716572 (Why is no real title available?)
• scientific article; zbMATH DE number 1034042 (Why is no real title available?)
• scientific article; zbMATH DE number 1941665 (Why is no real title available?)
• scientific article; zbMATH DE number 795286 (Why is no real title available?)
• scientific article; zbMATH DE number 5224911 (Why is no real title available?)
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• A unified approach to model selection and sparse recovery using regularized least squares
• Adaptive sampling for Bayesian variable selection
• Approximations and consistency of Bayes factors as model dimension grows
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Bayes Model Averaging with Selection of Regressors
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• Bayesian Variable Selection and Regularization for Time–Frequency Surface Estimation
• Bayesian Wavelet Regression on Curves With Application to a Spectroscopic Calibration Problem
• Bayesian model selection in high-dimensional settings
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• Bayesian variable selection in structured high-dimensional covariate spaces with applications in genomics
• Benchmark priors for Bayesian model averaging.
• Calibration and empirical Bayes variable selection
• Consistency of Bayes factors for intrinsic priors in normal linear models
• Consistency of Bayesian linear model selection with a growing number of parameters
• Consistency of Bayesian procedures for variable selection
• Consistency of objective Bayes factors as the model dimension grows
• Consistency of objective Bayes factors for nonnested linear models and increasing model dimension
• Consistent high-dimensional Bayesian variable selection via penalized credible regions
• Empirical Bayes False Coverage Rate Controlling Confidence Intervals
• Flexible Empirical Bayes Estimation for Wavelets
• High-dimensional generalized linear models and the lasso
• High-dimensional graphs and variable selection with the Lasso
• High-dimensional variable selection
• Lasso-type recovery of sparse representations for high-dimensional data
• Likelihood-based selection and sharp parameter estimation
• Mixtures of g Priors for Bayesian Variable Selection
• Mixtures of \(g\)-priors for Bayesian model averaging with economic applications
• Multiple shrinkage and subset selection in wavelets
• Nonparametric regression using Bayesian variable selection
• Objective Bayesian model selection in Gaussian graphical models
• Sequential Monte Carlo Samplers
• Shrinkage tuning parameter selection with a diverging number of parameters
• Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
• Sure independence screening in generalized linear models with NP-dimensionality
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• The Intrinsic Bayes Factor for Model Selection and Prediction
• The risk inflation criterion for multiple regression
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm

Cited in

(8)

• On numerical aspects of Bayesian model selection in high and ultrahigh-dimensional settings
• Consistent high-dimensional Bayesian variable selection via penalized credible regions
• Bayesian model selection in high-dimensional settings
• High-dimensional Bayesian inference in nonparametric additive models
• Selective inference via marginal screening for high dimensional classification
• Bayesian model selection via composite likelihood for high-dimensional data integration
• Bayesian model selection for high-dimensional data
• Bayesian variable selection with shrinking and diffusing priors

Describes a project that uses

Uses Software

• BayesDA

This page was built for publication: Bayesian high-dimensional screening via MCMC

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q466528)