Bayesian high-dimensional screening via MCMC

DOI10.1016/J.JSPI.2014.07.002zbMATH Open1307.62075arXiv1302.1154OpenAlexW2016133507MaRDI QIDQ466528FDOQ466528

Authors: Zuofeng Shang, Ping Li 

Publication date: 27 October 2014

Published in: Journal of Statistical Planning and Inference (Search for Journal in Brave)

Abstract: We explore the theoretical and numerical property of a fully Bayesian model selection method in sparse ultrahigh-dimensional settings, i.e., $pggn$, where $p$ is the number of covariates and $n$ is the sample size. Our method consists of (1) a hierarchical Bayesian model with a novel prior placed over the model space which includes a hyperparameter $t_n$ controlling the model size, and (2) an efficient MCMC algorithm for automatic and stochastic search of the models. Our theory shows that, when specifying $t_n$ correctly, the proposed method yields selection consistency, i.e., the posterior probability of the true model asymptotically approaches one; when $t_n$ is misspecified, the selected model is still asymptotically nested in the true model. The theory also reveals insensitivity of the selection result with respect to the choice of $t_n$. In implementations, a reasonable prior is further assumed on $t_n$ which allows us to draw its samples stochastically. Our approach conducts selection, estimation and even inference in a unified framework. No additional prescreening or dimension reduction step is needed. Two novel $g$-priors are proposed to make our approach more flexible. A simulation study is given to display the numerical advantage of our method.

Full work available at URL: https://arxiv.org/abs/1302.1154

Recommendations

• Bayesian model selection in high-dimensional settings
• Bayesian estimation in a high dimensional parameter framework
• Model Based Screening Embedded Bayesian Variable Selection for Ultra-high Dimensional Settings
• Bayesian model selection for high-dimensional data
• High-dimensional Bayesian inference via the unadjusted Langevin algorithm
• BAYESIAN DYNAMIC VARIABLE SELECTION IN HIGH DIMENSIONS
• Selective inference via marginal screening for high dimensional classification
• On numerical aspects of Bayesian model selection in high and ultrahigh-dimensional settings
• High-Dimensional Filtering Using Nested Sequential Monte Carlo
• On the computational complexity of high-dimensional Bayesian variable selection

zbMATH Keywords

model selectionhigh-dimensionalityposterior consistencyconstrained blockwise Gibbs samplerfully Bayesian methodgeneralized hyper-\(g\) priorgeneralized Zellner-Siow priorsize-control prior on model space

Mathematics Subject Classification ID

Bayesian inference (62F15) Linear regression; mixed models (62J05)

Cites Work

• Variable selection in high-dimensional linear models: partially faithful distributions and the PC-simple algorithm
• Sure independence screening in generalized linear models with NP-dimensionality
• A unified approach to model selection and sparse recovery using regularized least squares
• Sequential Monte Carlo Samplers
• Lasso-type recovery of sparse representations for high-dimensional data
• High-dimensional generalized linear models and the lasso
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• High-dimensional graphs and variable selection with the Lasso
• Title not available (Why is that?)
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• Nonparametric regression using Bayesian variable selection
• Approximations and consistency of Bayes factors as model dimension grows
• Shrinkage tuning parameter selection with a diverging number of parameters
• A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)
• Mixtures of g Priors for Bayesian Variable Selection
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Bayesian Variable Selection and Regularization for Time–Frequency Surface Estimation
• Title not available (Why is that?)
• Bayes and empirical-Bayes multiplicity adjustment in the variable-selection problem
• High-dimensional variable selection
• The sparsity and bias of the LASSO selection in high-dimensional linear regression
• The risk inflation criterion for multiple regression
• Bayesian variable selection for high dimensional generalized linear models: convergence rates of the fitted densities
• Calibration and empirical Bayes variable selection
• The Intrinsic Bayes Factor for Model Selection and Prediction
• Consistent high-dimensional Bayesian variable selection via penalized credible regions
• Likelihood-Based Selection and Sharp Parameter Estimation
• Bayesian Model Selection in High-Dimensional Settings
• Consistency of objective Bayes factors as the model dimension grows
• Asymptotic properties of bridge estimators in sparse high-dimensional regression models
• Bayes Model Averaging with Selection of Regressors
• Benchmark priors for Bayesian model averaging.
• Consistency of Bayesian procedures for variable selection
• Consistency of Bayesian linear model selection with a growing number of parameters
• Consistency of Bayes factors for intrinsic priors in normal linear models
• Multiple shrinkage and subset selection in wavelets
• Flexible Empirical Bayes Estimation for Wavelets
• Mixtures of \(g\)-priors for Bayesian model averaging with economic applications
• Consistency of objective Bayes factors for nonnested linear models and increasing model dimension
• Objective Bayesian model selection in Gaussian graphical models
• Adaptive sampling for Bayesian variable selection
• Title not available (Why is that?)
• Bayesian Variable Selection in Structured High-Dimensional Covariate Spaces With Applications in Genomics
• Bayesian Wavelet Regression on Curves With Application to a Spectroscopic Calibration Problem
• Title not available (Why is that?)
• Empirical Bayes False Coverage Rate Controlling Confidence Intervals

Cited In (4)

• Consistent high-dimensional Bayesian variable selection via penalized credible regions
• Bayesian model selection via composite likelihood for high-dimensional data integration
• High-dimensional Bayesian inference in nonparametric additive models
• Selective inference via marginal screening for high dimensional classification

Uses Software

• BayesDA

This page was built for publication: Bayesian high-dimensional screening via MCMC

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q466528)