High-dimensional posterior consistency for hierarchical non-local priors in regression

DOI10.1214/19-BA1154zbMATH Open1437.62253arXiv1709.06607OpenAlexW2962789209WikidataQ128017132 ScholiaQ128017132MaRDI QIDQ2297241FDOQ2297241

Authors: Xuan Cao, Kshitij Khare, Malay Ghosh 

Publication date: 18 February 2020

Published in: Bayesian Analysis (Search for Journal in Brave)

Abstract: The choice of tuning parameters in Bayesian variable selection is a critical problem in modern statistics. In particular, for Bayesian linear regression with non-local priors, the scale parameter in the non-local prior density is an important tuning parameter which reflects the dispersion of the non-local prior density around zero, and implicitly determines the size of the regression coefficients that will be shrunk to zero. Current approaches treat the scale parameter as given, and suggest choices based on prior coverage/asymptotic considerations. In this paper, we consider the fully Bayesian approach introduced in (Wu, 2016) with the pMOM non-local prior and an appropriate Inverse-Gamma prior on the tuning parameter to analyze the underlying theoretical property. Under standard regularity assumptions, we establish strong model selection consistency in a high-dimensional setting, where $p$ is allowed to increase at a polynomial rate with n$or even at a sub-exponential rate with n. Through simulation studies, we demonstrate that our model selection procedure can outperform other Bayesian methods which treat the scale parameter as given, and commonly used penalized likelihood methods, in a range of simulation settings.

Full work available at URL: https://arxiv.org/abs/1709.06607

Recommendations

• Bayesian variable selection with shrinking and diffusing priors
• Scalable Bayesian variable selection using nonlocal prior densities in ultrahigh-dimensional settings
• A novel Bayesian approach for variable selection in linear regression models
• On Model Selection Consistency of Bayesian Method for Normal Linear Models
• Variable selection consistency of Gaussian process regression

zbMATH Keywords

model selectionhigh-dimensional datamultivariate regressionposterior consistencynon-local prior

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07)

Cites Work

• Scalable Bayesian variable selection using nonlocal prior densities in ultrahigh-dimensional settings
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• Mixtures of g Priors for Bayesian Variable Selection
• Title not available (Why is that?)
• Bayesian variable selection with shrinking and diffusing priors
• Spike and slab variable selection: frequentist and Bayesian strategies
• Bayesian model selection in high-dimensional settings
• Efficient Empirical Bayes Variable Selection and Estimation in Linear Models
• Bayesian linear regression with sparse priors
• On the use of Non-Local Prior Densities in Bayesian Hypothesis Tests
• Posterior graph selection and estimation consistency for high-dimensional Bayesian DAG models
• High-dimensional variable selection with reciprocal \(L_{1}\)-regularization

Cited In (13)

• Strong selection consistency of Bayesian vector autoregressive models based on a pseudo-likelihood approach
• Hierarchical shrinkage priors for regression models
• Joint Bayesian Variable and DAG Selection Consistency for High-dimensional Regression Models with Network-structured Covariates
• On the non-local priors for sparsity selection in high-dimensional Gaussian DAG models
• Consistent Bayesian sparsity selection for high-dimensional Gaussian DAG models with multiplicative and beta-mixture priors
• Consistent skinny Gibbs in probit regression
• On the beta prime prior for scale parameters in high-dimensional Bayesian regression models
• Hierarchical shrinkage priors and model fitting for high-dimensional generalized linear models
• Variable selection consistency of Gaussian process regression
• Bayesian Approaches to Shrinkage and Sparse Estimation
• Disjunct support spike-and-slab priors for variable selection in regression under quasi-sparseness
• Development of network-guided transcriptomic risk score for disease prediction
• Consistent group selection using nonlocal priors in regression

Uses Software

• glmnet

This page was built for publication: High-dimensional posterior consistency for hierarchical non-local priors in regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2297241)