Finite sample posterior concentration in high-dimensional regression

DOI10.1093/IMAIAI/IAU003MaRDI QIDQ5177865zbMATH OpenOpenAlexFDO

Authors N. Strawn, Artin Armagan, Rayan Saab, Lawrence Carin, David Dunson

Publication date 6 March 2015

Published in Information and Inference: A Journal of the IMA (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1207.4854

zbMATH Keywords

asymptoticsBayesiancompressible priorhigh-dimensionalposterior contractionregressionshrinkage prior

Mathematics Subject Classification ID

Estimation in multivariate analysis (62H12) Linear regression; mixed models (62J05) Ridge regression; shrinkage estimators (Lasso) (62J07)

Abstract: We study the behavior of the posterior distribution in high-dimensional Bayesian Gaussian linear regression models having $pggn$, with $p$ the number of predictors and $n$ the sample size. Our focus is on obtaining quantitative finite sample bounds ensuring sufficient posterior probability assigned in neighborhoods of the true regression coefficient vector, $$, with high probability. We assume that $$ is approximately $S$-sparse and obtain universal bounds, which provide insight into the role of the prior in controlling concentration of the posterior. Based on these finite sample bounds, we examine the implied asymptotic contraction rates for several examples showing that sparsely-structured and heavy-tail shrinkage priors exhibit rapid contraction rates. We also demonstrate that a stronger result holds for the Uniform-Gaussianfootnote[2]{A binary vector of indicators ($gamma$) is drawn from the uniform distribution on the set of binary sequences with exactly $S$ ones, and then each $$ if $gamm{a}_i=1$ and $$ if $gamm{a}_i=0$.} prior. These types of finite sample bounds provide guidelines for designing and evaluating priors for high-dimensional problems.

Recommendations

• Bayesian linear regression with sparse priors
• High-dimensional multivariate posterior consistency under global-local shrinkage priors
• Posterior consistency in linear models under shrinkage priors
• Empirical Bayes posterior concentration in sparse high-dimensional linear models
• Ultra high-dimensional multivariate posterior contraction rate under shrinkage priors

Cited in

(10)

• High-dimensional multivariate posterior consistency under global-local shrinkage priors
• On the contraction properties of some high-dimensional quasi-posterior distributions
• Unified Bayesian theory of sparse linear regression with nuisance parameters
• Bayesian high-dimensional semi-parametric inference beyond sub-Gaussian errors
• Posterior consistency in linear models under shrinkage priors
• Impacts of high dimensionality in finite samples
• Bayesian regression based on principal components for high-dimensional data
• Prior influence in linear regression when the number of covariates increases to infinity
• On the beta prime prior for scale parameters in high-dimensional Bayesian regression models
• Bayesian linear regression with sparse priors

This page was built for publication: Finite sample posterior concentration in high-dimensional regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5177865)