An MCMC approach to empirical Bayes inference and Bayesian sensitivity analysis via empirical processes

DOI10.1214/17-AOS1597MaRDI QIDQ2413604zbMATH OpenOpenAlexWikidataFDO

Authors Hani Doss, Yeon Hee Park

Publication date 14 September 2018

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1807.02191

zbMATH Keywords

Donsker classgeometric ergodicityhyperparameter selectionlatent Dirichlet allocation modelregenerative simulation

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Computational methods in Markov chains (60J22) Empirical decision procedures; empirical Bayes procedures (62C12) Asymptotic properties of parametric estimators (62F12) Bayesian inference (62F15)

Abstract: Consider a Bayesian situation in which we observe $Ysim{p}_{heta}$, where $hetainTheta$, and we have a family $u_h,,hinmathcalH$ of potential prior distributions on $Theta$. Let $g$ be a real-valued function of $heta$, and let $I_g(h)$ be the posterior expectation of $g(heta)$ when the prior is $u_h$. We are interested in two problems: (i) selecting a particular value of $h$, and (ii) estimating the family of posterior expectations $I_g(h),,hinmathcalH$. Let $m_y(h)$ be the marginal likelihood of the hyperparameter $h$: $m_y(h)=int{p}_{heta}(y),u_h(dheta)$. The empirical Bayes estimate of $h$ is, by definition, the value of $h$ that maximizes $m_y(h)$. It turns out that it is typically possible to use Markov chain Monte Carlo to form point estimates for $m_y(h)$ and $I_g(h)$ for each individual $h$ in a continuum, and also confidence intervals for $m_y(h)$ and $I_g(h)$ that are valid pointwise. However, we are interested in forming estimates, with confidence statements, of the entire families of integrals $m_y(h),,hinmathcalH$ and $I_g(h),,hinmathcalH$: we need estimates of the first family in order to carry out empirical Bayes inference, and we need estimates of the second family in order to do Bayesian sensitivity analysis. We establish strong consistency and functional central limit theorems for estimates of these families by using tools from empirical process theory. We give two applications, one to Latent Dirichlet Allocation, which is used in topic modelling, and the other is to a model for Bayesian variable selection in linear regression.

Recommendations

• Computational approaches for empirical Bayes methods and Bayesian sensitivity analysis
• A computational framework for empirical Bayes inference
• Estimation of large families of Bayes factors from Markov chain output
• Bayes and empirical Bayes: do they merge?
• Posterior concentration rates for empirical Bayes procedures with applications to Dirichlet process mixtures

Cites work

• 10.1162/jmlr.2003.3.4-5.993
• -stable limit laws for harmonic mean estimators of marginal likelihoods
• An overview of robust Bayesian analysis. (With discussion)
• Annealing Markov Chain Monte Carlo with Applications to Ancestral Inference
• Batch means and spectral variance estimators in Markov chain Monte Carlo
• Bayes and empirical Bayes: do they merge?
• Bayesian Variable Selection in Linear Regression
• Calibration and empirical Bayes variable selection
• Computational approaches for empirical Bayes methods and Bayesian sensitivity analysis
• Convergence Rates and Asymptotic Standard Errors for Markov Chain Monte Carlo Algorithms for Bayesian Probit Regression
• Editorial: Bayesian model selection: some thoughts on future directions
• Estimates and Standard Errors for Ratios Of Normalizing Constants from Multiple Markov Chains Via Regeneration
• Fixed-Width Output Analysis for Markov Chain Monte Carlo
• scientific article; zbMATH DE number 5954262 (Why is no real title available?)
• scientific article; zbMATH DE number 4070082 (Why is no real title available?)
• scientific article; zbMATH DE number 509150 (Why is no real title available?)
• scientific article; zbMATH DE number 3349105 (Why is no real title available?)
• Markov chain Monte Carlo estimation of quantiles
• Markov chain Monte Carlo: can we trust the third significant figure?
• Markov chains and stochastic stability
• Mixtures of g Priors for Bayesian Variable Selection
• Monte Carlo likelihood inference for missing data models
• Nonparametric regression using Bayesian variable selection
• On the applicability of regenerative simulation in Markov chain Monte Carlo
• Regeneration in Markov Chain Samplers
• Semiparametric Regression
• Uniform limit theorems for Harris recurrent Markov chains

Cited in

(6)

• Estimation of large families of Bayes factors from Markov chain output
• Principled selection of hyperparameters in the latent Dirichlet allocation model
• A computational framework for empirical Bayes inference
• Computational approaches for empirical Bayes methods and Bayesian sensitivity analysis
• Scalable empirical Bayes inference and Bayesian sensitivity analysis
• From EM to data augmentation: the emergence of MCMC Bayesian computation in the 1980s

Describes a project that uses

Uses Software

• bernor
• gensim
• SemiPar

This page was built for publication: An MCMC approach to empirical Bayes inference and Bayesian sensitivity analysis via empirical processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2413604)