Fast and accurate estimation of non-nested binomial hierarchical models using variational inference

DOI10.1214/21-BA1266arXiv2007.12300OpenAlexW3158248884MaRDI QIDQ6202924FDOQ6202924

Authors: Max Goplerud 

Publication date: 27 February 2024

Published in: Bayesian Analysis (Search for Journal in Brave)

Abstract: Non-linear hierarchical models are commonly used in many disciplines. However, inference in the presence of non-nested effects and on large datasets is challenging and computationally burdensome. This paper provides two contributions to scalable and accurate inference. First, I derive a new mean-field variational algorithm for estimating binomial logistic hierarchical models with an arbitrary number of non-nested random effects. Second, I propose "marginally augmented variational Bayes" (MAVB) that further improves the initial approximation through a step of Bayesian post-processing. I prove that MAVB provides a guaranteed improvement in the approximation quality at low computational cost and induces dependencies that were assumed away by the initial factorization assumptions. I apply these techniques to a study of voter behavior using a high-dimensional application of the popular approach of multilevel regression and post-stratification (MRP). Existing estimation took hours whereas the algorithms proposed run in minutes. The posterior means are well-recovered even under strong factorization assumptions. Applying MAVB further improves the approximation by partially correcting the under-estimated variance. The proposed methodology is implemented in an open source software package.

Full work available at URL: https://arxiv.org/abs/2007.12300

zbMATH Keywords

variational Bayeshierarchical modelsmarginal augmentationscalable statistical methodology

Mathematics Subject Classification ID

Statistics (62-XX)

Cites Work

• Automatic differentiation variational inference
• Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
• Understanding predictive information criteria for Bayesian models
• Generalized multilevel structural equation modeling
• Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
• Parameter expansion to accelerate EM: the PX-EM algorithm
• Parameter Expansion for Data Augmentation
• Super Learner
• Simple marginally noninformative prior distributions for covariance matrices
• General design Bayesian generalized linear mixed models
• Variational inference for generalized linear mixed models using partially noncentered parametrizations
• Variational Bayesian Inference for Parametric and Nonparametric Regression With Missing Data
• Theory of Gaussian variational approximation for a Poisson mixed model
• Multilevel and longitudinal modeling using Stata
• A variational maximization-maximization algorithm for generalized linear mixed models with crossed random effects
• Use of Model Reparametrization to Improve Variational Bayes
• Fast and accurate binary response mixed model analysis via expectation propagation

Cited In (2)

• Scalable Bayesian computation for crossed and nested hierarchical models
• Efficient data augmentation techniques for some classes of state space models

This page was built for publication: Fast and accurate estimation of non-nested binomial hierarchical models using variational inference

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6202924)