On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example

DOI10.1214/18-AOAS1169zbMATH Open1405.62165arXiv1711.11057WikidataQ129462751 ScholiaQ129462751MaRDI QIDQ1624811FDOQ1624811

Authors: Yen-Chi Chen, Y. Samuel Wang, Elena A. Erosheva 

Publication date: 16 November 2018

Published in: The Annals of Applied Statistics (Search for Journal in Brave)

Abstract: Variational inference is a general approach for approximating complex density functions, such as those arising in latent variable models, popular in machine learning. It has been applied to approximate the maximum likelihood estimator and to carry out Bayesian inference, however, quantification of uncertainty with variational inference remains challenging from both theoretical and practical perspectives. This paper is concerned with developing uncertainty measures for variational inference by using bootstrap procedures. We first develop two general bootstrap approaches for assessing the uncertainty of a variational estimate and the study the underlying bootstrap theory in both fixed- and increasing-dimension settings. We then use the bootstrap approach and our theoretical results in the context of mixed membership modeling with multivariate binary data on functional disability from the National Long Term Care Survey. We carry out a two-sample approach to test for changes in the repeated measures of functional disability for the subset of individuals present in 1989 and 1994 waves.

Full work available at URL: https://arxiv.org/abs/1711.11057

Recommendations

• Variational inference for multiplicative intensity models
• Consistency of variational Bayes inference for estimation and model selection in mixtures
• Variational inference for large-scale models of discrete choice
• Asymptotic normality and valid inference for Gaussian variational approximation
• Mean-field variational approximate Bayesian inference for latent variable models

zbMATH Keywords

bootstrapBayesian inferenceincreasing dimensionvariational inferencetwo-sample testmultivariate binary datamixed membership modelfunctional disability

Mathematics Subject Classification ID

Bayesian inference (62F15) Bootstrap, jackknife and other resampling methods (62F40) Applications of statistics to biology and medical sciences; meta analysis (62P10) Estimation in multivariate analysis (62H12)

Cites Work

• Graphical models, exponential families, and variational inference
• Asymptotic Statistics
• Title not available (Why is that?)
• Consistent Estimates Based on Partially Consistent Observations
• On the asymptotic accuracy of Efron's bootstrap
• Maximum likelihood estimates in exponential response models
• Consistency of maximum-likelihood and variational estimators in the stochastic block model
• Bootstrap and wild bootstrap for high dimensional linear models
• Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
• Mixed membership stochastic blockmodels
• Mixture Densities, Maximum Likelihood and the EM Algorithm
• Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels
• Inconsistency of the Bootstrap when a Parameter is on the Boundary of the Parameter Space
• Variational Bayesian inference and complexity control for stochastic block models
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• All of Nonparametric Statistics
• Variational inference for Dirichlet process mixtures
• Science and Statistics
• 10.1162/jmlr.2003.3.4-5.993
• Asymptotic behavior of M-estimators of p regression parameters when \(p^ 2/n\) is large. I. Consistency
• User-friendly tail bounds for sums of random matrices
• Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity
• The bootstrap and Edgeworth expansion
• An introduction to variational methods for graphical models
• Asymptotic behavior of M estimators of p regression parameters when \(p^ 2/n\) is large. II: Normal approximation
• Asymptotics with increasing dimension for robust regression with applications to the bootstrap
• Describing disability through individual-level mixture models for multivariate binary data
• Joint consistency of nonparametric item characteristic curve and ability estimation
• Theory of Gaussian variational approximation for a Poisson mixed model
• Inference on means using the bootstrap
• Asymptotic normality and valid inference for Gaussian variational approximation
• A variational EM method for mixed membership models with multivariate rank data: an analysis of public policy preferences
• Variational inference for latent variables and uncertain inputs in Gaussian processes
• Guarding against spurious discoveries in high dimensions
• Handbook of Mixed Membership Models and Their Applications

Cited In (5)

• Semiparametric regression using variational approximations
• Multivariate mixed membership modeling: inferring domain-specific risk profiles
• Safety signal detection with control of latent factors
• Inferring food intake from multiple biomarkers using a latent variable model
• A family of parsimonious mixtures of multivariate Poisson-lognormal distributions for clustering multivariate count data

Uses Software

• STRUCTURE

This page was built for publication: On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1624811)