On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example

DOI10.1214/18-AOAS1169MaRDI QIDQ1624811zbMATH OpenWikidataFDO

Authors Yen-Chi Chen, Y. Samuel Wang, Elena A. Erosheva

Publication date 16 November 2018

Published in The Annals of Applied Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1711.11057, https://projecteuclid.org/euclid.aoas/1532743479

zbMATH Keywords

bootstrapBayesian inferenceincreasing dimensionvariational inferencetwo-sample testmultivariate binary datamixed membership modelfunctional disability

Mathematics Subject Classification ID

Bayesian inference (62F15) Bootstrap, jackknife and other resampling methods (62F40) Applications of statistics to biology and medical sciences; meta analysis (62P10) Estimation in multivariate analysis (62H12)

Abstract: Variational inference is a general approach for approximating complex density functions, such as those arising in latent variable models, popular in machine learning. It has been applied to approximate the maximum likelihood estimator and to carry out Bayesian inference, however, quantification of uncertainty with variational inference remains challenging from both theoretical and practical perspectives. This paper is concerned with developing uncertainty measures for variational inference by using bootstrap procedures. We first develop two general bootstrap approaches for assessing the uncertainty of a variational estimate and the study the underlying bootstrap theory in both fixed- and increasing-dimension settings. We then use the bootstrap approach and our theoretical results in the context of mixed membership modeling with multivariate binary data on functional disability from the National Long Term Care Survey. We carry out a two-sample approach to test for changes in the repeated measures of functional disability for the subset of individuals present in 1989 and 1994 waves.

Recommendations

• Variational inference for multiplicative intensity models
• Consistency of variational Bayes inference for estimation and model selection in mixtures
• Variational inference for large-scale models of discrete choice
• Asymptotic normality and valid inference for Gaussian variational approximation
• Mean-field variational approximate Bayesian inference for latent variable models

Cites work

• scientific article; zbMATH DE number 3782216 (Why is no real title available?)
• 10.1162/jmlr.2003.3.4-5.993
• A variational EM method for mixed membership models with multivariate rank data: an analysis of public policy preferences
• All of Nonparametric Statistics
• An introduction to variational methods for graphical models
• Asymptotic Statistics
• Asymptotic behavior of M estimators of p regression parameters when \(p^ 2/n\) is large. II: Normal approximation
• Asymptotic behavior of M-estimators of p regression parameters when \(p^ 2/n\) is large. I. Consistency
• Asymptotic behavior of likelihood methods for exponential families when the number of parameters tends to infinity
• Asymptotic normality and valid inference for Gaussian variational approximation
• Asymptotic normality of maximum likelihood and its variational approximation for stochastic blockmodels
• Asymptotics with increasing dimension for robust regression with applications to the bootstrap
• Bootstrap and wild bootstrap for high dimensional linear models
• Consistency of maximum-likelihood and variational estimators in the stochastic block model
• Consistent Estimates Based on Partially Consistent Observations
• Describing disability through individual-level mixture models for multivariate binary data
• Gaussian approximations and multiplier bootstrap for maxima of sums of high-dimensional random vectors
• Graphical models, exponential families, and variational inference
• Guarding against spurious discoveries in high dimensions
• Handbook of Mixed Membership Models and Their Applications
• Inconsistency of the Bootstrap when a Parameter is on the Boundary of the Parameter Space
• Inference on means using the bootstrap
• Joint consistency of nonparametric item characteristic curve and ability estimation
• MODEL SELECTION AND INFERENCE: FACTS AND FICTION
• Maximum likelihood estimates in exponential response models
• Mixed membership stochastic blockmodels
• Mixture Densities, Maximum Likelihood and the EM Algorithm
• On the asymptotic accuracy of Efron's bootstrap
• Science and Statistics
• The bootstrap and Edgeworth expansion
• Theory of Gaussian variational approximation for a Poisson mixed model
• User-friendly tail bounds for sums of random matrices
• Variational Bayesian inference and complexity control for stochastic block models
• Variational inference for Dirichlet process mixtures
• Variational inference for latent variables and uncertain inputs in Gaussian processes

Cited in

(5)

• Inferring food intake from multiple biomarkers using a latent variable model
• Multivariate mixed membership modeling: inferring domain-specific risk profiles
• Safety signal detection with control of latent factors
• A family of parsimonious mixtures of multivariate Poisson-lognormal distributions for clustering multivariate count data
• Semiparametric regression using variational approximations

Describes a project that uses

Uses Software

• STRUCTURE

This page was built for publication: On the use of bootstrap with variational inference: theory, interpretation, and a two-sample test example

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1624811)