Conditionally conjugate mean-field variational Bayes for logistic models
From MaRDI portal
Publication:2292397
Abstract: Variational Bayes (VB) is a common strategy for approximate Bayesian inference, but simple methods are only available for specific classes of models including, in particular, representations having conditionally conjugate constructions within an exponential family. Models with logit components are an apparently notable exception to this class, due to the absence of conjugacy between the logistic likelihood and the Gaussian priors for the coefficients in the linear predictor. To facilitate approximate inference within this widely used class of models, Jaakkola and Jordan (2000) proposed a simple variational approach which relies on a family of tangent quadratic lower bounds of logistic log-likelihoods, thus restoring conjugacy between these approximate bounds and the Gaussian priors. This strategy is still implemented successfully, but less attempts have been made to formally understand the reasons underlying its excellent performance. To cover this key gap, we provide a formal connection between the above bound and a recent P'olya-gamma data augmentation for logistic regression. Such a result places the computational methods associated with the aforementioned bounds within the framework of variational inference for conditionally conjugate exponential family models, thereby allowing recent advances for this class to be inherited also by the methods relying on Jaakkola and Jordan (2000).
Recommendations
- Mean-field variational approximate Bayesian inference for latent variable models
- Variational multinomial logit Gaussian process
- Variational Bayes for regime-switching log-normal models
- Mean field variational Bayes for elaborate distributions
- Variational inference in nonconjugate models
- Conjugate priors and bias reduction for logistic regression models
- A note on mean-field variational approximations in Bayesian probit models
- A conditional density estimation partition model using logistic Gaussian processes
- Bayesian modeling and variational inference for logistic group sparse regression model
Cites work
- 10.1162/jmlr.2003.3.4-5.993
- A Stochastic Approximation Method
- An introduction to variational methods for graphical models
- Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
- Explaining variational approximations
- Gaussian processes for machine learning.
- scientific article; zbMATH DE number 6377992 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1059776 (Why is no real title available?)
- Introduction to Stochastic Search and Optimization
- Logistic stick-breaking process
- Mean field variational Bayes for elaborate distributions
- Mixed membership stochastic blockmodels
- Model based clustering of high-dimensional binary data
- Monotonicity of quadratic-approximation algorithms
- Multivariate sharp quadratic bounds via \(\boldsymbol{\Sigma}\)-strong convexity and the Fenchel connection
- New inequalities for hyperbolic functions and their applications
- On Information and Sufficiency
- Pattern recognition and machine learning.
- Sampling-Based Approaches to Calculating Marginal Densities
- Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies
- Sharp quadratic majorization in one dimension
- Sparse logistic principal components analysis for binary data
- The Pólya-gamma Gibbs sampler for Bayesian logistic regression is uniformly ergodic
- Variational inference for large-scale models of discrete choice
- Variational inference in nonconjugate models
Cited in
(21)- Tree based credible set estimation
- Computationally efficient Bayesian unit-level models for non-Gaussian data under informative sampling with application to estimation of health insurance coverage
- Approximate Riemannian conjugate gradient learning for fixed-form variational Bayes
- Variational Bayes for regime-switching log-normal models
- Stratified Stochastic Variational Inference for High-Dimensional Network Factor Model
- Variational inference in nonconjugate models
- Integrating sample similarities into latent class analysis: a tree‐structured shrinkage approach
- Bayesian Lesion Estimation with a Structured Spike-and-Slab Prior
- Conjugate priors and bias reduction for logistic regression models
- Variational Bayesian analysis of survival data using a log-logistic accelerated failure time model
- Hyperbolic secant representation of the logistic function: application to probabilistic multiple instance learning for CT intracranial hemorrhage detection
- Efficient posterior inference for spatio-temporal modelling of repeated cross-sectional data
- Partially factorized variational inference for high-dimensional mixed models
- Variational Inference based on a Subclass of Closed Skew Normals
- Fast Variational Inference for Bayesian Factor Analysis in Single and Multi-Study Settings
- Stochastic variational inference for clustering short text data with finite mixtures of Dirichlet-multinomial distributions
- Skewed Bernstein-von Mises theorem and skew-modal approximations
- Non-Conjugate Variational Bayes for Pseudo-Likelihood Mixed Effect Models
- Skew-symmetric approximations of posterior distributions
- Tractable Bayesian density regression via logit stick-breaking priors
- Fast Bayesian estimation of spatial count data models
This page was built for publication: Conditionally conjugate mean-field variational Bayes for logistic models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2292397)
