VAE: a stochastic process prior for Bayesian deep learning with MCMC

DOI10.1007/S11222-022-10151-WzbMATH Open1499.62027arXiv2002.06873OpenAlexW4306646271MaRDI QIDQ2103969FDOQ2103969

Authors: Swapnil Mishra, Seth R. Flaxman, Tresnia Berah, Harrison Zhu, Mikko S. Pakkanen, Samir Bhatt 

Publication date: 9 December 2022

Published in: Statistics and Computing (Search for Journal in Brave)

Abstract: Stochastic processes provide a mathematically elegant way model complex data. In theory, they provide flexible priors over function classes that can encode a wide range of interesting assumptions. In practice, however, efficient inference by optimisation or marginalisation is difficult, a problem further exacerbated with big data and high dimensional input spaces. We propose a novel variational autoencoder (VAE) called the prior encoding variational autoencoder ($pi$VAE). The $pi$VAE is finitely exchangeable and Kolmogorov consistent, and thus is a continuous stochastic process. We use $pi$VAE to learn low dimensional embeddings of function classes. We show that our framework can accurately learn expressive function classes such as Gaussian processes, but also properties of functions to enable statistical inference (such as the integral of a log Gaussian process). For popular tasks, such as spatial interpolation, $pi$VAE achieves state-of-the-art performance both in terms of accuracy and computational efficiency. Perhaps most usefully, we demonstrate that the low dimensional independently distributed latent space representation learnt provides an elegant and scalable means of performing Bayesian inference for stochastic processes within probabilistic programming languages such as Stan.

Full work available at URL: https://arxiv.org/abs/2002.06873

Recommendations

• Deep variational inference
• scientific article
• Generic inference in latent Gaussian process models
• Automatic differentiation variational inference
• Bayesian regression and classification using Gaussian process priors indexed by probability density functions

zbMATH Keywords

Bayesian inferenceMCMCspatio-temporalVAE

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Bayesian inference (62F15) Artificial neural networks and deep learning (68T07)

Cites Work

• Spectra of some self-exciting and mutually exciting point processes
• Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)
• Reducing the Dimensionality of Data with Neural Networks
• Gaussian processes for machine learning.
• Log Gaussian Cox Processes
• An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
• Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Stochastic processes and applications. Diffusion processes, the Fokker-Planck and Langevin equations
• Bayesian learning for neural networks
• Title not available (Why is that?)
• On the geometric ergodicity of Hamiltonian Monte Carlo
• An Introduction to Variational Autoencoders

Cited In (1)

• Learning variational autoencoders via MCMC speed measures

Uses Software

• Stan
• darch
• PyTorch
• Adam

This page was built for publication: \(\pi\) VAE: a stochastic process prior for Bayesian deep learning with MCMC

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2103969)