Distributed Bayesian learning with stochastic natural gradient expectation propagation and the posterior server

MaRDI QIDQ4637029zbMATH OpenFDO

Authors Leonard Q. Hasenclever, Stefan Webb, Thibaut Lienart, Balaji Lakshminarayanan, Charles Blundell, Sebastian J. Vollmer, Yee Whye Teh

Publication date 17 April 2018

Full work available at URL https://arxiv.org/abs/1512.09327, http://jmlr.csail.mit.edu/papers/v18/16-478.html

zbMATH Keywords

Bayesian learningdeep learningdistributed learningexpectation propagationlarge-scale learningMarkov chain Monte Carlonatural gradientparameter serverposterior serverstochastic approximationvariational inference

Mathematics Subject Classification ID

Applications of Markov chains and discrete-time Markov processes on general state spaces (social mobility, learning theory, industrial processes, etc.) (60J20) Bayesian inference (62F15) Learning and adaptive systems in artificial intelligence (68T05) Artificial neural networks and deep learning (68T07) Distributed algorithms (68W15)

Recommendations

• Expectation propagation as a way of life: a framework for Bayesian inference on partitioned data
• Expectation Propagation in the Large Data Limit
• Expectation propagation with factorizing distributions: a Gaussian approximation and performance results for simple models
• Scaling up Bayesian variational inference using distributed computing clusters
• Distributed Bayesian machine learning procedures

Cites work

• A partially linear framework for massive heterogeneous data
• A Scalable Bootstrap for Massive Data
• A Stochastic Approximation Method
• Adaptive thermostats for noisy gradient systems
• CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation
• Divide and conquer kernel ridge regression: a distributed algorithm with minimax optimal rates
• Expectation propagation as a way of life: a framework for Bayesian inference on partitioned data
• Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics
• Graphical models, exponential families, and variational inference
• scientific article; zbMATH DE number 6377992 (Why is no real title available?)
• scientific article; zbMATH DE number 5957313 (Why is no real title available?)
• scientific article; zbMATH DE number 3567782 (Why is no real title available?)
• scientific article; zbMATH DE number 1222290 (Why is no real title available?)
• scientific article; zbMATH DE number 1560711 (Why is no real title available?)
• scientific article; zbMATH DE number 840151 (Why is no real title available?)
• Mirror descent and nonlinear projected subgradient methods for convex optimization.
• Overcoming catastrophic forgetting in neural networks
• Sequential Monte Carlo Methods in Practice
• The Information Geometry of Mirror Descent
• The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo

Cited in

(7)

• A partitioned quasi-likelihood for distributed statistical inference
• Laplace approximation and natural gradient for Gaussian process regression with heteroscedastic Student-\(t\) model
• scientific article; zbMATH DE number 7365721 (Why is no real title available?)
• Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach
• Trust-region based stochastic variational inference for distributed and asynchronous networks
• Distributed event-triggered unadjusted Langevin algorithm for Bayesian learning
• A Bayesian practitioner's guide to expectation propagation

Describes a project that uses

Uses Software

• Stan
• CIFAR
• Adam
• NUTS

This page was built for publication: Distributed Bayesian learning with stochastic natural gradient expectation propagation and the posterior server

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4637029)