Bayesian Inference Based on Stationary Fokker-Planck Sampling

DOI10.1162/NECO.2010.01-09-943MaRDI QIDQ3568379zbMATH OpenOpenAlexWikidataFDO

Authors Arturo Berrones

Publication date 11 June 2010

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0901.1144

zbMATH Keywords

Bayesian learningcomplex inference models

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: A novel formalism for Bayesian learning in the context of complex inference models is proposed. The method is based on the use of the Stationary Fokker--Planck (SFP) approach to sample from the posterior density. Stationary Fokker--Planck sampling generalizes the Gibbs sampler algorithm for arbitrary and unknown conditional densities. By the SFP procedure approximate analytical expressions for the conditionals and marginals of the posterior can be constructed. At each stage of SFP, the approximate conditionals are used to define a Gibbs sampling process, which is convergent to the full joint posterior. By the analytical marginals efficient learning methods in the context of Artificial Neural Networks are outlined. Off--line and incremental Bayesian inference and Maximum Likelihood Estimation from the posterior is performed in classification and regression examples. A comparison of SFP with other Monte Carlo strategies in the general problem of sampling from arbitrary densities is also presented. It is shown that SFP is able to jump large low--probabilty regions without the need of a careful tuning of any step size parameter. In fact, the SFP method requires only a small set of meaningful parameters which can be selected following clear, problem--independent guidelines. The computation cost of SFP, measured in terms of loss function evaluations, grows linearly with the given model's dimension.

Recommendations

• Bayesian inference for stochastic processes
• scientific article; zbMATH DE number 1219015
• Fokker-Planck particle systems for Bayesian inference: computational approaches
• scientific article; zbMATH DE number 472921
• Empirical Bayes Gibbs sampling
• Bayesian inference using adaptive sampling
• Bayesian statistical inference for sampling a finite population
• Bayesian estimation under informative sampling

Cites work

• scientific article; zbMATH DE number 3806623 (Why is no real title available?)
• Hyperparameter estimation for satellite image restoration using a MCMC maximum-likelihood method
• Langevin diffusions and Metropolis-Hastings algorithms
• Markov chain Monte Carlo methods for stochastic volatility models.
• Nonlinear Time Series Analysis
• Variational Bayes Solution of Linear Neural Networks and Its Generalization Performance

Cited in

(3)

• Bayesian learning via neural Schrödinger-Föllmer flows
• Rao-blackwellized sampling for batch and recursive Bayesian inference of piecewise affine models
• Fokker-Planck particle systems for Bayesian inference: computational approaches

This page was built for publication: Bayesian Inference Based on Stationary Fokker-Planck Sampling

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3568379)