Hamiltonian Monte Carlo acceleration using surrogate functions with random bases
From MaRDI portal
Abstract: For big data analysis, high computational cost for Bayesian methods often limits their applications in practice. In recent years, there have been many attempts to improve computational efficiency of Bayesian inference. Here we propose an efficient and scalable computational technique for a state-of-the-art Markov Chain Monte Carlo (MCMC) methods, namely, Hamiltonian Monte Carlo (HMC). The key idea is to explore and exploit the structure and regularity in parameter space for the underlying probabilistic model to construct an effective approximation of its geometric properties. To this end, we build a surrogate function to approximate the target distribution using properly chosen random bases and an efficient optimization process. The resulting method provides a flexible, scalable, and efficient sampling algorithm, which converges to the correct target distribution. We show that by choosing the basis functions and optimization process differently, our method can be related to other approaches for the construction of surrogate functions such as generalized additive models or Gaussian process models. Experiments based on simulated and real data show that our approach leads to substantially more efficient sampling algorithms compared to existing state-of-the art methods.
Recommendations
- Precomputing strategy for Hamiltonian Monte Carlo method based on regularity in parameter space
- Neural network gradient Hamiltonian Monte Carlo
- Extended stochastic gradient Markov chain Monte Carlo for large-scale Bayesian variable selection
- scientific article; zbMATH DE number 6781368
- Modified Hamiltonian Monte Carlo for Bayesian inference
Cites work
- A unifying view of sparse approximate Gaussian process regression
- Bayesian learning for neural networks
- Contribution to the Discussion of the Paper ‘Geodesic Monte Carlo on Embedded Manifolds’
- Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
- Equation of state calculations by fast computing machines
- Estimation of non-normalized statistical models by score matching
- Gaussian processes for machine learning (GPML) toolbox
- scientific article; zbMATH DE number 3652509 (Why is no real title available?)
- scientific article; zbMATH DE number 1522714 (Why is no real title available?)
- scientific article; zbMATH DE number 1560711 (Why is no real title available?)
- MCMC using Hamiltonian dynamics
- Monte Carlo strategies in scientific computing
- Practical Markov Chain Monte Carlo
- Precomputing strategy for Hamiltonian Monte Carlo method based on regularity in parameter space
- Riemann manifold Langevin and Hamiltonian Monte Carlo methods. With discussion and authors' reply
- Self-organization and associative memory.
- Some Applications of the Pseudoinverse of a Matrix
- Split Hamiltonian Monte Carlo
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
- Trust region Newton method for logistic regression
- Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem
Cited in
(13)- Variational Hamiltonian Monte Carlo via score matching
- Neural network gradient Hamiltonian Monte Carlo
- Accelerating MCMC via Kriging-based adaptive independent proposals and delayed rejection
- Emulation-accelerated Hamiltonian Monte Carlo algorithms for parameter estimation and uncertainty quantification in differential equation models
- Adaptive dimension reduction to accelerate infinite-dimensional geometric Markov chain Monte Carlo
- Modified Hamiltonian Monte Carlo for Bayesian inference
- Hamiltonian Monte Carlo based on evidence framework for Bayesian learning to neural network
- Precomputing strategy for Hamiltonian Monte Carlo method based on regularity in parameter space
- Extended stochastic gradient Markov chain Monte Carlo for large-scale Bayesian variable selection
- Laplacian smoothing stochastic gradient Markov chain Monte Carlo
- On the accept-reject mechanism for Metropolis-Hastings algorithms
- A data-driven and model-based accelerated Hamiltonian Monte Carlo method for Bayesian elliptic inverse problems
- A Hamiltonian Monte Carlo EM algorithm for generalized linear mixed models with spatial skew latent variables
This page was built for publication: Hamiltonian Monte Carlo acceleration using surrogate functions with random bases
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1703832)
