Auxiliary Gradient-Based Sampling Algorithms
From MaRDI portal
Abstract: We introduce a new family of MCMC samplers that combine auxiliary variables, Gibbs sampling and Taylor expansions of the target density. Our approach permits the marginalisation over the auxiliary variables yielding marginal samplers, or the augmentation of the auxiliary variables, yielding auxiliary samplers. The well-known Metropolis-adjusted Langevin algorithm (MALA) and preconditioned Crank-Nicolson Langevin (pCNL) algorithm are shown to be special cases. We prove that marginal samplers are superior in terms of asymptotic variance and demonstrate cases where they are slower in computing time compared to auxiliary samplers. In the context of latent Gaussian models we propose new auxiliary and marginal samplers whose implementation requires a single tuning parameter, which can be found automatically during the transient phase. Extensive experimentation shows that the increase in efficiency (measured as effective sample size per unit of computing time) relative to (optimised implementations of) pCNL, elliptical slice sampling and MALA ranges from 10-fold in binary classification problems to 25-fold in log-Gaussian Cox processes to 100-fold in Gaussian process regression, and it is on par with Riemann manifold Hamiltonian Monte Carlo in an example where the latter has the same complexity as the aforementioned algorithms. We explain this remarkable improvement in terms of the way alternative samplers try to approximate the eigenvalues of the target. We introduce a novel MCMC sampling scheme for hyperparameter learning that builds upon the auxiliary samplers. The MATLAB code for reproducing the experiments in the article is publicly available and a Supplement to this article contains additional experiments and implementation details.
Recommendations
- An adaptive gradient sampling algorithm for non-smooth optimization
- From Optimization to Sampling Through Gradient Flows
- Gradient-based adaptive importance samplers
- Adaptive sampling for incremental optimization using stochastic gradient descent
- Non-asymptotic guarantees for sampling by stochastic gradient descent
- On the differentiability check in gradient sampling methods
- A conjugate gradient sampling method for nonsmooth optimization
- Stochastic gradient descent, weighted sampling, and the randomized Kaczmarz algorithm
- Sampling Gaussian distributions in Krylov spaces with conjugate gradients
- A Robust Gradient Sampling Algorithm for Nonsmooth, Nonconvex Optimization
Cited in
(16)- f-SAEM: a fast stochastic approximation of the EM algorithm for nonlinear mixed effects models
- Posterior inference for sparse hierarchical non-stationary models
- Scalable inference for a full multivariate stochastic volatility model
- On the differentiability check in gradient sampling methods
- Auxiliary Variable Methods for Markov Chain Monte Carlo with Applications
- Gibbs Sampling for Bayesian Non-Conjugate and Hierarchical Models by Using Auxiliary Variables
- Informed proposals for local MCMC in discrete spaces
- Non-reversible guided Metropolis kernel
- Hamiltonian-Assisted Metropolis Sampling
- Scalable Bayesian computation for crossed and nested hierarchical models
- Bayesian prediction of jumps in large panels of time series data
- Auxiliary MCMC samplers for parallelisable inference in high-dimensional latent dynamical systems
- Fast Gibbs sampling for the local-seasonal-global trend Bayesian exponential smoothing model
- Faster high-accuracy log-concave sampling via algorithmic warm starts
- A localized consensus-based sampling algorithm
- Preconditioned discrete-HAMS: a second-order irreversible discrete sampler
This page was built for publication: Auxiliary Gradient-Based Sampling Algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4962088)
