Laplacian smoothing stochastic gradient Markov chain Monte Carlo
From MaRDI portal
(Redirected from Publication:5856684)
Abstract: As an important Markov Chain Monte Carlo (MCMC) method, stochastic gradient Langevin dynamics (SGLD) algorithm has achieved great success in Bayesian learning and posterior sampling. However, SGLD typically suffers from slow convergence rate due to its large variance caused by the stochastic gradient. In order to alleviate these drawbacks, we leverage the recently developed Laplacian Smoothing (LS) technique and propose a Laplacian smoothing stochastic gradient Langevin dynamics (LS-SGLD) algorithm. We prove that for sampling from both log-concave and non-log-concave densities, LS-SGLD achieves strictly smaller discretization error in -Wasserstein distance, although its mixing rate can be slightly slower. Experiments on both synthetic and real datasets verify our theoretical results, and demonstrate the superior performance of LS-SGLD on different machine learning tasks including posterior sampling, Bayesian logistic regression and training Bayesian convolutional neural networks. The code is available at url{https://github.com/BaoWangMath/LS-MCMC}.
Recommendations
- Laplacian smoothing gradient descent
- Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics
- Consistency and fluctuations for stochastic gradient Langevin dynamics
- Analysis of Langevin Monte Carlo via convex optimization
- Hybrid deterministic-stochastic gradient Langevin dynamics for Bayesian learning
Cites work
- A Bayesian Approach to Estimating Background Flows from a Passive Scalar
- Analysis and geometry of Markov diffusion operators
- Consistency and fluctuations for stochastic gradient Langevin dynamics
- Diffusion for Global Optimization in $\mathbb{R}^n $
- Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.
- Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics
- Fast mixing of Metropolized Hamiltonian Monte Carlo: benefits of multi-step gradients
- Hamiltonian Monte Carlo acceleration using surrogate functions with random bases
- Hamiltonian Monte Carlo with energy conserving subsampling
- scientific article; zbMATH DE number 54145 (Why is no real title available?)
- scientific article; zbMATH DE number 6781368 (Why is no real title available?)
- scientific article; zbMATH DE number 961607 (Why is no real title available?)
- Log-concave sampling: Metropolis-Hastings algorithms are fast
- MCMC using Hamiltonian dynamics
- Metastability in reversible diffusion processes. I: Sharp asymptotics for capacities and exit times
- Mimicking the one-dimensional marginal distributions of processes having an Ito differential
- Nonasymptotic convergence analysis for the unadjusted Langevin algorithm
- On the trend to equilibrium for the Fokker-Planck equation: an interplay between physics and functional analysis.
- Theoretical Guarantees for Approximate Sampling from Smooth and Log-Concave Densities
- User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradient
- Weighted Csiszár-Kullback-Pinsker inequalities and applications to transportation inequalities
Cited in
(13)- LS-MCMC
- Natural Langevin dynamics for neural networks
- Stochastic gradient descent with Polyak's learning rate
- An adaptively weighted stochastic gradient MCMC algorithm for Monte Carlo simulation and global optimization
- Laplacian smoothing gradient descent
- Weak approximation of transformed stochastic gradient MCMC
- Consistency and fluctuations for stochastic gradient Langevin dynamics
- Exploration of the (non-)asymptotic bias and variance of stochastic gradient Langevin dynamics
- scientific article; zbMATH DE number 7626720 (Why is no real title available?)
- Smoothing unadjusted Langevin algorithms for nonsmooth composite potential functions
- A deterministic gradient-based approach to avoid saddle points
- Stochastic gradient Langevin dynamics for (weakly) log-concave posterior distributions
- AEGD: adaptive gradient descent with energy
This page was built for publication: Laplacian smoothing stochastic gradient Markov chain Monte Carlo
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5856684)
