Gradient-Based Markov Chain Monte Carlo for Bayesian Inference With Non-differentiable Priors

DOI10.1080/01621459.2021.1909600MaRDI QIDQ6110732zbMATH OpenOpenAlexFDO

Authors Jacob Vorstrup Goldman, Sumeetpal S. Singh

Publication date 6 July 2023

Published in Journal of the American Statistical Association (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2103.09017

zbMATH Keywords

Bayesian imagingMarkov chain Monte CarloMarkov processespiece-wise deterministicproximal operators

Mathematics Subject Classification ID

Bayesian inference (62F15) Random fields; image analysis (62M40)

Abstract: The use of non-differentiable priors in Bayesian statistics has become increasingly popular, in particular in Bayesian imaging analysis. Current state of the art methods are approximate in the sense that they replace the posterior with a smooth approximation via Moreau-Yosida envelopes, and apply gradient-based discretized diffusions to sample from the resulting distribution. We characterize the error of the Moreau-Yosida approximation and propose a novel implementation using underdamped Langevin dynamics. In misson-critical cases, however, replacing the posterior with an approximation may not be a viable option. Instead, we show that Piecewise-Deterministic Markov Processes (PDMP) can be utilized for exact posterior inference from distributions satisfying almost everywhere differentiability. Furthermore, in contrast with diffusion-based methods, the suggested PDMP-based samplers place no assumptions on the prior shape, nor require access to a computationally cheap proximal operator, and consequently have a much broader scope of application. Through detailed numerical examples, including a non-differentiable circular distribution and a non-convex genomics model, we elucidate the relative strengths of these sampling methods on problems of moderate to high dimensions, underlining the benefits of PDMP-based methods when accurate sampling is decisive.

Recommendations

• Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau
• Subgradient Langevin methods for sampling from nonsmooth potentials
• Proximal Markov chain Monte Carlo algorithms
• A proximal Markov chain Monte Carlo method for Bayesian inference in imaging inverse problems: when Langevin meets Moreau
• The Bouncy Particle Sampler: A Non-Reversible Rejection-Free Markov Chain Monte Carlo Method

Cites work

• A first-order primal-dual algorithm for convex problems with applications to imaging
• A Hamiltonian Monte Carlo Method for Non-Smooth Energy Sampling
• Accelerating Proximal Markov Chain Monte Carlo by Using an Explicit Stabilized Method
• An algorithm for total variation minimization and applications
• Bayesian Compressive Sensing Using Laplace Priors
• Circular Distributions Based on Nonnegative Trigonometric Sums
• Convex analysis and monotone operator theory in Hilbert spaces
• Dirichlet-Laplace priors for optimal shrinkage
• Efficient Bayesian computation by proximal Markov chain Monte Carlo: when Langevin meets Moreau
• Exponential convergence of Langevin distributions and their discrete approximations
• Exponential ergodicity of the bouncy particle sampler
• Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
• High-dimensional Bayesian inference via the unadjusted Langevin algorithm
• scientific article; zbMATH DE number 3901778 (Why is no real title available?)
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 845714 (Why is no real title available?)
• Is there an analog of Nesterov acceleration for gradient-based MCMC?
• Nonlinear total variation based noise removal algorithms
• Nonsmooth optimization and its applications. Based on the workshop ``Nonsmooth Optimization and its Applications, Bonn, Germany, May 15--19, 2017
• Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
• On the Numerical Solution of Heat Conduction Problems in Two and Three Space Variables
• Optimal Scaling of Discrete Approximations to Langevin Diffusions
• Point process theory and applications. Marked point and picewise deterministic processes.
• Proximal algorithms in statistics and machine learning
• Proximal Markov chain Monte Carlo algorithms
• Randomized Hamiltonian Monte Carlo
• Randomized Hamiltonian Monte Carlo as scaling limit of the bouncy particle sampler and dimension-free convergence rates
• Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information
• Simulation of nonhomogeneous poisson processes by thinning
• Sparse Bayesian Methods for Low-Rank Matrix Estimation
• The Barker Proposal: Combining Robustness and Efficiency in Gradient-Based MCMC
• The Bayesian Lasso
• The Bouncy Particle Sampler: A Non-Reversible Rejection-Free Markov Chain Monte Carlo Method
• The horseshoe estimator for sparse signals
• The zig-zag process and super-efficient sampling for Bayesian analysis of big data
• Theoretical Guarantees for Approximate Sampling from Smooth and Log-Concave Densities
• Two Metropolis--Hastings Algorithms for Posterior Measures with Non-Gaussian Priors in Infinite Dimensions
• Variational Bayesian Blind Deconvolution Using a Total Variation Prior

Cited in

(4)

• On performance potentials and conditional Monte Carlo for gradient estimation for Markov chains
• Efficient Bayesian Computation for Low-Photon Imaging Problems
• Optimal scaling results for Moreau-Yosida Metropolis-adjusted Langevin algorithms
• Variational Inference Using a Differentiable Multigrid Linear Solver

This page was built for publication: Gradient-Based Markov Chain Monte Carlo for Bayesian Inference With Non-differentiable Priors

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6110732)