Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions
From MaRDI portal
Abstract: In this paper we study algorithms to find a Gaussian approximation to a target measure defined on a Hilbert space of functions; the target measure itself is defined via its density with respect to a reference Gaussian measure. We employ the Kullback-Leibler divergence as a distance and find the best Gaussian approximation by minimizing this distance. It then follows that the approximate Gaussian must be equivalent to the Gaussian reference measure, defining a natural function space setting for the underlying calculus of variations problem. We introduce a computational algorithm which is well-adapted to the required minimization, seeking to find the mean as a function, and parameterizing the covariance in two different ways: through low rank perturbations of the reference covariance; and through Schr"odinger potential perturbations of the inverse reference covariance. Two applications are shown: to a nonlinear inverse problem in elliptic PDEs, and to a conditioned diffusion process. We also show how the Gaussian approximations we obtain may be used to produce improved pCN-MCMC methods which are not only well-adapted to the high-dimensional setting, but also behave well with respect to small observational noise (resp. small temperatures) in the inverse problem (resp. conditioned diffusion).
Recommendations
- Kullback-Leibler approximation for probability measures on infinite dimensional spaces
- Gaussian approximations for probability measures on \(\mathbb R^d\)
- Relative entropy minimization over Hilbert spaces via Robbins-Monro
- Gaussian approximations of small noise diffusions in Kullback-Leibler divergence
- Gaussian Kullback-Leibler approximate inference
Cites work
- A Stochastic Approximation Method
- A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion
- Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions
- An adaptive Metropolis algorithm
- Approximation Methods which Converge with Probability one
- Bayesian inference with optimal maps
- Computing a nearest symmetric positive semidefinite matrix
- Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds
- Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
- Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations
- scientific article; zbMATH DE number 1223843 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- Invariant measures of stochastic partial differential equations and conditioned diffusions
- Inverse problems: a Bayesian perspective
- Kullback-Leibler approximation for probability measures on infinite dimensional spaces
- MCMC methods for functions: modifying old algorithms to make them faster
- On H-valued Robbins-Monro processes
- On the ergodicity properties of some adaptive MCMC algorithms
- Optimal low-rank approximations of Bayesian linear inverse problems
- Optimization with PDE Constraints
- Pattern recognition and machine learning.
- Riemann manifold Langevin and Hamiltonian Monte Carlo methods. With discussion and authors' reply
- Sample size selection in optimization methods for machine learning
- Signal processing problems on function space: Bayesian formulation, stochastic PDEs and effective MCMC methods
- Spectral gaps for a Metropolis-Hastings algorithm in infinite dimensions
- Stochastic approximation revisited
- Stochastic simulation: Algorithms and analysis
- Stochastic spectral methods for efficient Bayesian solution of inverse problems
- The stochastic root-finding problem: overview, solutions, and open questions
Cited in
(37)- Geometric MCMC for infinite-dimensional inverse problems
- An estimator for the relative entropy rate of path measures for stochastic differential equations
- The stochastic quasi-chemical model for bacterial growth: variational Bayesian parameter update
- Gaussian approximations of small noise diffusions in Kullback-Leibler divergence
- On a generalization of the preconditioned Crank-Nicolson metropolis algorithm
- Fill's algorithm for absolutely continuous stochastically monotone kernels
- Regularized divergences between covariance operators and Gaussian measures on Hilbert spaces
- Finite space Kantorovich problem with an MCMC of table moves
- Relative entropy minimization over Hilbert spaces via Robbins-Monro
- Variational Bayesian approximation of inverse problems using sparse precision matrices
- Adaptive dimension reduction to accelerate infinite-dimensional geometric Markov chain Monte Carlo
- A theoretical examination of diffusive molecular dynamics
- Kullback-Leibler approximation for probability measures on infinite dimensional spaces
- Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions
- Gaussian approximations for probability measures on \(\mathbb R^d\)
- An Adaptive Independence Sampler MCMC Algorithm for Bayesian Inferences of Functions
- Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
- Diffusive optical tomography in the Bayesian framework
- Coarse-graining of overdamped Langevin dynamics via the Mori-Zwanzig formalism
- Non-stationary multi-layered Gaussian priors for Bayesian inversion
- Gaussian approximations for transition paths in Brownian dynamics
- Variational Bayes' Method for Functions with Applications to Some Inverse Problems
- Variational Inference for Stochastic Differential Equations
- Non-centered parametric variational Bayes’ approach for hierarchical inverse problems of partial differential equations
- Hierarchical off-diagonal low-rank approximation of Hessians in inverse problems, with application to ice sheet model initialization
- Nonparametric inference of stochastic differential equations based on the relative entropy rate
- Principal feature detection via -Sobolev inequalities
- hIPPYlib-MUQ: a Bayesian inference software framework for integration of data with complex predictive models under uncertainty
- On an adaptive preconditioned Crank-Nicolson MCMC algorithm for infinite dimensional Bayesian inference
- A statistical framework and analysis for perfect radar pulse compression
- Derivative-informed neural operator acceleration of geometric MCMC for infinite-dimensional Bayesian inverse problems
- Reversibility of elliptical slice sampling revisited
- Infinite-dimensional distances and divergences between positive definite operators, Gaussian measures, and Gaussian processes
- Learning Informative Prior with Infinite-Dimensional Continuous Normalizing Flow for Bayesian Inverse Problem
- Independent approximation in separable Hilbert spaces via spectral truncation
- Efficient sampling for sparse Bayesian learning using hierarchical prior normalization
- Sparse polynomial chaos expansions using variational relevance vector machines
This page was built for publication: Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3454461)
