Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
From MaRDI portal
Bayesian inference (62F15) Stochastic analysis in statistical mechanics (82M60) Numerical computation of solutions to systems of equations (65H10) Numerical methods for inverse problems for boundary value problems involving PDEs (65N21) Fokker-Planck equations (35Q84) PDEs with randomness, stochastic partial differential equations (35R60) Numerical solutions to stochastic differential and integral equations (65C30) Stochastic particle methods (65C35) Stochastic methods (Fokker-Planck, Langevin, etc.) applied to problems in time-dependent statistical mechanics (82C31)
Abstract: Solving inverse problems without the use of derivatives or adjoints of the forward model is highly desirable in many applications arising in science and engineering. In this paper, we propose a new version of such a methodology, a framework for its analysis, and numerical evidence of the practicality of the method proposed. Our starting point is an ensemble of over-damped Langevin diffusions which interact through a single preconditioner computed as the empirical ensemble covariance. We demonstrate that the nonlinear Fokker-Planck equation arising from the mean-field limit of the associated stochastic differential equation (SDE) has a novel gradient flow structure, built on the Wasserstein metric and the covariance matrix of the noisy flow. Using this structure, we investigate large time properties of the Fokker-Planck equation, showing that its invariant measure coincides with that of a single Langevin diffusion, and demonstrating exponential convergence to the invariant measure in a number of settings. We introduce a new noisy variant on ensemble Kalman inversion (EKI) algorithms found from the original SDE by replacing exact gradients with ensemble differences; this defines the ensemble Kalman sampler (EKS). Numerical results are presented which demonstrate its efficacy as a derivative-free approximate sampler for the Bayesian posterior arising from inverse problems.
Recommendations
- Derivative-Free Bayesian Inversion Using Multiscale Dynamics
- Ensemble Kalman inversion: mean-field limit and convergence analysis
- Convergence acceleration of ensemble Kalman inversion in nonlinear settings
- Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
- Ensemble sampler for infinite-dimensional inverse problems
Cites work
- scientific article; zbMATH DE number 4211245 (Why is no real title available?)
- scientific article; zbMATH DE number 2152346 (Why is no real title available?)
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- scientific article; zbMATH DE number 3894218 (Why is no real title available?)
- A dynamical systems framework for intermittent data assimilation
- A gentle stochastic thermostat for molecular dynamics
- A nonparametric ensemble transform method for Bayesian inference
- A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems
- Affine natural proximal learning
- An analytical framework for consensus-based global optimization method
- Analysis of the ensemble Kalman filter for inverse problems
- Analysis of the ensemble and polynomial chaos Kalman filters in Bayesian inverse problems
- Approximate McKean-Vlasov representations for a class of SPDEs
- Asymptotic analysis for the generalized Langevin equation
- Bayesian Measures of Model Complexity and Fit
- Bayesian inference with optimal maps
- Convergence analysis of ensemble Kalman inversion: the linear, noisy case
- Data Assimilation
- Diffusion limits of the random walk Metropolis algorithm in high dimensions
- Discrete gradients for computational Bayesian inference
- Energy-diminishing integration of gradient systems
- Ensemble Kalman inversion: a derivative-free technique for machine learning tasks
- Ensemble Kalman methods for inverse problems
- Ensemble preconditioning for Markov chain Monte Carlo simulation
- Ensemble samplers with affine invariance
- Exponential convergence toward equilibrium for homogeneous Fokker-Planck-type equations
- Feedback Particle Filter
- Filtering complex turbulent systems.
- Fluctuations and Irreversible Process. II. Systems with Kinetic Energy
- From particle to kinetic and hydrodynamic descriptions of flocking
- Geometric integration using discrete gradients
- Information geometry
- Investigation of the sampling performance of ensemble-based methods with a simple reservoir model
- Iterative regularization for ensemble data assimilation in reservoir models
- Kinetic methods for inverse problems
- Kinetic models of opinion formation
- Long-time stability and accuracy of the ensemble Kalman-Bucy filter for fully observed processes and small measurement noise
- MCMC methods for functions: modifying old algorithms to make them faster
- Natural gradient via optimal transport
- Noisy gradient flow from a random walk in Hilbert space
- ON CONVEX SOBOLEV INEQUALITIES AND THE RATE OF CONVERGENCE TO EQUILIBRIUM FOR FOKKER-PLANCK TYPE EQUATIONS
- On the stability and the uniform propagation of chaos of a class of extended ensemble Kalman-Bucy filters
- On the stability and the uniform propagation of chaos properties of ensemble Kalman-Bucy filters
- On the trend to equilibrium for the Fokker-Planck equation: an interplay between physics and functional analysis.
- Online natural gradient as a Kalman filter
- Optimal Scaling of Discrete Approximations to Langevin Diffusions
- Optimal Transport
- Optimal acceptance rates for Metropolis algorithms: Moving beyond 0.234
- Optimal scaling for various Metropolis-Hastings algorithms.
- Optimal scaling of Metropolis algorithms: Heading toward general target distributions
- Optimal scaling of random-walk Metropolis algorithms on general target distributions
- Particle, kinetic, and hydrodynamic models of swarming
- Poisson's equation in nonlinear filtering
- Probabilistic Forecasting and Bayesian Data Assimilation
- Reciprocal Relations in Irreversible Processes. I.
- Reciprocal Relations in Irreversible Processes. II.
- Riemann manifold Langevin and Hamiltonian Monte Carlo methods. With discussion and authors' reply
- Runge–Kutta Methods for Dissipative and Gradient Dynamical Systems
- Scaling Limit of the Stein Variational Gradient Descent: The Mean Field Regime
- Statistical and computational inverse problems.
- Stochastic processes and applications. Diffusion processes, the Fokker-Planck and Langevin equations
- THE GEOMETRY OF DISSIPATIVE EVOLUTION EQUATIONS: THE POROUS MEDIUM EQUATION
- The Density Manifold and Configuration Space Quantization
- The Variational Formulation of the Fokker--Planck Equation
- The ensemble Kalman filter for combined state and parameter estimation
- Tikhonov regularization within ensemble Kalman inversion
- Time integration and discrete Hamiltonian systems
- Variance reduction using nonreversible Langevin samplers
- Wasserstein proximal of GANs
- Weak convergence and optimal scaling of random walk Metropolis algorithms
- Weak convergence of Metropolis algorithms for non-I.I.D. target distributions
- Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time
Cited in
(99)- Diffeomorphic measure matching with kernels for generative modeling
- Wasserstein stability estimates for covariance-preconditioned Fokker–Planck equations
- Nesterov acceleration for ensemble Kalman inversion and variants
- Adaptive Ensemble Kalman Inversion with Statistical Linearization
- Scaling Up Bayesian Uncertainty Quantification for Inverse Problems Using Deep Neural Networks
- Metropolis-adjusted interacting particle sampling
- Differential equation-constrained optimization with stochasticity
- scientific article; zbMATH DE number 7626720 (Why is no real title available?)
- Recent trends on nonlinear filtering for inverse problems
- Efficient derivative-free Bayesian inference for large-scale inverse problems
- Computational mean-field information dynamics associated with reaction-diffusion equations
- Convergence acceleration of ensemble Kalman inversion in nonlinear settings
- Well-posedness and approximation of reflected McKean-Vlasov SDEs with applications
- Ensemble Kalman inversion for nonlinear problems: weights, consistency, and variance bounds
- Ensemble Kalman methods: a mean-field perspective
- EnKSGD: a class of preconditioned black box optimization and inversion algorithms
- On a Metropolis-Hastings importance sampling estimator
- Coarse-gradient Langevin algorithms for dynamic data integration and uncertainty quantification
- Projected Wasserstein Gradient Descent for High-Dimensional Bayesian Inference
- Conditional optimal transport on function spaces
- Semi-discrete optimization through semi-discrete optimal transport: a framework for neural architecture search
- Gradient-free Stein variational gradient descent with kernel approximation
- Deep JKO: time-implicit particle methods for general nonlinear gradient flows
- Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
- Generalized unnormalized optimal transport and its fast algorithms
- Filtering Methods for Coupled Inverse Problems
- The ensemble Kalman filter for dynamic inverse problems
- On the ensemble Kalman inversion under inequality constraints
- Constrained Consensus-Based Optimization
- Global Optimization via Schrödinger–Föllmer Diffusion
- Stein variational gradient descent with local approximations
- Less interaction with forward models in Langevin dynamics: enrichment and homotopy
- Hyperparameter optimization for randomized algorithms: a case study on random features
- Birth–death dynamics for sampling: global convergence, approximations and their asymptotics
- Constrained ensemble Langevin Monte Carlo
- Ensemble Kalman inversion for image guided guide wire navigation in vascular systems
- Consensus‐based sampling
- Bayesian spatiotemporal modeling for inverse problems
- Subsampling in ensemble Kalman inversion
- Ensemble variational Fokker-Planck methods for data assimilation
- Surrogate modeling for Bayesian inverse problems based on physics-informed neural networks
- Affine invariant ensemble transform methods to improve predictive uncertainty in neural networks
- Ensemble Kalman inversion: mean-field limit and convergence analysis
- Ensemble Kalman sampler: mean-field limit and convergence analysis
- Iterated Kalman methodology for inverse problems
- Stein variational gradient descent: many-particle and long-time asymptotics
- Controlled stochastic processes for simulated annealing
- A localized consensus-based sampling algorithm
- Ensemble-Based Gradient Inference for Particle Methods in Optimization and Sampling
- The ensemble Kalman filter for rare event estimation
- Ensemble Markov chain Monte Carlo with teleporting walkers
- Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
- Continuous Time Limit of the Stochastic Ensemble Kalman Inversion: Strong Convergence Analysis
- Mean field stochastic partial differential equations with nonlinear kernels
- Error analysis for probabilities of rare events with approximate models
- Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
- Accelerated information gradient flow
- Adaptive regularisation for ensemble Kalman inversion
- Gradient-adjusted underdamped Langevin dynamics for sampling
- Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance
- Exponential entropy dissipation for weakly self-consistent Vlasov-Fokker-Planck equations
- The mean-field ensemble Kalman filter: near-Gaussian setting
- Sharp propagation of chaos for the ensemble Langevin sampler
- Affine invariant interacting Langevin dynamics for Bayesian inference
- Iterative ensemble Kalman methods: a unified perspective with some new variants
- Ensemble slice sampling. Parallel, black-box and gradient-free inference for correlated \& multimodal distributions
- Ensemble Kalman inversion for sparse learning of dynamical systems from time-averaged data
- Reverse diffusion sampling with tensor train approximations of Hamilton-Jacobi-Bellman equations
- High order spatial discretization for variational time implicit schemes: Wasserstein gradient flows and reaction-diffusion systems
- Neural dynamical operator: continuous spatial-temporal model with gradient-based and derivative-free optimization methods
- The forward-backward envelope for sampling with the overdamped Langevin algorithm
- Preserving nonlinear constraints in variational flow filtering data assimilation
- Sequential Kalman tuning of the t-preconditioned Crank-Nicolson algorithm: efficient, adaptive and gradient-free inference for Bayesian inverse problems
- Tikhonov regularization within ensemble Kalman inversion
- Fokker-Planck particle systems for Bayesian inference: computational approaches
- Covariance-modulated optimal transport and gradient flows
- A convergent numerical algorithm for the stochastic growth-fragmentation problem
- A stabilization of a continuous limit of the ensemble Kalman inversion
- Particle dual averaging: optimization of mean field neural network with global convergence rate analysis*
- Gradient-free score-based sampling methods with ensembles
- Calibrate, emulate, sample
- Perspectives on locally weighted ensemble Kalman methods
- Adaptive operator learning for infinite-dimensional Bayesian inverse problems
- Derivative-Free Bayesian Inversion Using Multiscale Dynamics
- Complete Deterministic Dynamics and Spectral Decomposition of the Linear Ensemble Kalman Inversion
- Tensor train based sampling algorithms for approximating regularized Wasserstein proximal operators
- Hessian metric via transport information geometry
- Projected Newton method for large-scale Bayesian linear inverse problems
- GP-ETAS: semiparametric Bayesian inference for the spatio-temporal epidemic type aftershock sequence model
- Ensemble inference methods for models with noisy and expensive likelihoods
- Affine-invariant ensemble transform methods for logistic regression
- Stable derivative free Gaussian mixture variational inference for Bayesian inverse problems
- Data assimilation: from mathematical and statistical foundations to applications. Abstracts from the workshop held February 23--28, 2025
- Propagation of chaos: a review of models, methods and applications. II: Applications
- Wasserstein steepest descent flows of discrepancies with Riesz kernels
- Mean-field limits for consensus-based optimization and sampling
- Discrete-time formulations as time discretization strategies in data assimilation
- Accelerating optimization over the space of probability measures
- Discrete gradients for computational Bayesian inference
This page was built for publication: Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5109769)
