Consensus‐based sampling
From MaRDI portal
Abstract: We propose a novel method for sampling and optimization tasks based on a stochastic interacting particle system. We explain how this method can be used for the following two goals: (i) generating approximate samples from a given target distribution; (ii) optimizing a given objective function. The approach is derivative-free and affine invariant, and is therefore well-suited for solving inverse problems defined by complex forward models: (i) allows generation of samples from the Bayesian posterior and (ii) allows determination of the maximum a posteriori estimator. We investigate the properties of the proposed family of methods in terms of various parameter choices, both analytically and by means of numerical simulations. The analysis and numerical simulation establish that the method has potential for general purpose optimization tasks over Euclidean space; contraction properties of the algorithm are established under suitable conditions, and computational experiments demonstrate wide basins of attraction for various specific problems. The analysis and experiments also demonstrate the potential for the sampling methodology in regimes in which the target distribution is unimodal and close to Gaussian; indeed we prove that the method recovers a Laplace approximation to the measure in certain parametric regimes and provide numerical evidence that this Laplace approximation attracts a large set of initial conditions in a number of examples.
Recommendations
- Affine invariant interacting Langevin dynamics for Bayesian inference
- Randomized maximum likelihood based posterior sampling
- Ensemble-Based Gradient Inference for Particle Methods in Optimization and Sampling
- Randomize-then-optimize: a method for sampling from posterior distributions in nonlinear inverse problems
- Derivative-Free Bayesian Inversion Using Multiscale Dynamics
Cites work
- A consensus-based global optimization method for high dimensional machine learning problems
- A consensus-based model for global optimization and its mean-field limit
- A dynamical systems framework for intermittent data assimilation
- A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems
- A simple proof of the Cucker-Smale flocking dynamics and mean-field limit
- Affine invariant interacting Langevin dynamics for Bayesian inference
- An analytical framework for consensus-based global optimization method
- Analysis of the ensemble and polynomial chaos Kalman filters in Bayesian inverse problems
- Analysis of the ensemble Kalman filter for inverse problems
- Ant colony optimization theory: a survey
- Applied asymptotic analysis
- Asymptotic flocking dynamics for the kinetic Cucker-Smale model
- Asymptotic Statistics
- Calibrate, emulate, sample
- Consensus-based optimization on hypersurfaces: Well-posedness and mean-field limit
- Convergence analysis of ensemble Kalman inversion: the linear, noisy case
- Convergence of a first-order consensus-based global optimization algorithm
- Ensemble Kalman inversion: a derivative-free technique for machine learning tasks
- Ensemble Kalman methods for inverse problems
- Ensemble preconditioning for Markov chain Monte Carlo simulation
- Ensemble samplers with affine invariance
- Equation of state calculations by fast computing machines
- Feedback Particle Filter
- Fokker-Planck particle systems for Bayesian inference: computational approaches
- Gaussian approximations for probability measures on \(\mathbb R^d\)
- Heterophilious dynamics enhances consensus
- scientific article; zbMATH DE number 4211245 (Why is no real title available?)
- scientific article; zbMATH DE number 795286 (Why is no real title available?)
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
- Investigation of the sampling performance of ensemble-based methods with a simple reservoir model
- Kinetic methods for inverse problems
- Kinetic models of opinion formation
- MAP estimators and their consistency in Bayesian nonparametric inverse problems
- Monte Carlo sampling methods using Markov chains and their applications
- Nonlinear Diffusion: Geodesic Convexity is Equivalent to Wasserstein Contraction
- On stochastic mirror descent with interacting particles: convergence properties and variance reduction
- On the mathematics of emergence
- Partial differential equations and stochastic methods in molecular dynamics
- Particle, kinetic, and hydrodynamic models of swarming
- Probabilistic Forecasting and Bayesian Data Assimilation
- Random batch methods (RBM) for interacting particle systems
- Sequential Monte Carlo Samplers
- Statistical and computational inverse problems.
- Stochastic mean-field limit: non-Lipschitz forces and swarming
- Stochastic processes and applications. Diffusion processes, the Fokker-Planck and Langevin equations
- Wasserstein stability estimates for covariance-preconditioned Fokker–Planck equations
Cited in
(31)- scientific article; zbMATH DE number 6500549 (Why is no real title available?)
- CONSISTENT SAMPLING WITH MULTI-, PRE- AND POST-FILTERINGS
- Efficient derivative-free Bayesian inference for large-scale inverse problems
- Birth–death dynamics for sampling: global convergence, approximations and their asymptotics
- On the ensemble Kalman inversion under inequality constraints
- Control methods in hyperbolic PDEs. Abstracts from the workshop held November 5--10, 2023
- Leveraging memory effects and gradient information in consensus-based optimisation: on global convergence in mean-field law
- Port-Hamiltonian structure of interacting particle systems and its mean-field limit
- Adaptive operator learning for infinite-dimensional Bayesian inverse problems
- Optimization by linear kinetic equations and mean-field Langevin dynamics
- MirrorCBO: a consensus-based optimization method in the spirit of mirror descent
- Mean-field limits for consensus-based optimization and sampling
- Uniform-in-time mean-field limit estimate for the consensus-based optimization
- Linearization of ergodic McKean SDEs and applications
- A consensus-based algorithm for non-convex multiplayer games
- Well-posedness and approximation of reflected McKean-Vlasov SDEs with applications
- Ensemble Kalman methods: a mean-field perspective
- Model predictive control strategies using consensus-based optimization
- A PDE framework of consensus-based optimization for objectives with multiple global minimizers
- Polarized consensus-based dynamics for optimization and sampling
- Deep learning-enhanced reduced-order ensemble Kalman filter for efficient Bayesian data assimilation of parametric PDEs
- Metropolis-adjusted interacting particle sampling
- Consensus-based optimisation with truncated noise
- Controlled stochastic processes for simulated annealing
- A localized consensus-based sampling algorithm
- Gradient-free sequential Bayesian experimental design via interacting particle systems
- A particle consensus approach to solving nonconvex-nonconcave min-max problems
- Parallel computations for Metropolis Markov chains with Picard maps
- Kinetic variable-sample methods for stochastic optimization problems
- Gradient is all you need? How consensus-based optimization can be interpreted as a stochastic relaxation of gradient descent
- Consensus-based adaptive sampling and approximation for high-dimensional energy landscapes
This page was built for publication: Consensus‐based sampling
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6085783)
