Convergence acceleration of ensemble Kalman inversion in nonlinear settings
From MaRDI portal
Publication:5070540
Abstract: Many data-science problems can be formulated as an inverse problem, where the parameters are estimated by minimizing a proper loss function. When complicated black-box models are involved, derivative-free optimization tools are often needed. The ensemble Kalman filter (EnKF) is a particle-based derivative-free Bayesian algorithm originally designed for data assimilation. Recently, it has been applied to inverse problems for computational efficiency. The resulting algorithm, known as ensemble Kalman inversion (EKI), involves running an ensemble of particles with EnKF update rules so they can converge to a minimizer. In this article, we investigate EKI convergence in general nonlinear settings. To improve convergence speed and stability, we consider applying EKI with non-constant step-sizes and covariance inflation. We prove that EKI can hit critical points with finite steps in non-convex settings. We further prove that EKI converges to the global minimizer polynomially fast if the loss function is strongly convex. We verify the analysis presented with numerical experiments on two inverse problems.
Recommendations
- Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
- Continuous Time Limit of the Stochastic Ensemble Kalman Inversion: Strong Convergence Analysis
- Well posedness and convergence analysis of the ensemble Kalman inversion
- Ensemble Kalman inversion: mean-field limit and convergence analysis
- Ensemble Kalman methods for inverse problems
Cites work
- scientific article; zbMATH DE number 46855 (Why is no real title available?)
- scientific article; zbMATH DE number 176065 (Why is no real title available?)
- scientific article; zbMATH DE number 1274356 (Why is no real title available?)
- scientific article; zbMATH DE number 2161246 (Why is no real title available?)
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- A Stochastic Approximation Method
- A regularizing Levenberg - Marquardt scheme, with applications to inverse groundwater filtration problems
- A regularizing iterative ensemble Kalman method for PDE-constrained inverse problems
- Accelerating Metropolis-within-Gibbs sampler with localized computations of differential equations
- Acceleration of Stochastic Approximation by Averaging
- Adaptive subgradient methods for online learning and stochastic optimization
- An introduction to computational stochastic PDEs
- Analysis of the ensemble Kalman filter for inverse problems
- Approximate Gauss–Newton Methods for Nonlinear Least Squares Problems
- Bayesian inversion in resin transfer molding
- Convergence of the square root ensemble Kalman filter in the large ensemble limit
- Convergence rates of convex variational regularization
- Data Assimilation
- Data assimilation: the Schrödinger perspective
- Derivative-free optimization: a review of algorithms and comparison of software implementations
- Ensemble Kalman inversion: a derivative-free technique for machine learning tasks
- Ensemble Kalman methods for inverse problems
- Ensemble filter techniques for intermittent data assimilation - a survey
- Filtering complex turbulent systems.
- Incremental Least Squares Methods and the Extended Kalman Filter
- Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
- Introduction to uncertainty quantification
- Introductory lectures on convex optimization. A basic course.
- Inverse problems: a Bayesian perspective
- Iteratively Regularized Gauss–Newton Method for Nonlinear Inverse Problems with Random Noise
- Kalman-based stochastic gradient method with stop condition and insensitivity to conditioning
- Large sample asymptotics for the ensemble Kalman filter
- Learning rate adaptation in stochastic gradient descent.
- Localization for MCMC: sampling high-dimensional posterior distributions with local structure
- MAP estimators and their consistency in Bayesian nonparametric inverse problems
- Modern regularization methods for inverse problems
- Nonlinear stability of the ensemble Kalman filter with adaptive covariance inflation
- On the continuous time limit of ensemble square root filters
- On the continuous time limit of the ensemble Kalman filter
- On the convergence of the ensemble Kalman filter.
- On the stability of sequential Monte Carlo methods in high dimensions
- Optimization by simulated annealing
- Parameterizations for ensemble Kalman inversion
- Performance analysis of local ensemble Kalman filter
- Performance of ensemble Kalman filters in large dimensions
- Probabilistic Forecasting and Bayesian Data Assimilation
- Sequential Monte Carlo Samplers
- Sequential Monte Carlo methods for high-dimensional inverse problems: a case study for the Navier-Stokes equations
- Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold Hamiltonian Monte Carlo
- Statistical inference for model parameters in stochastic gradient descent
- Stochastic stability of the discrete-time extended Kalman filter
- The ensemble Kalman filter for combined state and parameter estimation
- The incremental Gauss-Newton algorithm with adaptive stepsize rule
- The iterated Kalman filter update as a Gauss-Newton method
- Tikhonov regularization within ensemble Kalman inversion
- Unbiased ensemble square root filters
- Well posedness and convergence analysis of the ensemble Kalman inversion
- Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time
Cited in
(30)- Ensemble Kalman filter for neural network-based one-shot inversion
- Convergence analysis of ensemble Kalman inversion: the linear, noisy case
- Ensemble Kalman inversion for nonlinear problems: weights, consistency, and variance bounds
- A stabilization of a continuous limit of the ensemble Kalman inversion
- Efficient Bayesian physics informed neural networks for inverse problems via ensemble Kalman inversion
- Sequential Kalman tuning of the \(t\)-preconditioned Crank-Nicolson algorithm: efficient, adaptive and gradient-free inference for Bayesian inverse problems
- A strongly convergent numerical scheme from ensemble Kalman inversion
- Ensemble Kalman inversion: mean-field limit and convergence analysis
- On a Dynamic Variant of the Iteratively Regularized Gauss–Newton Method with Sequential Data
- Component-wise iterative ensemble Kalman inversion for static Bayesian models with unknown measurement error covariance
- Well posedness and convergence analysis of the ensemble Kalman inversion
- Tikhonov regularization within ensemble Kalman inversion
- Interacting Langevin diffusions: gradient structure and ensemble Kalman sampler
- Parameterizations for ensemble Kalman inversion
- Ensemble Kalman inversion: a derivative-free technique for machine learning tasks
- Hierarchical ensemble Kalman methods with sparsity-promoting generalized gamma hyperpriors
- Complete Deterministic Dynamics and Spectral Decomposition of the Linear Ensemble Kalman Inversion
- Adaptive Tikhonov strategies for stochastic ensemble Kalman inversion
- Iterative ensemble Kalman methods: a unified perspective with some new variants
- Ensemble Kalman inversion for general likelihoods
- Adaptive Ensemble Kalman Inversion with Statistical Linearization
- On convergence rates of adaptive ensemble Kalman inversion for linear ill-posed problems
- Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
- A stochastic iteratively regularized Gauss-Newton method
- EnKSGD: a class of preconditioned black box optimization and inversion algorithms
- Iterated Kalman methodology for inverse problems
- Adaptive operator learning for infinite-dimensional Bayesian inverse problems
- Localized ensemble Kalman inversion
- On the ensemble Kalman inversion under inequality constraints
- Ensemble Kalman sampler: mean-field limit and convergence analysis
This page was built for publication: Convergence acceleration of ensemble Kalman inversion in nonlinear settings
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5070540)
