Deterministic mean-field ensemble Kalman filtering

DOI10.1137/140984415MaRDI QIDQ2805009zbMATH OpenOpenAlexWikidataFDO

Authors Hamidou Tembine, R. Tempone, K. J. H. Law

Publication date 9 May 2016

Published in SIAM Journal on Scientific Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1409.0628

zbMATH Keywords

Gaussian bias termKalman filteringmean field limitrandom measurestotal variation metric

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Gaussian processes (60G15) Signal detection and filtering (aspects of stochastic processes) (60G35) Random measures (60G57) Discrete-time Markov processes on general state spaces (60J05) Inference from stochastic processes and prediction (62M20) Probabilistic models, generic numerical methods in probability and statistics (65C20) Stochastic particle methods (65C35) Filtering in stochastic control theory (93E11)

Abstract: The proof of convergence of the standard ensemble Kalman filter (EnKF) from Legland etal. (2011) is extended to non-Gaussian state space models. A density-based deterministic approximation of the mean-field limit EnKF (DMFEnKF) is proposed, consisting of a PDE solver and a quadrature rule. Given a certain minimal order of convergence $kappa$ between the two, this extends to the deterministic filter approximation, which is therefore asymptotically superior to standard EnKF when the dimension $d<2kappa$. The fidelity of approximation of the true distribution is also established using an extension of total variation metric to random measures. This is limited by a Gaussian bias term arising from non-linearity/non-Gaussianity of the model, which exists for both DMFEnKF and standard EnKF. Numerical results support and extend the theory.

Recommendations

• Large sample asymptotics for the ensemble Kalman filter
• On the convergence of the ensemble Kalman filter.
• Convergence of the square root ensemble Kalman filter in the large ensemble limit
• Mean field limit of ensemble square root filters -- discrete and continuous time
• On the continuous time limit of the ensemble Kalman filter

Cites work

• A deterministic filter for non-Gaussian Bayesian estimation -- Applications to dynamical system estimation with noisy measurements
• A generalized polynomial chaos based ensemble Kalman filter with high accuracy
• A hybrid sparse-grid approach for nonlinear filtering problems based on adaptive-domain of the Zakai equation approximations
• A mean field approximation in data assimilation for nonlinear dynamics
• Accuracy and stability of filters for dissipative PDEs
• Accuracy and stability of the continuous-time 3DVAR filter for the Navier-Stokes equation
• An introduction to stochastic differential equations
• Bayesian inference with optimal maps
• Bayesian inverse problems and Kalman filters
• Blended reduced subspace algorithms for uncertainty quantification of quadratic systems with a stable mean state
• Can local particle filters beat the curse of dimensionality?
• Convergence of the square root ensemble Kalman filter in the large ensemble limit
• Data Assimilation
• Data assimilation: Mathematical and statistical perspectives
• Dimensional reduction for a Bayesian filter
• Discrete data assimilation in the Lorenz and 2D Navier-Stokes equations
• Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.
• Fundamental limitations of polynomial chaos for uncertainty quantification in systems with intermittent instabilities
• Fundamentals of stochastic filtering
• scientific article; zbMATH DE number 1181255 (Why is no real title available?)
• scientific article; zbMATH DE number 2190526 (Why is no real title available?)
• scientific article; zbMATH DE number 3281211 (Why is no real title available?)
• Implicit particle filters for data assimilation
• Inference in hidden Markov models.
• Inverse problems: a Bayesian perspective
• Large sample asymptotics for the ensemble Kalman filter
• On the convergence of the ensemble Kalman filter.
• On the stability of interacting processes with applications to filtering and genetic algorithms
• On the trend to equilibrium for the Fokker-Planck equation: an interplay between physics and functional analysis.
• Pathwise accuracy and ergodicity of metropolized integrators for SDEs
• Polynomial chaos for the approximation of uncertainties: Chances and limits
• Pushing the Limits of Contemporary Statistics: Contributions in Honor of Jayanta K. Ghosh
• Sequential Monte Carlo Methods in Practice
• Sparse grids
• Statistical and computational inverse problems.
• Stochastic differential equations. An introduction with applications.
• Stochastic processes and filtering theory
• The Fokker-Planck equation. Methods of solution and applications.
• Variational ensemble Kalman filtering using limited memory BFGS
• Well-posedness and accuracy of the ensemble Kalman filter in discrete and continuous time

Cited in

(48)

• Perturbations and projections of Kalman-Bucy semigroups
• Performance analysis of local ensemble Kalman filter
• On the convergence of a non-linear ensemble Kalman smoother
• On one-dimensional Riccati diffusions
• Multilevel ensemble Kalman filtering for spatio-temporal processes
• Ensemble Kalman inversion: mean-field limit and convergence analysis
• Ensemble Kalman inversion for nonlinear problems: weights, consistency, and variance bounds
• Score matching filters for Gaussian Markov random fields with a linear model of the precision matrix
• Multi-index ensemble Kalman filtering
• Affine-mapping based variational ensemble Kalman filter
• Mean-field-type games with jump and regime switching
• A perturbation analysis of stochastic matrix Riccati diffusions
• COVID-19: data-driven mean-field-type game perspective
• Mean-field-type games
• Ergodicity and accuracy of optimal particle filters for Bayesian data assimilation
• Multilevel ensemble Kalman filtering
• scientific article; zbMATH DE number 6611875 (Why is no real title available?)
• Large sample asymptotics for the ensemble Kalman filter
• Continuous limits for constrained ensemble Kalman filter
• Analysis of the ensemble and polynomial chaos Kalman filters in Bayesian inverse problems
• A deterministic filter for non-Gaussian Bayesian estimation -- Applications to dynamical system estimation with noisy measurements
• A strongly convergent numerical scheme from ensemble Kalman inversion
• Numerical approximation of the Frobenius-Perron operator using the finite volume method
• Autodifferentiable ensemble Kalman filters
• Multilevel ensemble Kalman-Bucy filters
• Ensemble Kalman methods for high-dimensional hierarchical dynamic space-time models
• Analysis of a localised nonlinear ensemble Kalman Bucy filter with complete and accurate observations
• Well posedness and convergence analysis of the ensemble Kalman inversion
• Analysis of the ensemble Kalman filter for inverse problems
• Gradient flow structure and convergence analysis of the ensemble Kalman inversion for nonlinear forward models
• Continuous Time Limit of the Stochastic Ensemble Kalman Inversion: Strong Convergence Analysis
• A Stochastic Approximation-Langevinized Ensemble Kalman Filter Algorithm for State Space Models with Unknown Parameters
• On the mathematical theory of ensemble (linear-Gaussian) Kalman-Bucy filtering
• Degeneracy-Free Particle Filter: Ensemble Kalman Smoother Multiple Distribution Estimation Filter
• Advanced Multilevel Monte Carlo Methods
• Identification of factors impacting on the transmission and mortality of COVID-19
• Complete Deterministic Dynamics and Spectral Decomposition of the Linear Ensemble Kalman Inversion
• Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilation
• A langevinized ensemble Kalman filter for large-scale dynamic learning
• EnKSGD: a class of preconditioned black box optimization and inversion algorithms
• The mean-field ensemble Kalman filter: near-Gaussian setting
• An extended Langevinized ensemble Kalman filter for non-Gaussian dynamic systems
• Ensemble collapse analysis of Tikhonov-regularized ensemble Kalman inversion in parameter estimation for nonlinear models
• Importance sampling for rare event tracking within the ensemble Kalman filtering framework
• Probabilistic data-driven turbulence closure modeling by assimilating statistics
• Ensemble Kalman methods: a mean-field perspective
• Local sensitivity analysis for Bayesian inverse problems
• Filtered Markovian projection: dimensionality reduction in filtering for stochastic reaction networks

This page was built for publication: Deterministic mean-field ensemble Kalman filtering

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2805009)