Coupling techniques for nonlinear ensemble filtering

DOI10.1137/20M1312204zbMATH Open1503.93045OpenAlexW2953480975MaRDI QIDQ5044993FDOQ5044993

Authors: Alessio Spantini, Ricardo Baptista, Youssef M. Marzouk 

Publication date: 3 November 2022

Published in: SIAM Review (Search for Journal in Brave)

Abstract: We consider filtering in high-dimensional non-Gaussian state-space models with intractable transition kernels, nonlinear and possibly chaotic dynamics, and sparse observations in space and time. We propose a novel filtering methodology that harnesses transportation of measures, convex optimization, and ideas from probabilistic graphical models to yield robust ensemble approximations of the filtering distribution in high dimensions. Our approach can be understood as the natural generalization of the ensemble Kalman filter (EnKF) to nonlinear updates, using stochastic or deterministic couplings. The use of nonlinear updates can reduce the intrinsic bias of the EnKF at a marginal increase in computational cost. We avoid any form of importance sampling and introduce non-Gaussian localization approaches for dimension scalability. Our framework achieves state-of-the-art tracking performance on challenging configurations of the Lorenz-96 model in the chaotic regime.

Full work available at URL: https://arxiv.org/abs/1907.00389

Recommendations

• Bridging the ensemble Kalman and particle filters
• Nonlinear data assimilation
• Mixture ensemble Kalman filters
• A local ensemble data assimilation algorithm for nonlinear geophysical models
• Filtering with state space localized Kalman gain

zbMATH Keywords

approximate Bayesian computationgraphical modelsstate-space modelslocalizationnonlinear filteringcouplingsensemble Kalman filtertransport maps

Mathematics Subject Classification ID

Filtering in stochastic control theory (93E11) Nonlinear systems in control theory (93C10)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• A nonparametric ensemble transform method for Bayesian inference
• A random map implementation of implicit filters
• A survey of convergence results on particle filtering methods for practitioners
• An ensemble Kalman filter implementation based on modified Cholesky decomposition for inverse covariance matrix estimation
• An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
• Approximate Bayesian computational methods
• Bayesian inference with optimal maps
• Bridging the ensemble Kalman and particle filters
• Data Assimilation
• Deterministic Nonperiodic Flow
• Ensemble Kalman filtering with shrinkage regression techniques
• Ensemble Kalman methods for high-dimensional hierarchical dynamic space-time models
• Feedback Particle Filter
• Filtering complex turbulent systems.
• Gibbs Flow for Approximate Transport with Applications to Bayesian Computation
• Inference via low-dimensional couplings
• Likelihood-informed dimension reduction for nonlinear inverse problems
• Multilevel Monte Carlo for smoothing via transport methods
• Nonparametric estimation of large covariance matrices of longitudinal data
• Numerical Optimization
• On the convergence of the ensemble Kalman filter.
• On the stability and the uniform propagation of chaos of a class of extended ensemble Kalman-Bucy filters
• Optimal low-rank approximations of Bayesian linear inverse problems
• Optimal transport for applied mathematicians. Calculus of variations, PDEs, and modeling
• Probabilistic Forecasting and Bayesian Data Assimilation
• Probabilistic Forecasts, Calibration and Sharpness
• Probabilistic graphical models.
• Regularized estimation of large covariance matrices
• Remarks on a Multivariate Transformation
• Second-order Accurate Ensemble Transform Particle Filters
• Sequential Monte Carlo with kernel embedded mappings: the mapping particle filter
• The frontier of simulation-based inference
• Time series analysis by state space methods.
• Transport map accelerated Markov chain Monte Carlo
• Triangular transformations of measures

Cited In (18)

• Affine-mapping based variational ensemble Kalman filter
• Multivariate feedback particle filter rederived from the splitting-up scheme
• Ensemble transport smoothing. II: Nonlinear updates
• Ensemble transport smoothing. I: Unified framework
• Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilation
• On the representation and learning of monotone triangular transport maps
• Conditional sampling with monotone GANs: from generative models to likelihood-free inference
• A score-based filter for nonlinear data assimilation
• Nonlinear parameter estimation by a coupling ensemble Kalman filter
• Filtering with state space localized Kalman gain
• Unscented/ensemble transform-based variational filter
• Bridging the ensemble Kalman and particle filters
• Mixture ensemble Kalman filters
• Inference via low-dimensional couplings
• A local ensemble data assimilation algorithm for nonlinear geophysical models
• Feature-informed data assimilation
• An ensemble score filter for tracking high-dimensional nonlinear dynamical systems
• Efficient quadratures for high-dimensional Bayesian data assimilation

Uses Software

• ElemStatLearn
• EnKF-MC

This page was built for publication: Coupling techniques for nonlinear ensemble filtering

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5044993)