Coupling techniques for nonlinear ensemble filtering

DOI10.1137/20M1312204zbMATH Open1503.93045arXiv1907.00389OpenAlexW2953480975MaRDI QIDQ5044993FDOQ5044993

Authors: Alessio Spantini, Ricardo Baptista, Youssef M. Marzouk 

Publication date: 3 November 2022

Published in: SIAM Review (Search for Journal in Brave)

Abstract: We consider filtering in high-dimensional non-Gaussian state-space models with intractable transition kernels, nonlinear and possibly chaotic dynamics, and sparse observations in space and time. We propose a novel filtering methodology that harnesses transportation of measures, convex optimization, and ideas from probabilistic graphical models to yield robust ensemble approximations of the filtering distribution in high dimensions. Our approach can be understood as the natural generalization of the ensemble Kalman filter (EnKF) to nonlinear updates, using stochastic or deterministic couplings. The use of nonlinear updates can reduce the intrinsic bias of the EnKF at a marginal increase in computational cost. We avoid any form of importance sampling and introduce non-Gaussian localization approaches for dimension scalability. Our framework achieves state-of-the-art tracking performance on challenging configurations of the Lorenz-96 model in the chaotic regime.

Full work available at URL: https://arxiv.org/abs/1907.00389

Recommendations

• Bridging the ensemble Kalman and particle filters
• Nonlinear data assimilation
• Mixture ensemble Kalman filters
• A local ensemble data assimilation algorithm for nonlinear geophysical models
• Filtering with state space localized Kalman gain

zbMATH Keywords

approximate Bayesian computationgraphical modelsstate-space modelslocalizationnonlinear filteringcouplingsensemble Kalman filtertransport maps

Mathematics Subject Classification ID

Filtering in stochastic control theory (93E11) Nonlinear systems in control theory (93C10)

Cites Work

• Numerical Optimization
• Probabilistic Forecasts, Calibration and Sharpness
• Title not available (Why is that?)
• Remarks on a Multivariate Transformation
• An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
• Optimal transport for applied mathematicians. Calculus of variations, PDEs, and modeling
• Regularized estimation of large covariance matrices
• A nonparametric ensemble transform method for Bayesian inference
• Title not available (Why is that?)
• A random map implementation of implicit filters
• Title not available (Why is that?)
• Second-order Accurate Ensemble Transform Particle Filters
• Filtering complex turbulent systems.
• Approximate Bayesian computational methods
• Probabilistic graphical models.
• Nonparametric estimation of large covariance matrices of longitudinal data
• Deterministic Nonperiodic Flow
• Title not available (Why is that?)
• A survey of convergence results on particle filtering methods for practitioners
• Data Assimilation
• Bayesian inference with optimal maps
• Time series analysis by state space methods.
• Feedback Particle Filter
• Ensemble Kalman filtering with shrinkage regression techniques
• Bridging the ensemble Kalman and particle filters
• Triangular transformations of measures
• On the convergence of the ensemble Kalman filter.
• Likelihood-informed dimension reduction for nonlinear inverse problems
• Optimal Low-rank Approximations of Bayesian Linear Inverse Problems
• On the Stability and the Uniform Propagation of Chaos of a Class of Extended Ensemble Kalman--Bucy Filters
• Probabilistic Forecasting and Bayesian Data Assimilation
• Title not available (Why is that?)
• Transport Map Accelerated Markov Chain Monte Carlo
• An Ensemble Kalman Filter Implementation Based on Modified Cholesky Decomposition for Inverse Covariance Matrix Estimation
• Sequential Monte Carlo with kernel embedded mappings: the mapping particle filter
• Title not available (Why is that?)
• The frontier of simulation-based inference
• Ensemble Kalman Methods for High-Dimensional Hierarchical Dynamic Space-Time Models
• Gibbs Flow for Approximate Transport with Applications to Bayesian Computation
• Multilevel Monte Carlo for Smoothing via Transport Methods

Cited In (13)

• Affine-mapping based variational ensemble Kalman filter
• Multivariate feedback particle filter rederived from the splitting-up scheme
• Ensemble transport smoothing. II: Nonlinear updates
• Ensemble transport smoothing. I: Unified framework
• Machine learning-based conditional mean filter: a generalization of the ensemble Kalman filter for nonlinear data assimilation
• On the representation and learning of monotone triangular transport maps
• Conditional sampling with monotone GANs: from generative models to likelihood-free inference
• Filtering with state space localized Kalman gain
• Unscented/ensemble transform-based variational filter
• A local ensemble data assimilation algorithm for nonlinear geophysical models
• Feature-informed data assimilation
• An ensemble score filter for tracking high-dimensional nonlinear dynamical systems
• Efficient quadratures for high-dimensional Bayesian data assimilation

Uses Software

• ElemStatLearn
• EnKF-MC

This page was built for publication: Coupling techniques for nonlinear ensemble filtering

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5044993)