Can local particle filters beat the curse of dimensionality?
From MaRDI portal
Publication:81243
DOI10.1214/14-AAP1061zbMATH Open1325.60058arXiv1301.6585OpenAlexW1822984620MaRDI QIDQ81243FDOQ81243
Authors: Patrick Rebeschini, Ramon Van Handel, Patrick Rebeschini, Ramon van Handel 
Publication date: 1 October 2015
Published in: The Annals of Applied Probability (Search for Journal in Brave)
Abstract: The discovery of particle filtering methods has enabled the use of nonlinear filtering in a wide array of applications. Unfortunately, the approximation error of particle filters typically grows exponentially in the dimension of the underlying model. This phenomenon has rendered particle filters of limited use in complex data assimilation problems. In this paper, we argue that it is often possible, at least in principle, to develop local particle filtering algorithms whose approximation error is dimension-free. The key to such developments is the decay of correlations property, which is a spatial counterpart of the much better understood stability property of nonlinear filters. For the simplest possible algorithm of this type, our results provide under suitable assumptions an approximation error bound that is uniform both in time and in the model dimension. More broadly, our results provide a framework for the investigation of filtering problems and algorithms in high dimension.
Full work available at URL: https://arxiv.org/abs/1301.6585
Recommendations
- Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems
- A stable particle filter for a class of high-dimensional state-space models
- Localization in high-dimensional Monte Carlo filtering
- How to avoid the curse of dimensionality: scalability of particle filters with and without importance weights
- Filtering with state space localized Kalman gain
curse of dimensionalitydata assimilationdecay of correlationsfilter stabilityinteracting Markov chainslocal particle filters
Monte Carlo methods (65C05) Inference from stochastic processes and prediction (62M20) Signal detection and filtering (aspects of stochastic processes) (60G35) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Interacting random processes; statistical mechanics type models; percolation theory (60K35) Probability in computer science (algorithm analysis, random structures, phase transitions, etc.) (68Q87)
Cites Work
- Statistical mechanics of probabilistic cellular automata.
- Inference in hidden Markov models.
- Phase transitions in nonlinear filtering
- Mathematical and algorithmic aspects of atmosphere-ocean data assimilation. Abstracts from the workshop held December 2--8, 2012.
- Title not available (Why is that?)
- Title not available (Why is that?)
- Conditional ergodicity in infinite dimension
- Comparison theorems for Gibbs measures
- Gibbs measures and phase transitions.
- Filtering complex turbulent systems.
- Error bounds and normalising constants for sequential Monte Carlo samplers in high dimensions
- Data assimilation: Mathematical and statistical perspectives
- On the stability of interacting processes with applications to filtering and genetic algorithms
Cited In (59)
- A fast distributed data-assimilation algorithm for divergence-free advection
- Sequential estimation of temporally evolving latent space network models
- A method for high-dimensional smoothing
- Error bounds and normalising constants for sequential Monte Carlo samplers in high dimensions
- Title not available (Why is that?)
- A lagged particle filter for stable filtering of certain high-dimensional state-space models
- Multilevel bootstrap particle filter
- Accuracy of some approximate Gaussian filters for the Navier-Stokes equation in the presence of model error
- Ergodicity and accuracy of optimal particle filters for Bayesian data assimilation
- Polynomial Propagation of Moments in Stochastic Differential Equations
- Sequential Bayesian inference for implicit hidden Markov models and current limitations
- A hybrid ensemble transform particle filter for nonlinear and spatially extended dynamical systems
- Qualitative robustness in Bayesian inference
- Importance sampling: intrinsic dimension and computational cost
- Particle Filters for nonlinear data assimilation in high-dimensional systems
- Iterative importance sampling algorithms for parameter estimation
- Multilevel sequential Monte Carlo for Bayesian inverse problems
- On the mathematical theory of ensemble (linear-Gaussian) Kalman-Bucy filtering
- Filtering with state space localized Kalman gain
- Multilevel ensemble Kalman filtering for spatio-temporal processes
- Title not available (Why is that?)
- Learning biological dynamics from spatio-temporal data by Gaussian processes
- A comparison of nonlinear extensions to the ensemble Kalman filter. Gaussian anamorphosis and two-step ensemble filters
- Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems
- Posterior consistency for Gaussian process approximations of Bayesian posterior distributions
- Dimension-free Wasserstein contraction of nonlinear filters
- Design and analysis of infectious disease studies. Abstracts from the workshop held February 19--25, 2023
- On the stability of Kalman-Bucy diffusion processes
- Perturbations and projections of Kalman-Bucy semigroups
- Conditional sequential Monte Carlo in high dimensions
- Localization for MCMC: sampling high-dimensional posterior distributions with local structure
- Performance analysis of local ensemble Kalman filter
- Particle filters with nudging in multiscale chaotic systems: with application to the Lorenz '96 atmospheric model
- Scalable inference for a full multivariate stochastic volatility model
- A 4D-Var method with flow-dependent background covariances for the shallow-water equations
- Localization in high-dimensional Monte Carlo filtering
- Spatiotemporal blocking of the bouncy particle sampler for efficient inference in state-space models
- Sequential Monte Carlo methods for Bayesian elliptic inverse problems
- Inference on high-dimensional implicit dynamic models using a guided intermediate resampling filter
- A stable particle filter for a class of high-dimensional state-space models
- How to avoid the curse of dimensionality: scalability of particle filters with and without importance weights
- Comparison theorems for Gibbs measures
- Deterministic mean-field ensemble Kalman filtering
- Multilevel ensemble transform particle filtering
- Bagged Filters for Partially Observed Interacting Systems
- Advanced Multilevel Monte Carlo Methods
- Long-time asymptotics of the filtering distribution for partially observed chaotic dynamical systems
- Analysis of error propagation in particle filters with approximation
- Learning a tree-structured Ising model in order to make predictions
- Real-time estimation and prediction of unsteady flows using reduced-order models coupled with few measurements
- Perron–Frobenius Operator Filter for Stochastic Dynamical Systems
- A divide and conquer sequential Monte Carlo approach to high dimensional filtering
- An iterated block particle filter for inference on coupled dynamic systems with shared and unit-specific parameters
- Seasonal count time series
- Principal feature detection via \(\phi \)-Sobolev inequalities
- Adaptive online variance estimation in particle filters: the ALVar estimator
- Bayesian Dynamic Feature Partitioning in High-Dimensional Regression With Big Data
- The mean-field ensemble Kalman filter: near-Gaussian setting
- On log-concave approximations of high-dimensional posterior measures and stability properties in non-linear inverse problems
This page was built for publication: Can local particle filters beat the curse of dimensionality?
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q81243)
