Can local particle filters beat the curse of dimensionality?
From MaRDI portal
(Redirected from Publication:81243)
curse of dimensionalitydata assimilationdecay of correlationsfilter stabilityinteracting Markov chainslocal particle filters
Monte Carlo methods (65C05) Inference from stochastic processes and prediction (62M20) Signal detection and filtering (aspects of stochastic processes) (60G35) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Interacting random processes; statistical mechanics type models; percolation theory (60K35) Probability in computer science (algorithm analysis, random structures, phase transitions, etc.) (68Q87)
Abstract: The discovery of particle filtering methods has enabled the use of nonlinear filtering in a wide array of applications. Unfortunately, the approximation error of particle filters typically grows exponentially in the dimension of the underlying model. This phenomenon has rendered particle filters of limited use in complex data assimilation problems. In this paper, we argue that it is often possible, at least in principle, to develop local particle filtering algorithms whose approximation error is dimension-free. The key to such developments is the decay of correlations property, which is a spatial counterpart of the much better understood stability property of nonlinear filters. For the simplest possible algorithm of this type, our results provide under suitable assumptions an approximation error bound that is uniform both in time and in the model dimension. More broadly, our results provide a framework for the investigation of filtering problems and algorithms in high dimension.
Recommendations
- Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems
- A stable particle filter for a class of high-dimensional state-space models
- Localization in high-dimensional Monte Carlo filtering
- How to avoid the curse of dimensionality: scalability of particle filters with and without importance weights
- Filtering with state space localized Kalman gain
Cites work
- scientific article; zbMATH DE number 1014071 (Why is no real title available?)
- scientific article; zbMATH DE number 2042289 (Why is no real title available?)
- Comparison theorems for Gibbs measures
- Conditional ergodicity in infinite dimension
- Data assimilation: Mathematical and statistical perspectives
- Error bounds and normalising constants for sequential Monte Carlo samplers in high dimensions
- Filtering complex turbulent systems.
- Gibbs measures and phase transitions.
- Inference in hidden Markov models.
- Mathematical and algorithmic aspects of atmosphere-ocean data assimilation. Abstracts from the workshop held December 2--8, 2012.
- On the stability of interacting processes with applications to filtering and genetic algorithms
- Phase transitions in nonlinear filtering
- Statistical mechanics of probabilistic cellular automata.
Cited in
(59)- Perron–Frobenius Operator Filter for Stochastic Dynamical Systems
- A divide and conquer sequential Monte Carlo approach to high dimensional filtering
- An iterated block particle filter for inference on coupled dynamic systems with shared and unit-specific parameters
- A method for high-dimensional smoothing
- A fast distributed data-assimilation algorithm for divergence-free advection
- Sequential estimation of temporally evolving latent space network models
- Seasonal count time series
- Error bounds and normalising constants for sequential Monte Carlo samplers in high dimensions
- scientific article; zbMATH DE number 7625157 (Why is no real title available?)
- Multilevel bootstrap particle filter
- A lagged particle filter for stable filtering of certain high-dimensional state-space models
- Ergodicity and accuracy of optimal particle filters for Bayesian data assimilation
- Accuracy of some approximate Gaussian filters for the Navier-Stokes equation in the presence of model error
- Sequential Bayesian inference for implicit hidden Markov models and current limitations
- Polynomial Propagation of Moments in Stochastic Differential Equations
- Qualitative robustness in Bayesian inference
- A hybrid ensemble transform particle filter for nonlinear and spatially extended dynamical systems
- Importance sampling: intrinsic dimension and computational cost
- Particle Filters for nonlinear data assimilation in high-dimensional systems
- Iterative importance sampling algorithms for parameter estimation
- Principal feature detection via \(\phi \)-Sobolev inequalities
- Multilevel sequential Monte Carlo for Bayesian inverse problems
- On the mathematical theory of ensemble (linear-Gaussian) Kalman-Bucy filtering
- Filtering with state space localized Kalman gain
- Multilevel ensemble Kalman filtering for spatio-temporal processes
- Adaptive online variance estimation in particle filters: the ALVar estimator
- Learning biological dynamics from spatio-temporal data by Gaussian processes
- scientific article; zbMATH DE number 7370576 (Why is no real title available?)
- A comparison of nonlinear extensions to the ensemble Kalman filter. Gaussian anamorphosis and two-step ensemble filters
- Curse-of-dimensionality revisited: Collapse of the particle filter in very large scale systems
- Dimension-free Wasserstein contraction of nonlinear filters
- Perturbations and projections of Kalman-Bucy semigroups
- Posterior consistency for Gaussian process approximations of Bayesian posterior distributions
- On the stability of Kalman-Bucy diffusion processes
- Design and analysis of infectious disease studies. Abstracts from the workshop held February 19--25, 2023
- Localization for MCMC: sampling high-dimensional posterior distributions with local structure
- Conditional sequential Monte Carlo in high dimensions
- Performance analysis of local ensemble Kalman filter
- Bayesian Dynamic Feature Partitioning in High-Dimensional Regression With Big Data
- Particle filters with nudging in multiscale chaotic systems: with application to the Lorenz '96 atmospheric model
- The mean-field ensemble Kalman filter: near-Gaussian setting
- A 4D-Var method with flow-dependent background covariances for the shallow-water equations
- Scalable inference for a full multivariate stochastic volatility model
- Spatiotemporal blocking of the bouncy particle sampler for efficient inference in state-space models
- Localization in high-dimensional Monte Carlo filtering
- Sequential Monte Carlo methods for Bayesian elliptic inverse problems
- Inference on high-dimensional implicit dynamic models using a guided intermediate resampling filter
- A stable particle filter for a class of high-dimensional state-space models
- How to avoid the curse of dimensionality: scalability of particle filters with and without importance weights
- Comparison theorems for Gibbs measures
- Deterministic mean-field ensemble Kalman filtering
- Multilevel ensemble transform particle filtering
- On log-concave approximations of high-dimensional posterior measures and stability properties in non-linear inverse problems
- Bagged Filters for Partially Observed Interacting Systems
- Analysis of error propagation in particle filters with approximation
- Long-time asymptotics of the filtering distribution for partially observed chaotic dynamical systems
- Advanced Multilevel Monte Carlo Methods
- Learning a tree-structured Ising model in order to make predictions
- Real-time estimation and prediction of unsteady flows using reduced-order models coupled with few measurements
This page was built for publication: Can local particle filters beat the curse of dimensionality?
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q81243)
