An ensemble Kalman filter implementation based on modified Cholesky decomposition for inverse covariance matrix estimation

DOI10.1137/16M1097031zbMATH Open1454.62278arXiv1605.08875OpenAlexW2962699938WikidataQ130107109 ScholiaQ130107109MaRDI QIDQ4610145FDOQ4610145

Authors: Elias D. Nino Ruiz, Adrian Sandu, Xinwei Deng 

Publication date: 5 April 2018

Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)

Abstract: This paper develops an efficient implementation of the ensemble Kalman filter based on a modified Cholesky decomposition for inverse covariance matrix estimation. This implementation is named EnKF-MC. Background errors corresponding to distant model components with respect to some radius of influence are assumed to be conditionally independent. This allows to obtain sparse estimators of the inverse background error covariance matrix. The computational effort of the proposed method is discussed and different formulations based on various matrix identities are provided. Furthermore, an asymptotic proof of convergence with regard to the ensemble size is presented. In order to assess the performance and the accuracy of the proposed method, experiments are performed making use of the Atmospheric General Circulation Model SPEEDY. The results are compared against those obtained using the local ensemble transform Kalman filter (LETKF). Tests are performed for dense observations ($100\mathrm{\%}$ and $50\mathrm{\%}$ of the model components are observed) as well as for sparse observations (only $12\mathrm{\%}$, $6\mathrm{\%}$, and $4\mathrm{\%}$ of model components are observed). The results reveal that the use of modified Cholesky for inverse covariance matrix estimation can reduce the impact of spurious correlations during the assimilation cycle, i.e., the results of the proposed method are of better quality than those obtained via the LETKF in terms of root mean square error.

Full work available at URL: https://arxiv.org/abs/1605.08875

Recommendations

• An ensemble Kalman filter implementation based on the Ledoit and Wolf covariance matrix estimator
• An efficient implementation of the ensemble Kalman filter based on an iterative Sherman-Morrison formula
• An efficient ensemble Kalman filter implementation via shrinkage covariance matrix estimation: exploiting prior knowledge
• A sparse matrix formulation of model-based ensemble Kalman filter
• Error estimate for the ensemble Kalman filter analysis step

zbMATH Keywords

modified Cholesky decompositionensemble Kalman filterspurious correlationsbackground error covariance estimation

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Monte Carlo methods (65C05) Inference from stochastic processes and prediction (62M20) Direct numerical methods for linear systems and matrix inversion (65F05) Sequential estimation (62L12)

Cites Work

• Regularized estimation of large covariance matrices
• A prior-free framework of coherent inference and its derivation of simple shrinkage estimators
• Truncated Singular Value Decomposition Solutions to Discrete Ill-Posed Problems with Ill-Determined Numerical Rank
• Adaptive modeling, adaptive data assimilation and adaptive sampling
• An efficient implementation of the ensemble Kalman filter based on an iterative Sherman-Morrison formula
• Iterative algorithms for computing the singular subspace of a matrix associated with its smallest singular values
• Solving total least-squares problems in information retrieval
• Probabilistic Forecasting and Bayesian Data Assimilation

Cited In (12)

• Non-linear data assimilation via trust region optimization
• Coupling techniques for nonlinear ensemble filtering
• Ensemble transport smoothing. II: Nonlinear updates
• An adjoint-free four-dimensional variational data assimilation method via a modified Cholesky decomposition and an iterative Woodbury matrix formula
• A reduced-space line-search method for unconstrained optimization via random descent directions
• An ensemble forecast method based on observational errors
• A sparse matrix formulation of model-based ensemble Kalman filter
• An efficient ensemble Kalman filter implementation via shrinkage covariance matrix estimation: exploiting prior knowledge
• An efficient implementation of the ensemble Kalman filter based on an iterative Sherman-Morrison formula
• An ensemble Kalman filter implementation based on the Ledoit and Wolf covariance matrix estimator
• Regional weather downscaling using ensemble Kalman filter with singular vector
• A multifidelity ensemble Kalman filter with reduced order control variates

Uses Software

• EnKF
• EnKF-MC

This page was built for publication: An ensemble Kalman filter implementation based on modified Cholesky decomposition for inverse covariance matrix estimation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4610145)