Scalable Physics-Based Maximum Likelihood Estimation Using Hierarchical Matrices

DOI10.1137/21M1458880MaRDI QIDQ6177922zbMATH OpenFDO

Authors Yi-An Chen, Mihai Anitescu

Publication date 31 August 2023

Published in SIAM/ASA Journal on Uncertainty Quantification (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2303.10102

zbMATH Keywords

fast algorithmsGaussian processhierarchical matricesmaximum likelihood estimationparameter estimationuncertainty quantification

Mathematics Subject Classification ID

Determinants, permanents, traces, other special matrix functions (15A15) Computational methods for problems pertaining to statistics (62-08) Point estimation (62F10) Inference from spatial processes (62M30) Numerical methods for low-rank matrix approximation; matrix compression (65F55)

Abstract: Physics-based covariance models provide a systematic way to construct covariance models that are consistent with the underlying physical laws in Gaussian process analysis. The unknown parameters in the covariance models can be estimated using maximum likelihood estimation, but direct construction of the covariance matrix and classical strategies of computing with it requires $n$ physical model runs, $n^2$ storage complexity, and $n^3$ computational complexity. To address such challenges, we propose to approximate the discretized covariance function using hierarchical matrices. By utilizing randomized range sketching for individual off-diagonal blocks, the construction process of the hierarchical covariance approximation requires $O(logn)$ physical model applications and the maximum likelihood computations require $O(nlo{g}^{2}n)$ effort per iteration. We propose a new approach to compute exactly the trace of products of hierarchical matrices which results in the expected Fischer information matrix being computable in $O(nlo{g}^{2}n)$ as well. The construction is totally matrix-free and the derivatives of the covariance matrix can then be approximated in the same hierarchical structure by differentiating the whole process. Numerical results are provided to demonstrate the effectiveness, accuracy, and efficiency of the proposed method for parameter estimations and uncertainty quantification.

Recommendations

• Likelihood approximation with hierarchical matrices for large spatial datasets
• A matrix-free approach for solving the parametric Gaussian process maximum likelihood problem
• Local inversion-free estimation of spatial Gaussian processes
• Linear-Cost Covariance Functions for Gaussian Random Fields
• Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizations

Cites work

• A Class of Methods for Solving Nonlinear Simultaneous Equations
• A computational framework for infinite-dimensional Bayesian inverse problems. I: The linearized case, with application to global seismic inversion
• A fast randomized algorithm for computing a hierarchically semiseparable representation of a matrix
• A fast randomized algorithm for the approximation of matrices
• A matrix-free approach for solving the parametric Gaussian process maximum likelihood problem
• Advanced Lectures on Machine Learning
• An \(\mathcal O(N\log N)\) fast direct solver for partial hierarchically semi-separable matrices. With application to radial basis function interpolation
• An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
• Compressing Rank-Structured Matrices via Randomized Sampling
• Covariance tapering for likelihood-based estimation in large spatial data sets
• Data-sparse approximation of non-local operators by H^2-matrices
• Directional \(\mathcal{H}^2\)-matrix compression for high-frequency problems.
• Fast algorithms for hierarchically semiseparable matrices
• Fast construction of hierarchical matrix representation from matrix-vector multiplication
• Fast spatial Gaussian process maximum likelihood estimation via skeletonization factorizations
• Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions
• Fitting Gaussian Markov Random Fields to Gaussian Fields
• Fixed Rank Kriging for Very Large Spatial Data Sets
• Gaussian Markov Random Fields
• Gaussian Predictive Process Models for Large Spatial Data Sets
• Geostatistics. Modeling spatial uncertainty.
• Hierarchical Decompositions for the Computation of High-Dimensional Multivariate Normal Probabilities
• Hierarchical matrices: algorithms and analysis
• Hierarchical Matrix Approximations of Hessians Arising in Inverse Problems Governed by PDEs
• Hierarchical-block conditioning approximations for high-dimensional multivariate normal probabilities
• Hm-toolbox: MATLAB software for HODLR and HSS matrices
• scientific article; zbMATH DE number 3507807 (Why is no real title available?)
• scientific article; zbMATH DE number 1222301 (Why is no real title available?)
• scientific article; zbMATH DE number 1243473 (Why is no real title available?)
• scientific article; zbMATH DE number 1303701 (Why is no real title available?)
• scientific article; zbMATH DE number 1304257 (Why is no real title available?)
• scientific article; zbMATH DE number 1114428 (Why is no real title available?)
• scientific article; zbMATH DE number 1522697 (Why is no real title available?)
• scientific article; zbMATH DE number 3210432 (Why is no real title available?)
• scientific article; zbMATH DE number 6781341 (Why is no real title available?)
• Inconsistent Estimation and Asymptotically Equal Interpolations in Model-Based Geostatistics
• Introduction to Automatic Differentiation and MATLAB Object-Oriented Programming
• Kalman filters for non-linear systems: a comparison of performance
• Kriging and spatial design accelerated by orders of magnitude: combining low-rank covariance approximations with FFT-techniques
• Likelihood approximation with hierarchical matrices for large spatial datasets
• ON STATIONARY PROCESSES IN THE PLANE
• Optimal estimation and rank detection for sparse spiked covariance matrices
• Perturbation Analyses for the QR Factorization
• PHYSICS-BASED COVARIANCE MODELS FOR GAUSSIAN PROCESSES WITH MULTIPLE OUTPUTS
• Randomized algorithms for estimating the trace of an implicit symmetric positive semi-definite matrix
• Randomized algorithms for the low-rank approximation of matrices
• Randomized GPU Algorithms for the Construction of Hierarchical Matrices from Matrix-Vector Operations
• Scalable Gaussian Process Analysis for Implicit Physics-Based Covariance Models
• Scalable Gaussian Process Computations Using Hierarchical Matrices
• Spatiotemporal Hierarchical Bayesian Modeling Tropical Ocean Surface Winds
• Statistics for spatial data
• Stochastic stability of the discrete-time extended Kalman filter
• The Rational SPDE Approach for Gaussian Random Fields With General Smoothness
• Towards reconciling two asymptotic frameworks in spatial statistics

Cited in

(1)

• Covariance operator estimation: sparsity, lengthscale, and ensemble Kalman filters

This page was built for publication: Scalable Physics-Based Maximum Likelihood Estimation Using Hierarchical Matrices

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6177922)