Optimal low-rank approximations of Bayesian linear inverse problems
From MaRDI portal
Abstract: In the Bayesian approach to inverse problems, data are often informative, relative to the prior, only on a low-dimensional subspace of the parameter space. Significant computational savings can be achieved by using this subspace to characterize and approximate the posterior distribution of the parameters. We first investigate approximation of the posterior covariance matrix as a low-rank update of the prior covariance matrix. We prove optimality of a particular update, based on the leading eigendirections of the matrix pencil defined by the Hessian of the negative log-likelihood and the prior precision, for a broad class of loss functions. This class includes the F"{o}rstner metric for symmetric positive definite matrices, as well as the Kullback-Leibler divergence and the Hellinger distance between the associated distributions. We also propose two fast approximations of the posterior mean and prove their optimality with respect to a weighted Bayes risk under squared-error loss. These approximations are deployed in an offline-online manner, where a more costly but data-independent offline calculation is followed by fast online evaluations. As a result, these approximations are particularly useful when repeated posterior mean evaluations are required for multiple data sets. We demonstrate our theoretical results with several numerical examples, including high-dimensional X-ray tomography and an inverse heat conduction problem. In both of these examples, the intrinsic low-dimensional structure of the inference problem can be exploited while producing results that are essentially indistinguishable from solutions computed in the full space.
Recommendations
- Goal-oriented optimal approximations of Bayesian linear inverse problems
- Low-Rank Eigenvector Compression of Posterior Covariance Matrices for Linear Gaussian Inverse Problems
- Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations
- Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction
- Data-free likelihood-informed dimension reduction of Bayesian inverse problems
Cites work
- A computational framework for infinite-dimensional Bayesian inverse problems. I: The linearized case, with application to global seismic inversion
- A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion
- A Trace Minimization Algorithm for the Generalized Eigenvalue Problem
- Adaptive Hessian-Based Nonstationary Gaussian Process Response Surface Method for Probability Density Approximation with Application to Bayesian Solution of Large-Scale Inverse Problems
- An efficient approach for computing optimal low-rank regularized inverse matrices
- An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach
- An Inverse Free Preconditioned Krylov Subspace Method for Symmetric Generalized Eigenvalue Problems
- Analysis of the Hessian for inverse scattering problems. I: Inverse shape scattering of acoustic waves
- ARPACK Users' Guide
- Bayesian methods for data analysis.
- Bayesian preconditioned CGLS for source separation in MEG time series
- Computational Variants of the Lanczos Method for the Eigenproblem
- Convergence in variance of Chebyshev accelerated Gibbs samplers
- Data Assimilation
- Derivatives of Spectral Functions
- Designing optimal spectral filters for inverse problems
- Dimension-independent likelihood-informed MCMC
- Dimensionality reduction and polynomial chaos acceleration of Bayesian inference in inverse problems
- Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix
- Estimates for Some Computational Techniques in Linear Algebra
- Estimation with quadratic loss.
- Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations
- Fast and Exact Simulation of Stationary Gaussian Processes through Circulant Embedding of the Covariance Matrix
- Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions
- Generalized Rank-Constrained Matrix Approximations
- Generalizing the Singular Value Decomposition
- Hessian-based model reduction: large-scale inversion and prediction
- scientific article; zbMATH DE number 3945129 (Why is no real title available?)
- scientific article; zbMATH DE number 4052834 (Why is no real title available?)
- scientific article; zbMATH DE number 3538728 (Why is no real title available?)
- scientific article; zbMATH DE number 3633705 (Why is no real title available?)
- scientific article; zbMATH DE number 1220667 (Why is no real title available?)
- scientific article; zbMATH DE number 733664 (Why is no real title available?)
- scientific article; zbMATH DE number 783550 (Why is no real title available?)
- scientific article; zbMATH DE number 6159604 (Why is no real title available?)
- scientific article; zbMATH DE number 3227378 (Why is no real title available?)
- scientific article; zbMATH DE number 3390151 (Why is no real title available?)
- scientific article; zbMATH DE number 2221907 (Why is no real title available?)
- scientific article; zbMATH DE number 3070747 (Why is no real title available?)
- scientific article; zbMATH DE number 3111121 (Why is no real title available?)
- Inverse problems: a Bayesian perspective
- Large-scale Kalman filtering using the limited memory BFGS method
- Left and right preconditioning for electrical impedance tomography with structural information
- Likelihood-informed dimension reduction for nonlinear inverse problems
- LSRN: A parallel iterative solver for strongly over- or underdetermined systems
- Methods of conjugate gradients for solving linear systems
- Numerical Optimization
- On the limited memory BFGS method for large scale optimization
- On the Rates of Convergence of the Lanczos and the Block-Lanczos Methods
- On the regularizing properties of the GMRES method
- Optimal filters from calibration data for image deconvolution with data acquisition error
- Optimal regularized low rank inverse approximation
- Parameter and state model reduction for large-scale statistical inverse problems
- Preconditioned iterative methods for linear discrete ill-posed problems from a Bayesian inversion perspective
- Priorconditioners for linear systems
- Pseudoinversus and conjugate gradients
- Regularization methods for large-scale problems
- Regularization, GSVD and truncated GSVD
- Sampling Gaussian distributions in Krylov spaces with conjugate gradients
- Scaling limits in computational Bayesian inversion
- Simplified GSVD computations for the solution of linear discrete ill-posed problems
- Statistical and computational inverse problems.
- Templates for the Solution of Algebraic Eigenvalue Problems
- The Efficient Generation of Random Orthogonal Matrices with an Application to Condition Estimators
- The variational Kalman filter and an efficient implementation using limited memory BFGS
- Towards a Generalized Singular Value Decomposition
- Updating Quasi-Newton Matrices with Limited Storage
- Variational ensemble Kalman filtering using limited memory BFGS
Cited in
(67)- Reduced modeling of unknown trajectories
- Optimal projection of observations in a Bayesian setting
- A multiscale method for model order reduction in PDE parameter estimation
- Multimodal, high-dimensional, model-based, Bayesian inverse problems with applications in biomechanics
- Importance sampling: intrinsic dimension and computational cost
- Bayesian inference of random fields represented with the Karhunen-Loève expansion
- Deep composition of tensor-trains using squared inverse Rosenblatt transports
- Model reduction of linear dynamical systems via balancing for Bayesian inference
- Localization for MCMC: sampling high-dimensional posterior distributions with local structure
- Adaptive dimension reduction to accelerate infinite-dimensional geometric Markov chain Monte Carlo
- Spatial localization for nonlinear dynamical stochastic models for excitable media
- Hessian-based adaptive sparse quadrature for infinite-dimensional Bayesian inverse problems
- Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction
- On convergence rates of adaptive ensemble Kalman inversion for linear ill-posed problems
- A unified performance analysis of likelihood-informed subspace methods
- Thermal uncertainty analysis of a single particle model for a lithium-ion cell
- Accelerating Markov chain Monte Carlo with active subspaces
- Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations
- Multilevel sequential Monte Carlo with dimension-independent likelihood-informed proposals
- Low-Rank Eigenvector Compression of Posterior Covariance Matrices for Linear Gaussian Inverse Problems
- A multiscale strategy for Bayesian inference using transport maps
- MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure
- Algorithms for Kullback-Leibler approximation of probability measures in infinite dimensions
- scientific article; zbMATH DE number 1098882 (Why is no real title available?)
- An approximate empirical Bayesian method for large-scale linear-Gaussian inverse problems
- Iterative importance sampling algorithms for parameter estimation
- Efficient generalized Golub-Kahan based methods for dynamic inverse problems
- Inference via low-dimensional couplings
- Low-rank independence samplers in hierarchical Bayesian inverse problems
- Stein variational reduced basis Bayesian inversion
- Cross-Entropy-Based Importance Sampling with Failure-Informed Dimension Reduction for Rare Event Simulation
- Coupling techniques for nonlinear ensemble filtering
- scientific article; zbMATH DE number 7626767 (Why is no real title available?)
- Stochastic learning approach for binary optimization: application to Bayesian optimal design of experiments
- Alternating Linear Scheme in a Bayesian Framework for Low-Rank Tensor Approximation
- Certified dimension reduction in nonlinear Bayesian inverse problems
- Stein variational gradient descent on infinite-dimensional space and applications to statistical inverse problems
- Scalable Optimization-Based Sampling on Function Space
- Tensor train construction from tensor actions, with application to compression of large high order derivative tensors
- Linearized Bayesian inference for Young's modulus parameter field in an elastic model of slender structures
- Simultaneous sensing error recovery and tomographic inversion using an optimization-based approach
- Scalable Matrix-Free Adaptive Product-Convolution Approximation for Locally Translation-Invariant Operators
- Efficient Marginalization-Based MCMC Methods for Hierarchical Bayesian Inverse Problems
- Optimal regularized inverse matrices for inverse problems
- Goal-oriented optimal approximations of Bayesian linear inverse problems
- Data-free likelihood-informed dimension reduction of Bayesian inverse problems
- Time-limited balanced truncation for data assimilation problems
- Numerical linear algebra in data assimilation
- Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method
- Hierarchical off-diagonal low-rank approximation of Hessians in inverse problems, with application to ice sheet model initialization
- Multilevel dimension-independent likelihood-informed MCMC for large-scale inverse problems
- Point spread function approximation of high-rank Hessians with locally supported nonnegative integral kernels
- Certified coordinate selection for high-dimensional Bayesian inversion with Laplace prior
- Principal feature detection via -Sobolev inequalities
- Optimal experimental design: formulations and computations
- Bayesian identifiability analysis for infectious disease models: parameter reduction and model selection
- Uncertainty quantification. Abstracts from the workshop held April 20--25, 2025
- Sharp detection of low-dimensional structure in probability measures via dimensional logarithmic Sobolev inequalities
- Accuracy improvement in ensemble Kalman inversion through data-informed ensemble selection
- Building population-informed priors for Bayesian inference using data-consistent stochastic inversion
- Non-intrusive optimal experimental design for large-scale nonlinear Bayesian inverse problems using a Bayesian approximation error approach
- Distributionally robust Gaussian process regression and Bayesian inverse problems
- Estimating and using information in inverse problems
- Scalable and efficient algorithms for the propagation of uncertainty from data through inference to prediction for large-scale problems, with application to flow of the antarctic ice sheet
- Optimal low-rank posterior mean and distribution approximation in linear Gaussian inverse problems on Hilbert spaces
- Scalability of the second-order reliability method for stochastic differential equations with multiplicative noise
- Expected information gain estimation via density approximations: sample allocation and dimension reduction
Describes a project that uses
Uses Software
This page was built for publication: Optimal low-rank approximations of Bayesian linear inverse problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3452481)
