A unified performance analysis of likelihood-informed subspace methods
From MaRDI portal
Publication:2676941
Abstract: The likelihood-informed subspace (LIS) method offers a viable route to reducing the dimensionality of high-dimensional probability distributions arising in Bayesian inference. LIS identifies an intrinsic low-dimensional linear subspace where the target distribution differs the most from some tractable reference distribution. Such a subspace can be identified using the leading eigenvectors of a Gram matrix of the gradient of the log-likelihood function. Then, the original high-dimensional target distribution is approximated through various forms of marginalization of the likelihood function, in which the approximated likelihood only has support on the intrinsic low-dimensional subspace. This approximation enables the design of inference algorithms that can scale sub-linearly with the apparent dimensionality of the problem. Intuitively, the accuracy of the approximation, and hence the performance of the inference algorithms, are influenced by three factors -- the dimension truncation error in identifying the subspace, Monte Carlo error in estimating the Gram matrices, and Monte Carlo error in constructing marginalizations. %This work establishes a unified framework to analyze each of these three factors and their interplay. Under mild technical assumptions, we establish error bounds for a range of existing dimension reduction techniques based on the principle of LIS. Our error bounds also provide useful insights into the accuracy of these methods. In addition, we analyze the integration of LIS with sampling methods such as Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo (SMC). We also demonstrate the applicability of our analysis on a linear inverse problem with Gaussian prior, which shows that all the estimates can be dimension-independent if the prior covariance is a trace-class operator.
Recommendations
- Likelihood-informed dimension reduction for nonlinear inverse problems
- Data-free likelihood-informed dimension reduction of Bayesian inverse problems
- Scalable posterior approximations for large-scale Bayesian inverse problems via likelihood-informed parameter and state reduction
- Dimension-independent likelihood-informed MCMC
- Certified Dimension Reduction for Bayesian Updating with the Cross-Entropy Method
Cites work
- A computational framework for infinite-dimensional Bayesian inverse problems. I: The linearized case, with application to global seismic inversion
- A computational framework for infinite-dimensional Bayesian inverse problems. II: stochastic Newton MCMC with application to ice sheet flow inverse problems
- A Family of Nonparametric Density Estimation Algorithms
- A Simple Analytic Proof of an Inequality by P. Buser
- A stochastic Newton MCMC method for large-scale statistical inverse problems with application to seismic inversion
- A useful variant of the Davis-Kahan theorem for statisticians
- Accelerating Markov chain Monte Carlo with active subspaces
- Certified dimension reduction in nonlinear Bayesian inverse problems
- Conditional density estimation and simulation through optimal transport
- Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms
- Data-driven optimal transport
- Data-free likelihood-informed dimension reduction of Bayesian inverse problems
- Deep composition of tensor-trains using squared inverse Rosenblatt transports
- Dimension-independent likelihood-informed MCMC
- Error bounds for some approximate posterior measures in Bayesian inference
- Fast algorithms for Bayesian uncertainty quantification in large-scale linear inverse problems based on low-rank partial Hessian approximations
- From Brunn-Minkowski to Brascamp-Lieb and to logarithmic Sobolev inequalities
- Functional data analysis.
- Generalization of an inequality by Talagrand and links with the logarithmic Sobolev inequality
- Generalized bounds for active subspaces
- Geometric MCMC for infinite-dimensional inverse problems
- scientific article; zbMATH DE number 3599198 (Why is no real title available?)
- scientific article; zbMATH DE number 3046994 (Why is no real title available?)
- Importance Sampling and Necessary Sample Size: An Information Theory Approach
- Importance sampling: intrinsic dimension and computational cost
- Inference via low-dimensional couplings
- Inverse problems: a Bayesian perspective
- Isoperimetric and analytic inequalities for log-concave probability measures
- Likelihood-informed dimension reduction for nonlinear inverse problems
- Localization for MCMC: sampling high-dimensional posterior distributions with local structure
- Logarithmic Sobolev Inequalities
- Low-Rank Matrix Approximations Do Not Need a Singular Value Gap
- MALA-within-Gibbs samplers for high-dimensional distributions with sparse conditional structure
- Markov Chain Monte Carlo Methods for High Dimensional Inversion in Remote Sensing
- MCMC methods for functions: modifying old algorithms to make them faster
- Methodology and convergence rates for functional linear regression
- Multilevel Markov Chain Monte Carlo
- Multilevel sequential Monte Carlo with dimension-independent likelihood-informed proposals
- On a generalization of the preconditioned Crank-Nicolson metropolis algorithm
- On extensions of the Brunn-Minkowski and Prekopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation
- On the stability of sequential Monte Carlo methods in high dimensions
- Optimal low-rank approximations of Bayesian linear inverse problems
- Poincaré's inequalities and Talagrand's concentration phenomenon for the exponential distribution
- Prediction in functional linear regression
- Rates of contraction of posterior distributions based on \(p\)-exponential priors
- Statistical inversion and Monte Carlo sampling methods in electrical impedance tomography
- The Efficient Generation of Random Orthogonal Matrices with an Application to Condition Estimators
- The pseudo-marginal approach for efficient Monte Carlo computations
- Unbiased Monte Carlo: posterior estimation for intractable/infinite-dimensional models
- Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem
- Weighted Poincaré-type inequalities for Cauchy and other convex measures
- Well-posed Bayesian geometric inverse problems arising in subsurface flow
Cited in
(10)- Likelihood-informed dimension reduction for nonlinear inverse problems
- Prior normalization for certified likelihood-informed subspace detection of Bayesian inverse problems
- Deep Importance Sampling Using Tensor Trains with Application to a Priori and a Posteriori Rare Events
- Multilevel dimension-independent likelihood-informed MCMC for large-scale inverse problems
- Certified coordinate selection for high-dimensional Bayesian inversion with Laplace prior
- Principal feature detection via -Sobolev inequalities
- Optimal experimental design: formulations and computations
- Sharp detection of low-dimensional structure in probability measures via dimensional logarithmic Sobolev inequalities
- Continuous Gaussian mixture solution for linear Bayesian inversion with application to Laplace priors
- Optimal low-rank posterior mean and distribution approximation in linear Gaussian inverse problems on Hilbert spaces
This page was built for publication: A unified performance analysis of likelihood-informed subspace methods
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2676941)
