Unbiased estimation of the gradient of the log-likelihood in inverse problems
From MaRDI portal
Publication:2058724
Abstract: We consider the problem of estimating a parameter associated to a Bayesian inverse problem. Treating the unknown initial condition as a nuisance parameter, typically one must resort to a numerical approximation of gradient of the log-likelihood and also adopt a discretization of the problem in space and/or time. We develop a new methodology to unbiasedly estimate the gradient of the log-likelihood with respect to the unknown parameter, i.e. the expectation of the estimate has no discretization bias. Such a property is not only useful for estimation in terms of the original stochastic model of interest, but can be used in stochastic gradient algorithms which benefit from unbiased estimates. Under appropriate assumptions, we prove that our estimator is not only unbiased but of finite variance. In addition, when implemented on a single processor, we show that the cost to achieve a given level of error is comparable to multilevel Monte Carlo methods, both practically and theoretically. However, the new algorithm provides the possibility for parallel computation on arbitrarily many processors without any loss of efficiency, asymptotically. In practice, this means any precision can be achieved in a fixed, finite constant time, provided that enough processors are available.
Recommendations
- On Unbiased Estimation for Discretized Models
- Unbiased estimation of the gradient of the log-likelihood for a class of continuous-time state-space models
- Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs
- Gradient of the log-likelihood ratio for infinite-dimensional stochastic systems
- Unbiased Markov chain Monte Carlo for intractable target distributions
Cites work
- A general method for debiasing a Monte Carlo estimator
- scientific article; zbMATH DE number 48727 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- scientific article; zbMATH DE number 2106098 (Why is no real title available?)
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- Inference in hidden Markov models.
- Inverse problems: a Bayesian perspective
- Mean field simulation for Monte Carlo integration
- Multilevel sequential Monte Carlo samplers
- Multilevel sequential Monte Carlo with dimension-independent likelihood-informed proposals
- Multilevel sequential Monte Carlo: Mean square error bounds under verifiable conditions
- Sequential Monte Carlo Samplers
- The approximate solution of Fredholm integral equations of the first kind
- Unbiased estimation with square root convergence for SDE models
- Unbiased estimators and multilevel Monte Carlo
- Unbiased Monte Carlo: posterior estimation for intractable/infinite-dimensional models
- Uncertainty Quantification and Weak Approximation of an Elliptic Inverse Problem
- Well-posed stochastic extensions of ill-posed linear problems
Cited in
(13)- Efficient importance sampling for large sums of independent and identically distributed random variables
- Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization
- Unbiased estimation of the gradient of the log-likelihood for a class of continuous-time state-space models
- On unbiased backtransform of lognormal kriging estimates
- Coordinate Based Empirical Likelihood-Like Estimation in Ill-Conditioned Inverse Problems
- Unbiased Estimation Using Underdamped Langevin Dynamics
- On Unbiased Estimation for Discretized Models
- A randomized multi-index sequential Monte Carlo method
- Multi-index sequential Monte Carlo ratio estimators for Bayesian inverse problems
- Unbiased parameter estimation for partially observed diffusions
- Unbiased parameter estimation for Bayesian inverse problems
- Bayesian deep learning with multilevel trace-class neural networks
- Nudging state-space models for Bayesian filtering under misspecified dynamics
This page was built for publication: Unbiased estimation of the gradient of the log-likelihood in inverse problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2058724)
