Multilevel Monte Carlo estimation of expected information gains

DOI10.1080/07362994.2019.1705168MaRDI QIDQ3298097zbMATH OpenOpenAlexWikidataFDO

Authors Takashi Goda, Tomohiko Hironaka, Takeru Iwamoto

Publication date 21 July 2020

Published in Stochastic Analysis and Applications (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1811.07546

zbMATH Keywords

Bayesian experimental designexpected information gainmultilevel Monte Carlo

Mathematics Subject Classification ID

Bayesian inference (62F15) Optimal statistical designs (62K05) Monte Carlo methods (65C05) Measures of information, entropy (94A17)

Abstract: The expected information gain is an important quality criterion of Bayesian experimental designs, which measures how much the information entropy about uncertain quantity of interest $heta$ is reduced on average by collecting relevant data $Y$. However, estimating the expected information gain has been considered computationally challenging since it is defined as a nested expectation with an outer expectation with respect to $Y$ and an inner expectation with respect to $heta$. In fact, the standard, nested Monte Carlo method requires a total computational cost of $O(varepsilo{n}^{-3})$ to achieve a root-mean-square accuracy of $varepsilon$. In this paper we develop an efficient algorithm to estimate the expected information gain by applying a multilevel Monte Carlo (MLMC) method. To be precise, we introduce an antithetic MLMC estimator for the expected information gain and provide a sufficient condition on the data model under which the antithetic property of the MLMC estimator is well exploited such that optimal complexity of $O(varepsilo{n}^{-2})$ is achieved. Furthermore, we discuss how to incorporate importance sampling techniques within the MLMC estimator to avoid arithmetic underflow. Numerical experiments show the considerable computational cost savings compared to the nested Monte Carlo method for a simple test case and a more realistic pharmacokinetic model.

Recommendations

• Multilevel Monte Carlo estimation of the expected value of sample information
• Decision-making under uncertainty: using MLMC for efficient estimation of EVPPI
• Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs
• Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations
• An introduction to multilevel Monte Carlo methods

Cites work

• Antithetic multilevel Monte Carlo estimation for multi-dimensional SDEs without Lévy area simulation
• Bayesian Experimental Design: A Review
• Decision-making under uncertainty: using MLMC for efficient estimation of EVPPI
• Decision-theoretic sensitivity analysis for reservoir development under uncertainty using multilevel quasi-Monte Carlo methods
• Efficient Bayesian experimentation using an expected information gain lower bound
• Fast Bayesian experimental design: Laplace-based importance sampling for the expected information gain
• Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations
• Multi-index Monte Carlo: when sparsity meets sampling
• Multilevel Monte Carlo Path Simulation
• Multilevel nested simulation for efficient risk estimation
• Multilevel quasi-Monte Carlo path simulation
• Multilevel simulation of functionals of Bernoulli random variables with application to basket credit derivatives
• On a Measure of the Information Provided by an Experiment
• Towards Bayesian experimental design for nonlinear models that require a large number of sampling times

Cited in

(21)

• Decision-theoretic sensitivity analysis for reservoir development under uncertainty using multilevel quasi-Monte Carlo methods
• Optimal Bayesian experimental design for subsurface flow problems
• Multi-index ensemble Kalman filtering
• An efficient estimation of nested expectations without conditional sampling
• Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization
• Multimodal information gain in Bayesian design of experiments
• On the effective dimension and multilevel Monte Carlo
• Decision-making under uncertainty: using MLMC for efficient estimation of EVPPI
• Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations
• Unbiased MLMC stochastic gradient-based optimization of Bayesian experimental designs
• Unbiased MLMC-based variational Bayes for likelihood-free inference
• Multilevel Monte Carlo estimation of the expected value of sample information
• Modern Bayesian experimental design
• Multilevel double loop Monte Carlo and stochastic collocation methods with importance sampling for Bayesian optimal experimental design
• Optimal experimental design: formulations and computations
• Greedy selection of optimal location of sensors for uncertainty reduction in seismic moment tensor inversion
• Efficient estimation of expected information gain in Bayesian experimental design with multi-index Monte Carlo
• A multifidelity estimator of the expected information gain for Bayesian optimal experimental design
• Laplace-based strategies for Bayesian optimal experimental design with nuisance uncertainty
• Leveraging nested MLMC for sequential neural posterior estimation with intractable likelihoods
• Expected information gain estimation via density approximations: sample allocation and dimension reduction

This page was built for publication: Multilevel Monte Carlo estimation of expected information gains

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3298097)