On a Likelihood Approach for Monte Carlo Integration
The Monte Carlo integration by maximum likelihood estimator (MLE) is considered to evaluate the ratio \(Z/Z_1\) for the integrals of the kind \(Z_j = \int\limits_{\chi} q_j(x) d \mu_0\), where \(\mu_0\) is a nonnegative measure on a state space \(\chi\) and \(q_j(x)\) is a real-valued function (where the indices are omitted for \(Z\)). Importance sampling involves simulating observations from a single distribution. Generally, bridge sampling or stratified mixture sampling involves simulating observations from multiple distributions. In this case \(dP_j = (q_j(x)/Z_j) \, d \mu_0\) is the probability distribution and \(Z_j\) is the normalizing constant of the sampler \(P_j\), where \(1 \leq j \leq m\). Suppose that \(n_j\) independent observations \(\{ x_{j_1}, \ldots, x_{j_{n_j}} \}\) are available from \(P_j\) by a simulation technique, whereas totally there are \(n=\sum_{j=1}^m n_j\) observations. For importance sampling (\(m=1\)), the ratio \(Z/Z_1\) can be estimated by \(n^{-1} \sum_{i=1}^n \left[ q(x_i) / q_1(x_i) \right]\). The estimator is constructed via the identity \[ \frac{Z}{Z_1} = E_1 \left[ \frac{q(x)}{q_1(x)} \right] \;, \] where \(E_1\) denotes expectation under \(P_1\). In two different situations, the maximum likelihood estimator is obtained by estimating equations, which gives the smallest asymptotic variance. In the first situation one deals with the bridge sampling estimator \[ \frac{n_1^{-1} \sum_{i=1}^{n_1} \alpha(x_{1i})q_2(x_{1i})} {n_2^{-1} \sum_{i=1}^{n_2} \alpha(x_{2i})q_1(x_{2i})} \] for the ratio \(Z_2/Z_1\), including an optimizing function \(\alpha(x)\). In the second situation, the estimator for \(Z/Z_1\) of the form \[ \frac{1}{n} \sum\limits_{i=1}^n \frac{q(x_i) - {\mathbf b}^T {\mathbf g}(x_i)} {q_1(x_i)} \] is considered, where the components of vector \({\mathbf g}\) are real-valued functions \(g_1(x), \ldots, g_l(x)\) whose integrals with respect to \(\mu_0\) are zero, and \({\mathbf b} = (b_1, \ldots, b_l)^T\) is an arbitrary vector. It is known that the minimum variance is achieved asymptotically by the regression estimator, which corresponds to a special choice of \({\mathbf b}\). This problem is discussed in relation to a constrained MLE and then the results are generalised to the situation where more than one sampler is involved and control variates are used. Estimating equations including \textit{T. Hesterberg}'s stratified importance sampling estimator [Technometrics 37, No.~2, 185--194 (1995; Zbl 0822.62002)], \textit{E. Veach} and \textit{L. Guibas} multiple importance sampling estimator [``Optimally combining sampling techniques for Monte Carlo rendering in SIGGRAPH '95 Conference Proceedings, Reading MA: Addison-Wesley, 419--428 (1995)], and \textit{A. Owen} and \textit{Y. Zhou}'s method of control variates [J. Am. Stat. Assoc. 95, No.~449, 135--143 (2000; Zbl 0998.65003)] are considered.
- Importance sampling: intrinsic dimension and computational cost
- Calibrated path sampling and stepwise bridge sampling
- Product-form estimators: exploiting independence to scale up Monte Carlo
- Comment: unreasonable effectiveness of Monte Carlo
- Generalized multiple importance sampling
- Monte Carlo integration with Markov chain
- Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions
- A cluster-sample approach for Monte Carlo integration using multiple samplers
- Nonparametric likelihood and doubly robust estimating equations for marginal and nested structural models
- Second-order asymptotic theory for calibration estimators in sampling and missing-data problems
- A Theory of Statistical Models for Monte Carlo Integration
- A Perturbative Approach to Control Variates in Molecular Dynamics
- Two-Stage Importance Sampling With Mixture Proposals
- Partial differential equations and stochastic methods in molecular dynamics
- Warp Bridge Sampling: The Next Generation
- Selection of Proposal Distributions for Multiple Importance Sampling
- Computational approaches for empirical Bayes methods and Bayesian sensitivity analysis
- Times Square Sampling: An Adaptive Algorithm for Free Energy Estimation
- Error estimates and variance reduction for nonequilibrium stochastic dynamics
- Statistical inference with non-normalized models: score matching and noise contrastive estimation
- FEAT: free energy estimators with adaptive transport
- On estimation of conditional density models with two-phase sampling
This page was built for publication: On a Likelihood Approach for Monte Carlo Integration
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4651030)
