Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

DOI10.1007/S11222-014-9495-8MaRDI QIDQ2631369zbMATH OpenOpenAlexFDO

Authors I. Ntzoufras, Irini Moustaki, Vasiliki Silia Vitoratou

Publication date 29 July 2016

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1311.0656, http://eprints.lse.ac.uk/57685/

zbMATH Keywords

Bayes factormarginal likelihoodMonte Carlo integration

Mathematics Subject Classification ID

Bayesian inference (62F15) Generalized linear models (logistic models) (62J12)

Abstract: In latent variable models the parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample's divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures.

Recommendations

• Marginal likelihood estimation from the Metropolis output: tips and tricks for efficient implementation in generalized linear latent variable models
• On a Likelihood Approach for Monte Carlo Integration
• What are the advantages of MCMC based inference in latent variable models?
• A new Monte Carlo method for estimating marginal likelihoods
• Computation of marginal likelihoods with data-dependent support for latent variables

Cites work

• scientific article; zbMATH DE number 3815002 (Why is no real title available?)
• scientific article; zbMATH DE number 597911 (Why is no real title available?)
• scientific article; zbMATH DE number 2063756 (Why is no real title available?)
• scientific article; zbMATH DE number 947416 (Why is no real title available?)
• scientific article; zbMATH DE number 3297790 (Why is no real title available?)
• An investigation of hierarchical Bayes procedures in item response theory
• Applied Bayesian hierarchical methods.
• Batch Size Effects in the Analysis of Simulation Output
• Batch means and spectral variance estimators in Markov chain Monte Carlo
• Bayes Factors
• Bayes and empirical Bayes methods for data analysis.
• Bayes modal estimation in item response models
• Bayesian variable and link determination for generalised linear models
• Bayesian variable selection using cost-adjusted BIC, with application to cost-effective measurement of quality of health care
• Computing Bayes Factors by Combining Simulation and Asymptotic Approximations
• Estimating Bayes Factors via Posterior Simulation With the Laplace-Metropolis Estimator
• Estimation of Generalized Linear Latent Variable Models
• Fixed-Width Output Analysis for Markov Chain Monte Carlo
• Generalized latent trait models
• High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature
• Latent variable models and factor analysis. A unified approach
• Marginal Likelihood Estimation via Power Posteriors
• Marginal Likelihood From the Metropolis–Hastings Output
• Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects
• On the assessment of Monte Carlo error in simulation-based statistical analyses
• Simulating normalizing constants: From importance sampling to bridge sampling to path sampling
• The Variance of the Product of K Random Variables

Cited in

(1)

• On the use of marginal posteriors in marginal likelihood estimation via importance sampling

Describes a project that uses

Uses Software

• tsbridge

This page was built for publication: Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2631369)