Asymptotic accuracy of Bayesian estimation for a single latent variable

DOI10.1016/J.NEUNET.2015.04.014MaRDI QIDQ1669142zbMATH OpenOpenAlexDBLPWikidataFDO

Authors Keisuke Yamazaki

Publication date 30 August 2018

Published in Neural Networks (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1408.5661

zbMATH Keywords

unsupervised learninglatent variablehierarchical parametric modelsBayes estimation

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: In data science and machine learning, hierarchical parametric models, such as mixture models, are often used. They contain two kinds of variables: observable variables, which represent the parts of the data that can be directly measured, and latent variables, which represent the underlying processes that generate the data. Although there has been an increase in research on the estimation accuracy for observable variables, the theoretical analysis of estimating latent variables has not been thoroughly investigated. In a previous study, we determined the accuracy of a Bayes estimation for the joint probability of the latent variables in a dataset, and we proved that the Bayes method is asymptotically more accurate than the maximum-likelihood method. However, the accuracy of the Bayes estimation for a single latent variable remains unknown. In the present paper, we derive the asymptotic expansions of the error functions, which are defined by the Kullback-Leibler divergence, for two types of single-variable estimations when the statistical regularity is satisfied. Our results indicate that the accuracies of the Bayes and maximum-likelihood methods are asymptotically equivalent and clarify that the Bayes method is only advantageous for multivariable estimations.

Recommendations

• Bayesian estimation of multidimensional latent variables and its asymptotic accuracy
• Accuracy of latent-variable estimation in Bayesian semi-supervised learning
• Asymptotic accuracy of distribution-based estimation of latent variables
• Asymptotic accuracy of Bayes estimation for latent variables with redundancy
• Effects of additional data on Bayesian clustering

Cites work

• scientific article; zbMATH DE number 5957252 (Why is no real title available?)
• A new look at the statistical model identification
• Algebraic Geometry and Statistical Learning Theory
• Algebraic analysis for nonidentifiable learning machines
• An asymptotic behaviour of the marginal likelihood for general Markov models
• Asymptotic Statistics
• Bayesian predictive information criterion for the evaluation of hierarchical Bayesian and empirical Bayes models
• Convergence of estimates under dimensionality restrictions
• Convergence of latent mixing measures in finite and infinite mixture models
• Convergence rates of posterior distributions.
• Estimating the dimension of a model
• Information-theoretic asymptotics of Bayes methods
• Maximum Likelihood Estimation of Misspecified Models
• Note on the Consistency of the Maximum Likelihood Estimate
• Resolution of singularities of an algebraic variety over a field of characteristic zero. I
• Selecting models from data: artificial intelligence and statistics IV. Selected papers presented at the fourth international workshop on artificial intelligence and statistics held in January 1993
• Singularities in mixture models and upper bounds of stochastic complexity.

Cited in

(5)

• Bayesian estimation of multidimensional latent variables and its asymptotic accuracy
• Asymptotic accuracy of Bayes estimation for latent variables with redundancy
• Asymptotic accuracy of distribution-based estimation of latent variables
• Effects of additional data on Bayesian clustering
• Accuracy of latent-variable estimation in Bayesian semi-supervised learning

This page was built for publication: Asymptotic accuracy of Bayesian estimation for a single latent variable

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1669142)