Asymptotic accuracy of Bayes estimation for latent variables with redundancy

DOI10.1007/S10994-015-5482-3zbMATH Open1351.68241arXiv1205.3234OpenAlexW2034024264MaRDI QIDQ255357FDOQ255357

Authors: Keisuke Yamazaki 

Publication date: 9 March 2016

Published in: Machine Learning (Search for Journal in Brave)

Abstract: Hierarchical parametric models consisting of observable and latent variables are widely used for unsupervised learning tasks. For example, a mixture model is a representative hierarchical model for clustering. From the statistical point of view, the models can be regular or singular due to the distribution of data. In the regular case, the models have the identifiability; there is one-to-one relation between a probability density function for the model expression and the parameter. The Fisher information matrix is positive definite, and the estimation accuracy of both observable and latent variables has been studied. In the singular case, on the other hand, the models are not identifiable and the Fisher matrix is not positive definite. Conventional statistical analysis based on the inverse Fisher matrix is not applicable. Recently, an algebraic geometrical analysis has been developed and is used to elucidate the Bayes estimation of observable variables. The present paper applies this analysis to latent-variable estimation and determines its theoretical performance. Our results clarify behavior of the convergence of the posterior distribution. It is found that the posterior of the observable-variable estimation can be different from the one in the latent-variable estimation. Because of the difference, the Markov chain Monte Carlo method based on the parameter and the latent variable cannot construct the desired posterior distribution.

Full work available at URL: https://arxiv.org/abs/1205.3234

Recommendations

• Bayesian estimation of multidimensional latent variables and its asymptotic accuracy
• Asymptotic accuracy of distribution-based estimation of latent variables
• Asymptotic accuracy of Bayesian estimation for a single latent variable
• Accuracy of latent-variable estimation in Bayesian semi-supervised learning
• Equations of states in singular statistical estimation

zbMATH Keywords

unsupervised learningalgebraic geometryBayes statisticshierarchical parametric modelssingularities

Mathematics Subject Classification ID

Bayesian inference (62F15) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Estimating the dimension of a model
• A new look at the statistical model identification
• Hyper Markov laws in the statistical analysis of decomposable graphical models
• Convergence rates of posterior distributions.
• Identifiability of parameters in latent structure models with many observed variables
• Convergence of estimates under dimensionality restrictions
• Title not available (Why is that?)
• Stochastic complexity and modeling
• Singularities in mixture models and upper bounds of stochastic complexity.
• Asymptotic analysis of Bayesian generalization error with Newton diagram
• Convergence of latent mixing measures in finite and infinite mixture models
• Stochastic complexities of reduced rank regression in Bayesian estimation
• Resolution of singularities of an algebraic variety over a field of characteristic zero. I
• Algebraic analysis for nonidentifiable learning machines
• Stochastic complexity and generalization error of a restricted Boltzmann machine in Bayesian estimation
• Algebraic Geometry and Statistical Learning Theory
• Title not available (Why is that?)
• Title not available (Why is that?)
• Comparing two Bayes methods based on the free energy functions in Bernoulli mixtures
• An asymptotic behaviour of the marginal likelihood for general Markov models
• Resolution of Singularities and Division of Distributions

Cited In (8)

• Information criteria and cross validation for Bayesian inference in regular and singular cases
• The exact asymptotic form of Bayesian generalization error in latent Dirichlet allocation
• Recent advances in algebraic geometry and Bayesian statistics
• Accuracy of latent-variable estimation in Bayesian semi-supervised learning
• Asymptotic accuracy of Bayesian estimation for a single latent variable
• Asymptotic accuracy of distribution-based estimation of latent variables
• Bayesian estimation of multidimensional latent variables and its asymptotic accuracy
• Effects of additional data on Bayesian clustering

This page was built for publication: Asymptotic accuracy of Bayes estimation for latent variables with redundancy

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q255357)