Asymptotic accuracy of Bayes estimation for latent variables with redundancy
From MaRDI portal
(Redirected from Publication:255357)
Abstract: Hierarchical parametric models consisting of observable and latent variables are widely used for unsupervised learning tasks. For example, a mixture model is a representative hierarchical model for clustering. From the statistical point of view, the models can be regular or singular due to the distribution of data. In the regular case, the models have the identifiability; there is one-to-one relation between a probability density function for the model expression and the parameter. The Fisher information matrix is positive definite, and the estimation accuracy of both observable and latent variables has been studied. In the singular case, on the other hand, the models are not identifiable and the Fisher matrix is not positive definite. Conventional statistical analysis based on the inverse Fisher matrix is not applicable. Recently, an algebraic geometrical analysis has been developed and is used to elucidate the Bayes estimation of observable variables. The present paper applies this analysis to latent-variable estimation and determines its theoretical performance. Our results clarify behavior of the convergence of the posterior distribution. It is found that the posterior of the observable-variable estimation can be different from the one in the latent-variable estimation. Because of the difference, the Markov chain Monte Carlo method based on the parameter and the latent variable cannot construct the desired posterior distribution.
Recommendations
- Bayesian estimation of multidimensional latent variables and its asymptotic accuracy
- Asymptotic accuracy of distribution-based estimation of latent variables
- Asymptotic accuracy of Bayesian estimation for a single latent variable
- Accuracy of latent-variable estimation in Bayesian semi-supervised learning
- Equations of states in singular statistical estimation
Cites work
- scientific article; zbMATH DE number 5957252 (Why is no real title available?)
- scientific article; zbMATH DE number 3733065 (Why is no real title available?)
- scientific article; zbMATH DE number 1222289 (Why is no real title available?)
- A new look at the statistical model identification
- Algebraic Geometry and Statistical Learning Theory
- Algebraic analysis for nonidentifiable learning machines
- An asymptotic behaviour of the marginal likelihood for general Markov models
- Asymptotic analysis of Bayesian generalization error with Newton diagram
- Comparing two Bayes methods based on the free energy functions in Bernoulli mixtures
- Convergence of estimates under dimensionality restrictions
- Convergence of latent mixing measures in finite and infinite mixture models
- Convergence rates of posterior distributions.
- Estimating the dimension of a model
- Hyper Markov laws in the statistical analysis of decomposable graphical models
- Identifiability of parameters in latent structure models with many observed variables
- Resolution of Singularities and Division of Distributions
- Resolution of singularities of an algebraic variety over a field of characteristic zero. I
- Singularities in mixture models and upper bounds of stochastic complexity.
- Stochastic complexities of reduced rank regression in Bayesian estimation
- Stochastic complexity and generalization error of a restricted Boltzmann machine in Bayesian estimation
- Stochastic complexity and modeling
Cited in
(8)- Bayesian estimation of multidimensional latent variables and its asymptotic accuracy
- Information criteria and cross validation for Bayesian inference in regular and singular cases
- Asymptotic accuracy of distribution-based estimation of latent variables
- Recent advances in algebraic geometry and Bayesian statistics
- Effects of additional data on Bayesian clustering
- The exact asymptotic form of Bayesian generalization error in latent Dirichlet allocation
- Accuracy of latent-variable estimation in Bayesian semi-supervised learning
- Asymptotic accuracy of Bayesian estimation for a single latent variable
This page was built for publication: Asymptotic accuracy of Bayes estimation for latent variables with redundancy
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q255357)
