Dimension of Marginals of Kronecker Product Models

DOI10.1137/16M1077489zbMATH Open1383.14019arXiv1511.03570MaRDI QIDQ5347296FDOQ5347296

Guido Montúfar, Jason Morton

Publication date: 23 May 2017

Published in: SIAM Journal on Applied Algebra and Geometry (Search for Journal in Brave)

Abstract: A Kronecker product model is the set of visible marginal probability distributions of an exponential family whose sufficient statistics matrix factorizes as a Kronecker product of two matrices, one for the visible variables and one for the hidden variables. We estimate the dimension of these models by the maximum rank of the Jacobian in the limit of large parameters. The limit is described by the tropical morphism; a piecewise linear map with pieces corresponding to slicings of the visible matrix by the normal fan of the hidden matrix. We obtain combinatorial conditions under which the model has the expected dimension, equal to the minimum of the number of natural parameters and the dimension of the ambient probability simplex. Additionally, we prove that the binary restricted Boltzmann machine always has the expected dimension.

Full work available at URL: https://arxiv.org/abs/1511.03570

zbMATH Keywords

Kronecker productHadamard producttropical geometryrestricted Boltzmann machineparameter identifiabilitysecant varietyKhatri-Rao productexpected dimensionexponential family harmoniuminference function

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Characterization and structure theory of statistical distributions (62E10) Toric varieties, Newton polyhedra, Okounkov bodies (14M25) Combinatorial properties of polytopes and polyhedra (number of faces, shortest paths, etc.) (52B05)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Graphical Models, Exponential Families, and Variational Inference
• Learning Deep Architectures for AI
• On direct product matrices
• On the toric algebra of graphical models
• Lectures on algebraic statistics
• An implicitization challenge for binary factor analysis
• Stochastic complexity and generalization error of a restricted Boltzmann machine in Bayesian estimation
• Information Theory and Statistical Mechanics
• Secant varieties of ℙ¹×⋯×ℙ¹ (𝕟-times) are NOT defective for 𝕟≥5
• Secant varieties of Segre-Veronese varieties
• Induction for secant varieties of Segre varieties
• Support sets in exponential families and oriented matroid theory
• On the Alexander-Hirschowitz theorem
• A brief proof of a maximal rank theorem for generic double points in projective space
• Representational Power of Restricted Boltzmann Machines and Deep Belief Networks
• A tropical approach to secant dimensions
• Hierarchical models as marginals of hierarchical models
• Ranks of tensors, secant varieties of Segre varieties and fat points
• On the dimensions of secant varieties of Segre-Veronese varieties
• Decompositions of a Higher-Order Tensor in Block Terms—Part I: Lemmas for Partitioned Matrices
• Hierarchical Models, Marginal Polytopes, and Linear Codes
• Neighborliness of marginal polytopes
• Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines
• Mixture decompositions of exponential families using a decomposition of their sample spaces
• Discrete Restricted Boltzmann Machines
• When Does a Mixture of Products Contain a Product of Mixtures?

Recommendations

• Title not available (Why is that?) 👍 👎
• Title not available (Why is that?) 👍 👎
• Title not available (Why is that?) 👍 👎
• On Estimation of Covariance Matrices With Kronecker Product Structure 👍 👎
• Existence and uniqueness of the maximum likelihood estimator for models with a Kronecker product covariance structure 👍 👎
• Models with a Kronecker product covariance structure: estimation and testing 👍 👎
• Marginal-frechet-bounds for multidimensional distribution functions 👍 👎
• Dimension Estimation and Models 👍 👎
• On the dimension of product measures 👍 👎

This page was built for publication: Dimension of Marginals of Kronecker Product Models

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5347296)