Optimally approximating exponential families

MaRDI QIDQ5298775zbMATH OpenFDO

Authors Johannes Rauh

Publication date 14 June 2013

Full work available at URL https://arxiv.org/abs/1111.0483, http://www.kybernetika.cz/content/2013/2/199

zbMATH Keywords

exponential familyinformation divergence

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Information theory (general) (94A15) Measures of information, entropy (94A17)

Abstract: This article studies exponential families $mathcalE$ on finite sets such that the information divergence $D(P|mathcalE)$ of an arbitrary probability distribution from $mathcalE$ is bounded by some constant $D>0$. A particular class of low-dimensional exponential families that have low values of $D$ can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponential families. Exponential families where $D=log(2)$ are studied in detail. This case is special, because if $D<log(2)$, then $mathcalE$ contains all probability measures with full support.

Recommendations

• Optimality conditions for maximizers of the information divergence from an exponent family
• On limiting towards the boundaries of exponential families.
• The exponential family in abstract information theory
• scientific article; zbMATH DE number 1894364
• Closures of exponential families

Cites work

• An information-geometric approach to a theory of pragmatic structuring
• Finding the Maximizers of the Information Divergence From an Exponential Family
• Generalized maximum likelihood estimates for exponential families
• scientific article; zbMATH DE number 420868 (Why is no real title available?)
• scientific article; zbMATH DE number 44577 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• scientific article; zbMATH DE number 1134987 (Why is no real title available?)
• Information Theory and Statistical Mechanics
• Information Theory and Statistics: A Tutorial
• Lectures on algebraic statistics
• Locality of Global Stochastic Interaction in Directed Acyclic Networks
• Maximization of the information divergence from multinomial distributions
• On the toric algebra of graphical models
• Support sets in exponential families and oriented matroid theory

Cited in

(11)

• Approximation of density functions by sequences of exponential families
• Using specially designed exponential families for density estimation
• Rank verification for exponential families
• Closures of exponential families
• scientific article; zbMATH DE number 4143239 (Why is no real title available?)
• Mixture decompositions of exponential families using a decomposition of their sample spaces
• Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units
• Optimality conditions for maximizers of the information divergence from an exponent family
• Scaling of model approximation errors and expected entropy distances
• Maximum information divergence from linear and toric models
• Estimation in exponential families on permutations

This page was built for publication: Optimally approximating exponential families

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5298775)