On maximum entropy and minimum KL-divergence optimization by Gröbner basis methods

DOI10.1016/J.AMC.2012.05.052MaRDI QIDQ387708zbMATHOpenAlexFDO

Authors Ambedkar Dukkipati

Publication date 23 December 2013

Published in Applied Mathematics and Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0804.1083

zbMATH Keywords

Shannon entropyimplicitizationZariski closure

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Gröbner bases; other bases for ideals and modules (e.g., Janet and border bases) (13P10)

Abstract: We show that various formulations (e.g., dual and Kullback-Csiszar iterations) of estimation of maximum entropy (ME) models can be transformed to solving systems of polynomial equations in several variables for which one can use celebrated Grobner bases methods. Posing of ME estimation as solving polynomial equations is possible, in the cases where feature functions (sufficient statistic) that provides the information about the underlying random variable in the form of expectations are integer valued.

Recommendations

• An algebraic implicitization and specialization of minimum KL-divergence models
• Computational algebraic methods in efficient estimation
• The maximum likelihood degree
• Algebraic statistics and contingency table problems: log-linear models, likelihood estimatio, and disclosure limitation
• scientific article; zbMATH DE number 1894280

Cites work

• scientific article; zbMATH DE number 5777868 (Why is no real title available?)
• scientific article; zbMATH DE number 107482 (Why is no real title available?)
• scientific article; zbMATH DE number 1560708 (Why is no real title available?)
• scientific article; zbMATH DE number 835749 (Why is no real title available?)
• scientific article; zbMATH DE number 3241743 (Why is no real title available?)
• A geometric interpretation of Darroch and Ratcliff's generalized iterative scaling
• Algebraic Statistics for Computational Biology
• Algebraic algorithms for sampling from conditional distributions
• An algebraic implicitization and specialization of minimum KL-divergence models
• Contingency tables with given marginals
• Generalized Iterative Scaling for Log-Linear Models
• Gröbner bases for ideals in Laurent polynomial rings and their application to systems of difference equations
• I-divergence geometry of probability distributions and minimization problems
• Information Theory and Statistical Mechanics
• Information Theory and Statistics: A Tutorial
• Lifting standard bases in filtered structures
• Prior Probabilities
• Probability Densities with Given Marginals
• Properties of Gröbner bases under specializations
• Properties of cross-entropy minimization
• Solving the likelihood equations

Cited in

(4)

• On the computability of continuous maximum entropy distributions with applications
• An algebraic implicitization and specialization of minimum KL-divergence models
• Probabilistic knowledge representation using the principle of maximum entropy and Gröbner basis theory
• Computational algebraic methods in efficient estimation

This page was built for publication: On maximum entropy and minimum KL-divergence optimization by Gröbner basis methods

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q387708)