Minimization Problems Based on Relative <inline-formula> <tex-math notation="LaTeX">\alpha </tex-math></inline-formula>-Entropy I: Forward Projection

DOI10.1109/TIT.2015.2449311MaRDI QIDQ2977304zbMATH OpenOpenAlexFDO

Authors M. Ashok Kumar, Rajesh Sundaresan

Publication date 28 April 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1410.2346

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Abstract: Minimization problems with respect to a one-parameter family of generalized relative entropies are studied. These relative entropies, which we term relative $alpha$-entropies (denoted $mathscr{I}_{alpha}$), arise as redundancies under mismatched compression when cumulants of compressed lengths are considered instead of expected compressed lengths. These parametric relative entropies are a generalization of the usual relative entropy (Kullback-Leibler divergence). Just like relative entropy, these relative $alpha$-entropies behave like squared Euclidean distance and satisfy the Pythagorean property. Minimizers of these relative $alpha$-entropies on closed and convex sets are shown to exist. Such minimizations generalize the maximum R'{e}nyi or Tsallis entropy principle. The minimizing probability distribution (termed forward $mathscr{I}_{alpha}$-projection) for a linear family is shown to obey a power-law. Other results in connection with statistical inference, namely subspace transitivity and iterated projections, are also established. In a companion paper, a related minimization problem of interest in robust statistics that leads to a reverse $mathscr{I}_{alpha}$-projection is studied.

Recommendations

• Minimization Problems Based on Relative <inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-Entropy II: Reverse Projection
• Minimization under entropy conditions, with applications in lower bound problems
• Projection pursuit through relative entropy minimization
• Proximal averages for minimization of entropy functionals
• Minimization of entropy functionals
• Minimax entropy solutions of ill-posed problems
• Entropy function-based algorithms for solving a class of nonconvex minimization problems
• On entropy minimization and convergence
• Entropy-Like Minimization Methods Based On Modified Proximal Point Algorithm
• On the entropic regularization method for solving min-max problems with applications

Cited in

(5)

• Cramér-Rao lower bounds arising from generalized Csiszár divergences
• Projection theorems and estimating equations for power-law models
• Forward mean proximal pairs and zero entropy
• Optimal guessing under nonextensive framework and associated moment bounds
• A unified approach to the Pythagorean identity and projection theorem for a class of divergences based on M-estimations

This page was built for publication: Minimization Problems Based on Relative <inline-formula> <tex-math notation="LaTeX">$\alpha $ </tex-math></inline-formula>-Entropy I: Forward Projection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2977304)