A Precise Bare Simulation Approach to the Minimization of Some Distances. I. Foundations

DOI10.1109/TIT.2022.3215496MaRDI QIDQ6197485zbMATH OpenOpenAlexFDO

Authors M. Broniatowski, W. Stummer

Publication date 19 March 2024

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2107.01693

zbMATH Keywords

importance samplingTsallis (cross) entropiesShannon entropyRenyi entropiesRenyi divergencesrelative entropypower divergencesKullback-Leibler information distanceJensen-Shannon divergence/distance\(f\)-divergences of Csiszar-Ali-Silvey-Morimoto typeHellinger distancegeneralized maximum entropy methodEuclidean normsCressie-Read measuresBhattacharyya distanceBhattacharyya coefficientalpha-divergences

Mathematics Subject Classification ID

Information and communication theory, circuits (94-XX)

Abstract: In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively maximization) problems by a newly developed dimension-free bare (pure) simulation method, is the main goal of this paper. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). As a side effect, we also derive an innovative way of constructing new useful distances/divergences. To illustrate the core of our approach, we present numerous solved cases. The potential for widespread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences and entropies in various different research fields (which may also serve as an interdisciplinary interface).

Cited in

(5)

• On a cornerstone of bare-simulation distance/divergence optimization
• Empirical likelihood based confidence regions for functional of copulas
• Some smooth divergences for _1-approximations
• Shape analysis and comparison of audio patterns using divergence measures.
• Foundations of bare-simulation optimization of distances

This page was built for publication: A Precise Bare Simulation Approach to the Minimization of Some Distances. I. Foundations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6197485)