Entropy Bounds for Discrete Random Variables via Maximal Coupling

DOI10.1109/TIT.2013.2274515MaRDI QIDQ5346243zbMATH OpenFDO

Authors Igal Sason

Publication date 8 June 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1209.5259

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Measures of information, entropy (94A17)

Abstract: This paper derives new bounds on the difference of the entropies of two discrete random variables in terms of the local and total variation distances between their probability mass functions. The derivation of the bounds relies on maximal coupling, and they apply to discrete random variables which are defined over finite or countably infinite alphabets. Loosened versions of these bounds are demonstrated to reproduce some previously reported results. The use of the new bounds is exemplified for the Poisson approximation, where bounds on the local and total variation distances follow from Stein's method.

Cited in

(18)

• On coupling of probability distributions and estimating the divergence through variation
• An upper bound for entropy of discrete distributions having assigned moments
• Maximal correlation and monotonicity of free entropy and of Stein discrepancy
• Entropy and the fourth moment phenomenon
• On one extreme value problem for entropy and error probability
• Nonuniform bounds in the Poisson approximation with applications to informational distances. II
• On the entropy of couplings
• On some extremal problems for mutual information and entropy
• Improved lower bounds on the total variation distance for the Poisson approximation
• Coupling of probability distributions and an extremal problem for the divergence
• Tight uniform continuity bounds for quantum entropies: conditional entropy, relative entropy distance and energy constraints
• scientific article; zbMATH DE number 1103094 (Why is no real title available?)
• A bound on expectation values and variances of quantum observables via Rényi entropy and Tsallis entropy
• On the Maximum Entropy of a Sum of Independent Discrete Random Variables
• Coupling of several random variables
• Gaussian concentration bounds for stochastic chains of unbounded memory
• A note on three new bounds of Tsallis entropy
• Essential barrier height and a probabilistic approach in characterizing potential landscape

This page was built for publication: Entropy Bounds for Discrete Random Variables via Maximal Coupling

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5346243)