Minimum KL-Divergence on Complements of <inline-formula> <tex-math notation="TeX">L_{1} </tex-math></inline-formula> Balls

DOI10.1109/TIT.2014.2301446MaRDI QIDQ2986314zbMATH OpenOpenAlexFDO

Authors Daniel Berend, Peter Harremoës, Aryeh Kontorovich

Publication date 16 May 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1206.6544

Mathematics Subject Classification ID

Inequalities; stochastic orderings (60E15) Measures of information, entropy (94A17)

Abstract: Pinsker's widely used inequality upper-bounds the total variation distance $||P-Q|{|}_1$ in terms of the Kullback-Leibler divergence $D(P||Q)$. Although in general a bound in the reverse direction is impossible, in many applications the quantity of interest is actually $D^{\ast }(P,eps)$ --- defined, for an arbitrary fixed $P$, as the infimum of $D(P||Q)$ over all distributions $Q$ that are $eps$-far away from $P$ in total variation. We show that $D^{\ast }(P,eps)leCep{s}^2+O(ep{s}^3)$, where $C=C(P)=1/2$ for "balanced" distributions, thereby providing a kind of reverse Pinsker inequality. An application to large deviations is given, and some of the structural results may be of independent interest. Keywords: Pinsker inequality, Sanov's theorem, large deviations

Cited in

(1)

• Geographic spatiotemporal big data correlation analysis via the Hilbert-Huang transformation

This page was built for publication: Minimum KL-Divergence on Complements of <inline-formula> <tex-math notation="TeX">$L_{1}$ </tex-math></inline-formula> Balls

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2986314)