Minimum KL-Divergence on Complements of <inline-formula> <tex-math notation="TeX">L_{1} </tex-math></inline-formula> Balls

DOI10.1109/TIT.2014.2301446zbMATH Open1360.94142arXiv1206.6544OpenAlexW2027974842MaRDI QIDQ2986314FDOQ2986314

Authors: Daniel Berend, Peter Harremoës, Aryeh Kontorovich 

Publication date: 16 May 2017

Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)

Abstract: Pinsker's widely used inequality upper-bounds the total variation distance $||P-Q|{|}_1$ in terms of the Kullback-Leibler divergence $D(P||Q)$. Although in general a bound in the reverse direction is impossible, in many applications the quantity of interest is actually $D^{\ast }(P,eps)$ --- defined, for an arbitrary fixed $P$, as the infimum of $D(P||Q)$ over all distributions $Q$ that are $eps$-far away from $P$ in total variation. We show that $D^{\ast }(P,eps)leCep{s}^2+O(ep{s}^3)$, where $C=C(P)=1/2$ for "balanced" distributions, thereby providing a kind of reverse Pinsker inequality. An application to large deviations is given, and some of the structural results may be of independent interest. Keywords: Pinsker inequality, Sanov's theorem, large deviations

Full work available at URL: https://arxiv.org/abs/1206.6544

Mathematics Subject Classification ID

Inequalities; stochastic orderings (60E15) Measures of information, entropy (94A17)

Cited In (1)

• Geographic spatiotemporal big data correlation analysis via the Hilbert-Huang transformation

This page was built for publication: Minimum KL-Divergence on Complements of <inline-formula> <tex-math notation="TeX">$L_{1}$ </tex-math></inline-formula> Balls

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2986314)