Proximity Operators of Discrete Information Divergences

DOI10.1109/TIT.2017.2782789MaRDI QIDQ4566688zbMATH OpenFDO

Authors M. El-Gheche, Giovanni Chierchia, Jean-Christophe Pesquet

Publication date 27 June 2018

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1606.09552

Mathematics Subject Classification ID

Measures of information, entropy (94A17)

Abstract: Information divergences allow one to assess how close two distributions are from each other. Among the large panel of available measures, a special attention has been paid to convex $varphi$-divergences, such as Kullback-Leibler, Jeffreys-Kullback, Hellinger, Chi-Square, Renyi, and I${}_{alpha}$ divergences. While $varphi$-divergences have been extensively studied in convex analysis, their use in optimization problems often remains challenging. In this regard, one of the main shortcomings of existing methods is that the minimization of $varphi$-divergences is usually performed with respect to one of their arguments, possibly within alternating optimization techniques. In this paper, we overcome this limitation by deriving new closed-form expressions for the proximity operator of such two-variable functions. This makes it possible to employ standard proximal methods for efficiently solving a wide range of convex optimization problems involving $varphi$-divergences. In addition, we show that these proximity operators are useful to compute the epigraphical projection of several functions of practical interest. The proposed proximal tools are numerically validated in the context of optimal query execution within database management systems, where the problem of selectivity estimation plays a central role. Experiments are carried out on small to large scale scenarios.

Cited in

(4)

• Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists
• Enhanced computation of the proximity operator for perspective functions
• Proximity operators of perspective functions with nonlinear scaling
• Bregman proximal gradient method for linear optimization under entropic constraints

This page was built for publication: Proximity Operators of Discrete Information Divergences

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4566688)