Entropy and Source Coding for Integer-Dimensional Singular Random Variables

DOI10.1109/TIT.2016.2604248MaRDI QIDQ2976490zbMATH OpenOpenAlexFDO

Authors Günther Koliander, Georg Pichler, Erwin Riegler, Franz Hlawatsch

Publication date 28 April 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1505.03337

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Source coding (94A29)

Abstract: Entropy and differential entropy are important quantities in information theory. A tractable extension to singular random variables-which are neither discrete nor continuous-has not been available so far. Here, we present such an extension for the practically relevant class of integer-dimensional singular random variables. The proposed entropy definition contains the entropy of discrete random variables and the differential entropy of continuous random variables as special cases. We show that it transforms in a natural manner under Lipschitz functions, and that it is invariant under unitary transformations. We define joint entropy and conditional entropy for integer-dimensional singular random variables, and we show that the proposed entropy conveys useful expressions of the mutual information. As first applications of our entropy definition, we present a result on the minimal expected codeword length of quantized integer-dimensional singular sources and a Shannon lower bound for integer-dimensional singular sources.

Recommendations

• scientific article; zbMATH DE number 3920365
• The entropy theorem via coding bounds
• Source coding with escort distributions and Rényi entropy bounds
• On a generalized entropy and a coding theorem
• Generalized entropies in coding theory
• scientific article; zbMATH DE number 3930883
• Entropy encoding, Hilbert space, and Karhunen-Loève transforms
• A coding theorem on generalized R-norm entropy
• Entropy, universal coding, approximation, and bases properties
• A coding theorem connected on R-norm entropy

Cited in

(4)

• Entropy under disintegrations
• Entropy encoding, Hilbert space, and Karhunen-Loève transforms
• On the entropy of rectifiable and stratified measures
• The checkerboard copula and dependence concepts

This page was built for publication: Entropy and Source Coding for Integer-Dimensional Singular Random Variables

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2976490)