Information Theory With Finite Vector Spaces

DOI10.1109/TIT.2019.2907590MaRDI QIDQ5211397zbMATH OpenFDO

Authors Juan Pablo Vigneaux

Publication date 28 January 2020

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1807.05152

zbMATH Keywords

q-deformationquadratic entropyq-multinomial coefficientscounting of flags of finite vector spaces with prescribed length and dimensionscounting of vector subspaces of given dimensionTsallis 2-entropy

Mathematics Subject Classification ID

Measures of information, entropy (94A17) Coding theorems (Shannon theory) (94A24)

Abstract: Whereas Shannon entropy is related to the growth rate of multinomial coefficients, we show that the quadratic entropy (Tsallis 2-entropy) is connected to their $q$-deformation; when $q$ is a prime power, these $q$-multinomial coefficients count flags of finite vector spaces with prescribed length and dimensions. In particular, the $q$-binomial coefficients count vector subspaces of given dimension. We obtain this way a combinatorial explanation for the nonadditivity of the quadratic entropy, which arises from a recursive counting of flags. We show that statistical systems whose configurations are described by flags provide a frequentist justification for the maximum entropy principle with Tsallis statistics. We introduce then a discrete-time stochastic process associated to the $q$-binomial probability distribution, that generates at time $n$ a vector subspace of $mathbb{F}_q^n$ (here $mathbb{F}_q$ is the finite field of order $q$). The concentration of measure on certain "typical subspaces" allows us to extend the asymptotic equipartition property to this setting. The size of the typical set is quantified by the quadratic entropy. We discuss the applications to Shannon theory, particularly to source coding, when messages correspond to vector spaces.

Cited in

(1)

• A characterization of generalized multinomial coefficients related to the entropic chain rule

This page was built for publication: Information Theory With Finite Vector Spaces

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5211397)