On the Information Dimension of Stochastic Processes

DOI10.1109/TIT.2019.2922186MaRDI QIDQ5211458zbMATH OpenOpenAlexFDO

Authors Bernhard C. Geiger, Tobias Koch

Publication date 28 January 2020

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1702.00645

Mathematics Subject Classification ID

General theory of stochastic processes (60G07) Stationary stochastic processes (60G10) Gaussian processes (60G15) Measures of information, entropy (94A17)

Abstract: In 1959, R'enyi proposed the information dimension and the $d$-dimensional entropy to measure the information content of general random variables. This paper proposes a generalization of information dimension to stochastic processes by defining the information dimension rate as the entropy rate of the uniformly-quantized stochastic process divided by minus the logarithm of the quantizer step size $1/m$ in the limit as $moinfty$. It is demonstrated that the information dimension rate coincides with the rate-distortion dimension, defined as twice the rate-distortion function $R(D)$ of the stochastic process divided by $-log(D)$ in the limit as $Ddownarrow0$. It is further shown that, among all multivariate stationary processes with a given (matrix-valued) spectral distribution function (SDF), the Gaussian process has the largest information dimension rate, and that the information dimension rate of multivariate stationary Gaussian processes is given by the average rank of the derivative of the SDF. The presented results reveal that the fundamental limits of almost zero-distortion recovery via compressible signal pursuit and almost lossless analog compression are different in general.

Cited in

(10)

• On Rényi information for ergodic diffusion processes
• Proper complex random processes with applications to information theory
• Information upper bound for McKean–Vlasov stochastic differential equations
• Around the variational principle for metric mean dimension
• Information dimension of Galton board
• On Information/Entropy Flow in Stochastic Dynamical Systems
• Information rates of autoregressive processes
• Spatial ergodicity dimension: ergodic self-similarity of trajectories in discrete chaotic systems
• Regularity of almost-surely injective projections in Euclidean spaces
• Stochastic accumulation of information in discrete time: Comparing exact results and Wald approximations

This page was built for publication: On the Information Dimension of Stochastic Processes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5211458)