Joint Fixed-Rate Universal Lossy Coding and Identification of Continuous-AlphabetMemoryless Sources

DOI10.1109/TIT.2008.924669MaRDI QIDQ3604642zbMATHOpenAlexFDO

Authors Maxim Raginsky

Publication date 24 February 2009

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/cs/0512015

zbMATH Keywords

learningVapnik-Chervonenkis dimensionminimum-distance density estimationtwo-stage codesuniversal vector quantization

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Detection theory in information and communication theory (94A13) Bounds on codes (94B65)

Abstract: The problem of joint universal source coding and identification is considered in the setting of fixed-rate lossy coding of continuous-alphabet memoryless sources. For a wide class of bounded distortion measures, it is shown that any compactly parametrized family of $R^d$-valued i.i.d. sources with absolutely continuous distributions satisfying appropriate smoothness and Vapnik--Chervonenkis learnability conditions, admits a joint scheme for universal lossy block coding and parameter estimation, such that when the block length $n$ tends to infinity, the overhead per-letter rate and the distortion redundancies converge to zero as $O(n^{-1}logn)$ and $O(sqrt{n}^{-1}logn)$, respectively. Moreover, the active source can be determined at the decoder up to a ball of radius $O(sqrt{n}^{-1}logn)$ in variational distance, asymptotically almost surely. The system has finite memory length equal to the block length, and can be thought of as blockwise application of a time-invariant nonlinear filter with initial conditions determined from the previous block. Comparisons are presented with several existing schemes for universal vector quantization, which do not include parameter estimation explicitly, and an extension to unbounded distortion measures is outlined. Finally, finite mixture classes and exponential families are given as explicit examples of parametric sources admitting joint universal compression and modeling schemes of the kind studied here.

Recommendations

• Fixed-rate universal lossy source coding and rates of convergence for memoryless sources
• Strong universal source coding subject to a rate-distortion constraint
• The redundancy of source coding with a fidelity criterion .II. Coding at a fixed rate level with unknown statistics
• The Empirical Distribution of Rate-Constrained Source Codes
• Arbitrary source models and Bayesian codebooks in rate-distortion theory

Cited in

(2)

• Fixed-rate universal lossy source coding and rates of convergence for memoryless sources
• L-Identification for Uniformly Distributed Sources and the q-ary Identification Entropy of Second Order

This page was built for publication: Joint Fixed-Rate Universal Lossy Coding and Identification of Continuous-AlphabetMemoryless Sources

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3604642)