Coding on Countably Infinite Alphabets

DOI10.1109/TIT.2008.2008150MaRDI QIDQ4975701zbMATH OpenDBLPWikidataFDO

Authors Stéphane Boucheron, Elisabeth Gassiat, Aurélien Garivier

Publication date 8 August 2017

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0801.2456

Mathematics Subject Classification ID

Coding theorems (Shannon theory) (94A24) Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.) (aspects in computer science) (68P30) Bounds on codes (94B65)

Abstract: This paper describes universal lossless coding strategies for compressing sources on countably infinite alphabets. Classes of memoryless sources defined by an envelope condition on the marginal distribution provide benchmarks for coding techniques originating from the theory of universal coding over finite alphabets. We prove general upper-bounds on minimax regret and lower-bounds on minimax redundancy for such source classes. The general upper bounds emphasize the role of the Normalized Maximum Likelihood codes with respect to minimax regret in the infinite alphabet context. Lower bounds are derived by tailoring sharp bounds on the redundancy of Krichevsky-Trofimov coders for sources over finite alphabets. Up to logarithmic (resp. constant) factors the bounds are matching for source classes defined by algebraically declining (resp. exponentially vanishing) envelopes. Effective and (almost) adaptive coding techniques are described for the collection of source classes defined by algebraically vanishing envelopes. Those results extend ourknowledge concerning universal coding to contexts where the key tools from parametric inference

Cited in

(9)

• Some properties of Rényi entropy over countably infinite alphabets
• Universal coding for memoryless sources with countably infinite alphabets
• Alphabetic coding with exponential costs
• A Bernstein-von Mises theorem for discrete probability distributions
• A coding theorem for enumerable output machines
• The Universality of Grammar-Based Codes for Sources With Countably Infinite Alphabets
• Foundations of Software Science and Computation Structures
• On Finite Alphabets and Infinite Bases III: Simulation
• On finite alphabets and infinite bases

This page was built for publication: Coding on Countably Infinite Alphabets

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4975701)