Entropy and information theory

From MaRDI portal





The book is devoted to the theory of probability information measures and their applications to coding theorems for information sources and noisy channels. The author makes a general development of Shannon's mathematical theory of communication and develops tools and methods to prove the Shannon coding theorems. Many of the results on information measures for abstract alphabets and their convergence properties are extensions of generalizations of Shannon's original results. In considering standard alphabets rather than completely abstract alphabets, the mathematical models are somewhat less general, but this restriction, however, permits many strong results as well as the extensions to nonergodic processes. The information convergence results are combined with ergodic theorems to prove the general Shannon coding theorem for sources and channels. These results are sufficiently general to cover most systems encountered in applications. Much of the book is concerned with the properties of entropy, mutual information, conditional entropy, conditional information, and discrimination or relative entropy, along with the limiting normalized versions of these quantities such as entropy rate, and they provide an introduction to recent extensions requiring significant additional mathematical machinery. The book is self-contained with the exception of common results which may be found in the book ``Probability, random processes, and ergodic properties by the same author (New York 1988; Zbl 0644.60001).




Cited in
(only showing first 100 items - show all)








This page was built for publication: Entropy and information theory

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3997311)