Entropy and information theory
channelscoding theoremsconditional entropyconditional informationdiscriminationentropymutual informationprobability information measuresrelative entropyShannon's mathematical theory of communicationsources
Statistical aspects of information-theoretic topics (62B10) Introductory exposition (textbooks, tutorial papers, etc.) pertaining to information and communication theory (94-01) Information theory (general) (94A15) Measures of information, entropy (94A17) Coding theorems (Shannon theory) (94A24) Source coding (94A29) Channel models (including quantum) in information and communication theory (94A40)
The book is devoted to the theory of probability information measures and their applications to coding theorems for information sources and noisy channels. The author makes a general development of Shannon's mathematical theory of communication and develops tools and methods to prove the Shannon coding theorems. Many of the results on information measures for abstract alphabets and their convergence properties are extensions of generalizations of Shannon's original results. In considering standard alphabets rather than completely abstract alphabets, the mathematical models are somewhat less general, but this restriction, however, permits many strong results as well as the extensions to nonergodic processes. The information convergence results are combined with ergodic theorems to prove the general Shannon coding theorem for sources and channels. These results are sufficiently general to cover most systems encountered in applications. Much of the book is concerned with the properties of entropy, mutual information, conditional entropy, conditional information, and discrimination or relative entropy, along with the limiting normalized versions of these quantities such as entropy rate, and they provide an introduction to recent extensions requiring significant additional mathematical machinery. The book is self-contained with the exception of common results which may be found in the book ``Probability, random processes, and ergodic properties by the same author (New York 1988; Zbl 0644.60001).
- Entropy and information theory.
- Probability, Random Processes, and Ergodic Properties
- scientific article; zbMATH DE number 2131215
- Information-spectrum methods in information theory
- scientific article; zbMATH DE number 936158
- scientific article; zbMATH DE number 918233
- Abstract methods in information theory
- The interactions between ergodic theory and information theory
- On the different extensions of the ergodic theorem of information theory
- An introduction to single-user information theory
- Counting distinctions: on the conceptual foundations of Shannon's information theory
- Variational Bayesian identification and prediction of stochastic nonlinear dynamic causal models
- A note on the information content of a consistent pairwise comparison judgment matrix of an AHP decision maker
- Pricing strategy versus heterogeneous shopping behavior under market price dispersion
- Monitoring threshold functions over distributed data streams with node dependent constraints
- Evolution of altruism in spatial prisoner's dilemma: intra- and inter-cellular interactions
- Noise-induced chaos and phase space flux
- Optimal data compression algorithm
- The Markov approximation of the sequences of \(N\)-valued random variables and a class of small deviation theorems.
- Shannon information and power law analysis of the chromosome code
- Dynamics of Bayesian updating with dependent data and misspecified models
- Automatic human gait imitation and recognition in 3D from monocular video with an uncalibrated camera
- Random fractional generalized Airy differential equations: a probabilistic analysis using mean square calculus
- Euclidean forward-reverse Brascamp-Lieb inequalities: finiteness, structure, and extremals
- Information vs. entropy vs. probability
- Robustness to incorrect models and data-driven learning in average-cost optimal stochastic control
- On information gain, Kullback-Leibler divergence, entropy production and the involution kernel
- Fractional derivatives and negative probabilities
- Double variational principle for mean dimension with potential
- Multiscale fractional-order approximate entropy analysis of financial time series based on the cumulative distribution matrix
- Double variational principle for mean dimension
- Statistical mechanics of choice: MaxEnt estimation of population heterogeneity
- Asymptotics of input-constrained binary symmetric channel capacity
- A sufficient condition for a unique invariant distribution of a higher-order Markov chain
- Annealing and the normalized N-cut
- Gelfand-Yaglom-Perez theorem for generalized relative entropy functionals
- Recall and reasoning -- an information theoretical model of cognitive processes
- The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems
- Compressibility analysis of asymptotically mean stationary processes
- Shannon information entropy for a quantum nonlinear oscillator on a space of non-constant curvature
- Information-theoretic approach to the problem of automatic image recognition
- Information measures. Information and its description in science and engineering
- Stationary and ergodic properties of stochastic nonlinear systems controlled over communication channels
- On information theory and its applications
- Entropy and information theory.
- scientific article; zbMATH DE number 2131215 (Why is no real title available?)
- Abstract methods in information theory
- scientific article; zbMATH DE number 3136274 (Why is no real title available?)
- Two Remarks on the Basic Theorems of Information Theory.
- scientific article; zbMATH DE number 3173999 (Why is no real title available?)
- An inexact accelerated proximal gradient method and a dual Newton-CG method for the maximal entropy problem
- Information Theory, Relative Entropy and Statistics
- Variational characterization of the critical curve for pinning of random polymers
- scientific article; zbMATH DE number 4104849 (Why is no real title available?)
- Information dependency: strong consistency of Darbellay-Vajda partition estimators
- scientific article; zbMATH DE number 47293 (Why is no real title available?)
- scientific article; zbMATH DE number 53542 (Why is no real title available?)
- On the convergence of Shannon differential entropy, and its connections with density and entropy estimation
- scientific article; zbMATH DE number 1216200 (Why is no real title available?)
- scientific article; zbMATH DE number 1226591 (Why is no real title available?)
- Applying Shannon’s information theory to the measurement and analysis of financial statements
- The equality of Kolmogorov-Sinai entropy and metric permutation entropy generalized
- Decomposition of Kullback-Leibler risk and unbiasedness for parameter-free estimators
- scientific article; zbMATH DE number 1936712 (Why is no real title available?)
- Multiconcept classification of diagnostic knowledge to manufacturing systems: Analysis of incomplete data with continuous-valued attributes
- A multidimensional scaling analysis of musical sounds based on pseudo phase plane
- Application of wavelet entropy to predict atrial fibrillation progression from the surface ECG
- On Shannon's duality of a source and a channel and nonanticipative communication and communication for control
- Minimax Optimal Procedures for Locally Private Estimation
- Asymptotic stability of beneš filters
- Information-spectrum methods in information theory
- scientific article; zbMATH DE number 918233 (Why is no real title available?)
- Ergodicity conditions for controlled stochastic nonlinear systems under information constraints: a volume growth approach
- Symbolic dynamics in mean dimension theory
- A Medida de Informação de Shannon: Entropia
- Robustness to incorrect system models in stochastic control
- The resampling of entropies with the application of biodiversity
- On directed information theory and Granger causality graphs
- Robustness to approximations and model learning in MDPs and POMDPs
- Lumpings of Markov chains, entropy rate preservation, and higher-order lumpability
- Simple one-shot bounds for various source coding problems using smooth Rényi quantities
- Robustness to Incorrect Priors in Partially Observed Stochastic Control
- Some Shannon-McMillan approximation theorems for Markov chain field on the generalized Bethe tree
- Design of Information Channels for Optimization and Stabilization in Networked Control
- Privacy aware learning
- The Basic Theorems of Information Theory
- A Short Note on Almost Sure Convergence of Bayes Factors in the General Set-Up
- Varying c and particle horizons
- Distance, correlation and mutual information among portraits of organisms based on complete genomes
- On the tightness of linear policies for stabilization of linear systems over Gaussian networks
- Shannon's Entropy and Its Generalisations Towards Statistical Inference in Last Seven Decades
- Complexities of information sources
- Calculation of the anisotropic norm of the descriptor system
- Information Theory
- Some estimations of the Jensen difference and applications
- Approximation of stationary processes by hidden Markov models
- Mathematics of information. Theory and applications of Shannon-Wiener information. Translated from the German
- Sumsets and entropy revisited
- Multiphase flow modeling with density functional method
- The Fock-Darwin-Darboux system: eigenstates, information entropies and dispersion-like measures
- Rényi and Tsallis information entropies for the Darboux III quantum nonlinear oscillator
- Generalized twin Gaussian processes using Sharma-Mittal divergence
- On signal representations within the Bayes decision framework
- Algebraic and Shannon entropies of commutative hypergroups and their connection with information and permutation entropies and with calculation of entropy for chemical algebras
- Analysis of time series through complexity-entropy curves based on generalized fractional entropy
- Measuring synchronization in coupled model systems: a comparison of different approaches
- Entropy, search, complexity.
- Anisotropy-based robust performance analysis of finite horizon linear discrete time varying systems
- Solution of the stochastic \(H_{\infty }\)-optimization problem for discrete time linear systems under parametric uncertainty
- Computational mechanics of input-output processes: structured transformations and the \(\epsilon\)-transducer
This page was built for publication: Entropy and information theory
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3997311)