scientific article; zbMATH DE number 2061729
Point estimation (62F10) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Computer system organization (68Mxx) Coding and information theory (compaction, compression, models of communication, encoding schemes, etc.) (aspects in computer science) (68P30) Introductory exposition (textbooks, tutorial papers, etc.) pertaining to information and communication theory (94-01)
This textbook is geared toward upper-level undergraduate students and graduate students in engineering, science, mathematics, and computer science. Prerequisites for students using this book are a good working knowledge of calculus, probability theory and linear algebra. This textbook is not just any textbook on information theory. Besides the usual topics covered in a conventional course on information theory (Shannon theory, parctical solutions to communication problems, etc.), this text brings Bayesian data modelling, Monte Carlo methods, variational methods, clustering algorithms, and neural networks. The author takes the point-of-view that information theory and machine learning belong together. He manages to convince the reader of this viewpoint by continually showing the connections between seemingly unrelated topics. As Bob McEliece is quoted, ``An instant classic. ... You'll want two copies of this astonishing book, one for the office and one for the fireside at home. After some introductory chapters, the author turns to data compression, looking at the source coding theorem, symbol and stream codes, and codes for integers. Noisy-channel coding is the next main topic of interest, which includes a nice discussion of correlated random variables and their relation to communication. Other topics in information theory, such as hash codes, binary codes, and message passing, are also included. Students will certainly not want to miss Chapter 19, entitled, ``Why have sex? Part IV deals with probabilities and inference, using exact methods, deterministic approximations, and Monte Carlo methods to solve a variety of problems in information theory (decoding, inferring clusters from data, interpolation through noisy data, classifying patterns given labeled examples). This sets the scene for Part V on neural networks. The author presents some basic models coming from simple learning algorithms to whet the readers appetite for further independent reading. Hopfield networks, Boltzmann machines, and multilayer networks are included. This part concludes with a discussion on image denoising. The final part deals with sparse graph codes. The reader is introduced to the following codes for error-correction: LDPC (or Gallager), convolution, turbo, repeat accumulation, as well as to the digital fountain codes for erasure correction. After a discussion of each of these codes, the author arrives at the conclusion that the best solution to the communication problem is to ``combine a simple, pseudo-random code with a message-passing decoder. Appendices on notation, some physics (namely phase transitions), and some mathematics (a brief review of finite fields, eigenvectors, and perturbation theory) can be found at the end of the book.
- A new feature selection method for Gaussian mixture clustering
- On universal transfer learning
- Comparison of feature selection and classification algorithms in identifying malicious executables
- NML, Bayes and true distributions: a comment on Karabatsos and Walker (2006)
- A single-index model with multiple-links
- Simple method for inference in inverse Ising problem using full data
- Application of the maximum relative entropy method to the physics of ferromagnetic materials
- On the application of statistical physics to evolutionary biology
- Good cyclic codes and the uncertainty principle
- An information geometric perspective on the complexity of macroscopic predictions arising from incomplete information
- Sampling errors in nested sampling parameter estimation
- Gaussian process approximations for fast inference from infectious disease data
- Cooperative ``folding transition in the sequence space facilitates function-driven evolution of protein families
- Bayesian maximum entropy based algorithm for digital X-ray mammogram processing
- Open problems in universal induction \& intelligence
- The cumulative distribution transform and linear pattern classification
- Inexact Bayesian point pattern matching for linear transformations
- Scalable information inequalities for uncertainty quantification
- A new definition of entropy of belief functions in the Dempster-Shafer theory
- Parent oriented teacher selection causes language diversity
- Adjectival vagueness in a Bayesian model of interpretation
- Symmetry breaking and the emergence of path-dependence
- Prediction-error identification of LPV systems: a nonparametric Gaussian regression approach
- On the positive semi-definite property of similarity matrices
- Quantum algorithm design: techniques and applications
- Temporal variation and scale in movement-based resource selection functions
- A Bayesian approach to selecting hyperelastic constitutive models of soft tissue
- Cox processes for counting by detection
- Bayesian non-parametric approaches to reconstructing oscillatory systems and the Nyquist limit
- Bayesian variable selection for globally sparse probabilistic PCA
- On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods
- Online model learning of buildings using stochastic hybrid systems based on Gaussian processes
- MAP approximation to the variational Bayes Gaussian mixture model and application
- Consensus in topologically interacting swarms under communication constraints and time-delays
- Computer-aided optimal designs for improving neural network generalization
- An efficient shortest triangle paths algorithm applied to multi-camera self-calibration
- Worst-case global optimization of black-box functions through Kriging and relaxation
- Variational Bayesian methods for spatial data analysis
- Complexity-compression tradeoffs in lossy compression via efficient random codebooks and databases
- A framework for optimization under limited information
- Mean field inference for the Dirichlet process mixture model
- Pointwise comparison of PCM and \(\Sigma \Delta \) quantization
- FEM-cluster based reduction method for efficient numerical prediction of effective properties of heterogeneous material in nonlinear range
- An all-pair quantum SVM approach for big data multiclass classification
- The statistical finite element method (statFEM) for coherent synthesis of observation data and model predictions
- Entropy and information content of geostatistical models
- Tradeoff relation between mutual information and error probability in data classification problem
- An automatic Ockham's razor for Bayesians?
- A branch-cut-and-price algorithm for optimal decoding in digital communication systems
- A theory of incremental compression
- A resolution of St. Petersburg paradox
- Particle-based energetic variational inference
- Ensemble slice sampling. Parallel, black-box and gradient-free inference for correlated \& multimodal distributions
- Temporal mixture ensemble models for probabilistic forecasting of intraday cryptocurrency volume
- Towards quantum computing based community detection
- WIMP dark matter hidden behind its companion
- \(K\)-expectiles clustering
- Assessing the robustness of critical behavior in stochastic cellular automata
- Dynamics of macroscopic diffusion across meta-populations with top-down and bottom-up approaches: a review
- A taxonomy of surprise definitions
- Public announcements, public lies and recoveries
- Quantum linear system algorithm applied to communication systems
- Adaptive infinite dropout for noisy and sparse data streams
- Affine-mapping based variational ensemble Kalman filter
- Interpolating log-determinant and trace of the powers of matrix \(\mathbf{A}+ t\mathbf{B}\)
- Exploiting prior knowledge in compressed sensing to design robust systems for endoscopy image recovery
- Randomized maximum likelihood based posterior sampling
- An unsupervised machine-learning checkpoint-restart algorithm using Gaussian mixtures for particle-in-cell simulations
- Data driven Dirichlet sampling on manifolds
- An optimal uniform concentration inequality for discrete entropies on finite alphabets in the high-dimensional setting
- Variational Bayesian approximation of inverse problems using sparse precision matrices
- Universality classes for the Fisher metric derived from relative group entropy
- Parameter inference in a probabilistic model from data: regulation of transition rate in the Monte Carlo method
- Statistical physics and representations in real and artificial neural networks
- Discussion to: ``Bayesian graphical models for modern biological applications by Y. Ni, V. Baladandayuthapani, M. Vannucci and F. C. Stingo
- Parameter inference in a probabilistic model using clustered data
- Probabilistic programming with stochastic variational message passing
- Multi-fidelity classification using Gaussian processes: accelerating the prediction of large-scale computational models
- Towards understanding sparse filtering: a theoretical perspective
- Spiking time-dependent plasticity leads to efficient coding of predictions
- Uncertainty propagation in reduced order models based on crystal plasticity
- Understanding autoencoders with information theoretic concepts
- Entropy and compression: a simple proof of an inequality of Khinchin-Ornstein-Shields
- \textit{Kernel cuts}: kernel and spectral clustering meet regularization
- Constraints for generating graphs with imposed and forbidden patterns: an application to molecular graphs
- Step-by-step community detection in volume-regular graphs
- Combined state and parameter estimation in level-set methods
- Data-driven, variational model reduction of high-dimensional reaction networks
- Compressibility of random walker trajectories on growing networks
- A class of categorization methods for credit scoring models
- Constrained iterative ensemble smoother for multi solution search assisted history matching
- Solving a class of feature selection problems via fractional 0--1 programming
- Appearance of thermal time
- Matrix products with constraints on the sliding block relative frequencies of different factors
- Sample size determination for logistic regression
- A combination selection algorithm on forecasting
- Infotropism as the underlying principle of perceptual organization
- A meaning based information theory - informalogical space: basic concepts and convergence of information sequences
- Combining feature spaces for classification
- Fully probabilistic design of hierarchical Bayesian models
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4457477)
