Information-theoretic bounds and approximations in neural population coding

DOI10.1162/NECO_A_01056MaRDI QIDQ5157152zbMATH OpenOpenAlexDBLPWikidataFDO

Authors

Publication date 12 October 2021

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1611.01414, http://europepmc.org/articles/pmc6343676

zbMATH Keywords

information-theoretic asymptotic boundsneural population codingShannon's mutual information

Mathematics Subject Classification ID

Neural networks for/in biological studies, artificial life and related topics (92B20) Coding theorems (Shannon theory) (94A24)

Abstract: While Shannon's mutual information has widespread applications in many disciplines, for practical applications it is often difficult to calculate its value accurately for high-dimensional variables because of the curse of dimensionality. This paper is focused on effective approximation methods for evaluating mutual information in the context of neural population coding. For large but finite neural populations, we derive several information-theoretic asymptotic bounds and approximation formulas that remain valid in high-dimensional spaces. We prove that optimizing the population density distribution based on these approximation formulas is a convex optimization problem which allows efficient numerical solutions. Numerical simulation results confirmed that our asymptotic formulas were highly accurate for approximating mutual information for large neural populations. In special cases, the approximation formulas are exactly equal to the true mutual information. We also discuss techniques of variable transformation and dimensionality reduction to facilitate computation of the approximations.

Recommendations

• Quantifying information conveyed by large neuronal populations
• Mutual information, Fisher information, and efficient coding
• Mutual information calculation using empirical classification
• A kernel-based calculation of information on a metric space
• scientific article; zbMATH DE number 1928611

Cites work

• A Mathematical Theory of Communication
• Bayesian bounds for parameter estimation and nonlinear filtering/tracking
• Difficulty of Singularity in Population Coding
• Efficient sensory encoding and Bayesian inference with heterogeneous neural populations
• Elements of Information Theory
• Estimation of Entropy and Mutual Information
• Fisher and Shannon information in finite neural populations
• Fisher information and stochastic complexity
• scientific article; zbMATH DE number 1168332 (Why is no real title available?)
• scientific article; zbMATH DE number 2061729 (Why is no real title available?)
• scientific article; zbMATH DE number 2107836 (Why is no real title available?)
• scientific article; zbMATH DE number 3189754 (Why is no real title available?)
• scientific article; zbMATH DE number 3106666 (Why is no real title available?)
• Information-theoretic asymptotics of Bayes methods
• Mutual Information and Minimum Mean-Square Error in Gaussian Channels
• Mutual information, Fisher information, and efficient coding
• Optimal Short-Term Population Coding: When Fisher Information Fails

Cited in

(8)

• scientific article; zbMATH DE number 1154139 (Why is no real title available?)
• scientific article; zbMATH DE number 1164152 (Why is no real title available?)
• Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
• Multiple timescale online learning rules for information maximization with energetic constraints
• Quantifying information conveyed by large neuronal populations
• Coherent infomax as a computational goal for neural systems
• Mutual information, Fisher information, and efficient coding
• A fluctuation theory of communications

This page was built for publication: Information-theoretic bounds and approximations in neural population coding

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5157152)