Predictive coding, variational autoencoders, and biological connections
From MaRDI portal
Publication:5037129
Abstract: This paper reviews predictive coding, from theoretical neuroscience, and variational autoencoders, from machine learning, identifying the common origin and mathematical framework underlying both areas. As each area is prominent within its respective field, more firmly connecting these areas could prove useful in the dialogue between neuroscience and machine learning. After reviewing each area, we discuss two possible correspondences implied by this perspective: cortical pyramidal dendrites as analogous to (non-linear) deep networks and lateral inhibition as analogous to normalizing flows. These connections may provide new directions for further investigations in each field.
Recommendations
- Predictive coding approximates backprop along arbitrary computation graphs
- A predictive-coding network that is both discriminative and generative
- Predictive Coding and the Slowness Principle: An Information-Theoretic Approach
- Minimizing relative entropy in hierarchical predictive coding
- Learning probabilistic neural representations with randomly connected circuits
Cites work
- scientific article; zbMATH DE number 3118779 (Why is no real title available?)
- scientific article; zbMATH DE number 5284813 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1222283 (Why is no real title available?)
- scientific article; zbMATH DE number 1222290 (Why is no real title available?)
- scientific article; zbMATH DE number 794305 (Why is no real title available?)
- A Family of Nonparametric Density Estimation Algorithms
- A Mathematical Theory of Communication
- A logical calculus of the ideas immanent in nervous activity
- An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity
- Covariance estimation: the GLM and regularization perspectives
- Deep learning
- Equivalence of Backpropagation and Contrastive Hebbian Learning in a Layered Network
- Fusion, propagation, and structuring in belief networks
- Independent component analysis, a new concept?
- Learning representations by back-propagating errors
- Machine learning. A probabilistic perspective
- Neural networks and physical systems with emergent collective computational abilities
- Optimal Whitening and Decorrelation
- Probabilistic Principal Component Analysis
- Redundancy reduction with information-preserving nonlinear maps
- Supervised and unsupervised learning with two sites of synaptic integration
- Varieties of Helmholtz machine
Cited in
(5)- Toward optimality in scalable predictive coding
- Minimizing relative entropy in hierarchical predictive coding
- A predictive-coding network that is both discriminative and generative
- Predictive Coding and the Slowness Principle: An Information-Theoretic Approach
- Predictive coding approximates backprop along arbitrary computation graphs
This page was built for publication: Predictive coding, variational autoencoders, and biological connections
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5037129)
