A perturbative approach to nonlinearities in the information carried by a two layer neural network
From MaRDI portal
(Redirected from Publication:3583523)
Abstract: We evaluate the mutual information between the input and the output of a two layer network in the case of a noisy and non-linear analogue channel. In the case where the non-linearity is small with respect to the variability in the noise, we derive an exact expression for the contribution to the mutual information given by the non-linear term in first order of perturbation theory. Finally we show how the calculation can be simplified by means of a diagrammatic expansion. Our results suggest that the use of perturbation theories applied to neural systems might give an insight on the contribution of non-linearities to the information transmission and in general to the neuronal dynamics.
Recommendations
- Improved lower bound for the mutual information between signal and neural spike count
- The mutual information of a stochastic binary channel: validity of the replica symmetry ansatz
- Nonlinear neural networks. I: General theory.
- Maximization of mutual information in a linear noisy network: a detailed study
- Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer
Cites work
- A Mathematical Theory of Communication
- Information processing by a noisy binary channel
- Information processing by a perceptron in an unsupervised learning task
- Maximization of mutual information in a linear noisy network: a detailed study
- The mutual information of a stochastic binary channel: validity of the replica symmetry ansatz
- Threshold-linear formal neurons in auto-associative nets
Cited in
(2)
This page was built for publication: A perturbative approach to nonlinearities in the information carried by a two layer neural network
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3583523)
