Learning in the machine: to share or not to share?
From MaRDI portal
Publication:1980412
DOI10.1016/J.NEUNET.2020.03.016zbMATH Open1468.68186arXiv1909.11483OpenAlexW3013796640WikidataQ91664177 ScholiaQ91664177MaRDI QIDQ1980412FDOQ1980412
Authors: Jordan Ott, Erik Linstead, Nicholas LaHaye, Pierre Baldi 
Publication date: 8 September 2021
Published in: Neural Networks (Search for Journal in Brave)
Abstract: Weight-sharing is one of the pillars behind Convolutional Neural Networks and their successes. However, in physical neural systems such as the brain, weight-sharing is implausible. This discrepancy raises the fundamental question of whether weight-sharing is necessary. If so, to which degree of precision? If not, what are the alternatives? The goal of this study is to investigate these questions, primarily through simulations where the weight-sharing assumption is relaxed. Taking inspiration from neural circuitry, we explore the use of Free Convolutional Networks and neurons with variable connection patterns. Using Free Convolutional Networks, we show that while weight-sharing is a pragmatic optimization approach, it is not a necessity in computer vision applications. Furthermore, Free Convolutional Networks match the performance observed in standard architectures when trained using properly translated data (akin to video). Under the assumption of translationally augmented data, Free Convolutional Networks learn translationally invariant representations that yield an approximate form of weight sharing.
Full work available at URL: https://arxiv.org/abs/1909.11483
Recommendations
- Convolutional neural networks analyzed via convolutional sparse coding
- On the behavior of convolutional nets for feature extraction
- A Note on the Regularity of Images Generated by Convolutional Neural Networks
- Convolutional neural network models of V1 responses to complex patterns
- Error bounds for approximations using multichannel deep convolutional neural networks with downsampling
Cites Work
- Title not available (Why is that?)
- Neocognition: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position
- The dropout learning algorithm
- Learning in the machine: random backpropagation and the deep learning channel
- Learning in the machine: the symmetries of the deep learning channel
- Deep learning with dynamic spiking neurons and fixed feedback weights
Uses Software
This page was built for publication: Learning in the machine: to share or not to share?
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1980412)
