A network of spiking neurons for computing sparse representations in an energy-efficient way

DOI10.1162/NECO_A_00353zbMATH Open1268.92024arXiv1210.1530OpenAlexW2005918037WikidataQ42039320 ScholiaQ42039320MaRDI QIDQ2840871FDOQ2840871

Authors: Tao Hu, Dmitri B. Chklovskii, Alexander Genkin 

Publication date: 23 July 2013

Published in: Neural Computation (Search for Journal in Brave)

Abstract: Computing sparse redundant representations is an important problem both in applied mathematics and neuroscience. In many applications, this problem must be solved in an energy efficient way. Here, we propose a hybrid distributed algorithm (HDA), which solves this problem on a network of simple nodes communicating via low-bandwidth channels. HDA nodes perform both gradient-descent-like steps on analog internal variables and coordinate-descent-like steps via quantized external variables communicated to each other. Interestingly, such operation is equivalent to a network of integrate-and-fire neurons, suggesting that HDA may serve as a model of neural computation. We show that the numerical performance of HDA is on par with existing algorithms. In the asymptotic regime the representation error of HDA decays with time, t, as 1/t. HDA is stable against time-varying noise, specifically, the representation error decays as 1/sqrt(t) for Gaussian white noise.

Full work available at URL: https://arxiv.org/abs/1210.1530

Recommendations

• A common network architecture efficiently implements a variety of sparsity-based inference problems
• Efficient Computation Based on Stochastic Spikes
• Finding independent components using spikes: A natural result of Hebbian learning in a sparse spike coding scheme
• Lower Bounds for the Computational Power of Networks of Spiking Neurons
• Adapting spiking neural networks

Mathematics Subject Classification ID

Neural biology (92C20) Neural networks for/in biological studies, artificial life and related topics (92B20) Computational methods for problems pertaining to biology (92-08)

Cites Work

• Least angle regression. (With discussion)
• Pathwise coordinate optimization
• Atomic Decomposition by Basis Pursuit
• Regularization and Variable Selection Via the Elastic Net
• Convex optimization theory.
• Linearized Bregman iterations for compressed sensing
• Title not available (Why is that?)
• Coordinate descent optimization for \(l^{1}\) minimization with application to compressed sensing; a greedy algorithm
• Equality relating Euclidean distance cone to positive semidefinite cone
• Convergence of the linearized Bregman iteration for \(\ell _1\)-norm minimization
• Fast linearized Bregman iteration for compressive sensing and sparse denoising

Cited In (4)

• Short-term memory capacity in networks via the restricted isometry property
• A theoretical perspective on hyperdimensional computing
• A common network architecture efficiently implements a variety of sparsity-based inference problems
• ACCURATE, ENERGY-EFFICIENT CLASSIFICATION WITH SPIKING RANDOM NEURAL NETWORK

This page was built for publication: A network of spiking neurons for computing sparse representations in an energy-efficient way

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2840871)