Categorical perception: a groundwork for deep learning

DOI10.1162/NECO_A_01454MaRDI QIDQ5037158zbMATH OpenOpenAlexWikidataFDO

Authors Laurent Bonnasse-Gahot, Jean-Pierre Nadal

Publication date 25 February 2022

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2012.05549

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Cognitive psychology (91E10) Psychophysics and psychophysiology; perception (91E30)

Abstract: A well-known perceptual consequence of categorization in humans and other animals, called categorical perception, is notably characterized by a within-category compression and a between-category separation: two items, close in input space, are perceived closer if they belong to the same category than if they belong to different categories. Elaborating on experimental and theoretical results in cognitive science, here we study categorical effects in artificial neural networks. We combine a theoretical analysis that makes use of mutual and Fisher information quantities, and a series of numerical simulations on networks of increasing complexity. These formal and numerical analyses provide insights into the geometry of the neural representation in deep layers, with expansion of space near category boundaries and contraction far from category boundaries. We investigate categorical representation by using two complementary approaches: one mimics experiments in psychophysics and cognitive neuroscience by means of morphed continua between stimuli of different categories, while the other introduces a categoricality index that, for each layer in the network, quantifies the separability of the categories at the neural population level. We show on both shallow and deep neural networks that category learning automatically induces categorical perception. We further show that the deeper a layer, the stronger the categorical effects. As an outcome of our study, we propose a coherent view of the efficacy of different heuristic practices of the dropout regularization technique. More generally, our view, which finds echoes in the neuroscience literature, insists on the differential impact of noise in any given layer depending on the geometry of the neural representation that is being learned, i.e. on how this geometry reflects the structure of the categories.

Recommendations

• Visual categorization with random projection
• Categorization and effective perceptron learning in feed-forward neural networks
• Surprising properties of dropout in deep networks
• The dropout learning algorithm
• Emergence of invariance and disentanglement in deep representations

Cited in

(5)

• The structure of perceptual categories
• A self-organizing network for mixed category perception
• Perceive this as that -- Analogies, artificial perception, and category theory
• Visual categorization with random projection
• Training neural networks with structured noise improves classification and generalization

This page was built for publication: Categorical perception: a groundwork for deep learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5037158)