Modular representation of layered neural networks

DOI10.1016/J.NEUNET.2017.09.017zbMATH Open1434.68471arXiv1703.00168OpenAlexW2592004655WikidataQ47668396 ScholiaQ47668396MaRDI QIDQ2179094FDOQ2179094

Authors: Kaoru Hiramatsu, Kunio Kashino, Chihiro Watanabe 

Publication date: 12 May 2020

Published in: Neural Networks (Search for Journal in Brave)

Abstract: Layered neural networks have greatly improved the performance of various applications including image processing, speech recognition, natural language processing, and bioinformatics. However, it is still difficult to discover or interpret knowledge from the inference provided by a layered neural network, since its internal representation has many nonlinear and complex parameters embedded in hierarchical layers. Therefore, it becomes important to establish a new methodology by which layered neural networks can be understood. In this paper, we propose a new method for extracting a global and simplified structure from a layered neural network. Based on network analysis, the proposed method detects communities or clusters of units with similar connection patterns. We show its effectiveness by applying it to three use cases. (1) Network decomposition: it can decompose a trained neural network into multiple small independent networks thus dividing the problem and reducing the computation time. (2) Training assessment: the appropriateness of a trained result with a given hyperparameter or randomly chosen initial parameters can be evaluated by using a modularity index. And (3) data analysis: in practical data it reveals the community structure in the input, hidden, and output layers, which serves as a clue for discovering knowledge from a trained neural network.

Full work available at URL: https://arxiv.org/abs/1703.00168

Recommendations

• scientific article; zbMATH DE number 400585
• Layered Networks for Unsupervised Learning
• Symbolic knowledge extraction from trained neural networks: A sound approach
• Structural synthesis of modular weakly connected neural networks. II: Nuclear neural networks
• Extracting Rules from Neural Networks by Pruning and Hidden-Unit Splitting

zbMATH Keywords

community detectionnetwork analysislayered neural networks

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Title not available (Why is that?)
• Mixture models and exploratory analysis in networks
• Learning representations by back-propagating errors
• Natural language processing (almost) from scratch

Cited In (4)

• Title not available (Why is that?)
• Multilayer neural networks and polyhedral dichotomies
• Title not available (Why is that?)
• Compositionally-warped Gaussian processes

Uses Software

• ImageNet
• AlexNet

This page was built for publication: Modular representation of layered neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2179094)