Deep neural networks with a set of node-wise varying activation functions
From MaRDI portal
Publication:1980402
Abstract: This work presents deep asymmetric networks with a set of node-wise variant activation functions. The nodes' sensitivities are affected by activation function selections such that the nodes with smaller indices become increasingly more sensitive. As a result, features learned by the nodes are sorted by the node indices in the order of their importance. Asymmetric networks not only learn input features but also the importance of those features. Nodes of lesser importance in asymmetric networks can be pruned to reduce the complexity of the networks, and the pruned networks can be retrained without incurring performance losses. We validate the feature-sorting property using both shallow and deep asymmetric networks as well as deep asymmetric networks transferred from famous networks.
Recommendations
- A survey on modern trainable activation functions
- A representer theorem for deep neural networks
- Deep neural networks for rotation-invariance approximation and learning
- scientific article; zbMATH DE number 1843021
- Data-driven learning of feedforward neural networks with different activation functions
Cites work
Cited in
(2)
This page was built for publication: Deep neural networks with a set of node-wise varying activation functions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1980402)
