Improved weight initialization for deep and narrow feedforward neural network

DOI10.1016/J.NEUNET.2024.106362MaRDI QIDQ6558463FDOQ6558463

Authors: Hyun Woo Lee, Yunho Kim, Seung Yeop Yang, Hayoung Choi 

Publication date: 19 June 2024

Published in: Neural Networks (Search for Journal in Brave)

Recommendations

• A weight initialization based on the linear product structure for neural networks
• Structural, Syntactic, and Statistical Pattern Recognition
• Dying ReLU and initialization: theory and numerical examples
• Magnitude and angle dynamics in training single ReLU neurons
• A new initialization method based on normed statistical spaces in deep networks

zbMATH Keywords

deep learningfeedforward neural networksReLU activation functionweight initializationinitial weight matrix

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Cited In (1)

• Magnitude and angle dynamics in training single ReLU neurons

This page was built for publication: Improved weight initialization for deep and narrow feedforward neural network

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6558463)