Improved weight initialization for deep and narrow feedforward neural network

DOI10.1016/J.NEUNET.2024.106362MaRDI QIDQ6558463zbMATH OpenFDO

Authors Hyun Woo Lee, Yunho Kim, Seung Yeop Yang, Hayoung Choi

Publication date 19 June 2024

Published in Neural Networks (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1016/J.NEUNET.2024.106362

zbMATH Keywords

deep learningfeedforward neural networksinitial weight matrixReLU activation functionweight initialization

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Recommendations

• A weight initialization based on the linear product structure for neural networks
• Structural, Syntactic, and Statistical Pattern Recognition
• Dying ReLU and initialization: theory and numerical examples
• Magnitude and angle dynamics in training single ReLU neurons
• A new initialization method based on normed statistical spaces in deep networks

Cited in

(1)

• Magnitude and angle dynamics in training single ReLU neurons

This page was built for publication: Improved weight initialization for deep and narrow feedforward neural network

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6558463)