Magnitude and angle dynamics in training single ReLU neurons

DOI10.1016/J.NEUNET.2024.106435zbMATH Open1545.68114MaRDI QIDQ6587015FDOQ6587015

Authors: Sangmin Lee, Byeongsu Sim, Jong Chul Ye 

Publication date: 13 August 2024

Published in: Neural Networks (Search for Journal in Brave)

Recommendations

• The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality
• Non-convergence of stochastic gradient descent in the training of deep neural networks
• Improved weight initialization for deep and narrow feedforward neural network
• Training neural networks from an ergodic perspective
• On the Relation Between Loss Functions and T-Norms

zbMATH Keywords

gradient flowmagnitude and angle dynamicssingle ReLU neurons

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Cites Work

• Title not available (Why is that?)
• How Small Is a Unit Ball?
• The implicit bias of gradient descent on separable data
• Two models of double descent for weak features

Cited In (3)

• Improved weight initialization for deep and narrow feedforward neural network
• Training neural networks from an ergodic perspective
• Homotopy relaxation training algorithms for infinite-width two-layer ReLU neural networks

This page was built for publication: Magnitude and angle dynamics in training single ReLU neurons

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6587015)