Advancing neural network calibration: the role of gradient decay in large-margin Softmax optimization

DOI10.1016/J.NEUNET.2024.106457MaRDI QIDQ6587016zbMATH OpenFDO

Authors Siyuan Zhang, Linbo Xie

Publication date 13 August 2024

Published in Neural Networks (Search for Journal in Brave)

zbMATH Keywords

model calibrationcurriculum learninggradient decaylarge margin Softmaxlocal Lipschitz constraint

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Artificial neural networks and deep learning (68T07)

Recommendations

• A study on relationship between prediction uncertainty and robustness to noisy data
• AdaLo: adaptive learning rate optimizer with loss for classification
• Why does large batch training result in poor generalization? A comprehensive explanation and a better strategy from the viewpoint of stochastic optimization
• Homotopy relaxation training algorithms for infinite-width two-layer ReLU neural networks
• Levenberg-Marquardt multi-classification using hinge loss function

Cites work

• Accelerated methods for nonconvex optimization
• Robust Large Margin Deep Neural Networks
• Simple explanation of the no-free-lunch theorem and its implications
• The Elements of Statistical Learning
• The implicit bias of gradient descent on separable data

This page was built for publication: Advancing neural network calibration: the role of gradient decay in large-margin Softmax optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6587016)