Automatic, dynamic, and nearly optimal learning rate specification via local quadratic approximation

DOI10.1016/j.neunet.2021.03.025zbMath1521.68210arXiv2004.03260OpenAlexW3149827756MaRDI QIDQ6054924

Rui Wu, Danyang Huang, Yingqiu Zhu, Bo Zhang, Yu Chen, Yuan Gao, Hansheng Wang

Publication date: 28 September 2023

Published in: Neural Networks (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2004.03260

zbMATH Keywords

neural networksmachine learninggradient descentlearning rategradient-based optimizationlocal quadratic approximation

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Applications of mathematical programming (90C90)

Cites Work

• An optimal method for stochastic composite optimization
• Linear and nonlinear programming
• Algorithm 778: L-BFGS-B
• An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule
• Historical Development of the Newton–Raphson Method
• Learning representations by back-propagating errors
• A Stochastic Approximation Method
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: Automatic, dynamic, and nearly optimal learning rate specification via local quadratic approximation