Efficient learning rate adaptation based on hierarchical optimization approach

DOI10.1016/J.NEUNET.2022.02.014MaRDI QIDQ6072584zbMATH OpenOpenAlexWikidataFDO

Authors Gyoung S. Na

Publication date 13 October 2023

Published in Neural Networks (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Full work available at URL https://doi.org/10.1016/j.neunet.2022.02.014

zbMATH Keywords

computer visiondeep learningmathematical optimization

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Machine vision and scene understanding (68T45) Applications of mathematical programming (90C90)

Recommendations

• scientific article; zbMATH DE number 7583836
• Automatic, dynamic, and nearly optimal learning rate specification via local quadratic approximation
• A control theoretic framework for adaptive gradient optimizers
• Acceleration of the back propagation through dynamic adaptation of the learning rate
• A modification of adaptive moment estimation (Adam) for machine learning

Cites work

• A dual algorithm for the solution of nonlinear variational problems via finite element approximation
• A stochastic quasi-Newton method for large-scale optimization
• Adaptive subgradient methods for online learning and stochastic optimization
• Global convergence of unmodified 3-block ADMM for a class of convex minimization problems
• Linear Inversion of Band-Limited Reflection Seismograms
• Multiplier and gradient methods
• Noisy matrix decomposition via convex relaxation: optimal rates in high dimensions
• The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent

Cited in

(6)

• Efficient learning in adaptive processing of data structures
• Learning rate adaptation in stochastic gradient descent.
• scientific article; zbMATH DE number 7583836 (Why is no real title available?)
• A control theoretic framework for adaptive gradient optimizers
• An ADMM-LSTM framework for short-term load forecasting
• To raise or not to raise: the autonomous learning rate question

This page was built for publication: Efficient learning rate adaptation based on hierarchical optimization approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6072584)