torchopt
type: Item
instance of: R package
software version identifier: 0.1.3
software version identifier: 0.1.1
software version identifier: 0.1.2
software version identifier: 0.1.4
maintained by: Gilberto Camara
CRAN project: torchopt
programmed in: R
last update: 2023-06-06T00:00:00Z
description: Optimizers for 'torch' deep learning library. These functions include recent results published in the literature and are not part of the optimizers offered in 'torch'. Prospective users should test these optimizers with their data, since performance depends on the specific problem being solved. The packages includes the following optimizers: (a) 'adabelief' by Zhuang et al (2020), <arXiv:2010.07468>; (b) 'adabound' by Luo et al.(2019), <arXiv:1902.09843>; (c) 'adahessian' by Yao et al.(2021) <arXiv:2006.00719>; (d) 'adamw' by Loshchilov & Hutter (2019), <arXiv:1711.05101>; (e) 'madgrad' by Defazio and Jelassi (2021), <arXiv:2101.11075>; (f) 'nadam' by Dozat (2019), <https://openreview.net/pdf/OM0jvwB8jIp57ZJjtNEZ.pdf>; (g) 'qhadam' by Ma and Yarats(2019), <arXiv:1810.06801>; (h) 'radam' by Liu et al. (2019), <arXiv:1908.03265>; (i) 'swats' by Shekar and Sochee (2018), <arXiv:1712.07628>; (j) 'yogi' by Zaheer et al.(2019), <https://papers.nips.cc/paper/8186-adaptive-methods-for-nonconvex-optimization>.
author: Gilberto Camara
author: Rolf Simoes
author: Daniel Falbel
author: Felipe Souza
copyright license: Apache License
imports: graphics
imports: grDevices
imports: stats
imports: torch
cites work: AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients
cites work: Adaptive Gradient Methods with Dynamic Bound of Learning Rate
cites work: ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning
cites work: Decoupled Weight Decay Regularization
cites work: Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization
cites work: Quasi-hyperbolic momentum and Adam for deep learning
cites work: On the Variance of the Adaptive Learning Rate and Beyond
cites work: Improving Generalization Performance by Switching from Adam to SGD
MaRDI profile type: MaRDI software profile
depends on software: R
description: Advanced Optimizers for Torch