Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness (Q6145578)
From MaRDI portal
!
This is the item page for this Wikibase entity, intended for internal use and editing purposes. Please use the normal view instead:
scientific article; zbMATH DE number 7785652
| Language | Label | Description | Also known as |
|---|---|---|---|
| default for all languages | No label defined |
||
| English | Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness |
scientific article; zbMATH DE number 7785652 |
Statements
Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness (English)
0 references
9 January 2024
0 references
Convergence rate analyses of Adaptive Moment Estimation are widely used in nonconvex optimsation. This paper analyses this method and shows that the method performs well for small learning rates and hyperparameters close to 1. This has implications for nonconvex optimisation for deep learning applications.
0 references
adaptive moment estimation
0 references
batch size
0 references
hyperparameters
0 references
learning rate
0 references
nonconvex optimization
0 references
0 references
0.7935283184051514
0 references
0.7789536714553833
0 references
0.7769032716751099
0 references
