Understanding generalization error of SGD in nonconvex optimization

DOI10.1007/S10994-021-06056-WMaRDI QIDQ2127232zbMATH OpenOpenAlexFDO

Authors Yanyan Li

Publication date 20 April 2022

Published in Machine Learning (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/s10994-021-06056-w

zbMATH Keywords

generalization errornonconvex machine learningstochastic gradient descent

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Cites work

• 10.1162/153244302760200704
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• A theory of the learnable
• Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods
• Convex analysis and monotone operator theory in Hilbert spaces
• scientific article; zbMATH DE number 1332320 (Why is no real title available?)
• scientific article; zbMATH DE number 823069 (Why is no real title available?)
• scientific article; zbMATH DE number 3313108 (Why is no real title available?)
• scientific article; zbMATH DE number 3371284 (Why is no real title available?)
• Large-scale machine learning with stochastic gradient descent
• Learnability, stability and uniform convergence
• Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
• On complexity of stochastic programming problems
• Optimal rates for multi-pass stochastic gradient methods
• Rapid, robust, and reliable blind deconvolution via nonconvex optimization
• Robust Large Margin Deep Neural Networks
• Robust Stochastic Approximation Approach to Stochastic Programming
• Robustness and generalization
• Stability of randomized learning algorithms
• Understanding machine learning. From theory to algorithms

Cited in

(3)

• Stability analysis of stochastic gradient descent for homogeneous neural networks and linear classifiers
• Select without fear: almost all minibatch schedules generalize optimally
• Convergence of Adam for non-convex objectives: relaxed hyperparameters and non-ergodic case

Describes a project that uses

Uses Software

• CIFAR
• LIBSVM

This page was built for publication: Understanding generalization error of SGD in nonconvex optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2127232)