Optimal Rates for Multi-pass Stochastic Gradient Methods

zbMath1435.68272arXiv1605.08882MaRDI QIDQ4637012

Jun Hong Lin, Lorenzo Rosasco

Publication date: 17 April 2018

Full work available at URL: https://arxiv.org/abs/1605.08882

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) Learning and adaptive systems in artificial intelligence (68T05) Stochastic programming (90C15) Stochastic approximation (62L20)

Related Items (21)

Understanding generalization error of SGD in nonconvex optimization ⋮ Score-matching representative approach for big data analysis with generalized linear models ⋮ Graph-Dependent Implicit Regularisation for Distributed Stochastic Subgradient Descent ⋮ Generalization properties of doubly stochastic learning algorithms ⋮ Unnamed Item ⋮ Stochastic gradient descent for linear inverse problems in Hilbert spaces ⋮ Decentralized learning over a network with Nyström approximation using SGD ⋮ Unnamed Item ⋮ On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems ⋮ Online regularized learning algorithm for functional data ⋮ Optimal prediction for high-dimensional functional quantile regression in reproducing kernel Hilbert spaces ⋮ Kernel conjugate gradient methods with random projections ⋮ On the regularizing property of stochastic gradient descent ⋮ Optimal rates for spectral algorithms with least-squares regression over Hilbert spaces ⋮ Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression ⋮ Optimal Rates for Multi-pass Stochastic Gradient Methods ⋮ Unnamed Item ⋮ Dimension independent excess risk by stochastic gradient descent ⋮ An analysis of stochastic variance reduced gradient for linear inverse problems ^* ⋮ From inexact optimization to learning via gradient concentration ⋮ Regularization: From Inverse Problems to Large-Scale Machine Learning

Cites Work

• Nonparametric stochastic approximation with large step-sizes
• Kernel ridge vs. principal component regression: minimax bounds and the qualification of regularization operators
• On regularization algorithms in learning theory
• Online gradient descent learning algorithms
• Optimal rates for the regularized least-squares algorithm
• On early stopping in gradient descent learning
• Learning Theory
• On the Generalization Ability of On-Line Learning Algorithms
• CROSS-VALIDATION BASED ADAPTATION FOR REGULARIZATION OPERATORS IN LEARNING THEORY
• Cutting-set methods for robust convex optimization with pessimizing oracles
• Optimal Rates for Multi-pass Stochastic Gradient Methods
• Optimal Distributed Online Prediction using Mini-Batches
• Learning Bounds for Kernel Regression Using Effective Data Dimensionality

This page was built for publication: Optimal Rates for Multi-pass Stochastic Gradient Methods