Convergence in quadratic mean of averaged stochastic gradient algorithms without strong convexity nor bounded gradient

DOI10.1080/02331888.2023.2213371MaRDI QIDQ6168293zbMATH OpenOpenAlexFDO

Authors Antoine Godichon-Baggioni

Publication date 10 July 2023

Published in Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2107.12058

zbMATH Keywords

averagingnon-asymptotic convergenceonline learningstochastic gradient algorithmstochastic optimization

Mathematics Subject Classification ID

Sequential estimation (62L12) Stochastic approximation (62L20)

Abstract: Online averaged stochastic gradient algorithms are more and more studied since (i) they can deal quickly with large sample taking values in high dimensional spaces, (ii) they enable to treat data sequentially, (iii) they are known to be asymptotically efficient. In this paper, we focus on giving explicit bounds of the quadratic mean error of the estimates, and this, with very weak assumptions, i.e without supposing that the function we would like to minimize is strongly convex or admits a bounded gradient.

Cites work

• L^p and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective
• A generalization of the averaging procedure: the use of two-time-scale algorithms
• A Stochastic Approximation Method
• Acceleration of Stochastic Approximation by Averaging
• Adaptive subgradient methods for online learning and stochastic optimization
• Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
• Asymptotic Almost Sure Efficiency of Averaged Stochastic Algorithms
• Efficient and fast estimation of the geometric median in Hilbert spaces with an averaged stochastic gradient algorithm
• Estimating the geometric median in Hilbert spaces with stochastic gradient algorithms: L^p and almost sure rates of convergence
• Multivariate location estimation using extension of \(R\)-estimates through \(U\)-statistics type approach
• Non asymptotic controls on a recursive superquantile approximation
• On Projected Stochastic Gradient Descent Algorithm with Weighted Averaging for Least Squares Regression
• On the almost sure asymptotic behaviour of stochastic algorithm
• Online estimation of the asymptotic variance for averaged stochastic gradient algorithms
• Online estimation of the geometric median in Hilbert spaces: nonasymptotic confidence balls

This page was built for publication: Convergence in quadratic mean of averaged stochastic gradient algorithms without strong convexity nor bounded gradient

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6168293)