Optimal non-asymptotic analysis of the Ruppert-Polyak averaging stochastic algorithm

DOI10.1016/j.spa.2022.11.012OpenAlexW4310061700MaRDI QIDQ2680399

Fabien Panloup, Sébastien Gadat

Publication date: 2 January 2023

Published in: Stochastic Processes and their Applications (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1016/j.spa.2022.11.012

zbMATH Keywords

optimizationaveragingstochastic gradient descent

Mathematics Subject Classification ID

Nonparametric estimation (62G05) Convex programming (90C25) Stochastic approximation (62L20)

Related Items (3)

Stochastic heavy ball ⋮ On variance reduction for stochastic smooth convex optimization with multiplicative noise ⋮ An Efficient Stochastic Newton Algorithm for Parameter Estimation in Logistic Regressions

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Efficient and fast estimation of the geometric median in Hilbert spaces with an averaged stochastic gradient algorithm
• Estimating the geometric median in Hilbert spaces with stochastic gradient algorithms: \(L^p\) and almost sure rates of convergence
• On gradients of functions definable in o-minimal structures
• Introductory lectures on convex optimization. A basic course.
• From error bounds to the complexity of first-order descent methods for convex functions
• Stochastic heavy ball
• Spectral decompositions and \(\mathbb{L}^2\)-operator norms of toy hypocoercive semi-groups
• Online estimation of the geometric median in Hilbert spaces: nonasymptotic confidence balls
• Central limit theorems for stochastic approximation with controlled Markov chain dynamics
• Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
• Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity
• On the long time behavior of second order differential equations with asymptotically small dissipation
• Hypocoercivity
• Acceleration of Stochastic Approximation by Averaging
• Asymptotic Almost Sure Efficiency of Averaged Stochastic Algorithms
• Information-Theoretic Lower Bounds on the Oracle Complexity of Stochastic Convex Optimization
• Prediction, Learning, and Games
• A Stochastic Approximation Method
• L^p and almost sure rates of convergence of averaged stochastic gradient algorithms: locally strongly convex objective

This page was built for publication: Optimal non-asymptotic analysis of the Ruppert-Polyak averaging stochastic algorithm