Convergence Rate of Incremental Gradient and Incremental Newton Methods

DOI10.1137/17M1147846zbMath1428.90119arXiv1510.08562MaRDI QIDQ5237308

Mert Gürbüzbalaban, Asuman Ozdaglar, Pablo A. Parrilo

Publication date: 17 October 2019

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1510.08562

zbMATH Keywords

convex optimization; convergence rate; first-order methods; incremental algorithms

Mathematics Subject Classification ID

90C25: Convex programming

90C06: Large-scale problems in mathematical programming

90C30: Nonlinear programming

Related Items

Unnamed Item, On the Convergence Rate of Incremental Aggregated Gradient Algorithms, Convergence of Random Reshuffling under the Kurdyka–Łojasiewicz Inequality, Variable smoothing incremental aggregated gradient method for nonsmooth nonconvex regularized optimization, Accelerating incremental gradient optimization with curvature information, Why random reshuffling beats stochastic gradient descent

Uses Software

• Jellyfish
• Saga

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
• Logarithmic regret algorithms for online convex optimization
• Incremental gradient algorithms with stepsizes bounded away from zero
• The incremental Gauss-Newton algorithm with adaptive stepsize rule
• Introductory lectures on convex optimization. A basic course.
• Why random reshuffling beats stochastic gradient descent
• A globally convergent incremental Newton method
• Parallel stochastic gradient algorithms for large-scale matrix completion
• Incremental Subgradient Methods for Nondifferentiable Optimization
• Adaptivity of averaged stochastic gradient descent to local strong convexity for logistic regression
• Robust Stochastic Approximation Approach to Stochastic Programming
• Acceleration of Stochastic Approximation by Averaging
• A New Class of Incremental Gradient Methods for Least Squares Problems
• An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule
• Gradient Convergence in Gradient methods with Errors
• Global Convergence Rate of Proximal Incremental Aggregated Gradient Methods
• An Adaptive Associative Memory Principle
• Incremental Least Squares Methods and the Extended Kalman Filter
• Distributed Subgradient Methods for Multi-Agent Optimization
• A Collaborative Training Algorithm for Distributed Learning
• EXTRA: An Exact First-Order Algorithm for Decentralized Consensus Optimization
• On the Convergence Rate of Incremental Aggregated Gradient Algorithms
• A Convergent Incremental Gradient Method with a Constant Step Size
• On‐line learning for very large data sets
• On a Stochastic Approximation Method