Incremental gradient algorithms with stepsizes bounded away from zero
For the problem \[ \text{minimize}\quad \sum^k_{j= 1} f_j(x)\quad\text{subject to}\quad x\in\mathbb{R}^n, \] where \(f_1,\dots, f_k: \mathbb{R}^n\to \mathbb{R}\) are continuously differentiable functions, the author proposes the following incremental gradient algorithm: Choose any \(x^0\in \mathbb{R}^n\). Having \(x^i\), check a stopping criterion. If not satisfied, compute \(x^{i+ 1}= T(x^i,\eta_i)\), where \(T:\mathbb{R}^n\times \mathbb{R}^+\to \mathbb{R}^n\) is given by \[ T(x,\eta):= x-\eta \sum^k_{j=1}\nabla f_j(z^j) \] and \[ z^1= x,\quad z^{j+1}= z^j- \eta\nabla f_j(z^j),\quad j=1,\dots, k-1. \] The author presents convergence results for a class of incremental gradient algorithms with stepsizes bounded away from zero. Applications on neural network training are discussed.
- Error stability properties of generalized gradient-type algorithms
- Descent methods with linesearch in the presence of perturbations
- Convergence analysis of perturbed feasible descent methods
- On the convergence of a block-coordinate incremental gradient method
- Incrementally updated gradient methods for constrained and regularized optimization
- On the linear convergence of the stochastic gradient method with constant step-size
- A globally convergent incremental Newton method
- Sliced and Radon Wasserstein barycenters of measures
- Incremental gradient-free method for nonsmooth distributed optimization
- A framework for parallel second order incremental optimization algorithms for solving partially separable problems
- Global convergence of the Dai-Yuan conjugate gradient method with perturbations
- An incremental decomposition method for unconstrained optimization
- On perturbed steepest descent methods with inexact line search for bilevel convex optimization
- Network synchronization with convexity
- A New Class of Incremental Gradient Methods for Least Squares Problems
- An Incremental Gradient(-Projection) Method with Momentum Term and Adaptive Stepsize Rule
- String-averaging incremental stochastic subgradient algorithms
- Global convergence rate of proximal incremental aggregated gradient methods
- The Averaged Kaczmarz Iteration for Solving Inverse Problems
- A smooth inexact penalty reformulation of convex problems with linear constraints
- A scaled incremental gradient method
- Minimizing finite sums with the stochastic average gradient
- Convergence rate of incremental gradient and incremental Newton methods
- On the Convergence Rate of Incremental Aggregated Gradient Algorithms
- Approximation accuracy, gradient methods, and error bound for structured convex optimization
- Incremental subgradient algorithms with dynamic step sizes for separable convex optimizations
- Convergence of Random Reshuffling under the Kurdyka–Łojasiewicz Inequality
- Recent Theoretical Advances in Non-Convex Optimization
- Random algorithms for convex minimization problems
- Incremental proximal methods for large scale convex optimization
- Stochastic subgradient algorithm for nonsmooth nonconvex optimization
- Incremental quasi-Newton algorithms for solving a nonconvex, nonsmooth, finite-sum optimization problem
- Convergence properties of proximal (sub)gradient methods without convexity or smoothness of any of the functions
- Simultaneous incremental gradient method for inconsistent convex optimization problem
- Convergence of ease-controlled random reshuffling gradient algorithms under Lipschitz smoothness
- Robust inversion, dimensionality reduction, and randomized sampling
This page was built for publication: Incremental gradient algorithms with stepsizes bounded away from zero
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1273418)
