An analysis of stochastic variance reduced gradient for linear inverse problems *
From MaRDI portal
Publication:5019935
Abstract: Stochastic variance reduced gradient (SVRG) is a popular variance reduction technique for accelerating stochastic gradient descent (SGD). We provide a first analysis of the method for solving a class of linear inverse problems in the lens of the classical regularization theory. We prove that for a suitable constant step size schedule, the method can achieve an optimal convergence rate in terms of the noise level (under suitable regularity condition) and the variance of the SVRG iterate error is smaller than that by SGD. These theoretical findings are corroborated by a set of numerical experiments.
Recommendations
- Stochastic gradient descent for linear inverse problems in Hilbert spaces
- Stochastic asymptotical regularization for linear inverse problems
- On the Convergence of Stochastic Gradient Descent for Linear Inverse Problems in Banach Spaces
- A new regularized stochastic approximation framework for stochastic inverse problems
- A variational inequality based stochastic approximation for inverse problems in stochastic partial differential equations
- Stochastic variance reduced gradient methods using a trust-region-like scheme
- Stochastic inverse matrix computation with minimum variance of errors
- Stochastic reduced order models for inverse problems under uncertainty
- Stochastic variance-reduced cubic regularization methods
Cites work
- scientific article; zbMATH DE number 936298 (Why is no real title available?)
- A Stochastic Approximation Method
- A randomized Kaczmarz algorithm with exponential convergence
- Inverse problems. Tikhonov theory and algorithms
- Iterative regularization methods for nonlinear ill-posed problems
- Nonparametric stochastic approximation with large step-sizes
- On Nesterov acceleration for Landweber iteration of linear ill-posed problems
- On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems
- On the discrepancy principle for stochastic gradient descent
- On the regularizing property of stochastic gradient descent
- Online Learning as Stochastic Approximation of Regularization Paths: Optimality and Almost-Sure Convergence
- Online gradient descent learning algorithms
- Online learning in optical tomography: a stochastic approach
- Optimal rates for multi-pass stochastic gradient methods
- Optimal-order convergence of Nesterov acceleration for linear ill-posed problems
- Optimization methods for large-scale machine learning
- Regularization tools version \(4.0\) for matlab \(7.3\)
- Relaxation methods for image reconstruction
- Stochastic EM methods with variance reduction for penalised PET reconstructions
Cited in
(12)- On the Convergence of Stochastic Gradient Descent for Nonlinear Ill-Posed Problems
- Stochastic variance reduced gradient for affine rank minimization problem
- Improved SVRG for finite sum structure optimization with application to binary classification
- Stochastic variance reduced gradient methods using a trust-region-like scheme
- Asymptotic estimates for \(r\)-Whitney numbers of the second kind
- A stochastic variance reduced primal dual fixed point method for linearly constrained separable optimization
- Variance comparison between infinitesimal perturbation analysis and likelihood ratio estimators to stochastic gradient
- Cocoercivity, smoothness and bias in variance-reduced stochastic gradient methods
- On the Convergence of Stochastic Gradient Descent for Linear Inverse Problems in Banach Spaces
- Stochastic gradient descent for linear inverse problems in Hilbert spaces
- Analysis and improvement for a class of variance reduced methods
- A stochastic gradient descent approach with partitioned-truncated singular value decomposition for large-scale inverse problems of magnetic modulus data
This page was built for publication: An analysis of stochastic variance reduced gradient for linear inverse problems *
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5019935)
