On the Discrepancy Principle for Stochastic Gradient Descent

DOI10.1088/1361-6420/ABAA58MaRDI QIDQ6339736arXivFDO

Authors Tim Jahn, Bangti Jin

Publication date 30 April 2020

Published in arXiv

Full work available at URL https://doi.org/10.1088/1361-6420/ABAA58

Mathematics Subject Classification ID

General theory of linear operators (47Axx) Miscellaneous applications of operator theory (47Nxx) Nonparametric inference (62Gxx)

arXiv classification

cs.NAmath.NA

Abstract: Stochastic gradient descent (SGD) is a promising numerical method for solving large-scale inverse problems. However, its theoretical properties remain largely underexplored in the lens of classical regularization theory. In this note, we study the classical discrepancy principle, one of the most popular extit{a posteriori} choice rules, as the stopping criterion for SGD, and prove the finite iteration termination property and the convergence of the iterate in probability as the noise level tends to zero. The theoretical results are complemented with extensive numerical experiments.

Cited in

(3)

• Randomized Krylov-Projected Iterated Tikhonov Regularization for Large-Scale Ill-posed Problems Under A Posteriori Stopping Rule
• Early stopping of stochastic variance reduced gradient for linear inverse problems by the discrepancy principle
• On the convergence of doubly stochastic block gradient descent method for linear inverse problems

This page was built for publication: On the Discrepancy Principle for Stochastic Gradient Descent

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6339736)