Improving implicit regularization of SGD with preconditioning for least square problems

DOI10.1137/24M1710425MaRDI QIDQ7305221zbMATH OpenFDO

Authors Junwei Su, Chuan Wu, Le Zheng, Difan Zou

Publication date 29 July 2026

Published in SIAM Journal on Mathematics of Data Science (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1137/24M1710425

Mathematics Subject Classification ID

Linear regression; mixed models (62J05) Preconditioners for iterative methods (65F08)

Cites work

• Benign overfitting in linear regression
• Dimension free ridge regression
• Dimensionality Reduction, Regularization, and Generalization in Overparameterized Regressions
• Double Double Descent: On Generalization Errors in Transfer Learning between Linear Regression Tasks
• Faster kernel ridge regression using sketching and preconditioning
• Harder, Better, Faster, Stronger Convergence Rates for Least-Squares Regression
• High-Dimensional Analysis of Double Descent for Linear Regression with Random Projections
• High-dimensional asymptotics of prediction: ridge regression and classification
• scientific article; zbMATH DE number 3227378 (Why is no real title available?)
• Low-rank approximation and regression in input sparsity time
• Parallelizing stochastic gradient descent for least squares regression: mini-batching, averaging, and model misspecification
• Preconditioning techniques for large linear systems: A survey
• Rates of Convergence of Estimates and Test Statistics
• Sketching as a tool for numerical linear algebra
• Surprises in high-dimensional ridgeless least squares interpolation
• The Elements of Statistical Learning
• The optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization

This page was built for publication: Improving implicit regularization of SGD with preconditioning for least square problems

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7305221)