GPU-Enabled Large-Scale Optimization Using Randomized Linear Algebra

DOI10.48550/arXiv.2609.08136MaRDI QIDQ7288245arXivFDO

Authors Pratik Rathore, Parth Nobel, Madeleine Udell, Zachary Frangella, Xuning Hu

Publication date 8 September 2026

Published in arXiv

Tldr This paper introduces rlaopt, a PyTorch-based package for large-scale optimization using randomized numerical linear algebra that addresses the gap in combining GPU acceleration with simple interfaces for specifying problems; it includes solvers for positive-definite systems and convex empirical risk minimization and supports differentiation to enable hyperparameter tuning.

Full work available at URL https://doi.org/10.48550/arXiv.2609.08136

arXiv classification

cs.LGmath.OCstat.ML

Cites work

• A dual algorithm for the solution of nonlinear variational problems via finite element approximation
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• A Limited Memory Algorithm for Bound Constrained Optimization
• A Stochastic Approximation Method
• A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines
• Adam: A Method for Stochastic Optimization
• Approximating spectral densities of large matrices
• Clarabel: An interior-point solver for conic programs with quadratic objectives
• Conic optimization via operator splitting and homogeneous self-dual embedding
• CuClarabel: GPU Acceleration for a Conic Optimization Solver
• cuPDLP.jl: A GPU Implementation of Restarted Primal-Dual Hybrid Gradient for Linear Programming in Julia
• CVXPY: a Python-embedded modeling language for convex optimization
• Differentiable Convex Optimization Layers
• Distributed optimization and statistical learning via the alternating direction method of multipliers
• Don't Jump Through Hoops and Remove Those Loops: SVRG and Katyusha are Better Without the Outer Loop
• Fast estimation of \(\mathrm{tr}(f(A))\) via stochastic Lanczos quadrature
• Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions
• Gaussian Processes for Machine Learning
• Katyusha: the first direct acceleration of stochastic gradient methods
• Large Scale Kernel Learning using Block Coordinate Descent
• LSMR: An Iterative Algorithm for Sparse Least-Squares Problems
• LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares
• Methods of conjugate gradients for solving linear systems
• Optimal rates for the regularized least-squares algorithm
• OSQP: an operator splitting solver for quadratic programs
• Practical Large-Scale Linear Programming using Primal-Dual Hybrid Gradient
• PyHessian: Neural Networks Through the Lens of the Hessian
• Randomized iterative methods for linear systems
• Randomized Numerical Linear Algebra: Foundations & Algorithms
• Robust Stochastic Approximation Approach to Stochastic Programming
• SAGA: A Fast Incremental Gradient Method With Support for Non-Strongly Convex Composite Objectives
• Sharp analysis of low-rank kernel matrix approximations
• Sketching as a tool for numerical linear algebra
• SketchySGD: reliable stochastic optimization via randomized curvature estimates

This page was built for publication: GPU-Enabled Large-Scale Optimization Using Randomized Linear Algebra

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7288245)