An overview of stochastic quasi-Newton methods for large-scale machine learning

DOI10.1007/S40305-023-00453-9zbMath1524.90220OpenAlexW4321850195MaRDI QIDQ6097379

Cong-Ying Han, Tian-de Guo, Yan Liu

Publication date: 5 June 2023

Published in: Journal of the Operations Research Society of China (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s40305-023-00453-9

zbMATH Keywords

BFGSlarge-scale machine learningstochastic quasi-Newton methods

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Large-scale problems in mathematical programming (90C06) Applications of mathematical programming (90C90) Methods of quasi-Newton type (90C53)

Related Items (1)

A neural network transformation based global optimization algorithm

Cites Work

• A Stochastic Quasi-Newton Method for Large-Scale Optimization
• Accelerated gradient methods for nonconvex nonlinear and stochastic programming
• An optimal method for stochastic composite optimization
• Erratum to: ``Minimizing finite sums with the stochastic average gradient
• Sample size selection in optimization methods for machine learning
• On the limited memory BFGS method for large scale optimization
• Convergence of quasi-Newton matrices generated by the symmetric rank one update
• Sub-sampled Newton methods
• Momentum and stochastic momentum for stochastic gradient, Newton, proximal point and subspace descent methods
• Limited-memory BFGS with displacement aggregation
• A stochastic extra-step quasi-Newton method for nonsmooth nonconvex optimization
• Newton-type methods for non-convex optimization under inexact Hessian information
• Stochastic quasi-Newton with line-search regularisation
• On the Global Convergence of the BFGS Method for Nonconvex Unconstrained Optimization Problems
• Global Convergence of Online Limited Memory BFGS
• On the Influence of Momentum Acceleration on Online Learning
• A Quasi-Newton Approach to Nonsmooth Convex Optimization Problems in Machine Learning
• Probabilistic Interpretation of Linear Solvers
• Newton Sketch: A Near Linear-Time Optimization Algorithm with Linear-Quadratic Convergence
• On the Use of Stochastic Hessian Information in Optimization Methods for Machine Learning
• Sizing and Least-Change Secant Methods
• Fast Curvature Matrix-Vector Products for Second-Order Gradient Descent
• Updating Quasi-Newton Matrices with Limited Storage
• Algorithms for nonlinear constraints that use lagrangian functions
• Adaptive Sampling Strategies for Stochastic Optimization
• RES: Regularized Stochastic BFGS Algorithm
• Semi-stochastic coordinate descent
• Randomized Quasi-Newton Updates Are Linearly Convergent Matrix Inversion Algorithms
• First-Order Methods in Optimization
• Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration Strategies
• Probabilistic Line Searches for Stochastic Optimization
• Block BFGS Methods
• Optimization Methods for Large-Scale Machine Learning
• A Theoretical and Experimental Study of the Symmetric Rank-One Update
• Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization I: A Generic Algorithmic Framework
• A robust multi-batch L-BFGS method for machine learning
• A Noise-Tolerant Quasi-Newton Algorithm for Unconstrained Optimization
• Quasi-Newton methods for machine learning: forget the past, just sample
• A Variable Sample-Size Stochastic Quasi-Newton Method for Smooth and Nonsmooth Stochastic Convex Optimization
• On Stochastic and Deterministic Quasi-Newton Methods for Nonstrongly Convex Optimization: Asymptotic Convergence and Rate Analysis
• An investigation of Newton-Sketch and subsampled Newton methods
• Stochastic proximal quasi-Newton methods for non-convex composite optimization
• Analysis of the BFGS Method with Errors
• Adaptive, Limited-Memory BFGS Algorithms for Unconstrained Optimization
• A Rapidly Convergent Descent Method for Minimization
• Quasi-Newton Methods and their Application to Function Minimisation
• A Family of Variable-Metric Methods Derived by Variational Means
• Variations on Variable-Metric Methods
• A new approach to variable metric algorithms
• Conditioning of Quasi-Newton Methods for Function Minimization
• Stochastic Quasi-Newton Methods for Nonconvex Stochastic Optimization
• IQN: An Incremental Quasi-Newton Method with Local Superlinear Convergence Rate
• Adjustment of an Inverse Matrix Corresponding to a Change in One Element of a Given Matrix
• Methods of conjugate gradients for solving linear systems
• A Stochastic Approximation Method
• Exact and inexact subsampled Newton methods for optimization
• A distributed optimisation framework combining natural gradient with Hessian-free for discriminative sequence training
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item

This page was built for publication: An overview of stochastic quasi-Newton methods for large-scale machine learning