Stochastic three-term conjugate gradient method with variance technique for non-convex learning

DOI10.1007/S11222-024-10409-5MaRDI QIDQ6570331zbMATH OpenFDO

Authors Chen Ouyang, Chenkaixiang Lu, Xiong Zhao, Ruping Huang, Gonglin Yuan, Yiyan Jiang

Publication date 10 July 2024

Published in Statistics and Computing (Search for Journal in Brave)

zbMATH Keywords

machine learninglinear convergencestochastic conjugate gradientempirical risk loss function minimization

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Numerical mathematical programming methods (65K05) Nonconvex programming, global optimization (90C26) Methods of reduced gradient type (90C52)

Recommendations

• A mini-batch stochastic conjugate gradient algorithm with variance reduction
• Nonconvex optimization with inertial proximal stochastic variance reduction gradient
• Conjugate-gradient-based Adam for nonconvex stochastic optimization and its application to deep learning
• Stochastic optimization algorithm with variance reduction for solving non-smooth problems
• Stochastic momentum methods for non-convex learning without bounded assumptions

Cites work

• scientific article; zbMATH DE number 3278849 (Why is no real title available?)
• A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search
• A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property
• A Stochastic Approximation Method
• A class of new three-term descent conjugate gradient algorithms for large-scale unconstrained optimization and applications to image restoration problems
• A modified HZ conjugate gradient algorithm without gradient Lipschitz continuous condition for non convex functions
• Adaptive three-term PRP algorithms without gradient Lipschitz continuity condition for nonconvex functions
• Conjugate gradient methods with Armijo-type line searches.
• Convergence Conditions for Ascent Methods
• Deep learning
• Function minimization by conjugate gradients
• Large-scale machine learning with stochastic gradient descent
• Minimizing finite sums with the stochastic average gradient
• Optimization methods for large-scale machine learning
• Some descent three-term conjugate gradient methods and their global convergence
• The conjugate gradient method in extremal problems
• The global convergence of the Polak-Ribière-Polyak conjugate gradient algorithm under inexact line search for nonconvex functions
• The modified PRP conjugate gradient algorithm under a non-descent line search and its application in the Muskingum model and image restoration problems

This page was built for publication: Stochastic three-term conjugate gradient method with variance technique for non-convex learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6570331)