A sub-sampled tensor method for nonconvex optimization

DOI10.1093/IMANUM/DRAC057MaRDI QIDQ6190818zbMATH OpenOpenAlexFDO

Authors Jonas Köhler

Publication date 6 February 2024

Published in IMA Journal of Numerical Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1911.10367

zbMATH Keywords

nonlinear optimizationunconstrained optimizationnon-convex optimizationtrust-region methodscubic regularization

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Nonconvex programming, global optimization (90C26)

Abstract: We present a stochastic optimization method that uses a fourth-order regularized model to find local minima of smooth and potentially non-convex objective functions with a finite-sum structure. This algorithm uses sub-sampled derivatives instead of exact quantities. The proposed approach is shown to find an $(epsilo{n}_1,epsilo{n}_2,epsilo{n}_3)$-third-order critical point in at most $$ iterations, thereby matching the rate of deterministic approaches. In order to prove this result, we derive a novel tensor concentration inequality for sums of tensors of any order that makes explicit use of the finite-sum structure of the objective function.

Recommendations

• Implementable tensor methods in unconstrained convex optimization
• Accelerated methods for nonconvex optimization
• Newton-type methods for non-convex optimization under inexact Hessian information
• An optimal high-order tensor method for convex optimization
• Introduction to high-order optimization methods

Cited in

(1)

• Inexact tensor methods and their application to stochastic convex optimization

This page was built for publication: A sub-sampled tensor method for nonconvex optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6190818)