Approximating the diagonal of a Hessian: which sample set of points should be used

DOI10.1007/S11075-022-01304-ZMaRDI QIDQ2084264zbMATH OpenOpenAlexFDO

Authors Gabriel Jarry-Bolduc

Publication date 18 October 2022

Published in Numerical Algorithms (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2104.11821

zbMATH Keywords

centered simplex gradientcentered simplex Hessian diagonalderivative-free optimization

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Derivative-free methods and methods using generalized derivatives (90C56)

Abstract: An explicit formula to approximate the diagonal entries of the Hessian is introduced. When the derivative-free technique called emph{generalized centered simplex gradient} is used to approximate the gradient, then the formula can be computed for only one additional function evaluation. An error bound is introduced and provides information on the form of the sample set of points that should be used to approximate the diagonal of a Hessian. If the sample set of points is built in a specific manner, it is shown that the technique is $mathcalO(Delt{a}_S^2)$ accurate approximation of the diagonal entries of the Hessian where $Delt{a}_S$ is the radius of the sample set.

Recommendations

• Gradient and diagonal Hessian approximations using quadratic interpolation models and aligned regular bases
• Diagonal Hessian approximation for limited memory quasi-Newton via variational principle
• Diagonal approximation of the Hessian by finite differences for unconstrained optimization
• Error bounds for overdetermined and underdetermined generalized centred simplex gradients
• Improved Hessian approximation with modified quasi-Cauchy relation for gradient-type method

Cites work

• A survey on direct search methods for blackbox optimization and their applications
• Calculus identities for generalized simplex gradients: rules and applications
• Constrained global optimization of expensive black box functions using radial basis functions
• Derivative-free and blackbox optimization
• Derivative-free optimization methods
• Derivative-free optimization of expensive functions with computational error using weighted regression
• Error bounds for overdetermined and underdetermined generalized centred simplex gradients
• Generalized inverses: theory and computations
• Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation
• Global convergence of radial basis function trust region derivative-free algorithms
• Gradient and diagonal Hessian approximations using quadratic interpolation models and aligned regular bases
• scientific article; zbMATH DE number 47926 (Why is no real title available?)
• scientific article; zbMATH DE number 1274356 (Why is no real title available?)
• scientific article; zbMATH DE number 1301900 (Why is no real title available?)
• scientific article; zbMATH DE number 961607 (Why is no real title available?)
• Introduction to Derivative-Free Optimization
• Least Frobenius norm updating of quadratic models that satisfy interpolation conditions
• Multivariable calculus with applications
• The calculus of simplex gradients
• Using Sampling and Simplex Derivatives in Pattern Search Methods
• Variational Analysis

Cited in

(3)

• A matrix algebra approach to approximate Hessians
• Using generalized simplex methods to approximate derivatives
• Gradient and diagonal Hessian approximations using quadratic interpolation models and aligned regular bases

Describes a project that uses

Uses Software

• KELLEY
• Boosters
• IMFIL

This page was built for publication: Approximating the diagonal of a Hessian: which sample set of points should be used

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2084264)