Hermite least squares optimization: a modification of BOBYQA for optimization with limited derivative information

DOI10.1007/S11081-023-09795-YarXiv2204.05022OpenAlexW4365449150MaRDI QIDQ6088560FDOQ6088560

Authors: Mona Fuhrländer, Sebastian Schöps 

Publication date: 16 November 2023

Published in: Optimization and Engineering (Search for Journal in Brave)

Abstract: In this work we propose two Hermite-type optimization methods, Hermite least squares and Hermite BOBYQA, specialized for the case that some partial derivatives of the objective function are available and others are not. The main objective is to reduce the number of objective function calls by maintaining the convergence properties. Both methods are modifications of Powell's derivative-free BOBYQA algorithm. But instead of (underdetermined) interpolation for building the quadratic subproblem in each iteration, the training data is enriched with first and -- if possible -- second order derivatives and then (weighted) least squares regression is used. Proofs for global convergence are discussed and numerical results are presented. Further, the applicability is verified for a realistic test case in the context of yield optimization. Numerical tests show that the Hermite least squares approach outperforms classic BOBYQA if half or more partial derivatives are available. In addition, the Hermite-type approaches achieve more robustness and thus better performance in case of noisy objective functions.

Full work available at URL: https://arxiv.org/abs/2204.05022

Recommendations

• Improving the flexibility and robustness of model-based derivative-free optimization solvers
• On the numerical performance of finite-difference-based methods for derivative-free optimization
• A Derivative-Free Nonlinear Least Squares Solver
• Derivative-free optimization methods
• A derivative-free algorithm for least-squares minimization

zbMATH Keywords

noiseoptimizationleast squaresHermite interpolationderivative-freeBOBYQA

Mathematics Subject Classification ID

Nonlinear programming (90C30) Derivative-free methods and methods using generalized derivatives (90C56)

Cites Work

• UOBYQA: unconstrained optimization by quadratic approximation
• Title not available (Why is that?)
• Optimization by simulated annealing
• A Simplex Method for Function Minimization
• Lipschitzian optimization without the Lipschitz constant
• Title not available (Why is that?)
• Introduction to Derivative-Free Optimization
• Title not available (Why is that?)
• Title not available (Why is that?)
• Derivative-free optimization of expensive functions with computational error using weighted regression
• Derivative-free optimization: a review of algorithms and comparison of software implementations
• Geometry of interpolation sets in derivative free optimization
• On multivariate Hermite interpolation
• Geometry of sample sets in derivative-free optimization: polynomial regression and underdetermined interpolation
• On fast trust region methods for quadratic models with linear constraints
• Derivative-free and blackbox optimization
• YIELD OPTIMIZATION BASED ON ADAPTIVE NEWTON-MONTE CARLO AND POLYNOMIAL SURROGATES
• Improving the flexibility and robustness of model-based derivative-free optimization solvers
• Escaping local minima with local derivative-free methods: a numerical investigation

This page was built for publication: Hermite least squares optimization: a modification of BOBYQA for optimization with limited derivative information

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6088560)