Derivative-free optimization: a review of algorithms and comparison of software implementations

DOI10.1007/S10898-012-9951-YzbMath1272.90116OpenAlexW2160960847WikidataQ61160682 ScholiaQ61160682MaRDI QIDQ2392129

Luis Miguel Rios, Nikolaos V. Sahinidis

Publication date: 1 August 2013

Published in: Journal of Global Optimization (Search for Journal in Brave)

Full work available at URL: https://doi.org/10.1007/s10898-012-9951-y

zbMATH Keywords

direct search methodssurrogate modelsderivative-free algorithms

Mathematics Subject Classification ID

Nonlinear programming (90C30) Derivative-free methods and methods using generalized derivatives (90C56)

Related Items (only showing first 100 items - show all)

A parallel evolution strategy for an Earth imaging problem in geophysics ⋮ Using a derivative-free optimization method for multiple solutions of inverse transport problems ⋮ Calibration by optimization without using derivatives ⋮ ARGONAUT: algorithms for global optimization of constrained grey-box computational problems ⋮ On the numerical performance of finite-difference-based methods for derivative-free optimization ⋮ Full-low evaluation methods for derivative-free optimization ⋮ A discussion on variational analysis in derivative-free optimization ⋮ Enhancing quantum efficiency of thin-film silicon solar cells by Pareto optimality ⋮ Efficient unconstrained black box optimization ⋮ DAHash: distribution aware tuning of password hashing costs ⋮ A novel sequential approximate optimization approach using data mining for engineering design optimization ⋮ Empirical study of the improved UNIRANDI local search method ⋮ Basin hopping networks of continuous global optimization problems ⋮ Best practices for comparing optimization algorithms ⋮ Global optimization advances in mixed-integer nonlinear programming, MINLP, and constrained derivative-free optimization, CDFO ⋮ Convergence acceleration of ensemble Kalman inversion in nonlinear settings ⋮ How difficult is nonlinear optimization? A practical solver tuning approach, with illustrative results ⋮ A heuristic approach for optimal planning and operation of distribution systems ⋮ Pseudo expected improvement criterion for parallel EGO algorithm ⋮ Numerical methods for computing plausibility and belief distributions of consequences of a subjective model of object of research ⋮ Identifying material parameters in crystal plasticity by Bayesian optimization ⋮ Global Linear Convergence of Evolution Strategies on More than Smooth Strongly Convex Functions ⋮ A survey of deep network techniques all classifiers can adopt ⋮ Integrated experimental design and nonlinear optimization to handle computationally expensive models under resource constraints ⋮ Optimization of black-box problems using Smolyak grids and polynomial approximations ⋮ On \texttt{MATLAB} experience in accelerating \texttt{DIRECT-GLce} algorithm for constrained global optimization through dynamic data structures and parallelization ⋮ Single observation adaptive search for discrete and continuous stochastic optimization ⋮ Parallel distributed block coordinate descent methods based on pairwise comparison oracle ⋮ Escaping local minima with local derivative-free methods: a numerical investigation ⋮ Quality gain analysis of the weighted recombination evolution strategy on general convex quadratic functions ⋮ Machine learning-based surrogate modeling for data-driven optimization: a comparison of subset selection for regression techniques ⋮ Deterministic global derivative-free optimization of black-box problems with bounded Hessian ⋮ Globally convergent evolution strategies for constrained optimization ⋮ On the convergence rate issues of general Markov search for global minimum ⋮ Computationally efficient integrated design and predictive control of flexible energy systems using multi‐fidelity simulation‐based Bayesian optimization ⋮ Metaheuristic vs. deterministic global optimization algorithms: the univariate case ⋮ Manifold learning for accelerating coarse-grained optimization ⋮ Derivative-Free Optimization for Population Dynamic Models ⋮ MDTri: robust and efficient global mixed integer search of spaces of multiple ternary alloys. A DIRECT-inspired optimization algorithm for experimentally accessible computational material design ⋮ Pareto-aware strategies for faster convergence in multi-objective multi-scale search optimization ⋮ Augmented Lagrangian Pattern Search Based Multi-Agent Model Predictive Control of Rhine-Meuse Delta ⋮ Numerical simulation of polynomial-speed convergence phenomenon ⋮ GPU parameter tuning for tall and skinny dense linear least squares problems ⋮ Optimal undulatory swimming for a single fish-like body and for a pair of interacting swimmers ⋮ A deterministic global optimization using smooth diagonal auxiliary functions ⋮ Two-stage stochastic, large-scale optimization of a decentralized energy system: a case study focusing on solar PV, heat pumps and storage in a residential quarter ⋮ Global convergence of a derivative-free inexact restoration filter algorithm for nonlinear programming ⋮ Numerical continuation method for nonlinear system of scalar and functional equations ⋮ Globalizer: a novel supercomputer software system for solving time-consuming global optimization problems ⋮ Gradient-based cuckoo search for global optimization ⋮ A local search method for costly black-box problems and its application to CSP plant start-up optimization refinement ⋮ A process transfer model-based optimal compensation control strategy for batch process using just-in-time learning and trust region method ⋮ Gradient surfing: a new deterministic approach for low-dimensional global optimization ⋮ A modified DIRECT algorithm with bilevel partition ⋮ Unified approach for solving box-constrained models with continuous or discrete variables by non monotone direct search methods ⋮ Derivative-Free Optimization of Noisy Functions via Quasi-Newton Methods ⋮ Decomposition in derivative-free optimization ⋮ A risk-averse stochastic program for integrated system design and preventive maintenance planning ⋮ Derivative-free superiorization with component-wise perturbations ⋮ A direct search algorithm for global optimization ⋮ Globally convergent evolution strategies ⋮ RBFOpt: an open-source library for black-box optimization with costly function evaluations ⋮ Finding Effective SAT Partitionings Via Black-Box Optimization ⋮ Directed particle swarm optimization with Gaussian-process-based function forecasting ⋮ Hill-Climbing Algorithm with a Stick for Unconstrained Optimization Problems ⋮ Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption ⋮ ParEGO extensions for multi-objective optimization of expensive evaluation functions ⋮ CGRS -- an advanced hybrid method for global optimization of continuous functions closely coupling extended random search and conjugate gradient method ⋮ Auction optimization using regression trees and linear models as integer programs ⋮ DEFT-FUNNEL: an open-source global optimization solver for constrained grey-box and black-box problems ⋮ How to catch a lion in the desert: on the solution of the coverage directed generation (CDG) problem ⋮ Improving the convergence rate of the DIRECT global optimization algorithm ⋮ Default policies for global optimisation of noisy functions with severe noise ⋮ A simplicial homology algorithm for Lipschitz optimisation ⋮ Computation of the output of a function with fuzzy inputs based on a low-rank tensor approximation ⋮ The DIRECT algorithm: 25 years later ⋮ Tuning BARON using derivative-free optimization algorithms ⋮ Global optimization via inverse distance weighting and radial basis functions ⋮ Advantages of simplicial partitioning for Lipschitz optimization problems with linear constraints ⋮ A sequential quadratic programming algorithm for equality-constrained optimization without derivatives ⋮ A class of derivative-free nonmonotone optimization algorithms employing coordinate rotations and gradient approximations ⋮ A new \texttt{DIRECT-GLh} algorithm for global optimization with hidden constraints ⋮ Derivative-free superiorization: principle and algorithm ⋮ Simulation optimization: a review of algorithms and applications ⋮ On the choice of the low-dimensional domain for global optimization via random embeddings ⋮ Global optimization based on active preference learning with radial basis functions ⋮ A globally convergent trust-region algorithm for unconstrained derivative-free optimization ⋮ Combining Bayesian optimization and Lipschitz optimization ⋮ Recursive modified pattern search on high-dimensional simplex: a blackbox optimization technique ⋮ Derivative-free optimization methods ⋮ GOPS: efficient RBF surrogate global optimization algorithm with high dimensions and many parallel processors including application to multimodal water quality PDE model calibration ⋮ Data-driven spatial branch-and-bound algorithms for box-constrained simulation-based optimization ⋮ Optimized ellipse packings in regular polygons ⋮ The theory of variational hybrid quantum-classical algorithms ⋮ Deep learning volatility: a deep neural network perspective on pricing and calibration in (rough) volatility models ⋮ MrDIRECT: a multilevel robust DIRECT algorithm for global optimization problems ⋮ Efficient numerical methods for the optimisation of large kinetic reaction mechanisms ⋮ Model Development and Optimization for Space Engineering: Concepts, Tools, Applications, and Perspectives ⋮ Review and comparison of algorithms and software for mixed-integer derivative-free optimization ⋮ A derivative-free scaling memoryless DFP method for solving large scale nonlinear monotone equations

Uses Software

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• HOPSPACK
• TOMLAB
• NOMADm
• LINDO
• COIN-OR
• SPACE
• DFO
• BOBYQA
• Ipopt
• IMFIL
• KELLEY
• CMA-ES
• Spacemap
• DAKOTA
• SNOBFIT
• LINDOGlobal
• fminsearch
• glcCluster
• MultiMin
• OQNLP
• UOBYQA
• EGO
• OrthoMADS
• BRENT
• MCS
• ORBIT

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Optimization by Simulated Annealing
• Constrained global optimization of expensive black box functions using radial basis functions
• Spent potliner treatment process optimization using a MADS algorithm
• Additive scaling and the \texttt{DIRECT} algorithm
• Improved strategies for radial basis function methods for global optimization
• A hybrid simplex search and particle swarm optimization for unconstrained optimization
• The GLOBAL optimization method revisited
• Incorporating minimum Frobenius norm models in direct search
• A computational framework for derivative-free optimization of cardiovascular geometries
• An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization
• A limit theorem for a class of inhomogeneous Markov processes
• Efficient global optimization of expensive black-box functions
• Global optimization by multilevel coordinate search
• Lipschitzian optimization without the Lipschitz constant
• Handbook of global optimization
• Recent progress in unconstrained nonlinear optimization without derivatives
• Design and analysis of computer experiments. With comments and a rejoinder by the authors
• A polyhedral branch-and-cut approach to global optimization
• A comparison of complete global optimization solvers
• Genetic algorithms: Foundations and applications
• UOBYQA: unconstrained optimization by quadratic approximation
• A taxonomy of global optimization methods based on response surfaces
• Global optimization of costly nonconvex functions using radial basis functions
• Finding local optima of high-dimensional functions using direct search methods
• A particle swarm pattern search method for bound constrained global optimization
• Convergence results for generalized pattern search algorithms are tight
• Reliability based design optimization using design explorer
• Optimal aeroacoustic shape design using the surrogate management framework
• A theoretical framework for simulated annealing
• Optimizing an Empirical Scoring Function for Transmembrane Protein Structure Determination
• A stochastic method for global optimization
• Improved Algorithms for Convex Minimization in Relative Scale
• On the Convergence of Pattern Search Algorithms
• Quantitative Object Reconstruction Using Abel Transform X-Ray Tomography and Mixed Variable Optimization
• Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions
• The global solver in the LINDO API
• Trailing-edge noise reduction using derivative-free optimization and large-eddy simulation
• Convergence of Mesh Adaptive Direct Search to Second‐Order Stationary Points
• Using Sampling and Simplex Derivatives in Pattern Search Methods
• Developments of NEWUOA for minimization without derivatives
• Using simplex gradients of nonsmooth functions in direct search methods
• OrthoMADS: A Deterministic MADS Instance with Orthogonal Directions
• Introduction to Derivative-Free Optimization
• ORBIT: Optimization by Radial Basis Function Interpolation in Trust-Regions
• `` Direct Search Solution of Numerical and Statistical Problems
• On the Convergence of the Multidirectional Search Algorithm
• Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
• Convergence of the Nelder--Mead Simplex Method to a Nonstationary Point
• Comparison of public-domain software for black box global optimization^∗
• Pattern Search Methods for Linearly Constrained Minimization
• On the Convergence of Asynchronous Parallel Pattern Search
• Pattern Search Algorithms for Bound Constrained Minimization
• Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition
• A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization with General Constraints and Simple Bounds
• On the Global Convergence of Derivative-Free Methods for Unconstrained Optimization
• An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima
• Fortified-Descent Simplicial Search Method: A General Approach
• Benchmarking Derivative-Free Optimization Algorithms
• Global Convergence of General Derivative-Free Trust-Region Algorithms to First- and Second-Order Critical Points
• A Progressive Barrier for Derivative-Free Nonlinear Programming
• Equation of State Calculations by Fast Computing Machines
• Hit-and-Run Algorithms for Generating Multivariate Distributions
• Mesh Adaptive Direct Search Algorithms for Constrained Optimization
• A Simplex Method for Function Minimization
• A Sequential Method Seeking the Global Maximum of a Function
• Sequential Application of Simplex Designs in Optimisation and Evolutionary Operation
• A radial basis function method for global optimization
• A locally-biased form of the DIRECT algorithm.
• Finite-time performance analysis of static simulated annealing algorithms
• Using DIRECT to solve an aircraft routing problem

This page was built for publication: Derivative-free optimization: a review of algorithms and comparison of software implementations