Parallel Bayesian global optimization of expensive functions
From MaRDI portal
Abstract: We consider parallel global optimization of derivative-free expensive-to-evaluate functions, and propose an efficient method based on stochastic approximation for implementing a conceptual Bayesian optimization algorithm proposed by Ginsbourger et al. (2007). At the heart of this algorithm is maximizing the information criterion called the "multi-points expected improvement, or the q-EI. To accomplish this, we use infinitessimal perturbation analysis (IPA) to construct a stochastic gradient estimator and show that this estimator is unbiased. We also show that the stochastic gradient ascent algorithm using the constructed gradient estimator converges to a stationary point of the q-EI surface, and therefore, as the number of multiple starts of the gradient ascent algorithm and the number of steps for each start grow large, the one-step Bayes optimal set of points is recovered. We show in numerical experiments that our method for maximizing the q-EI is faster than methods based on closed-form evaluation using high-dimensional integration, when considering many parallel function evaluations, and is comparable in speed when considering few. We also show that the resulting one-step Bayes optimal algorithm for parallel global optimization finds high-quality solutions with fewer evaluations than a heuristic based on approximately maximizing the q-EI. A high-quality open source implementation of this algorithm is available in the open source Metrics Optimization Engine (MOE).
Recommendations
- A parallel Bayesian optimization method based on adaptive surrogate model
- Parallel simultaneous perturbation optimization
- Expected improvement for expensive optimization: a review
- Pseudo expected improvement criterion for parallel EGO algorithm
- Balancing global and local search in parallel efficient global optimization algorithms
Cites work
- A literature survey of benchmark functions for global optimisation problems
- An informational approach to the global optimization of expensive-to-evaluate functions
- Average performance of a class of adaptive algorithms for global optimization
- Bayesian optimization via simulation with pairwise sampling and correlated prior beliefs
- Bayesian stopping rules for multistart global optimization methods
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
- Design and analysis of computer experiments. With comments and a rejoinder by the authors
- Direct Search Methods on Parallel Machines
- Efficient global optimization of expensive black-box functions
- Gaussian processes for machine learning.
- Global optimization of stochastic black-box systems via sequential kriging meta-models
- scientific article; zbMATH DE number 3612796 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- On the condition number of covariance matrices in kriging, estimation, and simulation of random fields
- On the limited memory BFGS method for large scale optimization
- One-dimensional Global Optimization Based on Statistical Models
- One-dimensional global optimization for observations with noise
- Performance evaluation and perturbation analysis of discrete event dynamic systems
- The knowledge-gradient policy for correlated normal beliefs
Cited in
(27)- Stratified Bayesian optimization
- Constrained Bayesian optimization with noisy experiments
- Batch sequential adaptive designs for global optimization
- Parallel sequential Monte Carlo for stochastic gradient-free nonconvex optimization
- Combining Bayesian optimization and Lipschitz optimization
- Pseudo expected improvement criterion for parallel EGO algorithm
- Bayesian updating with two-step parallel Bayesian optimization and quadrature
- Multi-fidelity cost-aware Bayesian optimization
- A multicriteria generalization of Bayesian global optimization
- Bayesian Monte Carlo for the Global Optimization of Expensive Functions
- On parallel policies for ranking and selection problems
- Multiobjective Tree-Structured Parzen Estimator
- Are we forgetting about compositional optimisers in Bayesian optimisation?
- A parallel Bayesian optimization method based on adaptive surrogate model
- Balancing global and local search in parallel efficient global optimization algorithms
- Parallel simultaneous perturbation optimization
- Algorithm 1025: PARyOpt: A Software for P arallel A synchronous R emote Ba y esian Opt imization
- Parallel efficient global optimization by using the minimum energy criterion
- An asynchronous parallel high-throughput model calibration framework for crystal plasticity finite element constitutive models
- Scalable Bayesian optimization with generalized product of experts
- Optimal experimental design: formulations and computations
- A derivative-free approach to mixed integer constrained multiobjective nonsmooth black-box optimization
- Review of large-scale simulation optimization
- A bi-objective optimization based acquisition strategy for batch Bayesian global optimization
- A radial basis function method for noisy global optimisation
- A portfolio approach to massively parallel Bayesian optimization
- Batch Bayesian optimization via particle gradient flows
This page was built for publication: Parallel Bayesian global optimization of expensive functions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5144800)
