Convergence rates of efficient global optimization algorithms
From MaRDI portal
Abstract: Efficient global optimization is the problem of minimizing an unknown function f, using as few evaluations f(x) as possible. It can be considered as a continuum-armed bandit problem, with noiseless data and simple regret. Expected improvement is perhaps the most popular method for solving this problem; the algorithm performs well in experiments, but little is known about its theoretical properties. Implementing expected improvement requires a choice of Gaussian process prior, which determines an associated space of functions, its reproducing-kernel Hilbert space (RKHS). When the prior is fixed, expected improvement is known to converge on the minimum of any function in the RKHS. We begin by providing convergence rates for this procedure. The rates are optimal for functions of low smoothness, and we modify the algorithm to attain optimal rates for smoother functions. For practitioners, however, these results are somewhat misleading. Priors are typically not held fixed, but depend on parameters estimated from the data. For standard estimators, we show this procedure may never discover the minimum of f. We then propose alternative estimators, chosen to minimize the constants in the rate of convergence, and show these estimators retain the convergence rates of a fixed prior.
Recommendations
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
- On the convergence rates of expected improvement methods
- scientific article; zbMATH DE number 6276166
- No-regret Bayesian optimization with unknown hyperparameters
- An analysis of covariance parameters in Gaussian process-based optimization
Cited in
(83)- Training an artificial neural network for recognizing electron collision patterns
- pBO-2GP-3B: a batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics
- Convergence rate of a simulated annealing algorithm with noisy observations
- On the convergence rates of expected improvement methods
- Global optimization using Gaussian processes to estimate biological parameters from image data
- A theoretical framework for calibration in computer models: parametrization, estimation and convergence properties
- Generalized hierarchical expected improvement method based on black-box functions of adaptive search strategy
- A Hierarchical Expected Improvement Method for Bayesian Optimization
- Intergenerational risk sharing in a defined contribution pension system: analysis with Bayesian optimization
- Convergence rates of a global optimization algorithm
- Datadriven HOPGD based computational vademecum for welding parameter identification
- scientific article; zbMATH DE number 7306883 (Why is no real title available?)
- An asynchronous parallel high-throughput model calibration framework for crystal plasticity finite element constitutive models
- Parallel efficient global optimization by using the minimum energy criterion
- Constrained Bayesian optimization with noisy experiments
- Adjusted expected improvement for cumulative regret minimization in noisy Bayesian optimization
- Taking another step: a simple approach to high-dimensional Bayesian optimization
- Optimization on Manifolds via Graph Gaussian Processes
- Collaborative and adaptive Bayesian optimization for bounding variances and probabilities under hybrid uncertainties
- A new expected-improvement algorithm for continuous minimax optimization
- Global Convergence Rate Analysis of a Generic Line Search Algorithm with Noise
- An initialization strategy for high-dimensional surrogate-based expensive black-box optimization
- Combining Bayesian optimization and Lipschitz optimization
- A supermartingale approach to Gaussian process based sequential design of experiments
- Cross-validation-based adaptive sampling for Gaussian process models
- Expected improvement for expensive optimization: a review
- Gaussian process optimization with failures: classification and convergence proof
- An extended two-stage sequential optimization approach: properties and performance
- Deterministic global optimization with Gaussian processes embedded
- A model aggregation approach for high-dimensional large-scale optimization
- Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics
- Combined Global and Local Search for Optimization with Gaussian Process Models
- Rapid design of metamaterials via multitarget Bayesian optimization
- A Multilevel Simulation Optimization Approach for Quantile Functions
- Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation
- Convergence conditions and numerical comparison of global optimization methods based on dimensionality reduction schemes
- Examples of inconsistency in optimization by expected improvement
- No-regret Bayesian optimization with unknown hyperparameters
- Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions
- Robust randomized optimization with k nearest neighbors
- Random drift particle swarm optimization algorithm: convergence analysis and parameter selection
- Optimization of expensive black-box problems via gradient-enhanced Kriging
- Machine learning predictions of China commodity price indices
- Evaluation on different volume of fluid methods in unstructured solver under the optimized condition
- Voronoi candidates for Bayesian optimization
- Moderate deviations inequalities for Gaussian process regression
- Convergence Rates of Epsilon-Greedy Global Optimization Under Radial Basis Function Interpolation
- Distance-Distributed Design for Gaussian Process Surrogates
- FigBO: a generalized acquisition function framework with look-ahead capability for Bayesian optimization
- An analysis of covariance parameters in Gaussian process-based optimization
- Forecasts of wholesale food price indices through Gaussian process regressions
- Cross-validation based adaptive sampling for multilevel Gaussian process models
- Rates of Convergence for a Class of Global Stochastic Optimization Algorithms
- Bayesian optimization design for finding a maximum tolerated dose combination in phase I clinical trials
- Sequential Model-Based Optimization for Continuous Inputs with Finite Decision Space
- Effect of the subdivision strategy on convergence and efficiency of some global optimization algorithms
- On the convergence rate issues of general Markov search for global minimum
- Applying Bayesian optimization with Gaussian process regression to computational fluid dynamics problems
- TREGO: a trust-region framework for efficient global optimization
- Certified multifidelity zeroth-order optimization
- Complete expected improvement converges to an optimal budget allocation
- Algorithm 1025: PARyOpt: A Software for P arallel A synchronous R emote Ba y esian Opt imization
- Optimal learning for nonlinear parametric belief models over multidimensional continuous spaces
- Tracking global optima in dynamic environments with efficient global optimization
- BOB: Bayesian optimized bootstrap for approximate posterior sampling in Gaussian mixture models
- Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios
- Bayesian Optimization with Expensive Integrands
- A review of benchmark and test functions for global optimization algorithms and metaheuristics
- Small sample spaces for Gaussian processes
- Optimization of expensive black-box problems with penalized expected improvement
- Gaussian processes for history-matching: application to an unconventional gas reservoir
- Enhancing Gaussian process surrogates for optimization and posterior approximation via random exploration
- Expected improvement in efficient global optimization through bootstrapped Kriging
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
- Gaussian processes for computer experiments
- Bayesian optimization approaches for identifying the best genotype from a candidate population
- Lower bounds on the noiseless worst-case complexity of efficient global optimization
- Gaussian process bandits with adaptive discretization
- scientific article; zbMATH DE number 7370640 (Why is no real title available?)
- scientific article; zbMATH DE number 4020863 (Why is no real title available?)
- Derivative-free optimization methods
- An adaptive univariate global optimization algorithm and its convergence rate for twice continuously differentiable functions
- Improving the convergence rate of the DIRECT global optimization algorithm
This page was built for publication: Convergence rates of efficient global optimization algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5396713)
