Convergence rates of efficient global optimization algorithms
From MaRDI portal
Abstract: Efficient global optimization is the problem of minimizing an unknown function f, using as few evaluations f(x) as possible. It can be considered as a continuum-armed bandit problem, with noiseless data and simple regret. Expected improvement is perhaps the most popular method for solving this problem; the algorithm performs well in experiments, but little is known about its theoretical properties. Implementing expected improvement requires a choice of Gaussian process prior, which determines an associated space of functions, its reproducing-kernel Hilbert space (RKHS). When the prior is fixed, expected improvement is known to converge on the minimum of any function in the RKHS. We begin by providing convergence rates for this procedure. The rates are optimal for functions of low smoothness, and we modify the algorithm to attain optimal rates for smoother functions. For practitioners, however, these results are somewhat misleading. Priors are typically not held fixed, but depend on parameters estimated from the data. For standard estimators, we show this procedure may never discover the minimum of f. We then propose alternative estimators, chosen to minimize the constants in the rate of convergence, and show these estimators retain the convergence rates of a fixed prior.
Recommendations
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
- On the convergence rates of expected improvement methods
- scientific article; zbMATH DE number 6276166
- No-regret Bayesian optimization with unknown hyperparameters
- An analysis of covariance parameters in Gaussian process-based optimization
Cited in
(89)- Effect of the subdivision strategy on convergence and efficiency of some global optimization algorithms
- Convergence rates of a global optimization algorithm
- On the convergence rate issues of general Markov search for global minimum
- Gaussian processes for history-matching: application to an unconventional gas reservoir
- Gaussian process bandits with adaptive discretization
- Constrained Bayesian optimization with noisy experiments
- Average performance of passive algorithms for global optimization
- An adaptive univariate global optimization algorithm and its convergence rate for twice continuously differentiable functions
- pBO-2GP-3B: a batch parallel known/unknown constrained Bayesian optimization with feasibility classification and its applications in computational fluid dynamics
- Datadriven HOPGD based computational vademecum for welding parameter identification
- Gaussian process optimization with failures: classification and convergence proof
- Expected improvement for expensive optimization: a review
- An extended two-stage sequential optimization approach: properties and performance
- Deterministic global optimization with Gaussian processes embedded
- Bayesian optimization approaches for identifying the best genotype from a candidate population
- Generalized hierarchical expected improvement method based on black-box functions of adaptive search strategy
- Applying Bayesian optimization with Gaussian process regression to computational fluid dynamics problems
- Rapid design of metamaterials via multitarget Bayesian optimization
- Combining Bayesian optimization and Lipschitz optimization
- Optimization of expensive black-box problems via gradient-enhanced Kriging
- A supermartingale approach to Gaussian process based sequential design of experiments
- Global optimization using Gaussian processes to estimate biological parameters from image data
- Examples of inconsistency in optimization by expected improvement
- Convergence conditions and numerical comparison of global optimization methods based on dimensionality reduction schemes
- Tracking global optima in dynamic environments with efficient global optimization
- Small sample spaces for Gaussian processes
- A new expected-improvement algorithm for continuous minimax optimization
- On the convergence rates of expected improvement methods
- scientific article; zbMATH DE number 4020863 (Why is no real title available?)
- Expected improvement in efficient global optimization through bootstrapped Kriging
- Optimal learning for nonlinear parametric belief models over multidimensional continuous spaces
- Gaussian processes for computer experiments
- An analysis of covariance parameters in Gaussian process-based optimization
- No-regret Bayesian optimization with unknown hyperparameters
- Rates of Convergence for a Class of Global Stochastic Optimization Algorithms
- Global Convergence Rate Analysis of a Generic Line Search Algorithm with Noise
- Convergence guarantees for Gaussian process means with misspecified likelihoods and smoothness
- Bayesian Optimization with Expensive Integrands
- A Multilevel Simulation Optimization Approach for Quantile Functions
- Combined Global and Local Search for Optimization with Gaussian Process Models
- Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions
- Kriging prediction with isotropic Matérn correlations: robustness and experimental designs
- An initialization strategy for high-dimensional surrogate-based expensive black-box optimization
- Complete expected improvement converges to an optimal budget allocation
- Improving the convergence rate of the DIRECT global optimization algorithm
- Derivative-free optimization methods
- Robust randomized optimization with k nearest neighbors
- Convergence rate of a simulated annealing algorithm with noisy observations
- A theoretical framework for calibration in computer models: parametrization, estimation and convergence properties
- Cross-validation-based adaptive sampling for Gaussian process models
- Algorithm 1025: PARyOpt: A Software for P arallel A synchronous R emote Ba y esian Opt imization
- Bayesian Estimation and Optimization for Learning Sequential Regularized Portfolios
- Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics
- Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation
- TREGO: a trust-region framework for efficient global optimization
- Parallel efficient global optimization by using the minimum energy criterion
- Optimization on Manifolds via Graph Gaussian Processes
- Collaborative and adaptive Bayesian optimization for bounding variances and probabilities under hybrid uncertainties
- An asynchronous parallel high-throughput model calibration framework for crystal plasticity finite element constitutive models
- Moderate deviations inequalities for Gaussian process regression
- Convergence Rates of Epsilon-Greedy Global Optimization Under Radial Basis Function Interpolation
- Lower bounds on the noiseless worst-case complexity of efficient global optimization
- A Hierarchical Expected Improvement Method for Bayesian Optimization
- Intergenerational risk sharing in a defined contribution pension system: analysis with Bayesian optimization
- Taking another step: a simple approach to high-dimensional Bayesian optimization
- Evaluation on different volume of fluid methods in unstructured solver under the optimized condition
- Distance-Distributed Design for Gaussian Process Surrogates
- Sequential Model-Based Optimization for Continuous Inputs with Finite Decision Space
- Bayesian optimization design for finding a maximum tolerated dose combination in phase I clinical trials
- Certified multifidelity zeroth-order optimization
- Optimization of expensive black-box problems with penalized expected improvement
- Machine learning predictions of China commodity price indices
- Voronoi candidates for Bayesian optimization
- A model aggregation approach for high-dimensional large-scale optimization
- Forecasts of wholesale food price indices through Gaussian process regressions
- BOB: Bayesian optimized bootstrap for approximate posterior sampling in Gaussian mixture models
- A review of benchmark and test functions for global optimization algorithms and metaheuristics
- Enhancing Gaussian process surrogates for optimization and posterior approximation via random exploration
- Adjusted expected improvement for cumulative regret minimization in noisy Bayesian optimization
- FigBO: a generalized acquisition function framework with look-ahead capability for Bayesian optimization
- Cross-validation based adaptive sampling for multilevel Gaussian process models
- Simple-regret rates and minimax optimality of fixed-prior expected improvement in Matérn and squared-exponential RKHSs
- Relaxed Gaussian process interpolation: a goal-oriented approach to Bayesian optimization
- Securities transaction settlement optimization on superconducting quantum devices
- Alternating Bayesian mixed integer optimization for context window extension in large language models without tuning model parameters
- Forecasts of composite real estate price indices through Gaussian process regressions
- Training an artificial neural network for recognizing electron collision patterns
- Random drift particle swarm optimization algorithm: convergence analysis and parameter selection
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
This page was built for publication: Convergence rates of efficient global optimization algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5396713)
