Cross-validation-based adaptive sampling for Gaussian process models
From MaRDI portal
Publication:5862906
Abstract: In many real-world applications, we are interested in approximating black-box, costly functions as accurately as possible with the smallest number of function evaluations. A complex computer code is an example of such a function. In this work, a Gaussian process (GP) emulator is used to approximate the output of complex computer code. We consider the problem of extending an initial experiment (set of model runs) sequentially to improve the emulator. A sequential sampling approach based on leave-one-out (LOO) cross-validation is proposed that can be easily extended to a batch mode. This is a desirable property since it saves the user time when parallel computing is available. After fitting a GP to training data points, the expected squared LOO (ES-LOO) error is calculated at each design point. ES-LOO is used as a measure to identify important data points. More precisely, when this quantity is large at a point it means that the quality of prediction depends a great deal on that point and adding more samples nearby could improve the accuracy of the GP. As a result, it is reasonable to select the next sample where ES-LOO is maximised. However, ES-LOO is only known at the experimental design and needs to be estimated at unobserved points. To do this, a second GP is fitted to the ES-LOO errors and where the maximum of the modified expected improvement (EI) criterion occurs is chosen as the next sample. EI is a popular acquisition function in Bayesian optimisation and is used to trade-off between local/global search. However, it has a tendency towards exploitation, meaning that its maximum is close to the (current) "best" sample. To avoid clustering, a modified version of EI, called pseudo expected improvement, is employed which is more explorative than EI yet allows us to discover unexplored regions. Our results show that the proposed sampling method is promising.
Recommendations
- Gaussian process hyper-parameter estimation using parallel asymptotically independent Markov sampling
- Computer emulation with nonstationary Gaussian processes
- Exploiting Variance Reduction Potential in Local Gaussian Process Search
- Computer Experiments: Prediction Accuracy, Sample Size and Model Complexity Revisited
- Multiobjective optimization using Gaussian process emulators via stepwise uncertainty reduction
Cites work
- scientific article; zbMATH DE number 3163314 (Why is no real title available?)
- scientific article; zbMATH DE number 1209388 (Why is no real title available?)
- scientific article; zbMATH DE number 1522714 (Why is no real title available?)
- scientific article; zbMATH DE number 775913 (Why is no real title available?)
- A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output from a Computer Code
- A literature survey of benchmark functions for global optimisation problems
- A novel hybrid sequential design strategy for global surrogate modeling of computer experiments
- A survey of cross-validation procedures for model selection
- A taxonomy of global optimization methods based on response surfaces
- Accurate emulators for large-scale computer experiments
- Approximate leave-future-out cross-validation for Bayesian time series models
- Bayesian data analysis.
- Convergence properties of the expected improvement algorithm with fixed mean and covariance functions
- Convergence rates of efficient global optimization algorithms
- Cross validation and maximum likelihood estimations of hyper-parameters of Gaussian processes with model misspecification
- Cross-validation for selecting a model selection procedure
- Design and analysis of computer experiments. With comments and a rejoinder by the authors
- Design of computer experiments: space filling and beyond
- Diagnostics-driven nonstationary emulators using kernel mixtures
- Differential evolution -- a simple and efficient heuristic for global optimization over continuous spaces
- Efficient global optimization of expensive black-box functions
- Metamodels for computer-based engineering design: Survey and recommendations
- Multivariate adaptive regression splines
- Near-optimal sensor placements in Gaussian processes: theory, efficient algorithms and empirical studies
- On the convergence rates of expected improvement methods
- Pseudo expected improvement criterion for parallel EGO algorithm
- Quantification of Airfoil Geometry-Induced Aerodynamic Uncertainties---Comparison of Approaches
- Sequential Design with Mutual Information for Computer Experiments (MICE): Emulation of a Tsunami Model
- The design and analysis of computer experiments.
- Universal Prediction Distribution for Surrogate Models
Cited in
(6)- Cholesky-based experimental design for Gaussian process and kernel-based emulation and calibration
- Gradient and uncertainty enhanced sequential sampling for global fit
- Parallel cross-validation: a scalable fitting method for Gaussian process models
- Gaussian process hyper-parameter estimation using parallel asymptotically independent Markov sampling
- Adaptive Gaussian Process Approximation for Bayesian Inference with Expensive Likelihood Functions
- Adaptive multiple importance sampling for Gaussian processes
Describes a project that uses
Uses Software
This page was built for publication: Cross-validation-based adaptive sampling for Gaussian process models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5862906)
